/bin/bash - Proper Whitespace Handling - Whitespace Safety - End-of-Options Parameter Security
Supporting multiple command line parameters with spaces in wrapper scripts and End-of-Options Parameter (--
) for better security.
Safe ways to print[edit]
There is no safe usage of echo
, use printf '%s'
instead.
shellcheck
bug reports:
Please note that printf
does not have a default format specifier, but treats the first positional parameter as the format. When the format is missing, the data is treated as if the format specifier is %b
. It is always recommended to be explicit on the format being used to avoid this mistake.
Normally, there is no need to interpret the escape sequences of a variable, therefore use the printf
format specifier %s
when the data is not printed to the terminal:
var="$(printf '%s' "${untrusted_text}")"
printf '%s\n' "message here"
is the equivalent of echo "message here"
.
If you require escapes to be interpreted, interpret them on a per-need basis:
red="$(printf '%b' "\e[31m")" # red=$'\e[31m' # printf -v red '%b' "\e[31m" nocolor="$(printf '%b' "\e[m")" # nocolor=$'\e[m' # printf -v nocolor '%b' "\e[m"
Escapes that are already interpreted can be printed with %s
without making a difference:
var="$(printf '%s' "${red} ${untrusted_text} ${nocolor}")"
And this is why you should use stprint
when printing to the terminal, as it will sanitize unsafe characters (unicode) while simply using printf '%s'
is not safe when escapes are already interpreted:
stprint "${red} ${untrusted_text} ${nocolor}" printf '%s' "${red} ${untrusted_text} ${nocolor}" | stprint printf '%s' "${red} ${untrusted_text} ${nocolor}" | stprint | less -R
Rule of thumb:
echo
: Never!printf
: Whenever the printed data is not used by a terminal.- Format
%b
: Only for trusted data. - Format
%s
: With any data.
- Format
stecho
: Whenever the printed data is used by a terminal.- When not using
stecho
: Whenstecho
cannot reasonably be considered available such as during early build-steps when building Kicksecure from source code using derivative-maker.
- When not using
Resources:
- https://github.com/anordal/shellharden/blob/master/how_to_do_things_safely_in_bash.md#echo--printf
- https://unix.stackexchange.com/questions/65803/why-is-printf-better-than-echo
- https://pubs.opengroup.org/onlinepubs/9799919799/utilities/echo.html
Bash Proper Whitespace Handling[edit]
- Quote variables.
- Build parameters using arrays.
- Enforce
nounset
. - Use end-of-options.
- Style: use long option names.
#!/bin/bash ## https://yakking.branchable.com/posts/whitespace-safety/ #set -x set -o errexit set -o nounset set -o errtrace set -o pipefail lib_dir="/tmp/test/lib/program with space/something spacy" main_app_dir="/tmp/test/home/user/folder with space/abc" mkdir --parents -- "${lib_dir}" mkdir --parents -- "${main_app_dir}" declare -a cmd_list cmd_list+=("cp") cmd_list+=("--recursive") cmd_list+=("--") cmd_list+=("${lib_dir}") cmd_list+=("${main_app_dir}/") ## Execution example. ## Note: drop 'echo' echo "${cmd_list[@]}" ## 'for' loop example. for cmd_item in "${cmd_list[@]}"; do printf '%s\n' "cmd_item: '$cmd_item'" done ## Alternative. cmd_alt_list=( cp ## program --recursive ## recursive -- ## stop option parsing (protects against paths that begin with '-') "$lib_dir" ## source directory "$main_app_dir" ## destination ) ## 'for' loop example. for cmd_alt_item in "${cmd_alt_list[@]}"; do printf '%s\n' "cmd_alt_item: '$cmd_alt_item'" done
Why nounset
[edit]
Because it is better to be explicit if a variable should be empty or not:
rm --force -- "/$UNSET_VAR"
Will return:
rm: cannot remove '/': Is a directory
Setting UNSET_VAR=""
would not fix this issue, but that is another problem, checking if every used variable can be empty or not.
local[edit]
Error swallowing[edit]
Note:
local testbar=$(false)
expected: error
actual: no error
better:
local testvar testvar=$(false)
Unexpected scoping with functions[edit]
local
variables in one function will be accessible within nested function calls.
Example:
fn_01 () { local myvar myvar='supposedly local' printf '%s\n' "in fn_01, myvar is $myvar" fn_02 printf '%s\n' "in fn_01, myvar is now $myvar" } fn_02 () { printf '%s\n' "in fn_02, myvar is $myvar" myvar='not so local after all' printf '%s\n' "in fn_02, myvar is now $myvar" } fn_01
Output:
in fn_01, myvar is supposedly local in fn_02, myvar is supposedly local in fn_02, myvar is now not so local after all in fn_01, myvar is now not so local after all
To avoid problems from this, it's best to declare all function-local variables as local
at the head of a function. For example:
fn_01 () { local myvar myvar='local to fn_01' printf '%s\n' "in fn_01, myvar is $myvar" fn_02 printf '%s\n' "in fn_01, myvar is now $myvar" } fn_02 () { local myvar myvar='local to fn_02' printf '%s\n' "in fn_02, myvar is $myvar" } fn_01
Output:
in fn_01, myvar is local to fn_01 in fn_02, myvar is local to fn_02 in fn_01, myvar is now local to fn_01
POSIX array[edit]
On a POSIX shell, there is one array, the $@
, which have different scopes by function or main script. You can build it with set --
:
Add items to an array:
set -- a b c
Add items to the beginning or end of the array:
set -- b set -- a "$@" c
Use of End-of-Options Parameter (--)[edit]
The end-of-options parameter "--
" is crucial because otherwise inputs might be mistaken for command options. This might even be a security risk. Here are examples using the `sponge` command:
sponge -a testfilename </dev/null
Result: OK. This works because "testfilename
" doesn't look like an option.
sponge -a --testfilename </dev/null
Result: Fail. The command interprets "--testfilename
" as a series of options:
sponge: invalid option -- '-' sponge: invalid option -- 't' sponge: invalid option -- 'e' ...
sponge -a -- --testfilename </dev/null
Result: OK. The `--
` signals that "--testfilename
" is a filename, not an option.
Conclusion:
- The "
--
" parameter marks the end of command options. - Use "
--
" at the end of a command to prevent misinterpretation. - This technique is applicable to many Unix/Linux commands, not just
sponge
.
nounset - Check if Variable Exists[edit]
#!/bin/bash set -x set -o errexit set -o nounset set -o errtrace set -o pipefail ## Enable for testing. #unset HOME if [ -z "${HOME+x}" ]; then printf '%s\n' "Error: HOME is not set." >&2 fi printf '%s' "$HOME"
Safely Using Find with End-Of-Options[edit]
Example:
Note: Variable could be different. Could be for example --/usr
.
folder_name="/usr"
printf '%s' "${folder_name}" | find -files0-from - -perm /u=s,g=s -print0
Of if safe_echo_nonewline
is available from helper-scripts.
https://github.com/Kicksecure/helper-scripts/blob/master/usr/libexec/helper-scripts/safe_echo.sh
# shellcheck disable=SC1091 source /usr/libexec/helper-scripts/safe_echo.sh safe_echo_nonewline "${folder_name}" | find -files0-from - -perm /u=s,g=s -print0
use stprint
instead?
loops[edit]
magic subshells[edit]
Avoid piping data from a command into a loop. This spawns a subshell even without using $()
syntax. Bad code example:
str="abc def ghi" line_count=0 printf '%s\n' "${str}" | while read -r line; do ((line_count += 1)) done printf '%s\n' "${line_count}" ## Expected result: 3 ## Actual result: 0
Instead, redirect files or command output into the loop. Good code example:
str="abc def ghi" line_count=0 while read -r line; do ((line_count += 1)) done < <(printf '%s\n' "${str}") printf '%s\n' "${line_count}" ## Result: 3
stdin stealing[edit]
Commands that read from stdin can swallow data that was supposed to be processed by the read
component of a while read
loop. qrexec-client-vm
is an example, vim
is another example. Bad code example:
str="abc def ghi" while read -r line; do vim "$line" done < <(printf '%s\n' "${str}") ## Output: ## ## Vim: Warning: Input is not from a terminal ## Vim: Error reading input, exiting... ## Vim: preserving files... ## Vim: Finished.
Work around this by using alternative file descriptors and redirection. Good code example:
str="abc def ghi" while read -r line 0<&3; do vim "$line" done 3< <(printf '%s\n' "${str}") ## Result: Opens "abc" in Vim, then "def", then "ghi".
misc[edit]
base_name="${file_name##*/}" file_extension="${base_name##*.}"
coding style[edit]
- use long options rather than short options, for example use
grep --invert-match
instead ofgrep -i
, when sensible - no trailing whitespaces allowed in source code files
- all source code files must have a newline at the end
- no git style symlinks (git symlinks) (text file without newline at the end) because of past git symlink CVE
- Avoid unicode whenever possible. See alsp unicode-show.
- use:
shellcheck
- avoid
rm
, prefersafe-rm
[1] - avoid
wget
andcurl
, preferscurl
(Secure Downloads) - avoid
grep
, usestr_match
str_replace
append-once
overwrite
- use
${variable}
style - use shell options
set -o errexit set -o nounset set -o errtrace set -o pipefail
- do not use:
which
, usecommand -v
instead. This is becausewhich
is an external binary, whichcommand
is a built-in (a bit faster).
pipefail echo printf grep quiet[edit]
This combination can be an issue due to broken pipe.
#!/bin/bash ## problem set -x set -o errexit set -o nounset set -o errtrace set -o pipefail counter=0 for i in {1..10000}; do counter=$(( counter + 1 )) #printf "0\n" echo "0\n" done | grep --quiet "0"
Improved Error Handler[edit]
Inspired by https://github.com/pottmi/stringent.sh
if (( "$BASH_SUBSHELL" >= 1 )); then kill "$$" fi
Actually not needed. When a subshell detects an error (thanks to errexit and errtrace), it returns and the parent shell will also catch the non-zero exit code. The script terminating itself and not running the error handler twice is only useful in rare cases.
Resources[edit]
- https://github.com/anordal/shellharden/blob/master/how_to_do_things_safely_in_bash.md
- https://dwheeler.com/essays/fixing-unix-linux-filenames.html
- use with care:
Seat Belts and Airbags for bash
- use with care: https://github.com/pottmi/stringent.sh
- use with care: https://github.com/pottmi/stringent.sh
See Also[edit]

We believe security software like Kicksecure needs to remain Open Source and independent. Would you help sustain and grow the project? Learn more about our 13 year success story and maybe DONATE!