/bin/bash - Proper Whitespace Handling - Whitespace Safety - End-of-Options Parameter Security

From Kicksecure
< Dev
Jump to navigation Jump to search
Design Previous page: Dev/Licensing Index page: Design Next page: Dev/certification /bin/bash - Proper Whitespace Handling - Whitespace Safety - End-of-Options Parameter Security

Supporting multiple command line parameters with spaces in wrapper scripts and End-of-Options Parameter (--) for better security.

Safe ways to print[edit]

There is no safe usage of echo, use printf '%s' instead.

shellcheck bug reports:

Please note that printf does not have a default format specifier, but treats the first positional parameter as the format. When the format is missing, the data is treated as if the format specifier is %b. It is always recommended to be explicit on the format being used to avoid this mistake.

Normally, there is no need to interpret the escape sequences of a variable, therefore use the printf format specifier %s when the data is not printed to the terminal:

var="$(printf '%s' "${untrusted_text}")"

printf '%s\n' "message here" is the equivalent of echo "message here".

If you require escapes to be interpreted, interpret them on a per-need basis:

red="$(printf '%b' "\e[31m")" # red=$'\e[31m' # printf -v red '%b' "\e[31m" nocolor="$(printf '%b' "\e[m")" # nocolor=$'\e[m' # printf -v nocolor '%b' "\e[m"

Escapes that are already interpreted can be printed with %s without making a difference:

var="$(printf '%s' "${red} ${untrusted_text} ${nocolor}")"

And this is why you should use stprint when printing to the terminal, as it will sanitize unsafe characters (unicode) while simply using printf '%s' is not safe when escapes are already interpreted:

stprint "${red} ${untrusted_text} ${nocolor}" printf '%s' "${red} ${untrusted_text} ${nocolor}" | stprint printf '%s' "${red} ${untrusted_text} ${nocolor}" | stprint | less -R

Rule of thumb:

  • echo: Never!
  • printf: Whenever the printed data is not used by a terminal.
    • Format %b: Only for trusted data.
    • Format %s: With any data.
  • stecho: Whenever the printed data is used by a terminal.
    • When not using stecho: When stecho cannot reasonably be considered available such as during early build-steps when building Kicksecure from source code using derivative-maker.

Resources:

Bash Proper Whitespace Handling[edit]

  • Quote variables.
  • Build parameters using arrays.
  • Enforce nounset.
  • Use end-of-options.
  • Style: use long option names.
#!/bin/bash

## https://yakking.branchable.com/posts/whitespace-safety/

#set -x
set -o errexit
set -o nounset
set -o errtrace
set -o pipefail

lib_dir="/tmp/test/lib/program with space/something spacy"
main_app_dir="/tmp/test/home/user/folder with space/abc"

mkdir --parents -- "${lib_dir}"
mkdir --parents -- "${main_app_dir}"

declare -a cmd_list

cmd_list+=("cp")
cmd_list+=("--recursive")
cmd_list+=("--")
cmd_list+=("${lib_dir}")
cmd_list+=("${main_app_dir}/")

## Execution example.
## Note: drop 'echo'
echo "${cmd_list[@]}"

## 'for' loop example.
for cmd_item in "${cmd_list[@]}"; do
    printf '%s\n' "cmd_item: '$cmd_item'"
done

## Alternative.
cmd_alt_list=(
    cp               ## program
    --recursive      ## recursive
    --               ## stop option parsing (protects against paths that begin with '-')
    "$lib_dir"       ## source directory
    "$main_app_dir"  ## destination
)

## 'for' loop example.
for cmd_alt_item in "${cmd_alt_list[@]}"; do
    printf '%s\n' "cmd_alt_item: '$cmd_alt_item'"
done

Why nounset[edit]

Because it is better to be explicit if a variable should be empty or not:

rm --force -- "/$UNSET_VAR"

Will return:

rm: cannot remove '/': Is a directory

Setting UNSET_VAR="" would not fix this issue, but that is another problem, checking if every used variable can be empty or not.

local[edit]

Error swallowing[edit]

Note:

local testbar=$(false)

expected: error

actual: no error

better:

local testvar testvar=$(false)

Unexpected scoping with functions[edit]

local variables in one function will be accessible within nested function calls.

Example:

fn_01 () { local myvar myvar='supposedly local' printf '%s\n' "in fn_01, myvar is $myvar" fn_02 printf '%s\n' "in fn_01, myvar is now $myvar" } fn_02 () { printf '%s\n' "in fn_02, myvar is $myvar" myvar='not so local after all' printf '%s\n' "in fn_02, myvar is now $myvar" } fn_01

Output:

in fn_01, myvar is supposedly local
in fn_02, myvar is supposedly local
in fn_02, myvar is now not so local after all
in fn_01, myvar is now not so local after all

To avoid problems from this, it's best to declare all function-local variables as local at the head of a function. For example:

fn_01 () { local myvar myvar='local to fn_01' printf '%s\n' "in fn_01, myvar is $myvar" fn_02 printf '%s\n' "in fn_01, myvar is now $myvar" } fn_02 () { local myvar myvar='local to fn_02' printf '%s\n' "in fn_02, myvar is $myvar" } fn_01

Output:

in fn_01, myvar is local to fn_01
in fn_02, myvar is local to fn_02
in fn_01, myvar is now local to fn_01

POSIX array[edit]

On a POSIX shell, there is one array, the $@, which have different scopes by function or main script. You can build it with set --:

Add items to an array:

set -- a b c

Add items to the beginning or end of the array:

set -- b
set -- a "$@" c

Use of End-of-Options Parameter (--)[edit]

The end-of-options parameter "--" is crucial because otherwise inputs might be mistaken for command options. This might even be a security risk. Here are examples using the `sponge` command:

sponge -a testfilename </dev/null

Result: OK. This works because "testfilename" doesn't look like an option.

sponge -a --testfilename </dev/null

Result: Fail. The command interprets "--testfilename" as a series of options:

sponge: invalid option -- '-'
sponge: invalid option -- 't'
sponge: invalid option -- 'e'
...

sponge -a -- --testfilename </dev/null

Result: OK. The `--` signals that "--testfilename" is a filename, not an option.

Conclusion:

  • The "--" parameter marks the end of command options.
  • Use "--" at the end of a command to prevent misinterpretation.
  • This technique is applicable to many Unix/Linux commands, not just sponge.

nounset - Check if Variable Exists[edit]

#!/bin/bash

set -x
set -o errexit
set -o nounset
set -o errtrace
set -o pipefail

## Enable for testing.
#unset HOME

if [ -z "${HOME+x}" ]; then
    printf '%s\n' "Error: HOME is not set." >&2
fi

printf '%s' "$HOME"

Safely Using Find with End-Of-Options[edit]

Example:

Note: Variable could be different. Could be for example --/usr.

folder_name="/usr"

printf '%s' "${folder_name}" | find -files0-from - -perm /u=s,g=s -print0

Of if safe_echo_nonewline is available from helper-scripts.

https://github.com/Kicksecure/helper-scripts/blob/master/usr/libexec/helper-scripts/safe_echo.sharchive.org iconarchive.today icon

# shellcheck disable=SC1091 source /usr/libexec/helper-scripts/safe_echo.sh safe_echo_nonewline "${folder_name}" | find -files0-from - -perm /u=s,g=s -print0

use stprint instead?

loops[edit]

magic subshells[edit]

Avoid piping data from a command into a loop. This spawns a subshell even without using $() syntax. Bad code example:

str="abc
def
ghi"
line_count=0

printf '%s\n' "${str}" | while read -r line; do
  ((line_count += 1))
done

printf '%s\n' "${line_count}"

## Expected result: 3
## Actual result: 0

Instead, redirect files or command output into the loop. Good code example:

str="abc
def
ghi"
line_count=0

while read -r line; do
  ((line_count += 1))
done < <(printf '%s\n' "${str}")

printf '%s\n' "${line_count}"

## Result: 3

stdin stealing[edit]

Commands that read from stdin can swallow data that was supposed to be processed by the read component of a while read loop. qrexec-client-vm is an example, vim is another example. Bad code example:

str="abc
def
ghi"

while read -r line; do
  vim "$line"
done < <(printf '%s\n' "${str}")

## Output:
##
## Vim: Warning: Input is not from a terminal
## Vim: Error reading input, exiting...
## Vim: preserving files...
## Vim: Finished.

Work around this by using alternative file descriptors and redirection. Good code example:

str="abc
def
ghi"

while read -r line 0<&3; do
  vim "$line"
done 3< <(printf '%s\n' "${str}")

## Result: Opens "abc" in Vim, then "def", then "ghi".

misc[edit]

base_name="${file_name##*/}"
file_extension="${base_name##*.}"

coding style[edit]

  • use long options rather than short options, for example use grep --invert-match instead of grep -i, when sensible
  • no trailing whitespaces allowed in source code files
  • all source code files must have a newline at the end
  • no git style symlinks (git symlinks) (text file without newline at the end) because of past git symlink CVEarchive.org iconarchive.today icon
  • Avoid unicode whenever possible. See alsp unicode-show.
  • use:
    • shellcheck
    • avoid rm, prefer safe-rm [1]
    • avoid wget and curl, prefer scurl (Secure Downloads)
    • avoid grep, use str_match
    • str_replace
    • append-once
    • overwrite
  • use ${variable} style
  • use shell options
set -o errexit
set -o nounset
set -o errtrace
set -o pipefail
  • do not use:
    • which, use command -v instead. This is because which is an external binary, which command is a built-in (a bit faster).

pipefail echo printf grep quiet[edit]

This combination can be an issue due to broken pipe.

#!/bin/bash

## problem

set -x
set -o errexit
set -o nounset
set -o errtrace
set -o pipefail

counter=0
for i in {1..10000}; do
  counter=$(( counter + 1 ))
  #printf "0\n"
  echo "0\n"

done | grep --quiet "0"

Improved Error Handler[edit]

Inspired by https://github.com/pottmi/stringent.sharchive.org iconarchive.today icon

if (( "$BASH_SUBSHELL" >= 1 )); then kill "$$" fi

Actually not needed. When a subshell detects an error (thanks to errexit and errtrace), it returns and the parent shell will also catch the non-zero exit code. The script terminating itself and not running the error handler twice is only useful in rare cases.

Resources[edit]

See Also[edit]

Design Previous page: Dev/Licensing Index page: Design Next page: Dev/certification

Notification image

We believe security software like Kicksecure needs to remain Open Source and independent. Would you help sustain and grow the project? Learn more about our 13 year success story and maybe DONATE!