Dev/Licensing Design Dev/certification Previous page: Dev/Licensing Index page: Design Next page: Dev/certification /bin/bash - Proper Whitespace Handling - Whitespace Safety - End-of-Options Parameter Security

Supporting multiple command line parameters with spaces in wrapper scripts and End-of-Options Parameter (--) for better security.

Safe ways to print[edit]

There is no safe usage of echo, use printf '%s' instead.

shellcheck bug reports:

Please note that printf does not have a default format specifier, but treats the first positional parameter as the format. When the format is missing, the data is treated as if the format specifier is %b. It is always recommended to be explicit on the format being used to avoid this mistake.

Normally, there is no need to interpret the escape sequences of a variable, therefore use the printf format specifier %s when the data is not printed to the terminal:

var="$(printf '%s' "${untrusted_text}")"

printf '%s\n' "message here" is the equivalent of echo "message here".

If you require escapes to be interpreted, interpret them on a per-need basis:

red="$(printf '%b' "\e[31m")" # red=$'\e[31m' # printf -v red '%b' "\e[31m" nocolor="$(printf '%b' "\e[m")" # nocolor=$'\e[m' # printf -v nocolor '%b' "\e[m"

Escapes that are already interpreted can be printed with %s without making a difference:

var="$(printf '%s' "${red} ${untrusted_text} ${nocolor}")"

And this is why you should use stprint when printing to the terminal, as it will sanitize unsafe characters (unicode) while simply using printf '%s' is not safe when escapes are already interpreted:

stprint "${red} ${untrusted_text} ${nocolor}" printf '%s' "${red} ${untrusted_text} ${nocolor}" | stprint printf '%s' "${red} ${untrusted_text} ${nocolor}" | stprint | less -R

Rule of thumb:

echo: Never!
printf: Whenever the printed data is not used by a terminal.
- Format %b: Only for trusted data.
- Format %s: With any data.
stecho: Whenever the printed data is used by a terminal.
- When not using stecho: When stecho cannot reasonably be considered available such as during early build-steps when building Kicksecure from source code using derivative-maker.

Resources:

Bash Proper Whitespace Handling[edit]

Quote variables.
Build parameters using arrays.
Enforce nounset.
Use end-of-options.
Style: use long option names.

#!/bin/bash

## https://yakking.branchable.com/posts/whitespace-safety/

#set -x
set -o errexit
set -o nounset
set -o errtrace
set -o pipefail

lib_dir="/tmp/test/lib/program with space/something spacy"
main_app_dir="/tmp/test/home/user/folder with space/abc"

mkdir --parents -- "${lib_dir}"
mkdir --parents -- "${main_app_dir}"

declare -a cmd_list

cmd_list+=("cp")
cmd_list+=("--recursive")
cmd_list+=("--")
cmd_list+=("${lib_dir}")
cmd_list+=("${main_app_dir}/")

## Execution example.
## Note: drop 'echo'
echo "${cmd_list[@]}"

## 'for' loop example.
for cmd_item in "${cmd_list[@]}"; do
    printf '%s\n' "cmd_item: '$cmd_item'"
done

## Alternative.
cmd_alt_list=(
    cp               ## program
    --recursive      ## recursive
    --               ## stop option parsing (protects against paths that begin with '-')
    "$lib_dir"       ## source directory
    "$main_app_dir"  ## destination
)

## 'for' loop example.
for cmd_alt_item in "${cmd_alt_list[@]}"; do
    printf '%s\n' "cmd_alt_item: '$cmd_alt_item'"
done

Why `nounset`[edit]

Because it is better to be explicit if a variable should be empty or not:

rm --force -- "/$UNSET_VAR"

Will return:

rm: cannot remove '/': Is a directory

Setting UNSET_VAR="" would not fix this issue, but that is another problem, checking if every used variable can be empty or not.

local[edit]

Error swallowing[edit]

Note:

local testbar=$(false)

expected: error

actual: no error

better:

local testvar testvar=$(false)

Unexpected scoping with functions[edit]

local variables in one function will be accessible within nested function calls.

Example:

fn_01 () { local myvar myvar='supposedly local' printf '%s\n' "in fn_01, myvar is $myvar" fn_02 printf '%s\n' "in fn_01, myvar is now $myvar" } fn_02 () { printf '%s\n' "in fn_02, myvar is $myvar" myvar='not so local after all' printf '%s\n' "in fn_02, myvar is now $myvar" } fn_01

Output:

in fn_01, myvar is supposedly local
in fn_02, myvar is supposedly local
in fn_02, myvar is now not so local after all
in fn_01, myvar is now not so local after all

To avoid problems from this, it's best to declare all function-local variables as local at the head of a function. For example:

fn_01 () { local myvar myvar='local to fn_01' printf '%s\n' "in fn_01, myvar is $myvar" fn_02 printf '%s\n' "in fn_01, myvar is now $myvar" } fn_02 () { local myvar myvar='local to fn_02' printf '%s\n' "in fn_02, myvar is $myvar" } fn_01

Output:

in fn_01, myvar is local to fn_01
in fn_02, myvar is local to fn_02
in fn_01, myvar is now local to fn_01

POSIX array[edit]

On a POSIX shell, there is one array, the $@, which have different scopes by function or main script. You can build it with set --:

Add items to an array:

set -- a b c

Add items to the beginning or end of the array:

set -- b
set -- a "$@" c

Use of End-of-Options Parameter (--)[edit]

The end-of-options parameter "--" is crucial because otherwise inputs might be mistaken for command options. This might even be a security risk. Here are examples using the `sponge` command:

sponge -a testfilename </dev/null

Result: OK. This works because "testfilename" doesn't look like an option.

sponge -a --testfilename </dev/null

Result: Fail. The command interprets "--testfilename" as a series of options:

sponge: invalid option -- '-'
sponge: invalid option -- 't'
sponge: invalid option -- 'e'
...

sponge -a -- --testfilename </dev/null

Result: OK. The `--` signals that "--testfilename" is a filename, not an option.

Conclusion:

The "--" parameter marks the end of command options.
Use "--" at the end of a command to prevent misinterpretation.
This technique is applicable to many Unix/Linux commands, not just sponge.

nounset - Check if Variable Exists[edit]

#!/bin/bash

set -x
set -o errexit
set -o nounset
set -o errtrace
set -o pipefail

## Enable for testing.
#unset HOME

if [ -z "${HOME+x}" ]; then
    printf '%s\n' "Error: HOME is not set." >&2
fi

printf '%s' "$HOME"

Safely Using Find with End-Of-Options[edit]

Example:

Note: Variable could be different. Could be for example --/usr.

folder_name="/usr"

printf '%s' "${folder_name}" | find -files0-from - -perm /u=s,g=s -print0

Of if safe_echo_nonewline is available from helper-scripts.

https://github.com/Kicksecure/helper-scripts/blob/master/usr/libexec/helper-scripts/safe_echo.sh

# shellcheck disable=SC1091 source /usr/libexec/helper-scripts/safe_echo.sh safe_echo_nonewline "${folder_name}" | find -files0-from - -perm /u=s,g=s -print0

use stprint instead?

loops[edit]

magic subshells[edit]

Avoid piping data from a command into a loop. This spawns a subshell even without using $() syntax. Bad code example:

str="abc
def
ghi"
line_count=0

printf '%s\n' "${str}" | while read -r line; do
  ((line_count += 1))
done

printf '%s\n' "${line_count}"

## Expected result: 3
## Actual result: 0

Instead, redirect files or command output into the loop. Good code example:

str="abc
def
ghi"
line_count=0

while read -r line; do
  ((line_count += 1))
done < <(printf '%s\n' "${str}")

printf '%s\n' "${line_count}"

## Result: 3

stdin stealing[edit]

Commands that read from stdin can swallow data that was supposed to be processed by the read component of a while read loop. qrexec-client-vm is an example, vim is another example. Bad code example:

str="abc
def
ghi"

while read -r line; do
  vim "$line"
done < <(printf '%s\n' "${str}")

## Output:
##
## Vim: Warning: Input is not from a terminal
## Vim: Error reading input, exiting...
## Vim: preserving files...
## Vim: Finished.

Work around this by using alternative file descriptors and redirection. Good code example:

str="abc
def
ghi"

while read -r line 0<&3; do
  vim "$line"
done 3< <(printf '%s\n' "${str}")

## Result: Opens "abc" in Vim, then "def", then "ghi".

misc[edit]

base_name="${file_name##*/}"
file_extension="${base_name##*.}"

coding style[edit]

use long options rather than short options, for example use grep --invert-match instead of grep -i, when sensible
no trailing whitespaces allowed in source code files
all source code files must have a newline at the end
no git style symlinks (git symlinks) (text file without newline at the end) because of past git symlink CVE
Avoid unicode whenever possible. See alsp unicode-show.
use:
- shellcheck
- avoid rm, prefer safe-rm ^[1]
- avoid wget and curl, prefer scurl (Secure Downloads)
- avoid grep, use str_match
- str_replace
- append-once
- overwrite
use ${variable} style
use shell options

set -o errexit
set -o nounset
set -o errtrace
set -o pipefail

do not use:
- which, use command -v instead. This is because which is an external binary, which command is a built-in (a bit faster).

pipefail echo printf grep quiet[edit]

This combination can be an issue due to broken pipe.

#!/bin/bash

## problem

set -x
set -o errexit
set -o nounset
set -o errtrace
set -o pipefail

counter=0
for i in {1..10000}; do
  counter=$(( counter + 1 ))
  #printf "0\n"
  echo "0\n"

done | grep --quiet "0"

Improved Error Handler[edit]

Inspired by https://github.com/pottmi/stringent.sh

if (( "$BASH_SUBSHELL" >= 1 )); then kill "$$" fi

Actually not needed. When a subshell detects an error (thanks to errexit and errtrace), it returns and the parent shell will also catch the non-zero exit code. The script terminating itself and not running the error handler twice is only useful in rare cases.

/bin/bash - Proper Whitespace Handling - Whitespace Safety - End-of-Options Parameter Security

Contents

Safe ways to print[edit]

Bash Proper Whitespace Handling[edit]

Why `nounset`[edit]

local[edit]

Error swallowing[edit]

Unexpected scoping with functions[edit]

POSIX array[edit]

Use of End-of-Options Parameter (--)[edit]

nounset - Check if Variable Exists[edit]

Safely Using Find with End-Of-Options[edit]

loops[edit]

magic subshells[edit]

stdin stealing[edit]

misc[edit]

coding style[edit]

pipefail echo printf grep quiet[edit]

Improved Error Handler[edit]

Resources[edit]

See Also[edit]

Navigation menu

/bin/bash - Proper Whitespace Handling - Whitespace Safety - End-of-Options Parameter Security

Safe ways to print[edit]

Bash Proper Whitespace Handling[edit]

Why nounset[edit]

local[edit]

Error swallowing[edit]

Unexpected scoping with functions[edit]

POSIX array[edit]

Use of End-of-Options Parameter (--)[edit]

nounset - Check if Variable Exists[edit]

Safely Using Find with End-Of-Options[edit]

loops[edit]

magic subshells[edit]

stdin stealing[edit]

misc[edit]

coding style[edit]

pipefail echo printf grep quiet[edit]

Improved Error Handler[edit]

Resources[edit]

See Also[edit]

Navigation menu

Search

Why `nounset`[edit]