John EricsonandRobert Hensing
c11836126b
Use `set -u` and `set -o pipefail` to catch accidental mistakes and failures more strongly. - `set -u` catches the use of undefined variables - `set -o pipefail` catches failures (like `set -e`) earlier in the pipeline. This makes the tests a bit more robust. It is nice to read code not worrying about these spurious success paths (via uncaught) errors undermining the tests. Indeed, I caught some bugs doing this. There are a few tests where we run a command that should fail, and then search its output to make sure the failure message is one that we expect. Before, since the `grep` was the last command in the pipeline the exit code of those failing programs was silently ignored. Now with `set -o pipefail` it won't be, and we have to do something so the expected failure doesn't accidentally fail the test. To do that we use `expect` and a new `expectStderr` to check for the exact failing exit code. See the comments on each for why. `grep -q` is replaced with `grepQuiet`, see the comments on that function for why. `grep -v` when we just want the exit code is replaced with `grepInverse, see the comments on that function for why. `grep -q -v` together is, surprise surprise, replaced with `grepQuietInverse`, which is both combined. Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
51 lines
1.1 KiB
Bash
Executable File
51 lines
1.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
|
|
set -eu -o pipefail
|
|
|
|
red=""
|
|
green=""
|
|
yellow=""
|
|
normal=""
|
|
|
|
test=$1
|
|
|
|
dir="$(dirname "${BASH_SOURCE[0]}")"
|
|
source "$dir/common-test.sh"
|
|
|
|
post_run_msg="ran test $test..."
|
|
if [ -t 1 ]; then
|
|
red="[31;1m"
|
|
green="[32;1m"
|
|
yellow="[33;1m"
|
|
normal="[m"
|
|
fi
|
|
|
|
run_test () {
|
|
(init_test 2>/dev/null > /dev/null)
|
|
log="$(run_test_proper 2>&1)" && status=0 || status=$?
|
|
}
|
|
|
|
run_test
|
|
|
|
# Hack: Retry the test if it fails with “unexpected EOF reading a line” as these
|
|
# appear randomly without anyone knowing why.
|
|
# See https://github.com/NixOS/nix/issues/3605 for more info
|
|
if [[ $status -ne 0 && $status -ne 99 && \
|
|
"$(uname)" == "Darwin" && \
|
|
"$log" =~ "unexpected EOF reading a line" \
|
|
]]; then
|
|
echo "$post_run_msg [${yellow}FAIL$normal] (possibly flaky, so will be retried)"
|
|
echo "$log" | sed 's/^/ /'
|
|
run_test
|
|
fi
|
|
|
|
if [ $status -eq 0 ]; then
|
|
echo "$post_run_msg [${green}PASS$normal]"
|
|
elif [ $status -eq 99 ]; then
|
|
echo "$post_run_msg [${yellow}SKIP$normal]"
|
|
else
|
|
echo "$post_run_msg [${red}FAIL$normal]"
|
|
echo "$log" | sed 's/^/ /'
|
|
exit "$status"
|
|
fi
|