2015-07-30 02:37:31 +00:00
|
|
|
#!/usr/bin/env bash
|
2018-07-09 20:47:40 +00:00
|
|
|
# Including in script/integration and every t/t-*.sh file.
|
2015-05-18 23:28:58 +00:00
|
|
|
|
|
|
|
|
set -e
|
|
|
|
|
|
2016-03-18 15:03:05 +00:00
|
|
|
UNAME=$(uname -s)
|
|
|
|
|
IS_WINDOWS=0
|
|
|
|
|
IS_MAC=0
|
t/{t-path,testenv}.sh: refactor and fix path tests
In commits 74d5f2397f9abe4834bf1fe1fa02fd6c141b77ce and
10c4ffc6b888eee8f2134a7009a0db1bc393e17b the t/t-path.sh tests
were added to validate the remediations of the security issues
from CVE-2020-27955 and CVE-2021-21237, respectively. On Windows,
both of these tests make use of a "git.bat" script which stands in for
Git, and if the script is executed instead of the real Git during a
test, then that indicates that a security problem still exists.
However, in a previous commit we have now added a new helper program,
lfstest-badpathcheck, which can be used for the same purpose, and which
has the advantage of having a .exe extension on Windows. We will
make use of this helper program in a new test accompanying the
remediation of CVE-2022-24826.
The existence of this new helper also means we can now use it as a
fake "malicious" binary named "git.exe" in our existing t/t-path.sh
tests. This will help ensure that these tests are robust against
unexpected values of the Windows PATHEXT environment variable or
future changes to the set of executable file extensions we support
in Git LFS.
Using this new helper program instead of a "git.bat" script does mean
we need to be careful how and when it might run, because it mirrors
the name of the real Git executable. We therefore try to keep it out
of any possible execution path until we run our final concluding checks
in each test. In other words, rather than try to manipulate PATH (and
PATHEXT on Windows), we take steps to keep our "malicious" executable
out of any possible search path prior to when we want to explicitly
test Git LFS's behaviour in a known state relating to one of the CVEs.
(One challenge with manipulating PATH to try to remove paths that
resolve to the current directory is that it must be done iteratively
to deal with pathological cases like ":.:.:" and ":::". The "sed"
regular expressions in our existing tests would also need to use "\."
to match only the "." character, as well as additional expressions to
match "." at the beginning and end of PATH.)
It is easier, therefore, to simply avoid putting our "malicious"
executable into the current directory until the last moment. Also,
in the second test where we want to add it to a repository which
we will then clone, we can remove it as soon as we've run "git add",
and we can make sure to run the real Git (i.e., whatever is found
first using the extant PATH and PATHEXT variables) rather than ours
to perform that "git add".
When we reach the end of each test, where we want to explicitly run
Git LFS and see if it incorrectly searches in the current directory
even when not asked to, we now reset PATH and PATHEXT just for those
specific invocations of Git LFS. For PATH we use either BINPATH,
which contains only our compiled "git-lfs" binary, or BINPATH plus
the path to the real Git as returned by "command -v". For PATHEXT
we just use the primary executable file extension (if any) for the
current operating system.
To determine that primary executable file extension we add an X
variable which we set in t/testenv.sh and which parallels the one
set in the main Makefile. On Windows, we set X to contain ".exe",
and on Unix we set it to the empty string. We can then use this X
variable throughout our tests wherever we want to refer to a specific
executable's full filename.
With these changes, even when PATH includes "." as the first
directory to be searched, both of our tests should now always reach
their concluding checks and should function as expected at that
point. Specifically, they should fail by detecting the output of
our "malicious" Git program when run without the Git LFS code changes
corresponding to their respective CVEs, and should succeed otherwise.
(Technically, the second test will fail for a different reason
if the remediation for CVE-2020-27955 is not in effect than if only
the remediation for CVE-2021-21237 is not in effect. But the test
will fail at the same point in both cases, i.e., in its concluding
final check.)
In the final checks in both tests we search for the text string
"exploit" in the output log file captured after running a Git LFS
command, using the shell command "! grep -q 'exploit' output.log".
The "!" reverses the exit code from "grep", so if the word "exploit"
is found, the test should fail. This works in the first test because
the command is the last one in the test, so the inverted exit code
from "grep" is returned as the exit code from whole test's subshell.
However, in the second test several other commands follow this command,
and because "set -e" (which is standard at the top of every test)
ignores commands' exit codes when they are inverted with "!", the
test proceeds even if the word "exploit" is seen in the output log.
To resolve this problem we instead use a command pipeline and ensure
that when the "grep" succeeds, the exit code from the final command
in the pipeline is generated by "false". This successfully causes
the test to fail immediately when the word "exploit" is seen in the
output log file. Moreover, in both tests we now follow the "grep"
check with checks for the presence of a file named "exploit"; this
provides a second level of assurance that our "malicious" Git
program has not executed.
Finally, we add detailed comments regarding specific steps in both
tests where the intention and purpose may not be clear just from
the context.
2022-04-13 06:21:40 +00:00
|
|
|
X=""
|
2016-10-08 17:00:30 +00:00
|
|
|
SHASUM="shasum -a 256"
|
2017-08-08 17:46:12 +00:00
|
|
|
PATH_SEPARATOR="/"
|
2016-10-08 17:00:30 +00:00
|
|
|
|
2016-10-08 20:30:36 +00:00
|
|
|
if [[ $UNAME == MINGW* || $UNAME == MSYS* || $UNAME == CYGWIN* ]]
|
2016-03-18 15:03:05 +00:00
|
|
|
then
|
|
|
|
|
IS_WINDOWS=1
|
t/{t-path,testenv}.sh: refactor and fix path tests
In commits 74d5f2397f9abe4834bf1fe1fa02fd6c141b77ce and
10c4ffc6b888eee8f2134a7009a0db1bc393e17b the t/t-path.sh tests
were added to validate the remediations of the security issues
from CVE-2020-27955 and CVE-2021-21237, respectively. On Windows,
both of these tests make use of a "git.bat" script which stands in for
Git, and if the script is executed instead of the real Git during a
test, then that indicates that a security problem still exists.
However, in a previous commit we have now added a new helper program,
lfstest-badpathcheck, which can be used for the same purpose, and which
has the advantage of having a .exe extension on Windows. We will
make use of this helper program in a new test accompanying the
remediation of CVE-2022-24826.
The existence of this new helper also means we can now use it as a
fake "malicious" binary named "git.exe" in our existing t/t-path.sh
tests. This will help ensure that these tests are robust against
unexpected values of the Windows PATHEXT environment variable or
future changes to the set of executable file extensions we support
in Git LFS.
Using this new helper program instead of a "git.bat" script does mean
we need to be careful how and when it might run, because it mirrors
the name of the real Git executable. We therefore try to keep it out
of any possible execution path until we run our final concluding checks
in each test. In other words, rather than try to manipulate PATH (and
PATHEXT on Windows), we take steps to keep our "malicious" executable
out of any possible search path prior to when we want to explicitly
test Git LFS's behaviour in a known state relating to one of the CVEs.
(One challenge with manipulating PATH to try to remove paths that
resolve to the current directory is that it must be done iteratively
to deal with pathological cases like ":.:.:" and ":::". The "sed"
regular expressions in our existing tests would also need to use "\."
to match only the "." character, as well as additional expressions to
match "." at the beginning and end of PATH.)
It is easier, therefore, to simply avoid putting our "malicious"
executable into the current directory until the last moment. Also,
in the second test where we want to add it to a repository which
we will then clone, we can remove it as soon as we've run "git add",
and we can make sure to run the real Git (i.e., whatever is found
first using the extant PATH and PATHEXT variables) rather than ours
to perform that "git add".
When we reach the end of each test, where we want to explicitly run
Git LFS and see if it incorrectly searches in the current directory
even when not asked to, we now reset PATH and PATHEXT just for those
specific invocations of Git LFS. For PATH we use either BINPATH,
which contains only our compiled "git-lfs" binary, or BINPATH plus
the path to the real Git as returned by "command -v". For PATHEXT
we just use the primary executable file extension (if any) for the
current operating system.
To determine that primary executable file extension we add an X
variable which we set in t/testenv.sh and which parallels the one
set in the main Makefile. On Windows, we set X to contain ".exe",
and on Unix we set it to the empty string. We can then use this X
variable throughout our tests wherever we want to refer to a specific
executable's full filename.
With these changes, even when PATH includes "." as the first
directory to be searched, both of our tests should now always reach
their concluding checks and should function as expected at that
point. Specifically, they should fail by detecting the output of
our "malicious" Git program when run without the Git LFS code changes
corresponding to their respective CVEs, and should succeed otherwise.
(Technically, the second test will fail for a different reason
if the remediation for CVE-2020-27955 is not in effect than if only
the remediation for CVE-2021-21237 is not in effect. But the test
will fail at the same point in both cases, i.e., in its concluding
final check.)
In the final checks in both tests we search for the text string
"exploit" in the output log file captured after running a Git LFS
command, using the shell command "! grep -q 'exploit' output.log".
The "!" reverses the exit code from "grep", so if the word "exploit"
is found, the test should fail. This works in the first test because
the command is the last one in the test, so the inverted exit code
from "grep" is returned as the exit code from whole test's subshell.
However, in the second test several other commands follow this command,
and because "set -e" (which is standard at the top of every test)
ignores commands' exit codes when they are inverted with "!", the
test proceeds even if the word "exploit" is seen in the output log.
To resolve this problem we instead use a command pipeline and ensure
that when the "grep" succeeds, the exit code from the final command
in the pipeline is generated by "false". This successfully causes
the test to fail immediately when the word "exploit" is seen in the
output log file. Moreover, in both tests we now follow the "grep"
check with checks for the presence of a file named "exploit"; this
provides a second level of assurance that our "malicious" Git
program has not executed.
Finally, we add detailed comments regarding specific steps in both
tests where the intention and purpose may not be clear just from
the context.
2022-04-13 06:21:40 +00:00
|
|
|
X=".exe"
|
2016-10-08 17:00:30 +00:00
|
|
|
|
|
|
|
|
# Windows might be MSYS2 which does not have the shasum Perl wrapper
|
|
|
|
|
# script by default, so use sha256sum directly. MacOS on the other hand
|
|
|
|
|
# does not have sha256sum, so still use shasum as the default.
|
|
|
|
|
SHASUM="sha256sum"
|
2017-08-08 17:46:12 +00:00
|
|
|
PATH_SEPARATOR="\\"
|
2016-03-18 15:03:05 +00:00
|
|
|
elif [[ $UNAME == *Darwin* ]]
|
|
|
|
|
then
|
|
|
|
|
IS_MAC=1
|
|
|
|
|
fi
|
|
|
|
|
|
2020-06-16 15:23:40 +00:00
|
|
|
# Convert potentially MinGW bash paths to native Windows paths
|
|
|
|
|
# Needed to match generic built paths in test scripts to native paths generated from Go
|
|
|
|
|
native_path() {
|
|
|
|
|
local arg=$1
|
|
|
|
|
if [ $IS_WINDOWS -eq 1 ]; then
|
|
|
|
|
# Use params form to avoid interpreting any '\' characters
|
|
|
|
|
printf '%s' "$(cygpath -w $arg)"
|
|
|
|
|
else
|
|
|
|
|
printf '%s' "$arg"
|
|
|
|
|
fi
|
|
|
|
|
}
|
|
|
|
|
|
2016-03-18 15:03:05 +00:00
|
|
|
resolve_symlink() {
|
|
|
|
|
local arg=$1
|
2016-10-08 16:47:33 +00:00
|
|
|
if [ $IS_WINDOWS -eq 1 ]; then
|
2016-03-18 15:03:05 +00:00
|
|
|
printf '%s' "$arg"
|
2016-10-08 16:47:33 +00:00
|
|
|
elif [ $IS_MAC -eq 1 ]; then
|
2016-03-18 15:03:05 +00:00
|
|
|
# no readlink -f on Mac
|
|
|
|
|
local oldwd=$(pwd)
|
|
|
|
|
local target=$arg
|
|
|
|
|
|
|
|
|
|
cd `dirname $target`
|
|
|
|
|
target=`basename $target`
|
|
|
|
|
while [ -L "$target" ]
|
|
|
|
|
do
|
|
|
|
|
target=`readlink $target`
|
|
|
|
|
cd `dirname $target`
|
|
|
|
|
target=`basename $target`
|
|
|
|
|
done
|
|
|
|
|
|
|
|
|
|
local resolveddir=`pwd -P`
|
|
|
|
|
cd "$oldwd"
|
|
|
|
|
printf '%s' "$resolveddir/$target"
|
|
|
|
|
|
|
|
|
|
else
|
|
|
|
|
readlink -f "$arg"
|
|
|
|
|
fi
|
|
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
2015-06-19 17:06:55 +00:00
|
|
|
# The root directory for the git-lfs repository by default.
|
|
|
|
|
if [ -z "$ROOTDIR" ]; then
|
2018-07-12 17:39:39 +00:00
|
|
|
ROOTDIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd -P)"
|
2015-06-19 17:06:55 +00:00
|
|
|
fi
|
2015-05-19 22:54:15 +00:00
|
|
|
|
|
|
|
|
# Where Git LFS outputs the compiled binaries
|
2015-05-18 23:28:58 +00:00
|
|
|
BINPATH="$ROOTDIR/bin"
|
2015-05-19 22:54:15 +00:00
|
|
|
|
|
|
|
|
# Put bin path on PATH
|
2015-05-18 23:28:58 +00:00
|
|
|
PATH="$BINPATH:$PATH"
|
|
|
|
|
|
2016-03-18 15:04:49 +00:00
|
|
|
# Always provide a test dir outside our git repo if not specified
|
2016-03-24 17:41:42 +00:00
|
|
|
TEMPDIR_PREFIX="git-lfs_TEMP.XXXXXX"
|
2016-03-18 15:04:49 +00:00
|
|
|
if [ -z "$GIT_LFS_TEST_DIR" ]; then
|
|
|
|
|
GIT_LFS_TEST_DIR=$(mktemp -d -t "$TEMPDIR_PREFIX")
|
|
|
|
|
GIT_LFS_TEST_DIR=$(resolve_symlink $GIT_LFS_TEST_DIR)
|
|
|
|
|
# cleanup either after single test or at end of integration (except on fail)
|
|
|
|
|
RM_GIT_LFS_TEST_DIR=yes
|
2016-07-15 21:38:02 +00:00
|
|
|
fi
|
2015-05-18 23:28:58 +00:00
|
|
|
# create a temporary work space
|
2016-03-18 15:04:49 +00:00
|
|
|
TMPDIR=$GIT_LFS_TEST_DIR
|
2015-05-19 22:54:15 +00:00
|
|
|
|
|
|
|
|
# This is unique to every test file, and cleared after every test run.
|
2015-05-18 23:28:58 +00:00
|
|
|
TRASHDIR="$TMPDIR/$(basename "$0")-$$"
|
|
|
|
|
|
2015-05-19 22:54:15 +00:00
|
|
|
# The directory that the test Git server works from. This cleared at the
|
|
|
|
|
# beginning of every test run.
|
2018-07-09 20:47:40 +00:00
|
|
|
REMOTEDIR="$ROOTDIR/t/remote"
|
2015-05-19 22:54:15 +00:00
|
|
|
|
2015-08-04 16:06:30 +00:00
|
|
|
# The directory that stores credentials. Credentials are stored in files with
|
|
|
|
|
# the username:password with filenames identifying the host (port numbers are
|
|
|
|
|
# ignored).
|
|
|
|
|
#
|
|
|
|
|
# # stores the credentials for http://127.0.0.1:*
|
|
|
|
|
# $CREDSDIR/127.0.0.1
|
|
|
|
|
#
|
|
|
|
|
# # stores the credentials for http://git-server.com
|
|
|
|
|
# $CREDSDIR/git-server.com
|
|
|
|
|
#
|
2018-09-17 18:38:16 +00:00
|
|
|
CREDSDIR="$REMOTEDIR/creds/"
|
2015-08-04 16:06:30 +00:00
|
|
|
|
2015-05-19 22:54:15 +00:00
|
|
|
# This is the prefix for Git config files. See the "Test Suite" section in
|
2018-07-09 20:47:40 +00:00
|
|
|
# t/README.md
|
2015-05-18 23:28:58 +00:00
|
|
|
LFS_CONFIG="$REMOTEDIR/config"
|
|
|
|
|
|
2015-05-19 22:54:15 +00:00
|
|
|
# This file contains the URL of the test Git server. See the "Test Suite"
|
2018-07-09 20:47:40 +00:00
|
|
|
# section in t/README.md
|
2015-05-19 22:54:15 +00:00
|
|
|
LFS_URL_FILE="$REMOTEDIR/url"
|
|
|
|
|
|
2016-03-09 17:31:04 +00:00
|
|
|
# This file contains the SSL URL of the test Git server. See the "Test Suite"
|
2018-07-09 20:47:40 +00:00
|
|
|
# section in t/README.md
|
2016-03-09 17:31:04 +00:00
|
|
|
LFS_SSL_URL_FILE="$REMOTEDIR/sslurl"
|
|
|
|
|
|
2017-01-24 14:53:43 +00:00
|
|
|
# This file contains the client cert SSL URL of the test Git server. See the "Test Suite"
|
2018-07-09 20:47:40 +00:00
|
|
|
# section in t/README.md
|
2017-01-24 14:53:43 +00:00
|
|
|
LFS_CLIENT_CERT_URL_FILE="$REMOTEDIR/clientcerturl"
|
|
|
|
|
|
2016-03-09 17:31:04 +00:00
|
|
|
# This file contains the self-signed SSL cert of the TLS endpoint of the test Git server.
|
|
|
|
|
LFS_CERT_FILE="$REMOTEDIR/cert"
|
|
|
|
|
|
2017-01-24 14:53:43 +00:00
|
|
|
# This file contains the client certificate of the client cert endpoint of the test Git server.
|
|
|
|
|
LFS_CLIENT_CERT_FILE="$REMOTEDIR/client.crt"
|
|
|
|
|
|
|
|
|
|
# This file contains the client key of the client cert endpoint of the test Git server.
|
|
|
|
|
LFS_CLIENT_KEY_FILE="$REMOTEDIR/client.key"
|
|
|
|
|
|
2018-09-17 18:41:43 +00:00
|
|
|
# This file contains the client key of the client cert endpoint of the test Git server.
|
|
|
|
|
LFS_CLIENT_KEY_FILE_ENCRYPTED="$REMOTEDIR/client.enc.key"
|
|
|
|
|
|
2015-07-30 02:03:57 +00:00
|
|
|
# the fake home dir used for the initial setup
|
|
|
|
|
TESTHOME="$REMOTEDIR/home"
|
|
|
|
|
|
2018-10-30 08:06:55 +00:00
|
|
|
GIT_LFS_FORCE_PROGRESS=1
|
2015-05-26 15:47:57 +00:00
|
|
|
GIT_CONFIG_NOSYSTEM=1
|
2016-07-15 21:38:02 +00:00
|
|
|
GIT_TERMINAL_PROMPT=0
|
2017-05-19 15:25:22 +00:00
|
|
|
GIT_SSH=lfs-ssh-echo
|
2020-06-16 15:23:40 +00:00
|
|
|
GIT_TEMPLATE_DIR="$(native_path "$ROOTDIR/t/fixtures/templates")"
|
2017-01-04 20:27:05 +00:00
|
|
|
APPVEYOR_REPO_COMMIT_MESSAGE="test: env test should look for GIT_SSH too"
|
t/testenv: set LC_ALL to C
When we run the test suite, we search for specific output in Git's
output, but this output is dependent on the locale. Consequently,
running the test suite in a language for which Git has translations,
such as Spanish, French, or Russian, can cause test failures.
Let's set the locale to C, which will result in English output, so that
the tests pass. Ideally we'd use C.UTF-8, but not all systems have that
locale, so we need to stick with C.
2020-05-13 20:37:01 +00:00
|
|
|
LC_ALL=C
|
2015-05-26 15:39:04 +00:00
|
|
|
|
2015-08-11 20:18:30 +00:00
|
|
|
export CREDSDIR
|
2018-10-30 08:06:55 +00:00
|
|
|
export GIT_LFS_FORCE_PROGRESS
|
2016-09-11 15:41:06 +00:00
|
|
|
export GIT_CONFIG_NOSYSTEM
|
2017-01-04 18:16:07 +00:00
|
|
|
export GIT_SSH
|
2020-06-16 15:23:40 +00:00
|
|
|
export GIT_TEMPLATE_DIR
|
2017-01-04 20:27:05 +00:00
|
|
|
export APPVEYOR_REPO_COMMIT_MESSAGE
|
t/testenv: set LC_ALL to C
When we run the test suite, we search for specific output in Git's
output, but this output is dependent on the locale. Consequently,
running the test suite in a language for which Git has translations,
such as Spanish, French, or Russian, can cause test failures.
Let's set the locale to C, which will result in English output, so that
the tests pass. Ideally we'd use C.UTF-8, but not all systems have that
locale, so we need to stick with C.
2020-05-13 20:37:01 +00:00
|
|
|
export LC_ALL
|
2015-08-11 20:18:30 +00:00
|
|
|
|
2018-09-17 19:47:09 +00:00
|
|
|
# Don't fail if run under git rebase -x.
|
|
|
|
|
unset GIT_DIR
|
|
|
|
|
unset GIT_WORK_TREE
|
|
|
|
|
unset GIT_EXEC_PATH
|
|
|
|
|
unset GIT_CHERRY_PICK_HELP
|
|
|
|
|
|
2015-07-30 02:03:57 +00:00
|
|
|
mkdir -p "$TMPDIR"
|
2015-05-18 23:28:58 +00:00
|
|
|
mkdir -p "$TRASHDIR"
|
|
|
|
|
|
2016-10-08 16:47:33 +00:00
|
|
|
if [ $IS_WINDOWS -eq 1 ]; then
|
2016-07-15 13:53:02 +00:00
|
|
|
# prevent Windows OpenSSH from opening GUI prompts
|
|
|
|
|
SSH_ASKPASS=""
|
|
|
|
|
fi
|
2018-07-12 17:39:39 +00:00
|
|
|
|
|
|
|
|
. "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/testhelpers.sh"
|
