Go to file

Chris Darroch 762ccd4a49 subprocess: report errors when finding executables On Windows, when spawning a process, Go first looks for an executable file with the correct name in the current directory, and only if it fails to find one there does it look in the directories listed in the PATH environment variable. We would prefer not to replicate this behaviour and instead search only in the directories in PATH. Therefore, starting with the mitigation of CVE-2020-27955 in commit 74d5f2397f9abe4834bf1fe1fa02fd6c141b77ce, we resolve paths to executables ourselves rather than rely on Go to do so. The subprocess.LookPath() function we introduced in that change is adapted from Go's os/exec package. When it cannot find an executable in PATH, it returns an empty path string and an exec.ErrNotFound error. When that happens, we do not detect the condition and simply set the command path to the empty string. This can lead to undesirable behaviour in which a bug in the Go os/exec library causes it to run another executable other than the one we intended. When we set the command path to the empty string and then ask to execute the command, the native Go version of the LookPath() function for Windows is run with an empty path because filepath.Base() returns "." when passed the empty string and because we have left the current working directory also set to an empty string: `1724077b78/src/os/exec/exec.go (L348-L353)` Since the path string does not contain any path separator characters the current working directory is searched to try to find a file with a matching name and executable file extension (e.g., ".exe" or ".com"). To search the current working directory, the "." directory name is prepended to the given path with filepath.Join(): `1724077b78/src/os/exec/lp_windows.go (L84)` The filepath.Join() function ignores empty arguments, so this results in an incorrect filename of ".". All potential executable file extensions from the PATHEXT environment variable (or from a fixed list if that variable is not defined) are then appended to this filename, including their leading dot separator characters, thus producing filenames like "..com" and "..exe": `1724077b78/src/os/exec/lp_windows.go (L46-L50)` Should a file with one of these names exist in the current working directory, its name will be returned, which means that it will be executed instead of the command we expected to run. This is obviously undesirable and presents a risk to users. (Note that batch files named "..bat" and "..cmd" will also be found in the same manner, but they will not actually be executed due to an undocumented feature of the Windows API's family of CreateProcess() functions, which are used by Go to spawn processes. When passed an lpApplicationName argument ending with a ".bat" or ".cmd" extension, the CreateProcess() functions appear to instead execute "cmd.exe" and construct a "/c" command string from the lpCommandLine argument. The value of that argument is set using the full command line we are attempting to execute, and as such its first element is the actual name of the executable we intended to run; therefore, the command interpreter attempts to run the executable as a batch script and fails, and the "..bat" or "..cmd" file is effectively ignored.) To recap, when Git LFS attempts to execute a program on Windows, if the executable is not found anywhere in PATH but does exist in the current working directory, then when we call exec.Command() Go's internal LookPath() function will find the executable and not set the internal lookPathErr flag: `1724077b78/src/os/exec/exec.go (L174-L179)` Since we do not want to run executables in the current working directory, we subsequently run our own LookPath() function, which we use to reset the cmd.Path field. However, when our LookPath() returns an empty string, we do not detect that and instead set cmd.Path to that value. Then when we ask Go to run the command, because lookPathErr is nil, it proceeds, and the empty path causes it to find and run the first matching file in the working directory named "..com" or "..exe" (or any similar name with an executable file extension except for "..bat" and "..cmd"). We can prevent this behaviour by detecting when our LookPath() function returns an error and propagating it upwards to all callers of our subprocess.ExecCommand() function. We also add checks for this error at appropriate points and log or report the error as necessary. One particular circumstance of note occurs when a user has a Cygwin-installed "uname" in their PATH but not "cygpath", but "cygpath.exe" does exist in their current working directory. Then we will attempt to execute "cygpath" because we use the presence of "uname" as an indication that Cygwin is fully installed. Should a "..exe" or similar file also be present in the working directory, then it will be executed instead of "cygpath.exe". As with all other instances where we call subprocess.ExecCommand(), in tools.translateCygwinPath() we will now check for a returned error before trying to actually execute "cygpath". Unlike many of the other cases, though, we do not need to report the error in this one; instead, we simply return the current path from translateCygwinPath() instead of canonicalizing it, just as we do already if the "cygpath" executable fails for some reason. Finally, we add a new test to t/t-path.sh which checks for the incorrect execution of a "..exe" binary on Windows when "git.exe" is not found in PATH but does exist in the current working directory. This test passes when run with a Git LFS binary that includes the remediations from this commit, and fails otherwise. For our "malicious" binary named "..exe" we make use of the lfstest-badpathcheck test helper we added in a previous commit. We only run this test on Windows because the underlying bug in Go is Windows-specific as it depends on path extensions from PATHEXT being appended to the file name ".". Co-authored-by: brian m. carlson <bk2204@github.com>		2022-04-19 09:45:20 -07:00
.github	.github/workflows/*.yml: build on Windows 2019	2022-02-17 13:54:19 -08:00
commands	subprocess: report errors when finding executables	2022-04-19 09:45:20 -07:00
config	release: v3.1.0	2022-02-08 21:09:48 +00:00
creds	subprocess: report errors when finding executables	2022-04-19 09:45:20 -07:00
debian	Makefile,debian,rpm: build man pages into dirs	2022-02-27 12:06:53 -08:00
docker	docker: remove DOCKER_AUTOPULL	2022-02-07 14:42:36 +00:00
docs	docs/man: note push --all only pushes local refs	2022-03-02 11:51:26 -08:00
errors	errors: make strings translatable	2022-01-18 17:05:02 +00:00
filepathfilter	make additional message strings translatable	2022-01-29 22:36:19 -08:00
fs	fs: make strings translatable	2022-01-18 17:05:02 +00:00
git	subprocess: report errors when finding executables	2022-04-19 09:45:20 -07:00
lfs	subprocess: report errors when finding executables	2022-04-19 09:45:20 -07:00
lfsapi	make additional message strings translatable	2022-01-29 22:36:19 -08:00
lfshttp	subprocess: report errors when finding executables	2022-04-19 09:45:20 -07:00
locking	make additional message strings translatable	2022-01-29 22:36:19 -08:00
po	Introduce a single translation	2021-11-15 14:59:20 +00:00
rpm	Makefile,debian,rpm: build man pages into dirs	2022-02-27 12:06:53 -08:00
script	script/upload: correct RHEL 8 package repo	2022-04-11 18:29:51 +00:00
ssh	subprocess: report errors when finding executables	2022-04-19 09:45:20 -07:00
subprocess	subprocess: report errors when finding executables	2022-04-19 09:45:20 -07:00
t	subprocess: report errors when finding executables	2022-04-19 09:45:20 -07:00
tasklog	make additional message strings translatable	2022-01-29 22:36:19 -08:00
tools	subprocess: report errors when finding executables	2022-04-19 09:45:20 -07:00
tq	subprocess: report errors when finding executables	2022-04-19 09:45:20 -07:00
tr	tr: add a tool to build locales into the binary	2021-11-10 14:30:29 +00:00
vendor	go.*,vendor: bump gitobj to v2.1.0	2022-01-24 17:17:08 -08:00
.gitattributes	Enable autocrlf	2015-08-22 21:03:44 -04:00
.gitignore	Makefile,docs/man: build dynamic ronn index file	2022-02-07 00:24:34 -08:00
.mailmap	Add myself to .mailmap	2017-02-24 21:10:42 +01:00
CHANGELOG.md	release: v3.1.0	2022-02-08 21:09:48 +00:00
CODE-OF-CONDUCT.md	embed the open code of conduct since the link is bad now	2016-05-06 05:50:14 -06:00
CONTRIBUTING.md	Fix typos	2022-01-05 08:49:08 +02:00
git-lfs_windows_arm64.go	Fix Windows arm64 build	2021-09-25 22:14:52 +02:00
git-lfs_windows.go	Add support for Windows arm64	2021-08-18 22:01:11 +02:00
git-lfs.go	make additional message strings translatable	2022-01-29 22:36:19 -08:00
go.mod	go.*,vendor: bump gitobj to v2.1.0	2022-01-24 17:17:08 -08:00
go.sum	go.*,vendor: bump gitobj to v2.1.0	2022-01-24 17:17:08 -08:00
INSTALLING.md	update other github/git-lfs references	2016-11-15 10:07:11 -07:00
LICENSE.md	Update license year	2021-06-01 14:53:58 +02:00
Makefile	Makefile: update Windows signing certificate hash	2022-04-19 08:14:20 -07:00
README.md	Call out destructive command in README	2022-02-15 19:31:38 -05:00
SECURITY.md	{README,SECURITY}.md: add security bug report docs	2020-09-12 00:16:29 -07:00
versioninfo.json	release: v3.1.0	2022-02-08 21:09:48 +00:00

README.md

Git Large File Storage

Git LFS is a command line extension and specification for managing large files with Git.

The client is written in Go, with pre-compiled binaries available for Mac, Windows, Linux, and FreeBSD. Check out the website for an overview of features.

Getting Started

Downloading

You can install the Git LFS client in several different ways, depending on your setup and preferences.

Linux users. Debian and RPM packages are available from PackageCloud.
macOS users. Homebrew bottles are distributed, and can be installed via brew install git-lfs.
Windows users. Git LFS is included in the distribution of Git for Windows. Alternatively, you can install a recent version of Git LFS from the Chocolatey package manager.
Binary packages. In addition, binary packages are available for Linux, macOS, Windows, and FreeBSD.
Building from source. This repository can also be built from source using the latest version of Go, and the available instructions in our Wiki.

Note that Debian and RPM packages are built for all OSes for amd64 and i386. For arm64, only Debian packages for the latest Debian release are built due to the cost of building in emulation.

Installing

From binary

The binary packages include a script which will:

Install Git LFS binaries onto the system $PATH
Run git lfs install to perform required global configuration changes.

$ ./install.sh

From source

Ensure you have the latest version of Go, GNU make, and a standard Unix-compatible build environment installed.
On Windows, install goversioninfo with go get github.com/josephspurrier/goversioninfo/cmd/goversioninfo.
Run make.
Place the git-lfs binary, which can be found in bin, on your system’s executable $PATH or equivalent.
Git LFS requires global configuration changes once per-machine. This can be done by running:

$ git lfs install

Verifying releases

Releases are signed with the OpenPGP key of one of the core team members. To get these keys, you can run the following command, which will print them to standard output:

$ curl -L https://api.github.com/repos/git-lfs/git-lfs/tarball/core-gpg-keys | tar -Ozxf -

Once you have the keys, you can download the sha256sums.asc file and verify the file you want like so:

$ gpg -d sha256sums.asc | grep git-lfs-linux-amd64-v2.10.0.tar.gz | shasum -a 256 -c

Example Usage

To begin using Git LFS within a Git repository that is not already configured for Git LFS, you can indicate which files you would like Git LFS to manage. This can be done by running the following from within a Git repository:

$ git lfs track "*.psd"

(Where *.psd is the pattern of filenames that you wish to track. You can read more about this pattern syntax here).

Note: the quotation marks surrounding the pattern are important to prevent the glob pattern from being expanded by the shell.

After any invocation of git-lfs-track(1) or git-lfs-untrack(1), you must commit changes to your .gitattributes file. This can be done by running:

$ git add .gitattributes
$ git commit -m "track *.psd files using Git LFS"

You can now interact with your Git repository as usual, and Git LFS will take care of managing your large files. For example, changing a file named my.psd (tracked above via *.psd):

$ git add my.psd
$ git commit -m "add psd"

Tip: if you have large files already in your repository's history, git lfs track will not track them retroactively. To migrate existing large files in your history to use Git LFS, use git lfs migrate. For example:
$ git lfs migrate import --include="*.psd" --everything
Note that this will rewrite history and change all of the Git object IDs in your repository, just like the export version of this command.

For more information, read git-lfs-migrate(1).

You can confirm that Git LFS is managing your PSD file:

$ git lfs ls-files
3c2f7aedfb * my.psd

Once you've made your commits, push your files to the Git remote:

$ git push origin main
Uploading LFS objects: 100% (1/1), 810 B, 1.2 KB/s
# ...
To https://github.com/git-lfs/git-lfs-test
   67fcf6a..47b2002  main -> main

Note: Git LFS requires at least Git 1.8.2 on Linux or 1.8.5 on macOS.

Uninstalling

If you've decided that Git LFS isn't right for you, you can convert your repository back to a plain Git repository with git lfs migrate as well. For example:

$ git lfs migrate export --include="*.psd" --everything

Note that this will rewrite history and change all of the Git object IDs in your repository, just like the import version of this command.

If there's some reason that things aren't working out for you, please let us know in an issue, and we'll definitely try to help or get it fixed.

Limitations

Git LFS maintains a list of currently known limitations, which you can find and edit here.

Git LFS source code utilizes Go modules in its build system, and therefore this project contains a go.mod file with a defined Go module path. However, we do not maintain a stable Go language API or ABI, as Git LFS is intended to be used solely as a compiled binary utility. Please do not import the git-lfs module into other Go code and do not rely on it as a source code dependency.

Need Help?

You can get help on specific commands directly:

$ git lfs help <subcommand>

The official documentation has command references and specifications for the tool. There's also a FAQ on the wiki which answers some common questions.

If you have a question on how to use Git LFS, aren't sure about something, or are looking for input from others on tips about best practices or use cases, feel free to start a discussion.

You can always open an issue, and one of the Core Team members will respond to you. Please be sure to include:

The output of git lfs env, which displays helpful information about your Git repository useful in debugging.
Any failed commands re-run with GIT_TRACE=1 in the environment, which displays additional information pertaining to why a command crashed.

Contributing

See CONTRIBUTING.md for info on working on Git LFS and sending patches. Related projects are listed on the Implementations wiki page.

See also SECURITY.md for info on how to submit reports of security vulnerabilities.

Core Team

These are the humans that form the Git LFS core team, which runs the project.

In alphabetical order:

@bk2204	@chrisd8088	@larsxschneider

PGP 0223B187	PGP 088335A9	PGP A5795889

Alumni

These are the humans that have in the past formed the Git LFS core team, or have otherwise contributed a significant amount to the project. Git LFS would not be possible without them.