Since we're about to do a v3.0.0 release, let's bump the version to v3.
Make this change automatically with the following command to avoid any
missed items:
git grep -l github.com/git-lfs/git-lfs/v2 | \
xargs sed -i -e 's!github.com/git-lfs/git-lfs/v2!github.com/git-lfs/git-lfs/v3!g'
When our go.mod file was introduced in commit
114e85c2002091eb415040923d872f8e4a4bc636 in PR #3208, the module
path chosen did not include a trailing /v2 component. However,
the Go modules specification now advises that module paths must
have a "major version suffix" which matches the release version.
We therefore add a /v2 suffix to our module path and all its
instances in import paths.
See also https://golang.org/ref/mod#major-version-suffixes for
details regarding the Go module system's major version suffix rule.
When using the pure SSH-based protocol, we can get much higher speeds by
multiplexing multiple connections on the same SSH connection. If we're
using OpenSSH, let's enable the ControlMaster option unless
lfs.ssh.automultiplex is set to false, and multiplex these shell
operations over one connection.
We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we
can share many connections over one socket, but if that's not set, let's
default to creating a new temporary directory for the socket. On
Windows, where the native SSH client doesn't support ControlMaster,
we should fall back to using multiple connections since we use
ControlMaster=auto.
Note that the option exists because users may already be using SSH
multiplexing and we would want to provide a way for them to disable
this, in addition to the case where users have an old or broken OpenSSH
which cannot support this option.
We pass the connection object into each worker and adjust our transfer
code to pass it into each function we invoke. We also make sure to
properly terminate each connection at the end by reducing our connection
count to 0, which closes the extra (i.e., all) connections.
Co-authored-by: Chris Darroch <chrisd8088@github.com>
We'll want to access the context that was originally passed to us later
on in a future commit, so let's preserve it so we can extract it from
our API client.
In the future, we'll want to call into the SSH code from multiple
packages, so let's move it out of the lfshttp package into its own
package to avoid package import loops. While we're at it, rename the
function names to remove the "ssh" prefix, since it's implied by the
fact that they're in a package called "ssh".
Move the tests to their own package to prevent an import loop and expose
the private functions so we can test them there.
Right now, all of the SSH metadata for an endpoint is in the Endpoint
struct, but in the future we'd like to move the SSH code to its own
package. At that point, we'll want to avoid a dependency on the
Endpoint struct, so let's move the SSH metadata out into its own struct,
which we'll include in Endpoint.
While we're at it, let's adjust most of the SSH code to use this new
struct instead so we can easily move it in the future.
The fix for CVE-2020-27955 was incomplete because we did not consider
places outside of the subprocess code that invoke binaries. As a
result, there are still some places where an attacker can execute
arbitrary code by placing a malicious binary in the repository.
To make sure we've covered all the bases, let's just use the subprocess
code for executing all programs, which means that they'll be secure. As
of this commit, all users of exec.Command are in test code or the
subprocess code itself.
When using SSH cloning with a ControlMaster socket and a password on a
Unix system, we're unable to read from standard input and consequently
the SSH process fails. The redirect from standard input was intended to
fix an issue on Windows that is due to /dev/tty disappearing, but it
doesn't appear to work and causes issues on Unix systems. Revert this
change so we can explore other options.
This reverts commit 59ed04855afd7e8f1bb126eb6f6a5c5297f59253.
Some people have SSH prompt them for their passphrase when a key is
used. In such a situation, the user needs to be able to enter a password
on standard input so that they can use their key. When invoking ssh,
pass our standard input through to the ssh process so that it can prompt
the user if needed.
There are a small number of places where we'll want to pass data to the
shell. Add a function which formats a command name and arguments into a
command that invokes "sh -c" with the appropriate args. Additionally
add a form that quotes its arguments, and use this in the SSH code,
which wants to pass data to the shell.
In a future commit, we'll be adding some additional shell-related
handling which logically belongs to the subprocess package. To avoid an
import loop, move the ShellQuote function and its required variable from
the tools package to the subprocess package. Update the only caller of
this function.
This commit contains no functional change.
Extract more basic http-related functionality out of lfsapi and
into a new package, lfshttp. Everything is currently functional
aside from authorization.
