lunny/git-lfs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
46d1a93b22
git-lfs/ssh/ssh_test.go

566 lines
18 KiB
Go
Raw Normal View History

Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
package ssh_test
import (
"net/url"
"path/filepath"
ssh: disable concurrent transfers if no multiplexing Right now, we try to spawn multiple SSH connections using ControlMaster when making multiple requests. However, if multiplexing is not enabled, we can spawn multiple actual connections, which can be expensive and require lots of authentication requests. Instead, let's make sure we don't enable multiple connections if multiplexing is not enabled, since this may cause the user to be prompted for multiple SSH key connection attempts (say, if they're using a security key) and is substantially more heavyweight than simply spawning a new process over the same connection. Since the code has different behaviour if `XDG_RUNTIME_DIR` is set, make sure to unset it for the tests so that we always try to create a temporary directory and don't otherwise fail because that environment variable points somewhere unexpected.
2022-10-05 14:03:18 +00:00
"strings"
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
"testing"
Set package version to v3 Since we're about to do a v3.0.0 release, let's bump the version to v3. Make this change automatically with the following command to avoid any missed items: git grep -l github.com/git-lfs/git-lfs/v2 | \ xargs sed -i -e 's!github.com/git-lfs/git-lfs/v2!github.com/git-lfs/git-lfs/v3!g'
2021-09-01 19:41:10 +00:00
"github.com/git-lfs/git-lfs/v3/lfshttp"
"github.com/git-lfs/git-lfs/v3/ssh"
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestSSHGetLFSExeAndArgs(t *testing.T) {
cli, err := lfshttp.NewClient(nil)
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
meta.Path = "user/repo"
ssh: disable concurrent transfers if no multiplexing Right now, we try to spawn multiple SSH connections using ControlMaster when making multiple requests. However, if multiplexing is not enabled, we can spawn multiple actual connections, which can be expensive and require lots of authentication requests. Instead, let's make sure we don't enable multiple connections if multiplexing is not enabled, since this may cause the user to be prompted for multiple SSH key connection attempts (say, if they're using a security key) and is substantially more heavyweight than simply spawning a new process over the same connection. Since the code has different behaviour if `XDG_RUNTIME_DIR` is set, make sure to unset it for the tests so that we always try to create a temporary directory and don't otherwise fail because that environment variable points somewhere unexpected.
2022-10-05 14:03:18 +00:00
exe, args, _ := ssh.GetLFSExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, "git-lfs-authenticate", "download", false)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "ssh", exe)
assert.Equal(t, []string{
"user@foo.com",
"git-lfs-authenticate user/repo download",
}, args)
ssh: disable concurrent transfers if no multiplexing Right now, we try to spawn multiple SSH connections using ControlMaster when making multiple requests. However, if multiplexing is not enabled, we can spawn multiple actual connections, which can be expensive and require lots of authentication requests. Instead, let's make sure we don't enable multiple connections if multiplexing is not enabled, since this may cause the user to be prompted for multiple SSH key connection attempts (say, if they're using a security key) and is substantially more heavyweight than simply spawning a new process over the same connection. Since the code has different behaviour if `XDG_RUNTIME_DIR` is set, make sure to unset it for the tests so that we always try to create a temporary directory and don't otherwise fail because that environment variable points somewhere unexpected.
2022-10-05 14:03:18 +00:00
exe, args, _ = ssh.GetLFSExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, "git-lfs-authenticate", "upload", false)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "ssh", exe)
assert.Equal(t, []string{
"user@foo.com",
"git-lfs-authenticate user/repo upload",
}, args)
}
func TestSSHGetExeAndArgsSsh(t *testing.T) {
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": "",
"GIT_SSH": "",
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "ssh", exe)
ssh: avoid using -- where possible In Git LFS version 3.0, we added support for understanding the GIT_SSH_VARIANT environment variable and its corresponding configuration option. When we did so, we dropped the special-casing for program names and instead always used the -- setting for the default variant, "ssh". This variant represents OpenSSH, which as a program with normal POSIX option handling, supports -- just fine. Unfortunately, not all clients do. TeamCity's SSH client doesn't set the variant and also doesn't understand --. We need to handle this case to prevent option injection in case someone tries to use a URL like "ssh://-oProxyCommand=exploit/", which could otherwise lead to arbitrary code execution. However, we don't have to add this all the time, but only when the user-and-host portion starts with a dash. The reason is that the only other place an attacker could inject an option is in the path, and the path is not a separate option by itself. The path is always preceded by a command (either "git-lfs-authenticate" or "git-lfs-transfer" in a single option that contains the name of the command, the path, and the operation, separated by spaces. As a result, option injection is not possible in the path since those commands don't take options. Note that Git just dies in this case with a message like so: fatal: strange hostname '-oProxyCommand=exploit' blocked Let's adopt this approach of using -- less often so that we can more gracefully deal with this case and fix some SSH clients, even if they really should learn to understand this option. Update the test binary to expect the new results as well. Since we're adjusting the end-of-options delimiter such that it's only used for the default SSH variant and, in particular, only when the user/host parameter starts with a "-" character, we still need to accept that command argument format in our lfs-ssh-echo test helper program, so we keep that logic, but also simplify our logic for parsing the program's arguments so it will be, with luck, easier to maintain in the future. Note that we also remove the "$4" from one error message because it's no longer the case that that error condition always arises due to the fourth argument. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-11-15 15:53:03 +00:00
assert.Equal(t, []string{"user@foo.com"}, args)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
}
func TestSSHGetExeAndArgsSshCustomPort(t *testing.T) {
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": "",
"GIT_SSH": "",
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
meta.Port = "8888"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "ssh", exe)
ssh: avoid using -- where possible In Git LFS version 3.0, we added support for understanding the GIT_SSH_VARIANT environment variable and its corresponding configuration option. When we did so, we dropped the special-casing for program names and instead always used the -- setting for the default variant, "ssh". This variant represents OpenSSH, which as a program with normal POSIX option handling, supports -- just fine. Unfortunately, not all clients do. TeamCity's SSH client doesn't set the variant and also doesn't understand --. We need to handle this case to prevent option injection in case someone tries to use a URL like "ssh://-oProxyCommand=exploit/", which could otherwise lead to arbitrary code execution. However, we don't have to add this all the time, but only when the user-and-host portion starts with a dash. The reason is that the only other place an attacker could inject an option is in the path, and the path is not a separate option by itself. The path is always preceded by a command (either "git-lfs-authenticate" or "git-lfs-transfer" in a single option that contains the name of the command, the path, and the operation, separated by spaces. As a result, option injection is not possible in the path since those commands don't take options. Note that Git just dies in this case with a message like so: fatal: strange hostname '-oProxyCommand=exploit' blocked Let's adopt this approach of using -- less often so that we can more gracefully deal with this case and fix some SSH clients, even if they really should learn to understand this option. Update the test binary to expect the new results as well. Since we're adjusting the end-of-options delimiter such that it's only used for the default SSH variant and, in particular, only when the user/host parameter starts with a "-" character, we still need to accept that command argument format in our lfs-ssh-echo test helper program, so we keep that logic, but also simplify our logic for parsing the program's arguments so it will be, with luck, easier to maintain in the future. Note that we also remove the "$4" from one error message because it's no longer the case that that error condition always arises due to the fourth argument. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-11-15 15:53:03 +00:00
assert.Equal(t, []string{"-p", "8888", "user@foo.com"}, args)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
}
ssh: disable concurrent transfers if no multiplexing Right now, we try to spawn multiple SSH connections using ControlMaster when making multiple requests. However, if multiplexing is not enabled, we can spawn multiple actual connections, which can be expensive and require lots of authentication requests. Instead, let's make sure we don't enable multiple connections if multiplexing is not enabled, since this may cause the user to be prompted for multiple SSH key connection attempts (say, if they're using a security key) and is substantially more heavyweight than simply spawning a new process over the same connection. Since the code has different behaviour if `XDG_RUNTIME_DIR` is set, make sure to unset it for the tests so that we always try to create a temporary directory and don't otherwise fail because that environment variable points somewhere unexpected.
2022-10-05 14:03:18 +00:00
func TestSSHGetExeAndArgsSshMultiplexing(t *testing.T) {
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": "",
"GIT_SSH": "",
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
exe, baseargs, needShell, multiplexing := ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, true)
exe, args := ssh.FormatArgs(exe, baseargs, needShell, multiplexing)
assert.Equal(t, "ssh", exe)
assert.Equal(t, multiplexing, true)
assert.Equal(t, 3, len(args))
assert.Equal(t, "-oControlMaster=auto", args[0])
assert.True(t, strings.HasPrefix(args[1], "-oControlPath="))
assert.Equal(t, "user@foo.com", args[2])
}
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
func TestSSHGetExeAndArgsPlink(t *testing.T) {
plink := filepath.Join("Users", "joebloggs", "bin", "plink.exe")
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": "",
"GIT_SSH": plink,
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, plink, exe)
assert.Equal(t, []string{"user@foo.com"}, args)
}
func TestSSHGetExeAndArgsPlinkCustomPort(t *testing.T) {
plink := filepath.Join("Users", "joebloggs", "bin", "plink")
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": "",
"GIT_SSH": plink,
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
meta.Port = "8888"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, plink, exe)
assert.Equal(t, []string{"-P", "8888", "user@foo.com"}, args)
}
ssh: implement Git SSH variant selection We'd implemented Git's autodetection of SSH variants in the past, but we hadn't implemented the newer explicit SSH selection. Since we're about to pass some additional options to our ssh binaries, let's implement the proper variant using GIT_SSH_VARIANT and ssh.variant. Roughly, the algorithm is this: GIT_SSH_VARIANT overrides ssh.variant, and if neither is set, autodetection occurs. A user can specify either "ssh" (OpenSSH), "putty" (or its synonym "plink"), "tortoiseplink", or "simple", or, if they'd like the autodetection behavior, "auto". If the value is not any of these, then it is interpreted as "ssh". Remove the special-casing of certain command names in favor of the variant type. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-03-30 17:05:22 +00:00
func TestSSHGetExeAndArgsPlinkCustomPortExplicitEnvironment(t *testing.T) {
plink := filepath.Join("Users", "joebloggs", "bin", "ssh")
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": "",
"GIT_SSH": plink,
"GIT_SSH_VARIANT": "plink",
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
meta.Port = "8888"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
ssh: implement Git SSH variant selection We'd implemented Git's autodetection of SSH variants in the past, but we hadn't implemented the newer explicit SSH selection. Since we're about to pass some additional options to our ssh binaries, let's implement the proper variant using GIT_SSH_VARIANT and ssh.variant. Roughly, the algorithm is this: GIT_SSH_VARIANT overrides ssh.variant, and if neither is set, autodetection occurs. A user can specify either "ssh" (OpenSSH), "putty" (or its synonym "plink"), "tortoiseplink", or "simple", or, if they'd like the autodetection behavior, "auto". If the value is not any of these, then it is interpreted as "ssh". Remove the special-casing of certain command names in favor of the variant type. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-03-30 17:05:22 +00:00
assert.Equal(t, plink, exe)
assert.Equal(t, []string{"-P", "8888", "user@foo.com"}, args)
}
func TestSSHGetExeAndArgsPlinkCustomPortExplicitEnvironmentPutty(t *testing.T) {
plink := filepath.Join("Users", "joebloggs", "bin", "ssh")
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": "",
"GIT_SSH": plink,
"GIT_SSH_VARIANT": "putty",
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
meta.Port = "8888"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
ssh: implement Git SSH variant selection We'd implemented Git's autodetection of SSH variants in the past, but we hadn't implemented the newer explicit SSH selection. Since we're about to pass some additional options to our ssh binaries, let's implement the proper variant using GIT_SSH_VARIANT and ssh.variant. Roughly, the algorithm is this: GIT_SSH_VARIANT overrides ssh.variant, and if neither is set, autodetection occurs. A user can specify either "ssh" (OpenSSH), "putty" (or its synonym "plink"), "tortoiseplink", or "simple", or, if they'd like the autodetection behavior, "auto". If the value is not any of these, then it is interpreted as "ssh". Remove the special-casing of certain command names in favor of the variant type. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-03-30 17:05:22 +00:00
assert.Equal(t, plink, exe)
assert.Equal(t, []string{"-P", "8888", "user@foo.com"}, args)
}
func TestSSHGetExeAndArgsPlinkCustomPortExplicitEnvironmentSsh(t *testing.T) {
plink := filepath.Join("Users", "joebloggs", "bin", "ssh")
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": "",
"GIT_SSH": plink,
"GIT_SSH_VARIANT": "ssh",
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
meta.Port = "8888"
t: test batch transfers with git-lfs-transfer Add a test case to test batch transfers with git-lfs-transfer as well as git-lfs-authenticate. Label the test appropriately, so folks can distinguish between them.
2021-05-14 19:21:58 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
ssh: implement Git SSH variant selection We'd implemented Git's autodetection of SSH variants in the past, but we hadn't implemented the newer explicit SSH selection. Since we're about to pass some additional options to our ssh binaries, let's implement the proper variant using GIT_SSH_VARIANT and ssh.variant. Roughly, the algorithm is this: GIT_SSH_VARIANT overrides ssh.variant, and if neither is set, autodetection occurs. A user can specify either "ssh" (OpenSSH), "putty" (or its synonym "plink"), "tortoiseplink", or "simple", or, if they'd like the autodetection behavior, "auto". If the value is not any of these, then it is interpreted as "ssh". Remove the special-casing of certain command names in favor of the variant type. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-03-30 17:05:22 +00:00
assert.Equal(t, plink, exe)
ssh: avoid using -- where possible In Git LFS version 3.0, we added support for understanding the GIT_SSH_VARIANT environment variable and its corresponding configuration option. When we did so, we dropped the special-casing for program names and instead always used the -- setting for the default variant, "ssh". This variant represents OpenSSH, which as a program with normal POSIX option handling, supports -- just fine. Unfortunately, not all clients do. TeamCity's SSH client doesn't set the variant and also doesn't understand --. We need to handle this case to prevent option injection in case someone tries to use a URL like "ssh://-oProxyCommand=exploit/", which could otherwise lead to arbitrary code execution. However, we don't have to add this all the time, but only when the user-and-host portion starts with a dash. The reason is that the only other place an attacker could inject an option is in the path, and the path is not a separate option by itself. The path is always preceded by a command (either "git-lfs-authenticate" or "git-lfs-transfer" in a single option that contains the name of the command, the path, and the operation, separated by spaces. As a result, option injection is not possible in the path since those commands don't take options. Note that Git just dies in this case with a message like so: fatal: strange hostname '-oProxyCommand=exploit' blocked Let's adopt this approach of using -- less often so that we can more gracefully deal with this case and fix some SSH clients, even if they really should learn to understand this option. Update the test binary to expect the new results as well. Since we're adjusting the end-of-options delimiter such that it's only used for the default SSH variant and, in particular, only when the user/host parameter starts with a "-" character, we still need to accept that command argument format in our lfs-ssh-echo test helper program, so we keep that logic, but also simplify our logic for parsing the program's arguments so it will be, with luck, easier to maintain in the future. Note that we also remove the "$4" from one error message because it's no longer the case that that error condition always arises due to the fourth argument. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-11-15 15:53:03 +00:00
assert.Equal(t, []string{"-p", "8888", "user@foo.com"}, args)
ssh: implement Git SSH variant selection We'd implemented Git's autodetection of SSH variants in the past, but we hadn't implemented the newer explicit SSH selection. Since we're about to pass some additional options to our ssh binaries, let's implement the proper variant using GIT_SSH_VARIANT and ssh.variant. Roughly, the algorithm is this: GIT_SSH_VARIANT overrides ssh.variant, and if neither is set, autodetection occurs. A user can specify either "ssh" (OpenSSH), "putty" (or its synonym "plink"), "tortoiseplink", or "simple", or, if they'd like the autodetection behavior, "auto". If the value is not any of these, then it is interpreted as "ssh". Remove the special-casing of certain command names in favor of the variant type. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-03-30 17:05:22 +00:00
}
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
func TestSSHGetExeAndArgsTortoisePlink(t *testing.T) {
plink := filepath.Join("Users", "joebloggs", "bin", "tortoiseplink.exe")
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": "",
"GIT_SSH": plink,
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
ssh: implement Git SSH variant selection We'd implemented Git's autodetection of SSH variants in the past, but we hadn't implemented the newer explicit SSH selection. Since we're about to pass some additional options to our ssh binaries, let's implement the proper variant using GIT_SSH_VARIANT and ssh.variant. Roughly, the algorithm is this: GIT_SSH_VARIANT overrides ssh.variant, and if neither is set, autodetection occurs. A user can specify either "ssh" (OpenSSH), "putty" (or its synonym "plink"), "tortoiseplink", or "simple", or, if they'd like the autodetection behavior, "auto". If the value is not any of these, then it is interpreted as "ssh". Remove the special-casing of certain command names in favor of the variant type. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-03-30 17:05:22 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, plink, exe)
assert.Equal(t, []string{"-batch", "user@foo.com"}, args)
}
func TestSSHGetExeAndArgsTortoisePlinkCustomPort(t *testing.T) {
plink := filepath.Join("Users", "joebloggs", "bin", "tortoiseplink")
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": "",
"GIT_SSH": plink,
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
meta.Port = "8888"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, plink, exe)
assert.Equal(t, []string{"-batch", "-P", "8888", "user@foo.com"}, args)
}
ssh: implement Git SSH variant selection We'd implemented Git's autodetection of SSH variants in the past, but we hadn't implemented the newer explicit SSH selection. Since we're about to pass some additional options to our ssh binaries, let's implement the proper variant using GIT_SSH_VARIANT and ssh.variant. Roughly, the algorithm is this: GIT_SSH_VARIANT overrides ssh.variant, and if neither is set, autodetection occurs. A user can specify either "ssh" (OpenSSH), "putty" (or its synonym "plink"), "tortoiseplink", or "simple", or, if they'd like the autodetection behavior, "auto". If the value is not any of these, then it is interpreted as "ssh". Remove the special-casing of certain command names in favor of the variant type. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-03-30 17:05:22 +00:00
func TestSSHGetExeAndArgsTortoisePlinkCustomPortExplicitEnvironment(t *testing.T) {
plink := filepath.Join("Users", "joebloggs", "bin", "ssh")
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": "",
"GIT_SSH": plink,
"GIT_SSH_VARIANT": "tortoiseplink",
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
meta.Port = "8888"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
ssh: implement Git SSH variant selection We'd implemented Git's autodetection of SSH variants in the past, but we hadn't implemented the newer explicit SSH selection. Since we're about to pass some additional options to our ssh binaries, let's implement the proper variant using GIT_SSH_VARIANT and ssh.variant. Roughly, the algorithm is this: GIT_SSH_VARIANT overrides ssh.variant, and if neither is set, autodetection occurs. A user can specify either "ssh" (OpenSSH), "putty" (or its synonym "plink"), "tortoiseplink", or "simple", or, if they'd like the autodetection behavior, "auto". If the value is not any of these, then it is interpreted as "ssh". Remove the special-casing of certain command names in favor of the variant type. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-03-30 17:05:22 +00:00
assert.Equal(t, plink, exe)
assert.Equal(t, []string{"-batch", "-P", "8888", "user@foo.com"}, args)
}
func TestSSHGetExeAndArgsTortoisePlinkCustomPortExplicitConfig(t *testing.T) {
plink := filepath.Join("Users", "joebloggs", "bin", "ssh")
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": "",
"GIT_SSH": plink,
"GIT_SSH_VARIANT": "tortoiseplink",
}, map[string]string{
"ssh.variant": "tortoiseplink",
}))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
meta.Port = "8888"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
ssh: implement Git SSH variant selection We'd implemented Git's autodetection of SSH variants in the past, but we hadn't implemented the newer explicit SSH selection. Since we're about to pass some additional options to our ssh binaries, let's implement the proper variant using GIT_SSH_VARIANT and ssh.variant. Roughly, the algorithm is this: GIT_SSH_VARIANT overrides ssh.variant, and if neither is set, autodetection occurs. A user can specify either "ssh" (OpenSSH), "putty" (or its synonym "plink"), "tortoiseplink", or "simple", or, if they'd like the autodetection behavior, "auto". If the value is not any of these, then it is interpreted as "ssh". Remove the special-casing of certain command names in favor of the variant type. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-03-30 17:05:22 +00:00
assert.Equal(t, plink, exe)
assert.Equal(t, []string{"-batch", "-P", "8888", "user@foo.com"}, args)
}
func TestSSHGetExeAndArgsTortoisePlinkCustomPortExplicitConfigOverride(t *testing.T) {
plink := filepath.Join("Users", "joebloggs", "bin", "ssh")
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": "",
"GIT_SSH": plink,
}, map[string]string{
"ssh.variant": "putty",
}))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
meta.Port = "8888"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
ssh: implement Git SSH variant selection We'd implemented Git's autodetection of SSH variants in the past, but we hadn't implemented the newer explicit SSH selection. Since we're about to pass some additional options to our ssh binaries, let's implement the proper variant using GIT_SSH_VARIANT and ssh.variant. Roughly, the algorithm is this: GIT_SSH_VARIANT overrides ssh.variant, and if neither is set, autodetection occurs. A user can specify either "ssh" (OpenSSH), "putty" (or its synonym "plink"), "tortoiseplink", or "simple", or, if they'd like the autodetection behavior, "auto". If the value is not any of these, then it is interpreted as "ssh". Remove the special-casing of certain command names in favor of the variant type. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-03-30 17:05:22 +00:00
assert.Equal(t, plink, exe)
assert.Equal(t, []string{"-P", "8888", "user@foo.com"}, args)
}
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
func TestSSHGetExeAndArgsSshCommandPrecedence(t *testing.T) {
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": "sshcmd",
"GIT_SSH": "bad",
ssh: implement Git SSH variant selection We'd implemented Git's autodetection of SSH variants in the past, but we hadn't implemented the newer explicit SSH selection. Since we're about to pass some additional options to our ssh binaries, let's implement the proper variant using GIT_SSH_VARIANT and ssh.variant. Roughly, the algorithm is this: GIT_SSH_VARIANT overrides ssh.variant, and if neither is set, autodetection occurs. A user can specify either "ssh" (OpenSSH), "putty" (or its synonym "plink"), "tortoiseplink", or "simple", or, if they'd like the autodetection behavior, "auto". If the value is not any of these, then it is interpreted as "ssh". Remove the special-casing of certain command names in favor of the variant type. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-03-30 17:05:22 +00:00
"GIT_SSH_VARIANT": "simple",
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "sh", exe)
assert.Equal(t, []string{"-c", "sshcmd user@foo.com"}, args)
}
func TestSSHGetExeAndArgsSshCommandArgs(t *testing.T) {
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": "sshcmd --args 1",
ssh: implement Git SSH variant selection We'd implemented Git's autodetection of SSH variants in the past, but we hadn't implemented the newer explicit SSH selection. Since we're about to pass some additional options to our ssh binaries, let's implement the proper variant using GIT_SSH_VARIANT and ssh.variant. Roughly, the algorithm is this: GIT_SSH_VARIANT overrides ssh.variant, and if neither is set, autodetection occurs. A user can specify either "ssh" (OpenSSH), "putty" (or its synonym "plink"), "tortoiseplink", or "simple", or, if they'd like the autodetection behavior, "auto". If the value is not any of these, then it is interpreted as "ssh". Remove the special-casing of certain command names in favor of the variant type. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-03-30 17:05:22 +00:00
"GIT_SSH_VARIANT": "simple",
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "sh", exe)
assert.Equal(t, []string{"-c", "sshcmd --args 1 user@foo.com"}, args)
}
func TestSSHGetExeAndArgsSshCommandArgsWithMixedQuotes(t *testing.T) {
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": "sshcmd foo 'bar \"baz\"'",
ssh: implement Git SSH variant selection We'd implemented Git's autodetection of SSH variants in the past, but we hadn't implemented the newer explicit SSH selection. Since we're about to pass some additional options to our ssh binaries, let's implement the proper variant using GIT_SSH_VARIANT and ssh.variant. Roughly, the algorithm is this: GIT_SSH_VARIANT overrides ssh.variant, and if neither is set, autodetection occurs. A user can specify either "ssh" (OpenSSH), "putty" (or its synonym "plink"), "tortoiseplink", or "simple", or, if they'd like the autodetection behavior, "auto". If the value is not any of these, then it is interpreted as "ssh". Remove the special-casing of certain command names in favor of the variant type. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-03-30 17:05:22 +00:00
"GIT_SSH_VARIANT": "simple",
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "sh", exe)
assert.Equal(t, []string{"-c", "sshcmd foo 'bar \"baz\"' user@foo.com"}, args)
}
func TestSSHGetExeAndArgsSshCommandCustomPort(t *testing.T) {
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": "sshcmd",
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
meta.Port = "8888"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "sh", exe)
ssh: avoid using -- where possible In Git LFS version 3.0, we added support for understanding the GIT_SSH_VARIANT environment variable and its corresponding configuration option. When we did so, we dropped the special-casing for program names and instead always used the -- setting for the default variant, "ssh". This variant represents OpenSSH, which as a program with normal POSIX option handling, supports -- just fine. Unfortunately, not all clients do. TeamCity's SSH client doesn't set the variant and also doesn't understand --. We need to handle this case to prevent option injection in case someone tries to use a URL like "ssh://-oProxyCommand=exploit/", which could otherwise lead to arbitrary code execution. However, we don't have to add this all the time, but only when the user-and-host portion starts with a dash. The reason is that the only other place an attacker could inject an option is in the path, and the path is not a separate option by itself. The path is always preceded by a command (either "git-lfs-authenticate" or "git-lfs-transfer" in a single option that contains the name of the command, the path, and the operation, separated by spaces. As a result, option injection is not possible in the path since those commands don't take options. Note that Git just dies in this case with a message like so: fatal: strange hostname '-oProxyCommand=exploit' blocked Let's adopt this approach of using -- less often so that we can more gracefully deal with this case and fix some SSH clients, even if they really should learn to understand this option. Update the test binary to expect the new results as well. Since we're adjusting the end-of-options delimiter such that it's only used for the default SSH variant and, in particular, only when the user/host parameter starts with a "-" character, we still need to accept that command argument format in our lfs-ssh-echo test helper program, so we keep that logic, but also simplify our logic for parsing the program's arguments so it will be, with luck, easier to maintain in the future. Note that we also remove the "$4" from one error message because it's no longer the case that that error condition always arises due to the fourth argument. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-11-15 15:53:03 +00:00
assert.Equal(t, []string{"-c", "sshcmd -p 8888 user@foo.com"}, args)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
}
func TestSSHGetExeAndArgsCoreSshCommand(t *testing.T) {
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": "sshcmd --args 2",
}, map[string]string{
"core.sshcommand": "sshcmd --args 1",
}))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "sh", exe)
ssh: avoid using -- where possible In Git LFS version 3.0, we added support for understanding the GIT_SSH_VARIANT environment variable and its corresponding configuration option. When we did so, we dropped the special-casing for program names and instead always used the -- setting for the default variant, "ssh". This variant represents OpenSSH, which as a program with normal POSIX option handling, supports -- just fine. Unfortunately, not all clients do. TeamCity's SSH client doesn't set the variant and also doesn't understand --. We need to handle this case to prevent option injection in case someone tries to use a URL like "ssh://-oProxyCommand=exploit/", which could otherwise lead to arbitrary code execution. However, we don't have to add this all the time, but only when the user-and-host portion starts with a dash. The reason is that the only other place an attacker could inject an option is in the path, and the path is not a separate option by itself. The path is always preceded by a command (either "git-lfs-authenticate" or "git-lfs-transfer" in a single option that contains the name of the command, the path, and the operation, separated by spaces. As a result, option injection is not possible in the path since those commands don't take options. Note that Git just dies in this case with a message like so: fatal: strange hostname '-oProxyCommand=exploit' blocked Let's adopt this approach of using -- less often so that we can more gracefully deal with this case and fix some SSH clients, even if they really should learn to understand this option. Update the test binary to expect the new results as well. Since we're adjusting the end-of-options delimiter such that it's only used for the default SSH variant and, in particular, only when the user/host parameter starts with a "-" character, we still need to accept that command argument format in our lfs-ssh-echo test helper program, so we keep that logic, but also simplify our logic for parsing the program's arguments so it will be, with luck, easier to maintain in the future. Note that we also remove the "$4" from one error message because it's no longer the case that that error condition always arises due to the fourth argument. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-11-15 15:53:03 +00:00
assert.Equal(t, []string{"-c", "sshcmd --args 2 user@foo.com"}, args)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
}
func TestSSHGetExeAndArgsCoreSshCommandArgsWithMixedQuotes(t *testing.T) {
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, nil, map[string]string{
"core.sshcommand": "sshcmd foo 'bar \"baz\"'",
}))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "sh", exe)
ssh: avoid using -- where possible In Git LFS version 3.0, we added support for understanding the GIT_SSH_VARIANT environment variable and its corresponding configuration option. When we did so, we dropped the special-casing for program names and instead always used the -- setting for the default variant, "ssh". This variant represents OpenSSH, which as a program with normal POSIX option handling, supports -- just fine. Unfortunately, not all clients do. TeamCity's SSH client doesn't set the variant and also doesn't understand --. We need to handle this case to prevent option injection in case someone tries to use a URL like "ssh://-oProxyCommand=exploit/", which could otherwise lead to arbitrary code execution. However, we don't have to add this all the time, but only when the user-and-host portion starts with a dash. The reason is that the only other place an attacker could inject an option is in the path, and the path is not a separate option by itself. The path is always preceded by a command (either "git-lfs-authenticate" or "git-lfs-transfer" in a single option that contains the name of the command, the path, and the operation, separated by spaces. As a result, option injection is not possible in the path since those commands don't take options. Note that Git just dies in this case with a message like so: fatal: strange hostname '-oProxyCommand=exploit' blocked Let's adopt this approach of using -- less often so that we can more gracefully deal with this case and fix some SSH clients, even if they really should learn to understand this option. Update the test binary to expect the new results as well. Since we're adjusting the end-of-options delimiter such that it's only used for the default SSH variant and, in particular, only when the user/host parameter starts with a "-" character, we still need to accept that command argument format in our lfs-ssh-echo test helper program, so we keep that logic, but also simplify our logic for parsing the program's arguments so it will be, with luck, easier to maintain in the future. Note that we also remove the "$4" from one error message because it's no longer the case that that error condition always arises due to the fourth argument. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-11-15 15:53:03 +00:00
assert.Equal(t, []string{"-c", "sshcmd foo 'bar \"baz\"' user@foo.com"}, args)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
}
func TestSSHGetExeAndArgsConfigVersusEnv(t *testing.T) {
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, nil, map[string]string{
"core.sshcommand": "sshcmd --args 1",
}))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "sh", exe)
ssh: avoid using -- where possible In Git LFS version 3.0, we added support for understanding the GIT_SSH_VARIANT environment variable and its corresponding configuration option. When we did so, we dropped the special-casing for program names and instead always used the -- setting for the default variant, "ssh". This variant represents OpenSSH, which as a program with normal POSIX option handling, supports -- just fine. Unfortunately, not all clients do. TeamCity's SSH client doesn't set the variant and also doesn't understand --. We need to handle this case to prevent option injection in case someone tries to use a URL like "ssh://-oProxyCommand=exploit/", which could otherwise lead to arbitrary code execution. However, we don't have to add this all the time, but only when the user-and-host portion starts with a dash. The reason is that the only other place an attacker could inject an option is in the path, and the path is not a separate option by itself. The path is always preceded by a command (either "git-lfs-authenticate" or "git-lfs-transfer" in a single option that contains the name of the command, the path, and the operation, separated by spaces. As a result, option injection is not possible in the path since those commands don't take options. Note that Git just dies in this case with a message like so: fatal: strange hostname '-oProxyCommand=exploit' blocked Let's adopt this approach of using -- less often so that we can more gracefully deal with this case and fix some SSH clients, even if they really should learn to understand this option. Update the test binary to expect the new results as well. Since we're adjusting the end-of-options delimiter such that it's only used for the default SSH variant and, in particular, only when the user/host parameter starts with a "-" character, we still need to accept that command argument format in our lfs-ssh-echo test helper program, so we keep that logic, but also simplify our logic for parsing the program's arguments so it will be, with luck, easier to maintain in the future. Note that we also remove the "$4" from one error message because it's no longer the case that that error condition always arises due to the fourth argument. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-11-15 15:53:03 +00:00
assert.Equal(t, []string{"-c", "sshcmd --args 1 user@foo.com"}, args)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
}
func TestSSHGetExeAndArgsPlinkCommand(t *testing.T) {
plink := filepath.Join("Users", "joebloggs", "bin", "plink.exe")
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": plink,
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "sh", exe)
assert.Equal(t, []string{"-c", plink + " user@foo.com"}, args)
}
func TestSSHGetExeAndArgsPlinkCommandCustomPort(t *testing.T) {
plink := filepath.Join("Users", "joebloggs", "bin", "plink")
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": plink,
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
meta.Port = "8888"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "sh", exe)
assert.Equal(t, []string{"-c", plink + " -P 8888 user@foo.com"}, args)
}
func TestSSHGetExeAndArgsTortoisePlinkCommand(t *testing.T) {
plink := filepath.Join("Users", "joebloggs", "bin", "tortoiseplink.exe")
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": plink,
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "sh", exe)
assert.Equal(t, []string{"-c", plink + " -batch user@foo.com"}, args)
}
func TestSSHGetExeAndArgsTortoisePlinkCommandCustomPort(t *testing.T) {
plink := filepath.Join("Users", "joebloggs", "bin", "tortoiseplink")
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
"GIT_SSH_COMMAND": plink,
}, nil))
require.Nil(t, err)
meta := ssh.SSHMetadata{}
meta.UserAndHost = "user@foo.com"
meta.Port = "8888"
ssh: support concurrent transfers using the pure SSH protocol When using the pure SSH-based protocol, we can get much higher speeds by multiplexing multiple connections on the same SSH connection. If we're using OpenSSH, let's enable the ControlMaster option unless lfs.ssh.automultiplex is set to false, and multiplex these shell operations over one connection. We prefer XDG_RUNTIME_DIR because it's guaranteed to be private and we can share many connections over one socket, but if that's not set, let's default to creating a new temporary directory for the socket. On Windows, where the native SSH client doesn't support ControlMaster, we should fall back to using multiple connections since we use ControlMaster=auto. Note that the option exists because users may already be using SSH multiplexing and we would want to provide a way for them to disable this, in addition to the case where users have an old or broken OpenSSH which cannot support this option. We pass the connection object into each worker and adjust our transfer code to pass it into each function we invoke. We also make sure to properly terminate each connection at the end by reducing our connection count to 0, which closes the extra (i.e., all) connections. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-04-06 14:08:06 +00:00
exe, args := ssh.FormatArgs(ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &meta, false))
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "sh", exe)
assert.Equal(t, []string{"-c", plink + " -batch -P 8888 user@foo.com"}, args)
}
func TestSSHGetLFSExeAndArgsWithCustomSSH(t *testing.T) {
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
ssh: implement Git SSH variant selection We'd implemented Git's autodetection of SSH variants in the past, but we hadn't implemented the newer explicit SSH selection. Since we're about to pass some additional options to our ssh binaries, let's implement the proper variant using GIT_SSH_VARIANT and ssh.variant. Roughly, the algorithm is this: GIT_SSH_VARIANT overrides ssh.variant, and if neither is set, autodetection occurs. A user can specify either "ssh" (OpenSSH), "putty" (or its synonym "plink"), "tortoiseplink", or "simple", or, if they'd like the autodetection behavior, "auto". If the value is not any of these, then it is interpreted as "ssh". Remove the special-casing of certain command names in favor of the variant type. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-03-30 17:05:22 +00:00
"GIT_SSH": "not-ssh",
"GIT_SSH_VARIANT": "simple",
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
}, nil))
require.Nil(t, err)
u, err := url.Parse("ssh://git@host.com:12345/repo")
require.Nil(t, err)
e := lfshttp.EndpointFromSshUrl(u)
t.Logf("ENDPOINT: %+v", e)
assert.Equal(t, "12345", e.SSHMetadata.Port)
assert.Equal(t, "git@host.com", e.SSHMetadata.UserAndHost)
lfshttp: don't strip off initial slash on SSH commands When we process an SSH URL, we intentionally strip off the slash at the beginning of the URL. While that was convenient for git-lfs-authenticate, it also prevents us from handling an absolute path in git-lfs-transfer, since the path will have its leading slash stripped off and will therefore be relative. Instead, let's adopt Git's behavior, which is to not remove the leading slash. This is an incompatible change, but we're about to do a major release, so it's a good time to make it. This will affect both git-lfs-transfer and git-lfs-authenticate commands, but at least GitHub already supports the proper syntax. Note that since we process the non-URL form of SSH remotes by converting them to a URL and then parsing, let's strip off the leading slash when we process that form, since there we do have the ability to distinguish between absolute and relative paths. Update the lfs-ssh-echo binary to handle this new format.
2021-05-14 18:00:23 +00:00
assert.Equal(t, "/repo", e.SSHMetadata.Path)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
ssh: disable concurrent transfers if no multiplexing Right now, we try to spawn multiple SSH connections using ControlMaster when making multiple requests. However, if multiplexing is not enabled, we can spawn multiple actual connections, which can be expensive and require lots of authentication requests. Instead, let's make sure we don't enable multiple connections if multiplexing is not enabled, since this may cause the user to be prompted for multiple SSH key connection attempts (say, if they're using a security key) and is substantially more heavyweight than simply spawning a new process over the same connection. Since the code has different behaviour if `XDG_RUNTIME_DIR` is set, make sure to unset it for the tests so that we always try to create a temporary directory and don't otherwise fail because that environment variable points somewhere unexpected.
2022-10-05 14:03:18 +00:00
exe, args, _ := ssh.GetLFSExeAndArgs(cli.OSEnv(), cli.GitEnv(), &e.SSHMetadata, "git-lfs-authenticate", "download", false)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "not-ssh", exe)
lfshttp: don't strip off initial slash on SSH commands When we process an SSH URL, we intentionally strip off the slash at the beginning of the URL. While that was convenient for git-lfs-authenticate, it also prevents us from handling an absolute path in git-lfs-transfer, since the path will have its leading slash stripped off and will therefore be relative. Instead, let's adopt Git's behavior, which is to not remove the leading slash. This is an incompatible change, but we're about to do a major release, so it's a good time to make it. This will affect both git-lfs-transfer and git-lfs-authenticate commands, but at least GitHub already supports the proper syntax. Note that since we process the non-URL form of SSH remotes by converting them to a URL and then parsing, let's strip off the leading slash when we process that form, since there we do have the ability to distinguish between absolute and relative paths. Update the lfs-ssh-echo binary to handle this new format.
2021-05-14 18:00:23 +00:00
assert.Equal(t, []string{"-p", "12345", "git@host.com", "git-lfs-authenticate /repo download"}, args)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
}
func TestSSHGetLFSExeAndArgsInvalidOptionsAsHost(t *testing.T) {
cli, err := lfshttp.NewClient(nil)
require.Nil(t, err)
u, err := url.Parse("ssh://-oProxyCommand=gnome-calculator/repo")
require.Nil(t, err)
assert.Equal(t, "-oProxyCommand=gnome-calculator", u.Host)
e := lfshttp.EndpointFromSshUrl(u)
t.Logf("ENDPOINT: %+v", e)
assert.Equal(t, "-oProxyCommand=gnome-calculator", e.SSHMetadata.UserAndHost)
lfshttp: don't strip off initial slash on SSH commands When we process an SSH URL, we intentionally strip off the slash at the beginning of the URL. While that was convenient for git-lfs-authenticate, it also prevents us from handling an absolute path in git-lfs-transfer, since the path will have its leading slash stripped off and will therefore be relative. Instead, let's adopt Git's behavior, which is to not remove the leading slash. This is an incompatible change, but we're about to do a major release, so it's a good time to make it. This will affect both git-lfs-transfer and git-lfs-authenticate commands, but at least GitHub already supports the proper syntax. Note that since we process the non-URL form of SSH remotes by converting them to a URL and then parsing, let's strip off the leading slash when we process that form, since there we do have the ability to distinguish between absolute and relative paths. Update the lfs-ssh-echo binary to handle this new format.
2021-05-14 18:00:23 +00:00
assert.Equal(t, "/repo", e.SSHMetadata.Path)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
ssh: disable concurrent transfers if no multiplexing Right now, we try to spawn multiple SSH connections using ControlMaster when making multiple requests. However, if multiplexing is not enabled, we can spawn multiple actual connections, which can be expensive and require lots of authentication requests. Instead, let's make sure we don't enable multiple connections if multiplexing is not enabled, since this may cause the user to be prompted for multiple SSH key connection attempts (say, if they're using a security key) and is substantially more heavyweight than simply spawning a new process over the same connection. Since the code has different behaviour if `XDG_RUNTIME_DIR` is set, make sure to unset it for the tests so that we always try to create a temporary directory and don't otherwise fail because that environment variable points somewhere unexpected.
2022-10-05 14:03:18 +00:00
exe, args, _ := ssh.GetLFSExeAndArgs(cli.OSEnv(), cli.GitEnv(), &e.SSHMetadata, "git-lfs-authenticate", "download", false)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "ssh", exe)
lfshttp: don't strip off initial slash on SSH commands When we process an SSH URL, we intentionally strip off the slash at the beginning of the URL. While that was convenient for git-lfs-authenticate, it also prevents us from handling an absolute path in git-lfs-transfer, since the path will have its leading slash stripped off and will therefore be relative. Instead, let's adopt Git's behavior, which is to not remove the leading slash. This is an incompatible change, but we're about to do a major release, so it's a good time to make it. This will affect both git-lfs-transfer and git-lfs-authenticate commands, but at least GitHub already supports the proper syntax. Note that since we process the non-URL form of SSH remotes by converting them to a URL and then parsing, let's strip off the leading slash when we process that form, since there we do have the ability to distinguish between absolute and relative paths. Update the lfs-ssh-echo binary to handle this new format.
2021-05-14 18:00:23 +00:00
assert.Equal(t, []string{"--", "-oProxyCommand=gnome-calculator", "git-lfs-authenticate /repo download"}, args)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
}
func TestSSHGetLFSExeAndArgsInvalidOptionsAsHostWithCustomSSH(t *testing.T) {
cli, err := lfshttp.NewClient(lfshttp.NewContext(nil, map[string]string{
ssh: implement Git SSH variant selection We'd implemented Git's autodetection of SSH variants in the past, but we hadn't implemented the newer explicit SSH selection. Since we're about to pass some additional options to our ssh binaries, let's implement the proper variant using GIT_SSH_VARIANT and ssh.variant. Roughly, the algorithm is this: GIT_SSH_VARIANT overrides ssh.variant, and if neither is set, autodetection occurs. A user can specify either "ssh" (OpenSSH), "putty" (or its synonym "plink"), "tortoiseplink", or "simple", or, if they'd like the autodetection behavior, "auto". If the value is not any of these, then it is interpreted as "ssh". Remove the special-casing of certain command names in favor of the variant type. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-03-30 17:05:22 +00:00
"GIT_SSH": "not-ssh",
"GIT_SSH_VARIANT": "simple",
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
}, nil))
require.Nil(t, err)
u, err := url.Parse("ssh://--oProxyCommand=gnome-calculator/repo")
require.Nil(t, err)
assert.Equal(t, "--oProxyCommand=gnome-calculator", u.Host)
e := lfshttp.EndpointFromSshUrl(u)
t.Logf("ENDPOINT: %+v", e)
assert.Equal(t, "--oProxyCommand=gnome-calculator", e.SSHMetadata.UserAndHost)
lfshttp: don't strip off initial slash on SSH commands When we process an SSH URL, we intentionally strip off the slash at the beginning of the URL. While that was convenient for git-lfs-authenticate, it also prevents us from handling an absolute path in git-lfs-transfer, since the path will have its leading slash stripped off and will therefore be relative. Instead, let's adopt Git's behavior, which is to not remove the leading slash. This is an incompatible change, but we're about to do a major release, so it's a good time to make it. This will affect both git-lfs-transfer and git-lfs-authenticate commands, but at least GitHub already supports the proper syntax. Note that since we process the non-URL form of SSH remotes by converting them to a URL and then parsing, let's strip off the leading slash when we process that form, since there we do have the ability to distinguish between absolute and relative paths. Update the lfs-ssh-echo binary to handle this new format.
2021-05-14 18:00:23 +00:00
assert.Equal(t, "/repo", e.SSHMetadata.Path)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
ssh: disable concurrent transfers if no multiplexing Right now, we try to spawn multiple SSH connections using ControlMaster when making multiple requests. However, if multiplexing is not enabled, we can spawn multiple actual connections, which can be expensive and require lots of authentication requests. Instead, let's make sure we don't enable multiple connections if multiplexing is not enabled, since this may cause the user to be prompted for multiple SSH key connection attempts (say, if they're using a security key) and is substantially more heavyweight than simply spawning a new process over the same connection. Since the code has different behaviour if `XDG_RUNTIME_DIR` is set, make sure to unset it for the tests so that we always try to create a temporary directory and don't otherwise fail because that environment variable points somewhere unexpected.
2022-10-05 14:03:18 +00:00
exe, args, _ := ssh.GetLFSExeAndArgs(cli.OSEnv(), cli.GitEnv(), &e.SSHMetadata, "git-lfs-authenticate", "download", false)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "not-ssh", exe)
lfshttp: don't strip off initial slash on SSH commands When we process an SSH URL, we intentionally strip off the slash at the beginning of the URL. While that was convenient for git-lfs-authenticate, it also prevents us from handling an absolute path in git-lfs-transfer, since the path will have its leading slash stripped off and will therefore be relative. Instead, let's adopt Git's behavior, which is to not remove the leading slash. This is an incompatible change, but we're about to do a major release, so it's a good time to make it. This will affect both git-lfs-transfer and git-lfs-authenticate commands, but at least GitHub already supports the proper syntax. Note that since we process the non-URL form of SSH remotes by converting them to a URL and then parsing, let's strip off the leading slash when we process that form, since there we do have the ability to distinguish between absolute and relative paths. Update the lfs-ssh-echo binary to handle this new format.
2021-05-14 18:00:23 +00:00
assert.Equal(t, []string{"oProxyCommand=gnome-calculator", "git-lfs-authenticate /repo download"}, args)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
}
func TestSSHGetExeAndArgsInvalidOptionsAsHost(t *testing.T) {
cli, err := lfshttp.NewClient(nil)
require.Nil(t, err)
u, err := url.Parse("ssh://-oProxyCommand=gnome-calculator")
require.Nil(t, err)
assert.Equal(t, "-oProxyCommand=gnome-calculator", u.Host)
e := lfshttp.EndpointFromSshUrl(u)
t.Logf("ENDPOINT: %+v", e)
assert.Equal(t, "-oProxyCommand=gnome-calculator", e.SSHMetadata.UserAndHost)
assert.Equal(t, "", e.SSHMetadata.Path)
ssh: disable concurrent transfers if no multiplexing Right now, we try to spawn multiple SSH connections using ControlMaster when making multiple requests. However, if multiplexing is not enabled, we can spawn multiple actual connections, which can be expensive and require lots of authentication requests. Instead, let's make sure we don't enable multiple connections if multiplexing is not enabled, since this may cause the user to be prompted for multiple SSH key connection attempts (say, if they're using a security key) and is substantially more heavyweight than simply spawning a new process over the same connection. Since the code has different behaviour if `XDG_RUNTIME_DIR` is set, make sure to unset it for the tests so that we always try to create a temporary directory and don't otherwise fail because that environment variable points somewhere unexpected.
2022-10-05 14:03:18 +00:00
exe, args, needShell, _ := ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &e.SSHMetadata, false)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "ssh", exe)
assert.Equal(t, []string{"--", "-oProxyCommand=gnome-calculator"}, args)
assert.Equal(t, false, needShell)
}
func TestSSHGetExeAndArgsInvalidOptionsAsPath(t *testing.T) {
cli, err := lfshttp.NewClient(nil)
require.Nil(t, err)
u, err := url.Parse("ssh://git@git-host.com/-oProxyCommand=gnome-calculator")
require.Nil(t, err)
assert.Equal(t, "git-host.com", u.Host)
e := lfshttp.EndpointFromSshUrl(u)
t.Logf("ENDPOINT: %+v", e)
assert.Equal(t, "git@git-host.com", e.SSHMetadata.UserAndHost)
lfshttp: don't strip off initial slash on SSH commands When we process an SSH URL, we intentionally strip off the slash at the beginning of the URL. While that was convenient for git-lfs-authenticate, it also prevents us from handling an absolute path in git-lfs-transfer, since the path will have its leading slash stripped off and will therefore be relative. Instead, let's adopt Git's behavior, which is to not remove the leading slash. This is an incompatible change, but we're about to do a major release, so it's a good time to make it. This will affect both git-lfs-transfer and git-lfs-authenticate commands, but at least GitHub already supports the proper syntax. Note that since we process the non-URL form of SSH remotes by converting them to a URL and then parsing, let's strip off the leading slash when we process that form, since there we do have the ability to distinguish between absolute and relative paths. Update the lfs-ssh-echo binary to handle this new format.
2021-05-14 18:00:23 +00:00
assert.Equal(t, "/-oProxyCommand=gnome-calculator", e.SSHMetadata.Path)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
ssh: disable concurrent transfers if no multiplexing Right now, we try to spawn multiple SSH connections using ControlMaster when making multiple requests. However, if multiplexing is not enabled, we can spawn multiple actual connections, which can be expensive and require lots of authentication requests. Instead, let's make sure we don't enable multiple connections if multiplexing is not enabled, since this may cause the user to be prompted for multiple SSH key connection attempts (say, if they're using a security key) and is substantially more heavyweight than simply spawning a new process over the same connection. Since the code has different behaviour if `XDG_RUNTIME_DIR` is set, make sure to unset it for the tests so that we always try to create a temporary directory and don't otherwise fail because that environment variable points somewhere unexpected.
2022-10-05 14:03:18 +00:00
exe, args, needShell, _ := ssh.GetExeAndArgs(cli.OSEnv(), cli.GitEnv(), &e.SSHMetadata, false)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, "ssh", exe)
ssh: avoid using -- where possible In Git LFS version 3.0, we added support for understanding the GIT_SSH_VARIANT environment variable and its corresponding configuration option. When we did so, we dropped the special-casing for program names and instead always used the -- setting for the default variant, "ssh". This variant represents OpenSSH, which as a program with normal POSIX option handling, supports -- just fine. Unfortunately, not all clients do. TeamCity's SSH client doesn't set the variant and also doesn't understand --. We need to handle this case to prevent option injection in case someone tries to use a URL like "ssh://-oProxyCommand=exploit/", which could otherwise lead to arbitrary code execution. However, we don't have to add this all the time, but only when the user-and-host portion starts with a dash. The reason is that the only other place an attacker could inject an option is in the path, and the path is not a separate option by itself. The path is always preceded by a command (either "git-lfs-authenticate" or "git-lfs-transfer" in a single option that contains the name of the command, the path, and the operation, separated by spaces. As a result, option injection is not possible in the path since those commands don't take options. Note that Git just dies in this case with a message like so: fatal: strange hostname '-oProxyCommand=exploit' blocked Let's adopt this approach of using -- less often so that we can more gracefully deal with this case and fix some SSH clients, even if they really should learn to understand this option. Update the test binary to expect the new results as well. Since we're adjusting the end-of-options delimiter such that it's only used for the default SSH variant and, in particular, only when the user/host parameter starts with a "-" character, we still need to accept that command argument format in our lfs-ssh-echo test helper program, so we keep that logic, but also simplify our logic for parsing the program's arguments so it will be, with luck, easier to maintain in the future. Note that we also remove the "$4" from one error message because it's no longer the case that that error condition always arises due to the fourth argument. Co-authored-by: Chris Darroch <chrisd8088@github.com>
2021-11-15 15:53:03 +00:00
assert.Equal(t, []string{"git@git-host.com"}, args)
Move much of SSH code into a separate package In the future, we'll want to call into the SSH code from multiple packages, so let's move it out of the lfshttp package into its own package to avoid package import loops. While we're at it, rename the function names to remove the "ssh" prefix, since it's implied by the fact that they're in a package called "ssh". Move the tests to their own package to prevent an import loop and expose the private functions so we can test them there.
2021-02-03 22:27:01 +00:00
assert.Equal(t, false, needShell)
}
func TestParseBareSSHUrl(t *testing.T) {
e := lfshttp.EndpointFromBareSshUrl("git@git-host.com:repo.git")
t.Logf("endpoint: %+v", e)
assert.Equal(t, "git@git-host.com", e.SSHMetadata.UserAndHost)
assert.Equal(t, "repo.git", e.SSHMetadata.Path)
e = lfshttp.EndpointFromBareSshUrl("git@git-host.com/should-be-a-colon.git")
t.Logf("endpoint: %+v", e)
assert.Equal(t, "", e.SSHMetadata.UserAndHost)
assert.Equal(t, "", e.SSHMetadata.Path)
e = lfshttp.EndpointFromBareSshUrl("-oProxyCommand=gnome-calculator")
t.Logf("endpoint: %+v", e)
assert.Equal(t, "", e.SSHMetadata.UserAndHost)
assert.Equal(t, "", e.SSHMetadata.Path)
e = lfshttp.EndpointFromBareSshUrl("git@git-host.com:-oProxyCommand=gnome-calculator")
t.Logf("endpoint: %+v", e)
assert.Equal(t, "git@git-host.com", e.SSHMetadata.UserAndHost)
assert.Equal(t, "-oProxyCommand=gnome-calculator", e.SSHMetadata.Path)
}
Reference in New Issue Copy Permalink