We have a pktline repository that provides our pkt-line code from Git
LFS, but separated out into a separate, reusable repository. The
current code is private to the git package, and since we'll want to
use it in the future from multiple packages, it makes sense to just use
the separate repository, which we want to do anyway.
Let's include the repository in our go.mod and vendor the modules.
Remove the old code and replace its use with the new code.
When using Go 1.16 in CI, go get is upgrading some of our dependencies
automatically, and since these items are not in the go.sum file, Go
complains and refuses to build our binary. Let's bump the relevant
dependencies and vendor the resulting modules to make sure that Go can
build our binary successfully on a fresh Go 1.16.
This version uses a newer version of the Kerberos library, which should
work better in some environments. Tidy the go.mod and go.sum files and
vendor the dependencies.
Update x/text to v0.3.5 that fixes CVE-2020-28852.
The specific commands used for this update:
"go get golang.org/x/text@latest && go mod tidy && go mod vendor"
Update the vendored golang.org/x/net module to the latest
version, which in turn updates the vendored copy of the
golang.org/x/text module to 0.3.3. That version of x/text
includes a mitigation of CVE-2020-14040, which pertains to
processing UTF-16 data with a Byte Order Mark.
The Git LFS client should not be affected by that security
issue, since it does not open streams in UTF-16 mode, but
we update our modules to stay current with the upstream Go
sources and also to avoid being flagged by security scanners.
The specific commands run to perform this update were:
"go get golang.org/x/net@latest && go mod tidy && go mod vendor"
Use the IoctlFileClone ioctl wrapper provided by golang.org/x/sys/unix
instead of locally implementing it. This also fixes the ioctl on
GOARCHes where the value of FICLONE is different from the currently used
ioctlFiClone value (e.g. mips64/mips64le and ppc64/ppc64le).
This PR also bumps the version of golang.org/x/sys to get
IoctlFileClone and updates its vendored version by running
`go get golang.org/x/sys@latest && go mod tidy && go mod vendor`.
Previously, trying to compile `util_darwin.go` against a recent version
of `golang/sys` library failed with:
```
util_darwin.go:127:3: undefined: unix.SYS_CLONEFILEAT
```
Go v1.12 removed the ability to use direct syscalls. The clonefile
system calls was added to the latest `golang/sys` in
https://github.com/golang/go/issues/41366, so we can use that now.
Git will start to support SHA-256 as a hash for repositories in the near
future. Let's update gitobj to version 2 to support SHA-256
repositories properly. We initialize the repository based on the
extensions.objectFormat value, if one is provided, since this is the
configuration key that represents the hash algorithm.
Vendor the proper dependencies in place.
We'll need some additional dependencies to add support for Kerberos to
Git LFS. In order to support HTTP SPNEGO (Negotiate) support, add the
required modules and vendor their dependencies.
Testing showed that while race condition analysis in #3880 was correct, the way it tries to fix that
does not work for the *first* git-lfs process that will actually perform file move.
Instead, this commit performs multiple attempts when working with files in LFS storage.
Similar logic is already implemented in "cmd/go/internal/robustio" and "cmd/go/internal/renameio" packages.
However, they are not public, so we cannot use them.
The original parser that was used in #3825 brings in a lot of
dependencies that complicate packaging git-lfs. This replaces it with a
small parser I wrote with almost no dependencies. I've tested this as
extensively as i can and it seems to work correctly.
Update wildmatch to v1.0.4 to fix an issue with matching non-ASCII
patterns.
Note that the version number is v1.0.4 instead of v1.0.3 because the
master branch of wildmatch contains changes which are valuable but not
currently compatible with Git LFS and v1.0.3 was mistakenly tagged from
these changes. These changes would break semantic versioning, so the
tag was deleted and v1.0.4 was tagged with only compatible changes.
We can adopt these incompatible changes in the future, but it's better
to fix this bug now since it is causing pain for multiple users rather
than wait to address these changes until the new wildmatch can be
included.
Go 1.13 has become stricter and now requires that the timestamps in
go.mod reflect the commit timestamps. Update go.mod and go.sum to make
sure that Go 1.13 is happy.
Vendor the `golang.org/x/net` dependency. This will be used in a
future commit to add HTTP/2 transport support.
Note that due to the way go modules work, the `golang.org/x/sys`
dependency was updated as well when running `make vendor`.
The code which allocated and used a pty on Unix systems was unused, but
it did bring in a dependency on the github.com/kr/pty module. This
module is unmaintained and doesn't compile with gccgo. Since we no
longer need the pty code or this module, remove both of them.
The upstream of go-ntlm has archived its repository and is no longer
doing releases. Because this dependency is required for Git LFS, we've
created our own fork to ensure that the upstream repo doesn't disappear
on us. Use our own copy of go-ntlm within Git LFS.
In a future commit, we'll introduce a use of the semaphore package to
prevent goroutines from accidentally running us out of resources. Add
the package to go.mod and go.sum and vendor it in.
Update wildmatch to v1.0.2, which fixes the default case-sensitivity
settings such that Windows and Mac are case insensitive by default and
other Unices are case sensitive by default.
Since we have started tagging git-lfs/gitobj to be semantically
versioned, let's depend on that tag instead of the latest SHA-1 from
HEAD.
v1.0.0 is identical to the existing hash (1e97572956c1), so there are no
changes in vendor/, other than the pinned book-keeping.
In a future commit, we'll use the latest gitobj's untyped Object
function and storage functionality. Update the repository to the latest
master and vendor the dependencies.
We use package github.com/olekukonko/ts to issue IOCTL's to gather the
terminal size of the calling terminal in order to determine the length
of lines we attempt to print.
Until now, we have used revision ecf753e7c962, which is over 4 years
old. Since then, github.com/olekukonko/ts has introduced support for
Solaris, which we previously could not build or run on because package
syscall on Solaris does not export an IOCTL constant.
Let's upgrade to the latest version, and bring in support for Solaris.
In order to prefer semantically versioned tags of our dependent
repositories over Go's date & SHA-1 format, let's tag
github.com/git-lfs/wildmatch.git at v1.0.0 (on the latest master) and
vendor it as such here.
The only change between b31c34466d64 and v1.0.0 is:
* 83d2acb (Merge pull request #9 from git-lfs/ttaylorr/go-1.11,
2018-08-31)
* 2f71dd1 (.travis.yml: build on Go 1.11, 2018-08-30)
* 4bab7d7 (go.mod: initial commit, 2018-08-30)
Which is the introduction of the go.mod file (and related changes).
Thus, there are no code changes between the two, so this is a safe
change to make.
Since we are now building on Go 1.11 (as of 074a2d4f (all: use Go 1.11
in CI, 2018-08-28)) and Go 1.11 supports Go Modules [1], let's stop
using Glide, and begin using Go Modules.
This involves a few things:
* Teach the Makefile how to build go.sum files instead of glide.lock
files.
* Teach continuous integration services to build Git LFS in a
non-$GOPATH environment, since (without setting GO111MODULE=on
explicitly, which we choose not to do), this will break compiling
Git LFS, because Go 1.11 will ignore modules present in a Go
checkout beneath $GOPATH.
* In order to do the above, let's also make sure that we are
un-setting $GOCACHE in the environment, as this causes Go to work
without modules support [2].
* Because we're no longer building in a `$GOPATH`-based location,
let's instruct the CircleCI base image to archive the new location,
too.
* Similarly, teach the RPM spec to build in a non-$GOPATH location.
* By contrast, since we use dh_golang to build git-lfs binaries on
Debian, let's wait until the upstream dh_golang package is released
with support for Go 1.11 module support explicitly. Therefore, force
GO111MODULE to be on so that we can build a copy of Git LFS whose
checkout is within a $GOPATH.
Although the go.mod versions match the glide.yaml ones, the diff
attached is large because Go Modules do not vendor `_test.go` files,
whereas Glide does.
[1]: https://golang.org/doc/go1.11#modules
[2]: `GOCACHE=on` will be deprecated in Go 1.12, so this change makes
sense for that reason, too.
Co-authored-by: brian m. carlson <bk2204@github.com>
