Commit Graph

32 Commits

Author SHA1 Message Date
Eng Zer Jun
2ebe3ba361
Replace deprecated io/ioutil functions
The io/ioutil package has been deprecated as of Go 1.16 [1]. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

[1]: https://golang.org/doc/go1.16#ioutil
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2024-01-11 03:48:51 +08:00
brian m. carlson
0e543b8a1e
Revert "Call migrate() BlobFn on every blob"
This caused a substantial performance regression.  We should solve this
in a different way.

This reverts commit 3acbec20a6a3b211f4ef3e5ec83cecf0c1fc63b0.
2022-01-12 17:34:19 +00:00
brian m. carlson
9deb264979
Revert "Adapt unit tests to the new migrate() BlobFn behavior"
This caused a substantial performance regression.  We should solve this
in a different way.

This reverts commit 791690a11e189b71bfbc4888f50b4b69cc1370c2.
2022-01-12 17:33:53 +00:00
Yoann Congal
791690a11e Adapt unit tests to the new migrate() BlobFn behavior
TestRewriterDoesntVisitUnchangedSubtrees:
* Changed to TestRewriterDoesVisitUnchangedSubtrees (BlobFn will now be
  called on every blobs whether they changed or not)
* Assert changed to match that.

TestRewriterIgnoresPathsThatDontMatchFilter:
* Update "seen" count: "subdir/b.txt" should now be seen twice.

TestHistoryRewriterCallbacksSubtrees:
* Update asserted call order. BlobFn("a.txt") is now called twice.
2021-10-07 11:44:07 +02:00
Yoann Congal
3acbec20a6 Call migrate() BlobFn on every blob
Caching is now done inside the BlobFn function instead of in the caller.
To do this, the OID of the original blob is given as an argument to
BlobFn.

Adapt migrate() calls to this :
 * For "migrate info", a set of "seen" OID is cached to avoid counting
   them twice.
 * For "migrate import", the cleaned blob is cached and reused.

Fixes git-lfs/git-lfs#4628
2021-10-07 11:44:07 +02:00
brian m. carlson
087db1de70
Set package version to v3
Since we're about to do a v3.0.0 release, let's bump the version to v3.

Make this change automatically with the following command to avoid any
missed items:

  git grep -l github.com/git-lfs/git-lfs/v2 | \
  xargs sed -i -e 's!github.com/git-lfs/git-lfs/v2!github.com/git-lfs/git-lfs/v3!g'
2021-09-02 20:41:08 +00:00
brian m. carlson
22a7fbdb4a
filepathfilter: switch to strict pattern matching
There are two types of pattern matching with wildmatch patterns in Git:
gitignore and gitattributes patterns.  We'd like to support both, but
without the lazy matching that was the cause of so many problems before.

Add a type of pattern to use when creating a new filepathfilter and use
it to instantiate the wildmatch pattern with the proper flags.  Remove
the dirs flag, which is unused and has been for sometime, and also the
stripping of trailing slashes, which we want to use to indicate a
directory.  Drop the non-strict mode since our pattern matching is now
much improved and compatible with Git.
2021-08-26 15:48:54 +00:00
Chris Darroch
dd8e306e31 all: update go.mod module path with explicit v2
When our go.mod file was introduced in commit
114e85c2002091eb415040923d872f8e4a4bc636 in PR #3208, the module
path chosen did not include a trailing /v2 component.  However,
the Go modules specification now advises that module paths must
have a "major version suffix" which matches the release version.

We therefore add a /v2 suffix to our module path and all its
instances in import paths.

See also https://golang.org/ref/mod#major-version-suffixes for
details regarding the Go module system's major version suffix rule.
2021-08-09 23:18:38 -07:00
brian m. carlson
da4fdef00b
Support SHA-256 repositories
Git will start to support SHA-256 as a hash for repositories in the near
future.  Let's update gitobj to version 2 to support SHA-256
repositories properly.  We initialize the repository based on the
extensions.objectFormat value, if one is provided, since this is the
configuration key that represents the hash algorithm.

Vendor the proper dependencies in place.
2020-07-29 20:53:44 +00:00
Taylor Blau
22801fdaeb git/githistory/rewriter_test.go: ensure calls over subtrees 2018-07-09 12:14:34 -05:00
Taylor Blau
3c574d0ba1 git/githistory/rewriter_test.go: extract CallbackCall, collectCalls 2018-07-09 12:11:12 -05:00
Taylor Blau
92d8cf18a5 git/githistory: introduce TreePreCallbackFn
To determine the paths to migrate from a repository's .gitattributes, a
caller must do the following two things in order:

  1. Read the .gitattributes file(s) in a given tree contained within
  the repository.

  2. Rewrite blobs according to the attributes applied to their paths
  via the .gitattributes file(s) read in (1).

The framework for accomplishing the task necessary in (1) was written in
the previous commit. This commit introduces the rest of that mechanism
for (1).

Because a Git object's SHA-1 signature depends on its children, we must
visit the object graph in a topological ordering. This is not sufficient
for our purposes, since the patterns in a .gitattributes file cascade
downwards.

In other words, while we have to migrate from the leaves of the tree to
its root, we have to read the .gitattributes file(s) from root to
leaves.

To accomplish this, we introduce a new callback function in the
*githistory.RewriteOptions structure, TreePreCallbackFn, which is called
once as soon as a tree is opened for the first time, and before any
blobs or sub-trees are rewritten.

This provides the optimal time to inspect the repository's contents for
interesting .gitattributes files before migrating the blobs within.

We will use this new callback function in the following commit in order
to do precisely the task as described above.
2018-07-06 14:42:48 -05:00
Taylor Blau
b060df60f2 *: replace git/odb with vendored copy 2018-07-05 11:50:21 -05:00
Taylor Blau
1b48d251d3 git/odb: do not remove trailing newline 2018-06-08 14:53:42 -05:00
Taylor Blau
e7f4843e26 git/githistory: add debug() to aid in debugging 2018-06-08 14:53:10 -05:00
Taylor Blau
e35b429ec3 git/githistory: don't write to stdout 2018-06-08 11:21:22 -05:00
Taylor Blau
9e93c85684 git/githistory: only handle non-Windows paths 2018-02-15 15:50:21 -08:00
Taylor Blau
2f8c6ee0ff git/githistory: remove filepath.Join call 2018-02-15 15:50:20 -08:00
Taylor Blau
b90e969725 git/odb: don't print extra newline after commit message 2017-12-12 19:14:13 -08:00
Taylor Blau
28e2158bb9 git/githistory: add TestRewriterVisitsPackedObjects test 2017-09-06 14:23:30 -04:00
Taylor Blau
cf1eab01f5 git/githistory: fix incorrect pattern in test 2017-08-08 11:19:50 -06:00
Taylor Blau
f1850620a3 git/githistory: remove 'root' helper func 2017-08-08 11:19:36 -06:00
Taylor Blau
d09dad2ec8 Merge branch 'master' into githistory-ref-updater 2017-06-21 18:18:23 -06:00
Taylor Blau
c3fc92b22a git/githistory: teach Filter() to access filepathfilter used 2017-06-21 16:56:58 -06:00
Taylor Blau
d4931ae049 git/githistory: assert starting reference in *Rewriter tests 2017-06-20 10:58:54 -06:00
Taylor Blau
68b76005c9 git/githistory: teach UpdateRefs option 2017-06-20 10:41:00 -06:00
Taylor Blau
4f808e12c1 git/githistory: teach *Rewriter to link parents outside of migration 2017-06-08 14:49:14 -06:00
Taylor Blau
182b411d6a git,lfs: make *git.RevListScanner take Include, Exclude []string 2017-06-08 11:22:30 -06:00
Taylor Blau
5ed34d0fb4 git/githistory: call TreeCallbackFn before saving objects to database 2017-06-05 15:32:25 -06:00
Taylor Blau
23a61705fa git/githistory: make path include absolute root 2017-06-05 14:41:00 -06:00
Taylor Blau
97bc74aeb8 git/githistory: cull out subtrees/blobs based on filter match 2017-06-05 12:26:48 -06:00
Taylor Blau
f428960cd0 git/githistory: rename *HistoryRewriter to *Rewriter 2017-06-05 09:50:15 -06:00