commands/command_migrate.go: introduce '--fixup' flag on 'import'
A common invocation of the 'git lfs migrate import' command is with
'--include' and/or '--exclude' flag(s), which specify wildmatch
pattern(s) for which paths to migrate and/or not migrate.
This is useful for retroactively importing a set of files into Git LFS's
care, or fixing up a file that should have been tracked by Git LFS but
was accidentally committed as a large object instead.
In the later case, it is often the reality that a user will run 'git lfs
migrate --import' with an '--include' path that they believe will gather
the file (and the file alone). This approach is brittle because it
requires the user to infer not only the applicable pattern but the
meaning of that pattern. It also requires the user to run more than one
migration when fixing multiple types of files.
The .gitattributes file(s) contained within a repository provide an
authoritative source on what file(s) are considered by Git to be tracked
in Git LFS. We can use this information to infer the correct patterns to
``fix up'' a broken repository.
In the simplest case, if a repository's .gitattributes file contains the
following:
*.txt filter=lfs merge=lfs diff=lfs -text
But a .txt file matched by that pattern is not parse-able as an LFS
pointer, it will appear as unable to checkout.
Running 'git lfs migrate import --fixup --everything' will correctly
traverse history and find the affected .txt file, read it, create an
object file for it, and store it as an LFS pointer in history.
Thus, a user can run one command which will recognize arbitrarily
complex problems where a file should be tracked by Git LFS, but isn't.
Later, this feature could be combined with the new 'git lfs migrate
export' functionality to also clean files _out_ of Git LFS to object
files when they are not supposed to be tracked as Git LFS objects.
2018-07-06 19:20:02 +00:00
|
|
|
#!/usr/bin/env bash
|
|
|
|
|
2018-07-11 18:15:53 +00:00
|
|
|
. "$(dirname "$0")/fixtures/migrate.sh"
|
2018-07-10 18:48:02 +00:00
|
|
|
. "$(dirname "$0")/testlib.sh"
|
commands/command_migrate.go: introduce '--fixup' flag on 'import'
A common invocation of the 'git lfs migrate import' command is with
'--include' and/or '--exclude' flag(s), which specify wildmatch
pattern(s) for which paths to migrate and/or not migrate.
This is useful for retroactively importing a set of files into Git LFS's
care, or fixing up a file that should have been tracked by Git LFS but
was accidentally committed as a large object instead.
In the later case, it is often the reality that a user will run 'git lfs
migrate --import' with an '--include' path that they believe will gather
the file (and the file alone). This approach is brittle because it
requires the user to infer not only the applicable pattern but the
meaning of that pattern. It also requires the user to run more than one
migration when fixing multiple types of files.
The .gitattributes file(s) contained within a repository provide an
authoritative source on what file(s) are considered by Git to be tracked
in Git LFS. We can use this information to infer the correct patterns to
``fix up'' a broken repository.
In the simplest case, if a repository's .gitattributes file contains the
following:
*.txt filter=lfs merge=lfs diff=lfs -text
But a .txt file matched by that pattern is not parse-able as an LFS
pointer, it will appear as unable to checkout.
Running 'git lfs migrate import --fixup --everything' will correctly
traverse history and find the affected .txt file, read it, create an
object file for it, and store it as an LFS pointer in history.
Thus, a user can run one command which will recognize arbitrarily
complex problems where a file should be tracked by Git LFS, but isn't.
Later, this feature could be combined with the new 'git lfs migrate
export' functionality to also clean files _out_ of Git LFS to object
files when they are not supposed to be tracked as Git LFS objects.
2018-07-06 19:20:02 +00:00
|
|
|
|
|
|
|
begin_test "migrate import (--fixup)"
|
|
|
|
(
|
|
|
|
set -e
|
|
|
|
|
|
|
|
setup_single_local_branch_tracked_corrupt
|
|
|
|
|
|
|
|
txt_oid="$(calc_oid "$(git cat-file -p :a.txt)")"
|
|
|
|
|
commands: warn if working copy is dirty
In 'git lfs migrate import' and 'git lfs migrate export', Git LFS makes
destructive changes to a caller's repository and therefore invokes 'git
checkout --force', which throws away local changes.
To prevent this, let's introduce a check that notifies users when they
are going to throw away local changes, and allows for the user to abort
if they do not wish to discard their local changes.
With this, a user can safely migrate over dirty repositories (i.e., in
the case that they wanted to fix a file that should have been in Git
LFS, but wasn't) without having to finagle their repository to get it
into a migrate-able state.
For users with lots of pending migrations, also teach --yes, which
allows a user to avoid the check, and instead simply prints the warning
message to STDERR.
2018-07-10 15:18:46 +00:00
|
|
|
git lfs migrate import --everything --fixup --yes
|
commands/command_migrate.go: introduce '--fixup' flag on 'import'
A common invocation of the 'git lfs migrate import' command is with
'--include' and/or '--exclude' flag(s), which specify wildmatch
pattern(s) for which paths to migrate and/or not migrate.
This is useful for retroactively importing a set of files into Git LFS's
care, or fixing up a file that should have been tracked by Git LFS but
was accidentally committed as a large object instead.
In the later case, it is often the reality that a user will run 'git lfs
migrate --import' with an '--include' path that they believe will gather
the file (and the file alone). This approach is brittle because it
requires the user to infer not only the applicable pattern but the
meaning of that pattern. It also requires the user to run more than one
migration when fixing multiple types of files.
The .gitattributes file(s) contained within a repository provide an
authoritative source on what file(s) are considered by Git to be tracked
in Git LFS. We can use this information to infer the correct patterns to
``fix up'' a broken repository.
In the simplest case, if a repository's .gitattributes file contains the
following:
*.txt filter=lfs merge=lfs diff=lfs -text
But a .txt file matched by that pattern is not parse-able as an LFS
pointer, it will appear as unable to checkout.
Running 'git lfs migrate import --fixup --everything' will correctly
traverse history and find the affected .txt file, read it, create an
object file for it, and store it as an LFS pointer in history.
Thus, a user can run one command which will recognize arbitrarily
complex problems where a file should be tracked by Git LFS, but isn't.
Later, this feature could be combined with the new 'git lfs migrate
export' functionality to also clean files _out_ of Git LFS to object
files when they are not supposed to be tracked as Git LFS objects.
2018-07-06 19:20:02 +00:00
|
|
|
|
|
|
|
assert_pointer "refs/heads/master" "a.txt" "$txt_oid" "120"
|
|
|
|
assert_local_object "$txt_oid" "120"
|
|
|
|
|
|
|
|
master="$(git rev-parse refs/heads/master)"
|
|
|
|
master_attrs="$(git cat-file -p "$master:.gitattributes")"
|
|
|
|
echo "$master_attrs" | grep -q "*.txt filter=lfs diff=lfs merge=lfs"
|
|
|
|
)
|
|
|
|
end_test
|
|
|
|
|
2018-07-09 16:56:40 +00:00
|
|
|
begin_test "migrate import (--fixup, complex nested)"
|
|
|
|
(
|
|
|
|
set -e
|
|
|
|
|
|
|
|
setup_single_local_branch_complex_tracked
|
|
|
|
|
|
|
|
a_oid="$(calc_oid "$(git cat-file -p :a.txt)")"
|
|
|
|
b_oid="$(calc_oid "$(git cat-file -p :dir/b.txt)")"
|
|
|
|
|
commands: warn if working copy is dirty
In 'git lfs migrate import' and 'git lfs migrate export', Git LFS makes
destructive changes to a caller's repository and therefore invokes 'git
checkout --force', which throws away local changes.
To prevent this, let's introduce a check that notifies users when they
are going to throw away local changes, and allows for the user to abort
if they do not wish to discard their local changes.
With this, a user can safely migrate over dirty repositories (i.e., in
the case that they wanted to fix a file that should have been in Git
LFS, but wasn't) without having to finagle their repository to get it
into a migrate-able state.
For users with lots of pending migrations, also teach --yes, which
allows a user to avoid the check, and instead simply prints the warning
message to STDERR.
2018-07-10 15:18:46 +00:00
|
|
|
git lfs migrate import --everything --fixup --yes
|
2018-07-09 16:56:40 +00:00
|
|
|
|
|
|
|
assert_pointer "refs/heads/master" "a.txt" "$a_oid" "1"
|
|
|
|
refute_pointer "refs/heads/master" "b.txt"
|
|
|
|
|
|
|
|
assert_local_object "$a_oid" "1"
|
|
|
|
refute_local_object "$b_oid" "1"
|
|
|
|
|
|
|
|
master="$(git rev-parse refs/heads/master)"
|
|
|
|
master_attrs="$(git cat-file -p "$master:.gitattributes")"
|
|
|
|
master_dir_attrs="$(git cat-file -p "$master:dir/.gitattributes")"
|
|
|
|
echo "$master_attrs" | grep -q "*.txt filter=lfs diff=lfs merge=lfs"
|
|
|
|
echo "$master_dir_attrs" | grep -q "*.txt !filter !diff !merge"
|
|
|
|
)
|
|
|
|
end_test
|
|
|
|
|
commands/command_migrate.go: introduce '--fixup' flag on 'import'
A common invocation of the 'git lfs migrate import' command is with
'--include' and/or '--exclude' flag(s), which specify wildmatch
pattern(s) for which paths to migrate and/or not migrate.
This is useful for retroactively importing a set of files into Git LFS's
care, or fixing up a file that should have been tracked by Git LFS but
was accidentally committed as a large object instead.
In the later case, it is often the reality that a user will run 'git lfs
migrate --import' with an '--include' path that they believe will gather
the file (and the file alone). This approach is brittle because it
requires the user to infer not only the applicable pattern but the
meaning of that pattern. It also requires the user to run more than one
migration when fixing multiple types of files.
The .gitattributes file(s) contained within a repository provide an
authoritative source on what file(s) are considered by Git to be tracked
in Git LFS. We can use this information to infer the correct patterns to
``fix up'' a broken repository.
In the simplest case, if a repository's .gitattributes file contains the
following:
*.txt filter=lfs merge=lfs diff=lfs -text
But a .txt file matched by that pattern is not parse-able as an LFS
pointer, it will appear as unable to checkout.
Running 'git lfs migrate import --fixup --everything' will correctly
traverse history and find the affected .txt file, read it, create an
object file for it, and store it as an LFS pointer in history.
Thus, a user can run one command which will recognize arbitrarily
complex problems where a file should be tracked by Git LFS, but isn't.
Later, this feature could be combined with the new 'git lfs migrate
export' functionality to also clean files _out_ of Git LFS to object
files when they are not supposed to be tracked as Git LFS objects.
2018-07-06 19:20:02 +00:00
|
|
|
begin_test "migrate import (--fixup, --include)"
|
|
|
|
(
|
|
|
|
set -e
|
|
|
|
|
|
|
|
setup_single_local_branch_tracked_corrupt
|
|
|
|
|
commands: warn if working copy is dirty
In 'git lfs migrate import' and 'git lfs migrate export', Git LFS makes
destructive changes to a caller's repository and therefore invokes 'git
checkout --force', which throws away local changes.
To prevent this, let's introduce a check that notifies users when they
are going to throw away local changes, and allows for the user to abort
if they do not wish to discard their local changes.
With this, a user can safely migrate over dirty repositories (i.e., in
the case that they wanted to fix a file that should have been in Git
LFS, but wasn't) without having to finagle their repository to get it
into a migrate-able state.
For users with lots of pending migrations, also teach --yes, which
allows a user to avoid the check, and instead simply prints the warning
message to STDERR.
2018-07-10 15:18:46 +00:00
|
|
|
git lfs migrate import --everything --fixup --yes --include="*.txt" 2>&1 \
|
commands/command_migrate.go: introduce '--fixup' flag on 'import'
A common invocation of the 'git lfs migrate import' command is with
'--include' and/or '--exclude' flag(s), which specify wildmatch
pattern(s) for which paths to migrate and/or not migrate.
This is useful for retroactively importing a set of files into Git LFS's
care, or fixing up a file that should have been tracked by Git LFS but
was accidentally committed as a large object instead.
In the later case, it is often the reality that a user will run 'git lfs
migrate --import' with an '--include' path that they believe will gather
the file (and the file alone). This approach is brittle because it
requires the user to infer not only the applicable pattern but the
meaning of that pattern. It also requires the user to run more than one
migration when fixing multiple types of files.
The .gitattributes file(s) contained within a repository provide an
authoritative source on what file(s) are considered by Git to be tracked
in Git LFS. We can use this information to infer the correct patterns to
``fix up'' a broken repository.
In the simplest case, if a repository's .gitattributes file contains the
following:
*.txt filter=lfs merge=lfs diff=lfs -text
But a .txt file matched by that pattern is not parse-able as an LFS
pointer, it will appear as unable to checkout.
Running 'git lfs migrate import --fixup --everything' will correctly
traverse history and find the affected .txt file, read it, create an
object file for it, and store it as an LFS pointer in history.
Thus, a user can run one command which will recognize arbitrarily
complex problems where a file should be tracked by Git LFS, but isn't.
Later, this feature could be combined with the new 'git lfs migrate
export' functionality to also clean files _out_ of Git LFS to object
files when they are not supposed to be tracked as Git LFS objects.
2018-07-06 19:20:02 +00:00
|
|
|
| tee migrate.log
|
|
|
|
|
|
|
|
if [ "${PIPESTATUS[0]}" -eq 0 ]; then
|
|
|
|
echo >&2 "fatal: expected 'git lfs migrate ...' to fail, didn't ..."
|
|
|
|
exit 1
|
|
|
|
fi
|
|
|
|
|
|
|
|
grep -q "fatal: cannot use --fixup with --include, --exclude" migrate.log
|
|
|
|
)
|
|
|
|
end_test
|
|
|
|
|
|
|
|
begin_test "migrate import (--fixup, --exclude)"
|
|
|
|
(
|
|
|
|
set -e
|
|
|
|
|
|
|
|
setup_single_local_branch_tracked_corrupt
|
|
|
|
|
commands: warn if working copy is dirty
In 'git lfs migrate import' and 'git lfs migrate export', Git LFS makes
destructive changes to a caller's repository and therefore invokes 'git
checkout --force', which throws away local changes.
To prevent this, let's introduce a check that notifies users when they
are going to throw away local changes, and allows for the user to abort
if they do not wish to discard their local changes.
With this, a user can safely migrate over dirty repositories (i.e., in
the case that they wanted to fix a file that should have been in Git
LFS, but wasn't) without having to finagle their repository to get it
into a migrate-able state.
For users with lots of pending migrations, also teach --yes, which
allows a user to avoid the check, and instead simply prints the warning
message to STDERR.
2018-07-10 15:18:46 +00:00
|
|
|
git lfs migrate import --everything --fixup --yes --exclude="*.txt" 2>&1 \
|
commands/command_migrate.go: introduce '--fixup' flag on 'import'
A common invocation of the 'git lfs migrate import' command is with
'--include' and/or '--exclude' flag(s), which specify wildmatch
pattern(s) for which paths to migrate and/or not migrate.
This is useful for retroactively importing a set of files into Git LFS's
care, or fixing up a file that should have been tracked by Git LFS but
was accidentally committed as a large object instead.
In the later case, it is often the reality that a user will run 'git lfs
migrate --import' with an '--include' path that they believe will gather
the file (and the file alone). This approach is brittle because it
requires the user to infer not only the applicable pattern but the
meaning of that pattern. It also requires the user to run more than one
migration when fixing multiple types of files.
The .gitattributes file(s) contained within a repository provide an
authoritative source on what file(s) are considered by Git to be tracked
in Git LFS. We can use this information to infer the correct patterns to
``fix up'' a broken repository.
In the simplest case, if a repository's .gitattributes file contains the
following:
*.txt filter=lfs merge=lfs diff=lfs -text
But a .txt file matched by that pattern is not parse-able as an LFS
pointer, it will appear as unable to checkout.
Running 'git lfs migrate import --fixup --everything' will correctly
traverse history and find the affected .txt file, read it, create an
object file for it, and store it as an LFS pointer in history.
Thus, a user can run one command which will recognize arbitrarily
complex problems where a file should be tracked by Git LFS, but isn't.
Later, this feature could be combined with the new 'git lfs migrate
export' functionality to also clean files _out_ of Git LFS to object
files when they are not supposed to be tracked as Git LFS objects.
2018-07-06 19:20:02 +00:00
|
|
|
| tee migrate.log
|
|
|
|
|
|
|
|
if [ "${PIPESTATUS[0]}" -eq 0 ]; then
|
|
|
|
echo >&2 "fatal: expected 'git lfs migrate ...' to fail, didn't ..."
|
|
|
|
exit 1
|
|
|
|
fi
|
|
|
|
|
|
|
|
grep -q "fatal: cannot use --fixup with --include, --exclude" migrate.log
|
|
|
|
)
|
|
|
|
end_test
|
|
|
|
|
|
|
|
begin_test "migrate import (--fixup, --no-rewrite)"
|
|
|
|
(
|
|
|
|
set -e
|
|
|
|
|
|
|
|
setup_single_local_branch_tracked_corrupt
|
|
|
|
|
commands: warn if working copy is dirty
In 'git lfs migrate import' and 'git lfs migrate export', Git LFS makes
destructive changes to a caller's repository and therefore invokes 'git
checkout --force', which throws away local changes.
To prevent this, let's introduce a check that notifies users when they
are going to throw away local changes, and allows for the user to abort
if they do not wish to discard their local changes.
With this, a user can safely migrate over dirty repositories (i.e., in
the case that they wanted to fix a file that should have been in Git
LFS, but wasn't) without having to finagle their repository to get it
into a migrate-able state.
For users with lots of pending migrations, also teach --yes, which
allows a user to avoid the check, and instead simply prints the warning
message to STDERR.
2018-07-10 15:18:46 +00:00
|
|
|
git lfs migrate import --everything --fixup --yes --no-rewrite 2>&1 \
|
commands/command_migrate.go: introduce '--fixup' flag on 'import'
A common invocation of the 'git lfs migrate import' command is with
'--include' and/or '--exclude' flag(s), which specify wildmatch
pattern(s) for which paths to migrate and/or not migrate.
This is useful for retroactively importing a set of files into Git LFS's
care, or fixing up a file that should have been tracked by Git LFS but
was accidentally committed as a large object instead.
In the later case, it is often the reality that a user will run 'git lfs
migrate --import' with an '--include' path that they believe will gather
the file (and the file alone). This approach is brittle because it
requires the user to infer not only the applicable pattern but the
meaning of that pattern. It also requires the user to run more than one
migration when fixing multiple types of files.
The .gitattributes file(s) contained within a repository provide an
authoritative source on what file(s) are considered by Git to be tracked
in Git LFS. We can use this information to infer the correct patterns to
``fix up'' a broken repository.
In the simplest case, if a repository's .gitattributes file contains the
following:
*.txt filter=lfs merge=lfs diff=lfs -text
But a .txt file matched by that pattern is not parse-able as an LFS
pointer, it will appear as unable to checkout.
Running 'git lfs migrate import --fixup --everything' will correctly
traverse history and find the affected .txt file, read it, create an
object file for it, and store it as an LFS pointer in history.
Thus, a user can run one command which will recognize arbitrarily
complex problems where a file should be tracked by Git LFS, but isn't.
Later, this feature could be combined with the new 'git lfs migrate
export' functionality to also clean files _out_ of Git LFS to object
files when they are not supposed to be tracked as Git LFS objects.
2018-07-06 19:20:02 +00:00
|
|
|
| tee migrate.log
|
|
|
|
|
|
|
|
if [ "${PIPESTATUS[0]}" -eq 0 ]; then
|
|
|
|
echo >&2 "fatal: expected 'git lfs migrate ...' to fail, didn't ..."
|
|
|
|
exit 1
|
|
|
|
fi
|
|
|
|
|
|
|
|
grep -q "fatal: --no-rewrite and --fixup cannot be combined" migrate.log
|
|
|
|
)
|
|
|
|
end_test
|