git-lfs/docs/man/git-lfs-migrate.1.ronn

300 lines
9.9 KiB
Plaintext
Raw Normal View History

git-lfs-migrate(1) - Migrate history to or from git-lfs
=======================================================
## SYNOPSIS
`git lfs migrate` <mode> [options] [--] [branch ...]
## MODES
* `info`
Show information about repository size.
2017-09-13 16:40:57 +00:00
* `import`
Convert large Git objects to LFS pointers.
2019-10-21 00:56:02 +00:00
* `export`
2019-10-21 17:09:44 +00:00
Convert LFS pointers to large Git objects.
2019-10-21 00:56:02 +00:00
## OPTIONS
* `-I` <paths> `--include=`<paths>:
See [INCLUDE AND EXCLUDE].
* `-X` <paths> `--exclude=`<paths>:
See [INCLUDE AND EXCLUDE].
* `--include-ref`=<refname>:
See [INCLUDE AND EXCLUDE (REFS)].
* `--exclude-ref`=<refname>:
See [INCLUDE AND EXCLUDE (REFS)].
* `--skip-fetch`:
Assumes that the known set of remote references is complete, and should not
be refreshed when determining the set of "un-pushed" commits to migrate. Has
no effect when combined with `--include-ref` or `--exclude-ref`.
* `--everything`:
See [INCLUDE AND EXCLUDE (REFS)].
* `--yes`:
Assume a yes answer to any prompts, permitting noninteractive use.
Currently, the only such prompt is the one asking whether to overwrite
(destroy) any working copy changes. Thus, specifying this option may cause
data loss if you are not careful.
* [branch ...]:
Migrate only the set of branches listed. If not given, `git-lfs-migrate(1)`
will migrate the currently checked out branch.
References beginning with '^' will be excluded, whereas branches that do not
begin with '^' will be included.
If any of `--include-ref` or `--exclude-ref` are given, the checked out
branch will not be appended, but branches given explicitly will be appended.
### INFO
The 'info' mode has these additional options:
* `--above=<size>`
Only count files whose individual filesize is above the given size. 'size'
may be specified as a number of bytes, or a number followed by a storage
unit, e.g., "1b", "20 MB", "3 TiB", etc.
If a set of files sharing a common extension has no files in that set whose
individual size is above the given `--above` no files no entry for that set
will be shown.
* `--top=<n>`
Only include the top 'n' entries, ordered by how many total files match the
given pathspec. Default: top 5 entries.
2017-06-15 21:08:15 +00:00
* `--unit=<unit>`
Format the number of bytes in each entry as a quantity of the storage unit
provided. Valid units include:
* b, kib, mib, gib, tib, pib - for IEC storage units
* b, kb, mb, gb, tb, pb - for SI storage units
If a --unit is not specified, the largest unit that can fit the number of
counted bytes as a whole number quantity is chosen.
2017-09-13 16:40:57 +00:00
### IMPORT
The 'import' mode migrates large objects present in the Git history to pointer
files tracked and stored with Git LFS. It supports all the core 'migrate'
options and these additional ones:
* `--verbose`
Print the commit oid and filename of migrated files to STDOUT.
2017-09-13 16:40:57 +00:00
* `--above=<size>`
Only migrate files whose individual filesize is above the given size. 'size'
may be specified as a number of bytes, or a number followed by a storage
unit, e.g., "1b", "20 MB", "3 TiB", etc.
2018-03-20 01:12:10 +00:00
* `--object-map=<path>`
Write to 'path' a file with the mapping of each rewritten commits. The file
2018-03-21 01:27:15 +00:00
format is CSV with this pattern: `OLD-SHA`,`NEW-SHA`
2018-03-20 01:12:10 +00:00
* `--no-rewrite`
Migrate large objects to Git LFS in a new commit without rewriting git
history. Please note that when this option is used, the `migrate import`
command will expect a different argument list, specialized options will
become available, and the core `migrate` options will be ignored. See
[IMPORT (NO REWRITE)].
commands/command_migrate.go: introduce '--fixup' flag on 'import' A common invocation of the 'git lfs migrate import' command is with '--include' and/or '--exclude' flag(s), which specify wildmatch pattern(s) for which paths to migrate and/or not migrate. This is useful for retroactively importing a set of files into Git LFS's care, or fixing up a file that should have been tracked by Git LFS but was accidentally committed as a large object instead. In the later case, it is often the reality that a user will run 'git lfs migrate --import' with an '--include' path that they believe will gather the file (and the file alone). This approach is brittle because it requires the user to infer not only the applicable pattern but the meaning of that pattern. It also requires the user to run more than one migration when fixing multiple types of files. The .gitattributes file(s) contained within a repository provide an authoritative source on what file(s) are considered by Git to be tracked in Git LFS. We can use this information to infer the correct patterns to ``fix up'' a broken repository. In the simplest case, if a repository's .gitattributes file contains the following: *.txt filter=lfs merge=lfs diff=lfs -text But a .txt file matched by that pattern is not parse-able as an LFS pointer, it will appear as unable to checkout. Running 'git lfs migrate import --fixup --everything' will correctly traverse history and find the affected .txt file, read it, create an object file for it, and store it as an LFS pointer in history. Thus, a user can run one command which will recognize arbitrarily complex problems where a file should be tracked by Git LFS, but isn't. Later, this feature could be combined with the new 'git lfs migrate export' functionality to also clean files _out_ of Git LFS to object files when they are not supposed to be tracked as Git LFS objects.
2018-07-06 19:20:02 +00:00
* `--fixup`
Infer `--include` and `--exclude` filters on a per-commit basis based on the
.gitattributes files in a repository. In practice, this option imports any
filepaths which should be tracked by Git LFS according to the repository's
.gitattributes file(s), but aren't already pointers. This option is
incompatible with explicitly given `--include`, `--exclude` filters.
commands/command_migrate.go: introduce '--fixup' flag on 'import' A common invocation of the 'git lfs migrate import' command is with '--include' and/or '--exclude' flag(s), which specify wildmatch pattern(s) for which paths to migrate and/or not migrate. This is useful for retroactively importing a set of files into Git LFS's care, or fixing up a file that should have been tracked by Git LFS but was accidentally committed as a large object instead. In the later case, it is often the reality that a user will run 'git lfs migrate --import' with an '--include' path that they believe will gather the file (and the file alone). This approach is brittle because it requires the user to infer not only the applicable pattern but the meaning of that pattern. It also requires the user to run more than one migration when fixing multiple types of files. The .gitattributes file(s) contained within a repository provide an authoritative source on what file(s) are considered by Git to be tracked in Git LFS. We can use this information to infer the correct patterns to ``fix up'' a broken repository. In the simplest case, if a repository's .gitattributes file contains the following: *.txt filter=lfs merge=lfs diff=lfs -text But a .txt file matched by that pattern is not parse-able as an LFS pointer, it will appear as unable to checkout. Running 'git lfs migrate import --fixup --everything' will correctly traverse history and find the affected .txt file, read it, create an object file for it, and store it as an LFS pointer in history. Thus, a user can run one command which will recognize arbitrarily complex problems where a file should be tracked by Git LFS, but isn't. Later, this feature could be combined with the new 'git lfs migrate export' functionality to also clean files _out_ of Git LFS to object files when they are not supposed to be tracked as Git LFS objects.
2018-07-06 19:20:02 +00:00
If `--no-rewrite` is not provided and `--include` or `--exclude` (`-I`, `-X`,
respectively) are given, the .gitattributes will be modified to include any new
filepath patterns as given by those flags.
2017-09-13 16:40:57 +00:00
If `--no-rewrite` is not provided and neither of those flags are given, the
gitattributes will be incrementally modified to include new filepath extensions
as they are rewritten in history.
2017-09-13 16:40:57 +00:00
### IMPORT (NO REWRITE)
The `import` mode has a special sub-mode enabled by the `--no-rewrite` flag.
This sub-mode will migrate large objects to pointers as in the base `import`
mode, but will do so in a new commit without rewriting Git history. When using
this sub-mode, the base `migrate` options, such as `--include-ref`, will be
ignored, as will those for the base `import` mode. The `migrate` command will
also take a different argument list. As a result of these changes,
`--no-rewrite` will only operate on the current branch - any other interested
branches must have the generated commit merged in.
The `--no-rewrite` sub-mode supports the following options and arguments:
* `-m <message> --message=<message>`
Specifies a commit message for the newly created commit.
* [file ...]
The list of files to import. These files must be tracked by patterns
specified in the gitattributes.
If `--message` is given, the new commit will be created with the provided
message. If no message is given, a commit message will be generated based on the
file arguments.
2018-06-27 18:55:38 +00:00
### EXPORT
The 'export' mode migrates Git LFS pointer files present in the Git history out
of Git LFS, converting them into their corresponding object files. It supports
all the core 'migrate' options and these additional ones:
* `--verbose`
Print the commit oid and filename of migrated files to STDOUT.
* `--object-map=<path>`
Write to 'path' a file with the mapping of each rewritten commit. The file
format is CSV with this pattern: `OLD-SHA`,`NEW-SHA`
* `--remote=<git-remote>`
Download LFS objects from the provided 'git-remote' during the export. If
not provided, defaults to 'origin'.
The 'export' mode requires at minimum a pattern provided with the `--include`
argument to specify which files to export. Files matching the `--include`
patterns will be removed from Git LFS, while files matching the `--exclude`
patterns will retain their Git LFS status. The export command will modify the
.gitattributes to set/unset any filepath patterns as given by those flags.
2018-06-27 18:55:38 +00:00
## INCLUDE AND EXCLUDE
You can configure Git LFS to only migrate tree entries whose pathspec matches
the include glob and does not match the exclude glob, to reduce total migration
time or to only migrate part of your repo. Specify multiple patterns using the
comma as the delimiter.
Pattern matching is done as given to be functionally equivalent to pattern
matching as in .gitattributes.
## INCLUDE AND EXCLUDE (REFS)
You can configure Git LFS to only migrate commits reachable by references
include by `--include-ref` and not reachable by `--exclude-ref`.
```
D---E---F
/ \
A---B------C refs/heads/my-feature
\ \
\ refs/heads/master
\
refs/remotes/origin/master
```
In the above configuration, the following commits are reachable by each ref:
```
refs/heads/master: C, B, A
refs/heads/my-feature: F, E, D, B, A
refs/remote/origin/master: A
```
The following configuration:
```
--include-ref=refs/heads/my-feature
--include-ref=refs/heads/master
--exclude-ref=refs/remotes/origin/master
```
Would, therefore, include commits: F, E, D, C, B, but exclude commit A.
The presence of flag `--everything` indicates that all local and remote
references should be migrated.
2017-09-13 22:42:28 +00:00
## EXAMPLES
### Migrate unpushed commits
The migrate command's most common use case is to convert large git objects to
LFS before pushing your commits. By default, it only scans commits that don't
exist on any remote, so long as the repository is non-bare.
2017-09-13 22:42:28 +00:00
First, run `git lfs migrate info` to list the file types taking up the most
space in your repository.
```
$ git lfs migrate info
migrate: Fetching remote refs: ..., done
migrate: Sorting commits: ..., done
migrate: Examining commits: 100% (1/1), done
*.mp3 284 MB 1/1 files(s) 100%
*.pdf 42 MB 8/8 files(s) 100%
*.psd 9.8 MB 15/15 files(s) 100%
*.ipynb 6.9 MB 6/6 files(s) 100%
*.csv 5.8 MB 2/2 files(s) 100%
```
Now, you can run `git lfs migrate import` to convert some file types to LFS:
```
$ git lfs migrate import --include="*.mp3,*.psd"
2017-09-13 22:42:28 +00:00
migrate: Fetching remote refs: ..., done
migrate: Sorting commits: ..., done
migrate: Rewriting commits: 100% (1/1), done
2017-09-13 22:59:00 +00:00
master d2b959babd099fe70da1c1512e2475e8a24de163 -> 136e706bf1ae79643915c134e17a6c933fd53c61
2017-09-13 22:42:28 +00:00
migrate: Updating refs: ..., done
```
### Migrate local history
You can also migrate the entire history of your repository:
```
# Check for large files in your local master branch
$ git lfs migrate info --include-ref=master
# Check for large files in every branch
$ git lfs migrate info --everything
```
The same flags will work in `import` mode:
```
# Convert all zip files in your master branch
$ git lfs migrate import --include-ref=master --include="*.zip"
# Convert all zip files in every local branch
2017-09-18 22:56:26 +00:00
$ git lfs migrate import --everything --include="*.zip"
# Convert all files over 100K in every local branch
$ git lfs migrate import --everything --above=100Kb
2017-09-13 22:42:28 +00:00
```
Note: This will require a force push to any existing Git remotes.
### Migrate without rewriting local history
You can also migrate files without modifying the existing history of your
repository. Note that in the examples below, files in subdirectories are not
included because they are not explicitly specified.
commands/command_migrate.go: introduce '--fixup' flag on 'import' A common invocation of the 'git lfs migrate import' command is with '--include' and/or '--exclude' flag(s), which specify wildmatch pattern(s) for which paths to migrate and/or not migrate. This is useful for retroactively importing a set of files into Git LFS's care, or fixing up a file that should have been tracked by Git LFS but was accidentally committed as a large object instead. In the later case, it is often the reality that a user will run 'git lfs migrate --import' with an '--include' path that they believe will gather the file (and the file alone). This approach is brittle because it requires the user to infer not only the applicable pattern but the meaning of that pattern. It also requires the user to run more than one migration when fixing multiple types of files. The .gitattributes file(s) contained within a repository provide an authoritative source on what file(s) are considered by Git to be tracked in Git LFS. We can use this information to infer the correct patterns to ``fix up'' a broken repository. In the simplest case, if a repository's .gitattributes file contains the following: *.txt filter=lfs merge=lfs diff=lfs -text But a .txt file matched by that pattern is not parse-able as an LFS pointer, it will appear as unable to checkout. Running 'git lfs migrate import --fixup --everything' will correctly traverse history and find the affected .txt file, read it, create an object file for it, and store it as an LFS pointer in history. Thus, a user can run one command which will recognize arbitrarily complex problems where a file should be tracked by Git LFS, but isn't. Later, this feature could be combined with the new 'git lfs migrate export' functionality to also clean files _out_ of Git LFS to object files when they are not supposed to be tracked as Git LFS objects.
2018-07-06 19:20:02 +00:00
Without a specified commit message:
```
$ git lfs migrate import --no-rewrite test.zip *.mp3 *.psd
```
With a specified commit message:
```
$ git lfs migrate import --no-rewrite \
-m "Import test.zip, .mp3, .psd files in root of repo" \
test.zip *.mp3 *.psd
```
## SEE ALSO
Part of the git-lfs(1) suite.