197 lines
6.9 KiB
Markdown
197 lines
6.9 KiB
Markdown
# Git LFS Specification
|
|
|
|
This is a general guide for Git LFS clients. Typically it should be
|
|
implemented by a command line `git-lfs` tool, but the details may be useful
|
|
for other tools.
|
|
|
|
## The Pointer
|
|
|
|
The core Git LFS idea is that instead of writing large blobs to a Git repository,
|
|
only a pointer file is written.
|
|
|
|
* Pointer files are text files which MUST contain only UTF-8 characters.
|
|
* Each line MUST be of the format `{key} {value}\n` (trailing unix newline).
|
|
* Only a single space character between `{key}` and `{value}`.
|
|
* Keys MUST only use the characters `[a-z] [0-9] . -`.
|
|
* The first key is _always_ `version`.
|
|
* Lines of key/value pairs MUST be sorted alphabetically in ascending order
|
|
(with the exception of `version`, which is always first).
|
|
* Values MUST NOT contain return or newline characters.
|
|
* Pointer files SHOULD NOT have the executable bit set when checked out from Git.
|
|
|
|
The required keys are:
|
|
|
|
* `version` is a URL that identifies the pointer file spec. Parsers MUST use
|
|
simple string comparison on the version, without any URL parsing or
|
|
normalization. It is case sensitive, and %-encoding is discouraged.
|
|
* `oid` tracks the unique object id for the file, prefixed by its hashing
|
|
method: `{hash-method}:{hash}`. Currently, only `sha256` is supported.
|
|
* `size` is in bytes.
|
|
|
|
Example of a v1 text pointer:
|
|
|
|
```
|
|
version https://git-lfs.github.com/spec/v1
|
|
oid sha256:4d7a214614ab2935c943f9e0ff69d22eadbb8f32b1258daaa5e2ca24d17e2393
|
|
size 12345
|
|
(ending \n)
|
|
```
|
|
|
|
Blobs created with the pre-release version of the tool will generated files with
|
|
a different version URL. Git LFS can read these files, but writes them using
|
|
the version URL above.
|
|
|
|
```
|
|
version https://hawser.github.com/spec/v1
|
|
oid sha256:4d7a214614ab2935c943f9e0ff69d22eadbb8f32b1258daaa5e2ca24d17e2393
|
|
size 12345
|
|
(ending \n)
|
|
```
|
|
|
|
For testing compliance of any tool generating its own pointer files, the
|
|
reference is this official Git LFS tool:
|
|
|
|
NOTE: exact pointer command behavior TBD!
|
|
|
|
* Tools that parse and regenerate pointer files MUST preserve keys that they
|
|
don't know or care about.
|
|
* Run the `pointer` command to generate a pointer file for the given local
|
|
file:
|
|
|
|
$ git lfs pointer --file=path/to/file
|
|
Git LFS pointer for path/to/file:
|
|
|
|
version https://git-lfs.github.com/spec/v1
|
|
oid sha256:4d7a214614ab2935c943f9e0ff69d22eadbb8f32b1258daaa5e2ca24d17e2393
|
|
size 12345
|
|
|
|
* Run `pointer` to compare the blob OID of a pointer file built by Git LFS with
|
|
a pointer built by another tool.
|
|
|
|
* Write the other implementation's pointer to "other/pointer/file":
|
|
|
|
$ git lfs pointer --file=path/to/file --pointer=other/pointer/file
|
|
Git LFS pointer for path/to/file:
|
|
|
|
version https://git-lfs.github.com/spec/v1
|
|
oid sha256:4d7a214614ab2935c943f9e0ff69d22eadbb8f32b1258daaa5e2ca24d17e2393
|
|
size 12345
|
|
|
|
Blob OID: 60c8d8ab2adcf57a391163a7eeb0cdb8bf348e44
|
|
|
|
Pointer from other/pointer/file
|
|
version https://git-lfs.github.com/spec/v1
|
|
oid sha256 4d7a214614ab2935c943f9e0ff69d22eadbb8f32b1258daaa5e2ca24d17e2393
|
|
size 12345
|
|
|
|
Blob OID: 08e593eeaa1b6032e971684825b4b60517e0638d
|
|
|
|
Pointers do not match!
|
|
|
|
* It can also read STDIN to get the other implementation's pointer:
|
|
|
|
$ cat other/pointer/file | git lfs pointer --file=path/to/file --stdin
|
|
Git LFS pointer for path/to/file:
|
|
|
|
version https://git-lfs.github.com/spec/v1
|
|
oid sha256:4d7a214614ab2935c943f9e0ff69d22eadbb8f32b1258daaa5e2ca24d17e2393
|
|
size 12345
|
|
|
|
Blob OID: 60c8d8ab2adcf57a391163a7eeb0cdb8bf348e44
|
|
|
|
Pointer from STDIN
|
|
version https://git-lfs.github.com/spec/v1
|
|
oid sha256 4d7a214614ab2935c943f9e0ff69d22eadbb8f32b1258daaa5e2ca24d17e2393
|
|
size 12345
|
|
|
|
Blob OID: 08e593eeaa1b6032e971684825b4b60517e0638d
|
|
|
|
Pointers do not match!
|
|
|
|
## The Server
|
|
|
|
Git LFS needs a URL endpoint to talk to a remote server. A Git repository
|
|
can have different Git LFS endpoints for different remotes. Here is the list
|
|
of rules that Git LFS uses to determine a repository's Git LFS server:
|
|
|
|
1. The `lfs.url` string.
|
|
2. The `remote.{name}.lfsurl` string.
|
|
3. Append `/info/lfs` to the remote URL. Only works with HTTPS URLs.
|
|
|
|
Git LFS runs two `git config` commands to build up the list of values that it
|
|
uses:
|
|
|
|
1. `git config -l -f .gitconfig` - This file is checked into the repository and
|
|
can set defaults for every user that clones the repository.
|
|
2. `git config -l` - A user's local git configuration can override any settings
|
|
from `.gitconfig`.
|
|
|
|
Here's a sample Git config file with the optional remote and Git LFS
|
|
configuration options:
|
|
|
|
```
|
|
[core]
|
|
repositoryformatversion = 0
|
|
[lfs]
|
|
url = "https://github.com/github/git-lfs.git/info/lfs"
|
|
[remote "origin"]
|
|
url = https://github.com/github/git-lfs
|
|
fetch = +refs/heads/*:refs/remotes/origin/*
|
|
lfsurl = "https://github.com/github/git-lfs.git/info/lfs"
|
|
```
|
|
|
|
Git LFS uses `git credential` to fetch credentials for HTTPS requests. Setup
|
|
a credential cache helper to save passwords for future users.
|
|
|
|
## Intercepting Git
|
|
|
|
Git LFS uses the `clean` and `smudge` filters to decide which files use it. The
|
|
global filters can be set up with `git lfs init`:
|
|
|
|
```
|
|
$ git lfs init
|
|
```
|
|
|
|
These filters ensure that large files aren't written into the repository proper,
|
|
instead being stored locally at `.git/lfs/objects/{OID-PATH}` (where `{OID-PATH}`
|
|
is a sharded filepath of the form `OID[0:2]/OID[2:4]/OID`), synchronized with
|
|
the Git LFS server as necessary. Here is a sample path to a
|
|
|
|
.git/lfs/objects/4d/7a/4d7a214614ab2935c943f9e0ff69d22eadbb8f32b1258daaa5e2ca24d17e2393
|
|
|
|
The `clean` filter runs as files are added to repositories. Git sends the
|
|
content of the file being added as STDIN, and expects the content to write
|
|
to Git as STDOUT.
|
|
|
|
* Stream binary content from STDIN to a temp file, while calculating its SHA-256
|
|
signature.
|
|
* Atomically move the temp file to `.git/lfs/objects/{OID-PATH}` if it does not
|
|
exist, and the sha-256 signature of the contents matches the given OID.
|
|
* Delete the temp file.
|
|
* Write the pointer file to STDOUT.
|
|
|
|
Note that the `clean` filter does not push the file to the server. Use the
|
|
`git push` command to do that (lfs files are pushed before commits in a pre-push hook).
|
|
|
|
The `smudge` filter runs as files are being checked out from the Git repository
|
|
to the working directory. Git sends the content of the Git blob as STDIN, and
|
|
expects the content to write to the working directory as STDOUT.
|
|
|
|
* Read 100 bytes.
|
|
* If the content is ASCII and matches the pointer file format:
|
|
* Look for the file in `.git/lfs/objects/{OID-PATH}`.
|
|
* If it's not there, download it from the server.
|
|
* Read its contents to STDOUT
|
|
* Otherwise, simply pass the STDIN out through STDOUT.
|
|
|
|
The `.gitattributes` file controls when the filters run. Here's a sample file
|
|
runs all mp3 and zip files through Git LFS:
|
|
|
|
```
|
|
$ cat .gitattributes
|
|
*.mp3 filter=lfs -crlf
|
|
*.zip filter=lfs -crlf
|
|
```
|
|
|
|
Use the `git lfs track` command to view and add to `.gitattributes`.
|