2015-03-19 19:30:55 +00:00
|
|
|
# Git LFS Specification
|
2014-06-02 14:03:37 +00:00
|
|
|
|
2015-03-19 19:30:55 +00:00
|
|
|
This is a general guide for Git LFS clients. Typically it should be
|
|
|
|
implemented by a command line `git-lfs` tool, but the details may be useful
|
2014-06-02 14:03:37 +00:00
|
|
|
for other tools.
|
|
|
|
|
|
|
|
## The Pointer
|
|
|
|
|
2015-03-19 19:30:55 +00:00
|
|
|
The core Git LFS idea is that instead of writing large blobs to a Git repository,
|
2014-06-02 14:03:37 +00:00
|
|
|
only a pointer file is written.
|
|
|
|
|
|
|
|
```
|
2015-03-19 19:30:55 +00:00
|
|
|
version https://git-lfs.github.com/spec/v1
|
2014-08-14 17:15:22 +00:00
|
|
|
oid sha256:4d7a214614ab2935c943f9e0ff69d22eadbb8f32b1258daaa5e2ca24d17e2393
|
|
|
|
size 12345
|
2014-06-02 14:03:37 +00:00
|
|
|
(ending \n)
|
|
|
|
```
|
|
|
|
|
2014-07-21 19:41:45 +00:00
|
|
|
The pointer file should be small (less than 200 bytes), and consist of only
|
2014-08-14 17:15:22 +00:00
|
|
|
ASCII characters. Libraries that generate this should write the file
|
|
|
|
identically, so that different implementations write consistent pointers that
|
|
|
|
translate to the same Git blob OID. This means:
|
2014-07-24 21:45:41 +00:00
|
|
|
|
|
|
|
* Use properties "version", "oid", and "size" in that order.
|
2014-08-14 17:15:22 +00:00
|
|
|
* Separate the property from its value with a single space.
|
2014-07-24 21:45:41 +00:00
|
|
|
* Oid has a "sha256:" prefix. No other hashing methods are currently supported
|
2015-03-19 19:30:55 +00:00
|
|
|
for Git LFS oids.
|
2014-07-24 21:45:41 +00:00
|
|
|
* Size is in bytes.
|
2014-07-21 19:41:45 +00:00
|
|
|
|
|
|
|
Note: Earlier versions only contained the OID, with a `# comment` above it.
|
|
|
|
Here's some ruby code to parse older pointer files.
|
2014-06-02 14:03:37 +00:00
|
|
|
|
|
|
|
```
|
|
|
|
# data is a string of the content
|
|
|
|
# last full line contains the oid
|
|
|
|
return nil unless data.size < 100
|
|
|
|
lines = data.
|
|
|
|
strip. # strip ending whitespace
|
|
|
|
split("\n") # split by line breaks
|
|
|
|
|
|
|
|
# We look for a comment line, and the phrase `git-media` somewhere
|
|
|
|
lines[0] =~ /# (.*git-media|external)/ && lines.last
|
|
|
|
```
|
|
|
|
|
|
|
|
That code returns the OID, which should be on the last line. The OID is
|
|
|
|
generated from the SHA-256 signature of the file's contents.
|
|
|
|
|
|
|
|
## The Server
|
|
|
|
|
2015-03-19 19:30:55 +00:00
|
|
|
Git LFS needs a URL endpoint to talk to a remote server. A Git repository
|
|
|
|
can have different Git LFS endpoints for different remotes. Here is the list
|
2015-03-22 19:02:33 +00:00
|
|
|
of rules that Git LFS uses to determine a repository's Git LFS server:
|
2014-06-02 14:03:37 +00:00
|
|
|
|
2015-03-19 19:30:55 +00:00
|
|
|
1. The `lfs.url` string.
|
2015-04-15 19:24:44 +00:00
|
|
|
2. The `remote.{name}.lfsurl` string.
|
2015-03-19 20:18:31 +00:00
|
|
|
3. Append `/info/lfs` to the remote URL. Only works with HTTPS URLs.
|
2014-06-02 14:03:37 +00:00
|
|
|
|
2015-04-19 18:42:54 +00:00
|
|
|
Git LFS runs two `git config` commands to build up the list of values that it
|
|
|
|
uses:
|
|
|
|
|
|
|
|
1. `git config -l -f .gitconfig` - This file is checked into the repository and
|
|
|
|
can set defaults for every user that clones the repository.
|
|
|
|
2. `git config -l` - A user's local git configuration can override any settings
|
|
|
|
from `.gitconfig`.
|
|
|
|
|
2015-03-19 19:30:55 +00:00
|
|
|
Here's a sample Git config file with the optional remote and Git LFS
|
|
|
|
configuration options:
|
2014-06-02 14:03:37 +00:00
|
|
|
|
|
|
|
```
|
|
|
|
[core]
|
|
|
|
repositoryformatversion = 0
|
2015-03-19 19:30:55 +00:00
|
|
|
[lfs]
|
2015-04-17 13:09:33 +00:00
|
|
|
url = "https://github.com/github/git-lfs.git/info/lfs"
|
2014-06-02 14:03:37 +00:00
|
|
|
[remote "origin"]
|
2015-03-19 20:18:31 +00:00
|
|
|
url = https://github.com/github/git-lfs
|
2014-06-02 14:03:37 +00:00
|
|
|
fetch = +refs/heads/*:refs/remotes/origin/*
|
2015-04-17 13:09:33 +00:00
|
|
|
lfsurl = "https://github.com/github/git-lfs.git/info/lfs"
|
2014-06-02 14:03:37 +00:00
|
|
|
```
|
|
|
|
|
2015-03-19 19:30:55 +00:00
|
|
|
Git LFS uses `git credential` to fetch credentials for HTTPS requests. Setup
|
2014-06-02 14:03:37 +00:00
|
|
|
a credential cache helper to save passwords for future users.
|
|
|
|
|
|
|
|
## Intercepting Git
|
|
|
|
|
2015-03-19 19:30:55 +00:00
|
|
|
Git LFS uses the `clean` and `smudge` filters to decide which files use it. The
|
|
|
|
global filters can be set up with `git lfs init`:
|
2014-06-02 14:03:37 +00:00
|
|
|
|
|
|
|
```
|
2015-03-19 19:30:55 +00:00
|
|
|
$ git lfs init
|
2014-06-02 14:03:37 +00:00
|
|
|
```
|
|
|
|
|
2015-04-11 20:14:57 +00:00
|
|
|
These filters ensure that large files aren't written into the repository proper,
|
|
|
|
instead being stored locally at `.git/lfs/objects/{OID-PATH}` (where `{OID-PATH}`
|
|
|
|
is a sharded filepath of the form `OID[0:2]/OID[2:4]/OID`), synchronized with
|
2015-04-19 18:22:15 +00:00
|
|
|
the Git LFS server as necessary. Here is a sample path to a
|
|
|
|
|
|
|
|
.git/lfs/objects/4d/7a/4d7a214614ab2935c943f9e0ff69d22eadbb8f32b1258daaa5e2ca24d17e2393
|
2015-04-11 20:14:57 +00:00
|
|
|
|
2014-06-02 14:03:37 +00:00
|
|
|
The `clean` filter runs as files are added to repositories. Git sends the
|
|
|
|
content of the file being added as STDIN, and expects the content to write
|
|
|
|
to Git as STDOUT.
|
|
|
|
|
|
|
|
* Stream binary content from STDIN to a temp file, while calculating its SHA-256
|
|
|
|
signature.
|
2015-04-19 18:22:15 +00:00
|
|
|
* Atomically move the temp file to `.git/lfs/objects/{OID-PATH}` if it does not
|
|
|
|
exist, and the sha-256 signature of the contents matches the given OID.
|
2014-06-02 14:03:37 +00:00
|
|
|
* Delete the temp file.
|
|
|
|
* Write the pointer file to STDOUT.
|
|
|
|
|
|
|
|
Note that the `clean` filter does not push the file to the server. Use the
|
2015-03-19 19:30:55 +00:00
|
|
|
`git lfs sync` command to do that.
|
2014-06-02 14:03:37 +00:00
|
|
|
|
|
|
|
The `smudge` filter runs as files are being checked out from the Git repository
|
|
|
|
to the working directory. Git sends the content of the Git blob as STDIN, and
|
|
|
|
expects the content to write to the working directory as STDOUT.
|
|
|
|
|
|
|
|
* Read 100 bytes.
|
|
|
|
* If the content is ASCII and matches the pointer file format:
|
2015-04-10 13:25:47 +00:00
|
|
|
* Look for the file in `.git/lfs/objects/{OID-PATH}`.
|
2014-06-02 14:03:37 +00:00
|
|
|
* If it's not there, download it from the server.
|
|
|
|
* Read its contents to STDOUT
|
|
|
|
* Otherwise, simply pass the STDIN out through STDOUT.
|
|
|
|
|
|
|
|
The `.gitattributes` file controls when the filters run. Here's a sample file
|
2015-03-22 19:02:33 +00:00
|
|
|
runs all mp3 and zip files through Git LFS:
|
2014-06-02 14:03:37 +00:00
|
|
|
|
|
|
|
```
|
|
|
|
$ cat .gitattributes
|
2015-03-19 19:30:55 +00:00
|
|
|
*.mp3 filter=lfs -crlf
|
|
|
|
*.zip filter=lfs -crlf
|
2014-06-02 14:03:37 +00:00
|
|
|
```
|
2014-06-24 21:19:25 +00:00
|
|
|
|
2015-03-19 19:30:55 +00:00
|
|
|
Use the `git lfs path` command to view and add to `.gitattributes`.
|