lunny/git-lfs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

git/odb/pack: prevent unnecessary runtime.memmove

Browse Source 
Running 'go tool pprof' in CPU mode while Git LFS is running a migration
against a large repository shows that 4.73% percent of the CPU time is spent in
a function called 'runtime.memmove'. This is the function called when
append()-ing to a slice causes the slice to grow, and memory occupied by the
existing slice must be moved into a different contiguous group.

```
$ go tool pprof $(which git-lfs) git-lfs-1504820145.pprof dot
File: git-lfs
Type: cpu
Time: Sep 7, 2017 at 5:35pm (EDT)
Duration: 35.17s, Total samples = 30.42s (86.49%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) list runtime\.memmove
Total: 30.42s
ROUTINE ======================== runtime.memmove in ...
     1.44s      1.44s (flat, cum)  4.73% of Total
         .          .     30:// void runtime·memmove(void*, void*, uintptr)
         .          .     31:TEXT runtime·memmove(SB), NOSPLIT, $0-24
         .          .     32:
...
```

The implementation of 'runtime.memmove' is [fairly complex][1] and takes a
relatively long amount of time to execute. Unfortunately, the 'patch' function
in package 'git/odb/pack' is one of the smaller contributors:

```
(pprof) tree runtime\.memmove
----------------------------------------------------------+-------------
                                             0.01s   100% |   git/odb/pack.patch
         0     0%  4.73%      0.01s 0.033%                | runtime.growslice
                                             0.01s   100% |   runtime.gcAssistAlloc
----------------------------------------------------------+-------------
```

That said, the delta instructions do hint at the size of the patched result of
applying 'delta' to 'base', which we can use to eagerly allocate a contiguous
block of memory for.

[1]: https://github.com/golang/go/blob/go1.9/src/runtime/memmove_amd64.s
This commit is contained in:
Taylor Blau 2017-09-07 18:00:19 -04:00
parent 304707f25d
commit f4f8e76a5d
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
git/odb/pack/chain_delta.go
View File

@ -52,13 +52,13 @@ func patch(base, delta []byte) ([]byte, error) {
return nil, errors.New("git/odb/pack: invalid delta data")
}
var dest []byte
// The remainder of the delta header contains the destination size, and
// moves the "pos" offset to the correct position to begin the set of
// delta instructions.
destSize, pos := patchDeltaHeader(delta, pos)
dest := make([]byte, 0, destSize)
for pos < len(delta) {
c := int(delta[pos])
pos += 1