Commit Graph

150 Commits

Author SHA1 Message Date
Maciej Krüger
ba83787f0d
fetchgit: split off deterministic-git functions 2022-02-19 14:33:20 +01:00
Zhong Jianxin
36cc03e151 fetchgit: Add test for sparseCheckout 2022-01-24 11:23:56 +08:00
Zhong Jianxin
d03a07d5a7 fetchgit: Support sparse checkout
This allow git checkout small parts of a large repo, and avoid fetching
unnecessary blobs from server.
2022-01-24 11:23:56 +08:00
Tomas Bravo
80ab168c4a nix-prefetch-git: fix incorrect mktemp usage 2021-12-22 16:45:18 +01:00
Maciej Krüger
5c2b1b6a29
fetchgit: support hash parameter alongside sha256
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2021-11-09 07:40:20 +01:00
Sandro Jäckel
6f53c06748 fetchgit: add support for netrc file through impure NIX_GIT_SSL_CAINFO env 2021-09-28 15:00:11 +02:00
Robert Hensing
1648346444
Merge pull request #136022 from hercules-ci/fetcher-testing
Fetcher testing
2021-09-21 14:50:28 +02:00
Robert Hensing
0fa3d10d7d fetchgit.tests: init 2021-08-28 10:29:10 +02:00
Bjørn Forsman
9bd51f0db9 nix-prefetch-git: add fetchLFS flag to the JSON output
Similarly to the other flags (e.g. fetchSubmodules).
2021-08-28 10:14:48 +02:00
Bjørn Forsman
f43829a1d8 nix-prefetch-git: provide fallback for $TMPDIR
$TMPDIR is not set by default in NixOS (nor Ubuntu), so without this
change nix-prefetch-git fails when using git-lfs.
2021-08-28 10:14:48 +02:00
Atemu
11df41199b nix-prefetch-git: don't fetch everything when given a hash
It's hugely inefficient as we can't use shallow cloning (--depth=1).

This has been tested and adapted for quite a few hosts fetchgit is used on in
Nixpkgs. For those where fetching the hash directly doesn't work (most notably
git.savannah.gnu.org), we simply fall back to the old method.
2021-07-12 20:50:52 +02:00
Bjørn Forsman
2e8f6e0d93 fetchgit: document --fetch-lfs flag
Fixes: 74128561 ("fetchgit: add lfs support")
2021-05-07 15:38:54 -07:00
Sandro
d6df2c3fab
Merge pull request #111782 from Luflosi/fetchgit-escape-regex
fetchgit: escape dot in regex
2021-03-21 05:34:53 +01:00
taku0
a964d7094a
Merge pull request #104714 from codedownio/tree-hashes
fetchgit: support passing tree hashes as "rev"
2021-02-20 15:22:57 +09:00
Michael Weiss
2aadb9a53d
nix-prefetch-git: Run git-init with --initial-branch=master
The reason for this change is simply to avoid the following messages
that are unnecessary and can be confusing (and these messages will be
repeated for each submodule):
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint:   git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint:   git branch -m <name>

With this change the behaviour remains unchanged (apart from the
suppressed "warning" in the console output of course) and therefore this
doesn't cause any hashes to change and by default nix-prefetch-git uses
the "fetchgit" branch anyway (branchName can be set to override the
default):
Switched to a new branch 'fetchgit'

For that reason the initial branch name doesn't matter anyway and since
we're not relying on / hardcoding "master" we could simply switch to
"main" (which seems most common nowadays). See [0] for more details on
why this wouldn't break anything.
However, since the initial branch name doesn't matter and to avoid any
additional risks it was "decided" to keep using "master" (s. #113313).

[0]: https://github.com/NixOS/nixpkgs/pull/113313#issuecomment-780589516
2021-02-18 11:34:34 +01:00
Tom McLaughlin
6779902b32 fetchgit: support passing tree hashes as "rev" 2021-02-13 22:12:08 -08:00
Luflosi
e238ff073f
fetchgit: escape dot in regex
This regex should match files ending in `.git`, not any character and `git` after that.
2021-02-03 18:41:50 +01:00
Ben Siraphob
02ee14b9d5 treewide: stdenvNoCC.lib -> lib 2021-01-27 13:01:51 +07:00
Jonathan Ringer
741285611f fetchgit: add lfs support 2021-01-11 09:41:06 -08:00
Jörg Thalheim
2fb5dac372 nix-prefetch-git: also add store path to json output 2020-05-04 17:43:29 +05:30
Benjamin Hipple
259f3b3b68
Revert "fetchzip, fetchgit: cleanup handling of optional features and whitespace" 2020-03-26 00:35:40 -04:00
Benjamin Hipple
368ea2eb5e
Merge pull request #79581 from bhipple/feature/fetcher-whitespace-clean
fetchzip, fetchgit: cleanup handling of optional features and whitespace
2020-03-26 00:15:24 -04:00
Jan Tojnar
28faf5bc86
nix-prefetch-git: Fix output mangling
`git repack` and `git gc` sometimes print “Nothing new to pack.”
to stdout, which breaks redirecting output to JSON file.

Let’s move the stdout of all git calls where it is not used to stderr
so that we still receive the info but it does not pollute our output.
2020-02-14 22:15:36 +01:00
Jan Tojnar
f19a101a73
nix-prefetch-git: report deepClone & leaveDotGit
Just like in be30ba8e0e
we want `fetchgit (builtins.fromJSON (builtins.readFile ./src.json))` to work.
2020-02-14 06:05:43 +01:00
Benjamin Hipple
caff183402 fetchzip, fetchgit: cleanup handling of optional features and whitespace
No intended functional difference, just trying to polish to make this easier to
extend and more aligned with other nix conventions.
2020-02-08 21:51:46 -05:00
Dima
df7f3e0f98 nix-prefetch-git: list --branch-name in help
Listing the option and brief explanation.
Addresses: https://github.com/NixOS/nixpkgs/issues/74182
2019-11-26 20:43:23 +01:00
adisbladis
e578b8499f fetchgitPrivate: Remove fetcher
Since Nix 2.0 we have `builtins.fetchGit` which is a much better
option since it runs in the evaluator and has access to the regular
users ssh keys.
2019-11-14 09:34:12 -08:00
volth
7b8fb5c06c treewide: remove redundant quotes 2019-09-08 23:38:31 +00:00
Silvan Mosberger
89636b9a5e
nix-prefetch-git: Remove some pack file non-determinism 2019-08-31 01:16:50 +02:00
Matthew Bauer
c08cdfa781
Merge pull request #54543 from thefloweringash/git-prefetch-errors
nix-prefetch-git: propagate errors under --quiet
2019-06-18 11:20:28 -04:00
Andreas Rammhold
46f84adc0c
nix-prefetch-git: add -h as shortcut for --help to improve quality of life
Especially as a new user it is a much better experience to receive a
proper help response to `-h`. Currently passing `-h` will cause some
runtime error with the `git remote` error help being shown. Not very
helpful.

It doesn't hurt to be a bit more user friendly in this case.
2019-05-03 23:56:40 +02:00
Andrew Childs
597d6d2bcd nix-prefetch-git: propagate errors under --quiet 2019-01-25 01:52:26 +09:00
Jörg Thalheim
eac6797380
prefer-fetch-remote: an overlay to fetch on remote builders
This is useful when running tools like NixOps or nix-review
on workstations where the upload to the builder is significantly
slower then downloading the source on the builder itself.
2019-01-18 14:41:10 +00:00
Craig Younkins
8b12b17df3
treewide: Fix broken Gmane URLs 2018-12-25 22:34:55 -05:00
volth
52f53c69ce pkgs/*: remove unreferenced function arguments 2018-07-21 02:48:04 +00:00
Guillaume Maudoux
ea23a1ed4e fetchgit: adapt regex to git 2.18.0 2018-06-28 07:01:14 -05:00
Tim Cuthbertson
c5454933a1 nix-prefetch-git: fix output fetchSubmodules property 2018-06-17 21:19:27 +10:00
Lionello Lunesu
9fe26eed9e nix-prefetch-git: fix handling of submodules with spaces
The script would parse the output of `git submodule status` but
didn't handle paths with spaces in them. This would result in the
following error when trying to determine the URL of the submodule:

error: key does not contain a section: .url
2018-05-04 23:16:51 +08:00
Will Dietz
0e95bed017 nix-prefetch-git: fix extraction of submodule hashes on latest git
Summary:

According to git-submodule manpage,
"git submodule status" prefixes the hash with a '-' if it is not
initialized, and other chars in other circumstances.
(this is consistent on the various git versions tested)

nix-prefetch-git runs "git submodule init" which does you'd think,
but apparently despite this earlier versions of git before 2.16
would still give the hash the '-' suffix.
In particular this is the behavior when using 2.15 and 2.14.1
from the nixos-17.09 and nixos-17.03 channels respectively.

The script then used awk to drop the first char of the first field
which does the wrong thing when there is no prefix emitted:
while there is a space character before the hash, this is not
part of the field and so we ended up eating the first character
of the hash.

To fix this in a way that also works with the previous behavior,
this commit instead uses awk to grab the hash field
and uses tr to delete any '-' chars should they be present.

This seems to work in my testing, and for example can now
successfully fetch the source for "nginxModules.brotli"
where previously it would generate an error:

fatal: '22564a95d9ab58865a096b8d9f7324ea5f2e03e' is not a commit and a branch 'fetchgit' cannot be created from it

(we dropped a '2' from the beginning of the hash)
2018-01-24 20:18:59 +02:00
John Ericson
940c4fa3f5 treewide: Fetchers should use stdenvNoCC. 2018-01-10 11:18:44 -05:00
John Ericson
3d59b4d285 treewide: Fixed output fetch* derivations should use nativeBuildInputs 2018-01-09 20:14:46 -05:00
Frederik Rietdijk
f8eed5f7a5 fetchgitPrivate: put our custom ssh on PATH
Currently we wrap ssh so it can find the config file passed in by
<ssh-config-file>. If one however uses ProxyCommand ssh, then ssh that
is on PATH is taken (which is also unavailable when using nix-shell
--pure), which is the plain ${openssh}/bin/ssh.

This commit makes sure our wrapped ssh is available on PATH.
2017-11-07 14:07:52 +01:00
Eelco Dolstra
c3255fe8ec
fetchzip and friends: Set "name" to "source" by default
This makes them produce the same store paths as builtins.fetchgit,
builtins.fetchTarball etc. See
65b5f177b5.
2017-10-30 17:17:07 +01:00
Vaibhav Sagar
376f7aa761
Update nix-prefetch-git
s/understand/understood/
2017-10-29 12:02:26 +08:00
Vladimír Čunát
986c17727e
Merge: more compatibility for git* fetchers
They're additional commits from #26877.
Changing names of the fetched stuff was changing very many hashes,
and I think it's better to avoid that for the moment to reduce work
needed by nixpkgs users.  The fetchers are expected to be commonly
used even outside nixpkgs, and the current naming wasn't that bad
usually.

(commit analogical to d10c3cc5eedf58e80e2; I haven't noticed the part of
the PR has already got to master)
2017-07-09 10:31:24 +02:00
Volth
ab8dd33e5c support old naming, just add escaping of unsafe chars 2017-07-05 16:01:26 +00:00
Vladimír Čunát
9e1c7ddaae
Merge branch 'master' into staging 2017-07-05 09:53:53 +02:00
Volth
e7521e289c gitRepoToName: make it compatible with old fetchFromGitHub's "${repo}-${rev}-scr" to avoid mass rebuild 2017-07-04 20:22:13 +00:00
Volth
535a6c3520 fetchFromGitHub: fix fetchSubmodules=true when revision is not hexadecimal 2017-07-04 20:22:13 +00:00
Harmen
6dab2c72df nix-prefetch-git: add --quiet to --help 2017-07-03 11:44:33 +02:00