I was having a hard time testing new bootstrapFiles because
`make-bootstrap-tools.nix` imports `pkgspath` but does not pass anything
but the current system.
This is merely for convenience and I'm not entirely certain it's a
sensible thing to do, maybe generating new bootstrapFiles while
overriding the current bootstrapFiles isn't something you're supposed to
do?
The rpath structure for the bootstrap tools was reworked to minimize
the amount of rewriting required on unpack, but the test was not
updated to match the different structure.
Additionally [1] builds that use the bootstrap version of libc++
cannot find libc++abi if the reference includes the "lib"
component (ie, libc++ refers to libc++abi with
@rpath/lib/libc++abi.dylib).
[1] https://logs.nix.samueldr.com/nix-darwin/2021-05-18#4993282
Test failure observed on Hydra: https://hydra.nixos.org/build/143130126
Also begin to start work on cross compilation, though that will have to
be finished later.
The patches are based on the first version of
https://reviews.llvm.org/D99484. It's very annoying to do the
back-porting but the review has uncovered nothing super major so I'm
fine sticking with what I've got.
Beyond making the outputs work, I also strove to re-sync the packages,
as they have been drifting pointlessly apart for some time.
----
Other misc notes, highly incomplete
- lvm-config-native and llvm-config are put in `dev` because they are
tools just for build time.
- Clang no longer has an lld dep. That was introduced in
db29857eb391ed002046090851a44c452b80bdbd, but if clang needs help
finding lld when it is used we should just pass it flags / put in the
resource dir. Providing it at build time increases critical path
length for no good reason.
----
A note on `nativeCC`:
`stdenv` takes tools from the previous stage, so:
1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.stdenv.cc`: `(?0, ?1, x)`
while:
1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.targetPackages`: `(x, x, ?2)`
3. `pkgsBuildBuild.targetPackages.stdenv.cc`: `(?1, x, x)`
LTO is disabled during bootstrap to keep the bootstrap tools small and
avoid unnecessary LLVM rebuilds, but is enabled in the final stdenv
stage and should be usable by normal packages.
This also updates the bootstrap tool builder to LLVM 5, but not the ones
we actually use for bootstrap. I'll make that change in a subsequent commit
so as to provide traceable provenance of the bootstrap tools.
In the extremely unlikely case that our store hash path ends in several
digits (as is the case right now), the Darwin ld will try to interpret
those digits as a version number and barf. To avoid that, we pass in the
SDK version explicitly to stop it from trying to figure it out from iffy
context.
Introduce new abstraction, `stdenv/booter.nix` for composing bootstraping
stages, and use it everywhere for consistency. See that file for more doc.
Stdenvs besides Linux and Darwin are completely refactored to utilize this.
Those two, due to their size and complexity, are minimally edited for
easier reviewing.
No hashes should be changed.
- The darwin test can now force the use of the freshly-booted darwin stdenv
- The linux test now passes enough dummy arguments
This may make debugging harder, if so, check out #20889
Revert a revert of a merge that shouldn't have been in master but was intentionally in staging.
Next time I'll do this right after the revert instead of so far down the line...
This reverts commit 9adad8612b082bcbae30c81678a04b79a44079a4.
This partially reverts commit bddf4e21801d0629b608a8a0a46830c795e35ba4.
cc @copumpkin
Was breaking nix-build <nixpkgs/nixos/release-combined.nix> -A nixpkgs.tarball
See http://hydra.nixos.org/build/38886695/nixlog/2
This will break part of the bootstrap tools tests because the new tools
need some changes in the stdenv, but if I change them all at once, the
stdenv breaks with the old bootstrap tools. So I'm doing this first, then
will make changes to the stdenv once this bundle is built and I can use
it.
I also added some functionality to let me test one set of bootstrap tools
on another nixpkgs tree, which makes testing a lot more pleasant.
Was meant to go into staging, sorry
This reverts commit 57b2d1e9b0dcdd1d25bd2d450174764b9417ffc1, reversing
changes made to 760b2b9048ea775c319cb348d74447a20dea513e.
This reverts commit debd401b0f8bedae10c368abd0cacf75ef7f6350.
We must not use a single-binary build for the bootstrap since the common
binary gains a dynamic linkage to gmp (due to 'factor' and 'expr'
handling arbitrary-precision arithmetic).
Our coreutils now uses single-binary-build mode where, by default,
simple shebang scripts are used for all the binaries. That doesn't work
e.g. with the Linux unpacker which only handles standard binaries and
symlinks. Let's use the symlinked mode instead for boostrapping.
This does NOT change any stdenv hashes.
I only tested the case most important to me:
$ nix-build pkgs/top-level/release.nix -A stdenvBootstrapTools.x86_64-linux.test
The $lib output refers to the terminfo database in $out, which is about
10x larger than the ncurses shared library. Splitting these outputs
saves a small amount of space for any derivations that use the terminfo
database but not the ncurses library, but we do not have evidence that
any such exist.
