the default hasn't been changed since 2009
this can improve our test performances
nixos/tests: remove explicit memorySize <1024
1024MiB is now the default
The default has been unchanged for a decade. Space is cheaper and
software catches up with that. Let's not make our testing harder
than necessary by default.
qemu_kvm is only built for one architecture, so it's smaller and takes
MUCH less time to build if it has to be built from source. And this
module doesn't support running a VM for one architecture from another
architecture, so the one architecture is all we'll need.
pathsInNixDB isn't a very accurate name when a Nix store image is
built (virtualisation.useNixStoreImage); rename it to additionalPaths,
which should be general enough to cover both cases.
Add the `useNixStoreImage` option, allowing a disk image with the
necessary contents from the Nix store to be built using
make-disk-image.nix. The image will be mounted at `/nix/store` and
acts as a drop-in replacement for the usual 9p mounting of the host's
Nix store.
This removes the performance penalty of 9p, drastically improving
execution speed of applications which do lots of reads from the Nix
store. The caveats are increased disk space usage and image build
time.
I realized quite recently that running a test VM - as documented in the
manual - like
QEMU_NET_OPTS='hostfwd=tcp::8080-:80' ./result/bin/nixos-run-vms
doesn't work anymore on `master`. After bisecting I realized that the
introduction of a forward-port option[1] is the problem since it adds a
trailing comma even if no forwarding options are specified via
`virtualisation.forwardPorts`. In that case, the networking options
would look like `-netdev user,id=user.0,,hostfwd=tcp::8080-:80' which
confused QEMU and thus the VM refused to start.
Now, the trailing comma is only added if additional port forwards are
specified declaratively.
[1] b8bfc81d5b2d88b734a311f712fc0ba2b267f9e0
The current name is misleading: it doesn't contain cli arguments,
but several constants and utility functions related to qemu.
This commit also removes the use of `with import ...` for clarity.
... build-vm-with-bootloader" for EFI systems
This reverts commit 20257280d9a3591a2888299a1aabfa850a0d0d08, reversing
changes made to 926a1b20949ddd66e3a4e1327cb1c1010ab10442.
It broke nixosTests.installer.simpleUefiSystemdBoot
and right now channel is lagging behing for two weeks.
`nixos-rebuild build-vm-with-bootloader` currently fails with the
default NixOS EFI configuration:
$ cat >configuration.nix <<EOF
{
fileSystems."/".device = "/dev/sda1";
boot.loader.systemd-boot.enable = true;
boot.loader.efi.canTouchEfiVariables = true;
}
EOF
$ nixos-rebuild build-vm-with-bootloader -I nixos-config=$PWD/configuration.nix -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/nixos-20.09.tar.gz
[...]
insmod: ERROR: could not insert module /nix/store/1ibmgfr13r8b6xyn4f0wj115819f359c-linux-5.4.83/lib/modules/5.4.83/kernel/fs/efivarfs/efivarfs.ko.xz: No such device
mount: /sys/firmware/efi/efivars: mount point does not exist.
[ 1.908328] reboot: Power down
builder for '/nix/store/dx2ycclyknvibrskwmii42sgyalagjxa-nixos-boot-disk.drv' failed with exit code 32
[...]
Fix it by setting virtualisation.useEFIBoot = true in qemu-vm.nix, when
efi is needed.
And remove the now unneeded configuration in
./nixos/tests/systemd-boot.nix, since it's handled globally.
Before:
* release-20.03: successful build, unsuccessful run
* release-20.09 (and master): unsuccessful build
After:
* Successful build and run.
Fixes https://github.com/NixOS/nixpkgs/issues/107255
The toplevel derivations of systems that have `networking.hostName`
set to `""` (because they want their hostname to be set by DHCP) used
to be all named
`nixos-system-unnamed-${config.system.nixos.label}`.
This makes them hard to distinguish.
A similar problem existed in NixOS tests where `vmName` is used in the
`testScript` to refer to the VM. It defaulted to the
`networking.hostName` which when set to `""` won't allow you to refer
to the machine from the `testScript`.
This commit makes the `system.name` configurable. It still defaults to:
```
if config.networking.hostName == ""
then "unnamed"
else config.networking.hostName;
```
but in case `networking.hostName` needs to be to `""` the
`system.name` can be set to a distinguishable name.
Without this, systemd-boot does not add an EFI boot entry for itself.
The reason it worked before this fix is because it would fall back to
the default installed \EFI\BOOT\BOOTX64.EFI
boot.loader.grub.device` was hardcoded to `bootDevice`, which is
wrong, because that's the device for `/`, and with `useBootLoader`
the boot loader is not on that device.
This bug probably came into existence because of bad naming;
`virtualisation.bootDevice` has description
"The disk to be used for the root filesystem", which is very confusing;
it should be `.rootDevice` then!
Unfortunately, the description is right and the attribute name is wrong,
so it is not easy to change this without deprecation.
This commit ensures that even if you use `useBootLoader` and
`diskInterface == "scsi"`, the created VM can boot through, and can run
`nixos-rebuild afterwards.
It also adds extra commentary to explain what's going on in this module
in general in relation to `useBootLoader`.
xchg is advertised as a bidirectional exchange dir, but file content
transfer from host to VM fails due to caching:
If a file is read in the VM and then modified on the host, subsequent
re-reads in the VM can yield old, cached data.
This is caused by the use of 9p's cache=loose mode that is explicitly
meant for read-only mounts.
9p doesn't provide any suitable cache modes, so fix this by disabling
caching.
Also, remove a now unnecessary sync in the test driver.
As part of the networking.* name space cleanup, connman should be moved
to services.connman. The same will happen for example with
networkmanager in a separate PR.
Adds `virtualisation.qemu.drives` option to specify drives to be used by
qemu.
Also fix boot when `virtualisation.useBootLoader` is set to true. Since
the boot disk is second qemu doesn't boot on it. Added `bootindex=1` to
the boot disk device.
This module permits to preload Docker image in a VM in order to reduce
OIs on file copies. This module has to be only used in testing
environments, when the test requires several Docker images such as in
Kubernetes tests. In this case,
`virtualisation.dockerPreloader.images` can replace the
`services.kubernetes.kubelet.seedDockerImages` options.
The idea is to populate the /var/lib/docker directory by mounting qcow
files (we uses qcow file to avoid permission issues) that contain images.
For each image specified in
config.virtualisation.dockerPreloader.images:
1. The image is loaded by Docker in a VM
2. The resulting /var/lib/docker is written to a QCOW file
This set of QCOW files can then be used to populate the
/var/lib/docker:
1. Each QCOW is mounted in the VM
2. Symlink are created from these mount points to /var/lib/docker
3. A /var/lib/docker/image/overlay2/repositories.json file is generated
4. The docker daemon is started.
This reverts commit f777d2b719be0db4cb7819d5b6dbdd121db29a37.
cc #34409
This breaks evaluation of the tested job:
attribute 'diskInterface' missing, at /nix/store/5k9kk52bv6zsvsyyvpxhm8xmwyn2yjvx-source/pkgs/build-support/vm/default.nix:316:24
And don't need to source the uevent files anymore either since $MAJOR
or $MINOR aren't used elsewhere.
[dezgeg: The reason these are no longer needed is that 0d27df280f7ed5
switched /tmp to a devtmpfs which automatically creates such device
nodes]
Regression introduced by d4468bedb52bf03ff125d721634f83604bddd589.
No systemd messages are shown anymore during VM test runs, which is not
very helpful if you want to find out about failures.
There is a bit of a conflict between testing and the change that
introduced the regression. While the mentioned commit makes sure that
the primary console is tty0 for virtualisation.graphics = false, our VM
tests need to have the serial console as primary console.
So in order to support both, I added a new virtualisation.qemu.consoles
option, which allows to specify those options using the module system.
The default of this option is to use the changes that were introduced
and in test-instrumentation.nix we use only the serial console the same
way as before.
For test-instrumentation.nix I didn't add a baudrate to the serial
console because I can't find a reason on top of my head why it should
need it. There also wasn't a reason stated when that was introduced in
7499e4a5b917227696b7292ae1bd728d6b1978b0.
Signed-off-by: aszlig <aszlig@nix.build>
Cc: @flokli, @dezgeg, @edolstra
Always enable both tty and serial console, but set preferred console
depending on cfg.graphical.
Even in qemu graphical mode, you can switch to the serial console via
Ctrl+Alt+3.
With that being done, you also don't need to specify
`systemd.services."serial-getty@ttyS0".enable = true;` either as described in
https://nixos.wiki/wiki/Cheatsheet#Building_a_service_as_a_VM_.28for_testing.29,
as systemd automatically spawns a getty on consoles passwd via cmdline.
This also means, vms built by 'nixos-rebuild build-vm' can simply be run
properly in nographic mode by appending `-nographic` to `result/bin/run-*-vm`,
without the need to explicitly add platform-specific QEMU_KERNEL_PARAMS.
The ability to specify "-drive if=scsi" has been removed in QEMU version
2.12 (introduced in 3e3b39f173f9abc99da84084a1f4657c9de885bd).
Quote from https://wiki.qemu.org/ChangeLog/2.12#Incompatible_changes:
> The deprecated way of configuring SCSI devices with "-drive if=scsi"
> on x86 has been removed. Use an appropriate SCSI controller together
> "-device scsi-hd" or "-device scsi-cd" and a corresponding "-blockdev"
> parameter instead.
So whenever the diskInterface is "scsi" we use the new way to specify
the drive and fall back to the deprecated way for the time being. The
reason why I'm not using the new way for "virtio" and "ide" as well is
because there is no simple generic way anymore to specify these.
This also turns the type of the virtualisation.qemu.diskInterface option
to be an enum, so the user knows which values are allowed but we can
also make sure the right value is provided to prevent typos.
I've tested this against a few non-disk-related NixOS VM tests but also
the installer.grub1 test (because it uses "ide" as its drive interface),
the installer.simple test (just to be sure it still works with
"virtio") and all the tests in nixos/tests/boot.nix.
In order to be able to run the grub1 test I had to go back to
8b1cf100cd8badad6e1b6d4650b904b88aa870db (which is a known commit where
that test still works) and apply the QEMU update and this very commit,
because right now the test is broken.
Apart from the tests here in nixpkgs, I also ran another[1] test in
another repository which uses the "scsi" disk interface as well (in
comparison to most of the installer tests, this one actually failed
prior to this commit).
All of them now succeed.
[1]: 9b5a119972/tests/system/kernel/bfq.nix
Signed-off-by: aszlig <aszlig@nix.build>
Cc: @edostra, @grahamc, @dezgeg, @abbradar, @ts468
mke2fs has this annoying property that it uses getrandom() to get random
numbers (for whatever purposes) which blocks until the kernel's secure
RNG has sufficient entropy, which it usually doesn't in the early boot
(except if your CPU supports RDRAND) where we may need to create the
root disk.
So let's give the VM a virtio RNG to avoid the boot getting stuck at
mke2fs.
Allow out of band communication between qemu VMs and the host.
Useful to retrieve IPs of VMs from the host (for instance when libvirt can't analyze
DHCP requests because VMs are configured with static addresses or when
there is connectivity default).
Following legacy packing conventions, `isArm` was defined just for
32-bit ARM instruction set. This is confusing to non packagers though,
because Aarch64 is an ARM instruction set.
The official ARM overview for ARMv8[1] is surprisingly not confusing,
given the overall state of affairs for ARM naming conventions, and
offers us a solution. It divides the nomenclature into three levels:
```
ISA: ARMv8 {-A, -R, -M}
/ \
Mode: Aarch32 Aarch64
| / \
Encoding: A64 A32 T32
```
At the top is the overall v8 instruction set archicture. Second are the
two modes, defined by bitwidth but differing in other semantics too, and
buttom are the encodings, (hopefully?) isomorphic if they encode the
same mode.
The 32 bit encodings are mostly backwards compatible with previous
non-Thumb and Thumb encodings, and if so we can pun the mode names to
instead mean "sets of compatable or isomorphic encodings", and then
voilà we have nice names for 32-bit and 64-bit arm instruction sets
which do not use the word ARM so as to not confused either laymen or
experienced ARM packages.
[1]: https://developer.arm.com/products/architecture/a-profile
This is required on the ThunderX CPUs on the Packet.net Type-2A
machines that have a GICv3. For some reason the default is to create a
GICv2 independent of the host hardware...
Unlike pathsFromGraph, on Nix 1.12, this function produces a
registration file containing correct NAR hash/size information.
https://hydra.nixos.org/build/62832723
OVMF{,CODE,VARS}.fd are now available in a dedicated fd output, greatly
reducing the closure in the common case where only those files are used (a
few MBs versus several hundred MBs for the full OVMF).
Note: it's unclear why `dontPatchELF` is now necessary for the build to
pass (on my end, at any rate) but it doesn't make much sense to run this
fixup anyway,
Note: my reading of xen's INSTALL suggests that --with-system-ovmf should
point directly to the OVMF binary. As such, the previous invocation was
incorrect (it pointed to the root of the OVMF tree). In any case, I have
only built xen with `--with-system-ovmf`, I have not tested it.
Fixes https://github.com/NixOS/nixpkgs/issues/25854
Closes https://github.com/NixOS/nixpkgs/pull/25855
QEMU can allow guests to access more than one host core at a time.
Previously, this had to be done via ad-hoc arguments:
virtualisation.qemu.options = ["-smp 12"];
Now you can simply specify:
virtualisation.cores = 12;
Overlayfs is quite a bit faster, e.g. with it the KDE 5 test takes ~7m
instead of ~30m on my laptop (which is still not great, since plain
9pfs is ~4m30s).
This works around:
machine: must succeed: nix-store -qR /run/current-system | grep nixos-
machine# error: changing ownership of path ‘/nix/store’: Invalid argument
Probably Nix shouldn't be anal about the ownership of the store unless
it's trying to build/write to the store.
http://hydra.nixos.org/build/45093872/nixlog/17/raw
(cherry picked from commit 57a0f140643cde409022e297ed05e05f8d34d778)
Previously we were using two or three (qemu_kvm, qemu_test, and
qemu_test with a different dbus when minimal.nix is included).
(cherry picked from commit 8bfa4ce82ea7d23a1d4c6073bcc044e6bf9c4dbe)
This option is defined in qemu-vm.nix, but that module is not always
imported.
http://hydra.nixos.org/build/44817443
(cherry picked from commit 03c55005dfd6fbcd5cf8e00128a3bb6336b3bc0f)
- most nixos user only require time synchronisation,
while ntpd implements a battery-included ntp server (1,215 LOCs of C-Code vs 64,302)
- timesyncd support ntp server per interface (if configured through dhcp for instance)
- timesyncd is already included in the systemd package, switching to it would
save a little disk space (1,5M)
The reason to patch QEMU is that with latest Nix, tests like "printing"
or "misc" fail because they expect the store paths to be owned by uid 0
and gid 0.
Starting with NixOS/nix@5e51ffb1c2, Nix
builds inside of a new user namespace. Unfortunately this also means
that bind-mounted store paths that are part of the derivation's inputs
are no longer owned by uid 0 and gid 0 but by uid 65534 and gid 65534.
This in turn causes things like sudo or cups to fail with errors about
insecure file permissions.
So in order to avoid that, let's make sure the VM always gets files
owned by uid 0 and gid 0 and does a no-op when doing a chmod on a store
path.
In addition, this adds a virtualisation.qemu.program option so that we
can make sure that we only use the patched version if we're *really*
running NixOS VM tests (that is, whenever we have imported
test-instrumentation.nix).
Tested against the "misc" and "printing" tests.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
