Commit Graph

297 Commits

Author SHA1 Message Date
Domen Kožar
8a3b70791c vmTools.diskImages: add ubuntu 16.04 2016-04-29 11:50:27 +01:00
Tuomas Tynkkynen
01854a850a treewide: Replace module_init_tools -> kmod
The former is deprecated and doesn't handle compressed kernel modules,
so all current usages of it are broken.
2016-04-22 10:40:57 +03:00
Vladimír Čunát
30f14243c3 Merge branch 'master' into closure-size
Comparison to master evaluations on Hydra:
  - 1255515 for nixos
  - 1255502 for nixpkgs
2016-04-10 11:17:52 +02:00
Vladimír Čunát
d1df28f8e5 Merge 'staging' into closure-size
This is mainly to get the update of bootstrap tools.
Otherwise there were mysterious segfaults:
https://github.com/NixOS/nixpkgs/pull/7701#issuecomment-203389817
2016-04-07 14:40:51 +02:00
Vladimír Čunát
aa670eb503 vmTools: update debian jessie 8.3 -> 8.4
Their in-place updates break download hashes...
2016-04-05 14:32:04 +02:00
Lluís Batlle i Rossell
635c99ce87 vm: allow overriding QEMU_OPTS / memSize for images.
It's nice to be able to create disk images with -smp 4
in qemu.
2016-04-01 10:32:59 +02:00
Vladimír Čunát
ab15a62c68 Merge branch 'master' into closure-size
Beware that stdenv doesn't build. It seems something more will be needed
than just resolution of merge conflicts.
2016-04-01 10:06:01 +02:00
Lluís Batlle i Rossell
ab93f8c137 Making vm's qemu cache=unsafe. Faster.
I don't think it's unsafe, if it's meant for nix expressions.
2016-03-31 09:27:25 +02:00
Lluís Batlle i Rossell
e21dd19168 Making vm's interactive shell handle the terminal well. 2016-03-31 09:27:14 +02:00
Nicolas B. Pierron
6313a5698a Replace references to all-packages.nix, by references to the top-level of nixpkgs repository. 2016-03-13 18:25:52 +00:00
Vladimír Čunát
09af15654f Merge master into closure-size
The kde-5 stuff still didn't merge well.
I hand-fixed what I saw, but there may be more problems.
2016-03-08 09:58:19 +01:00
Lluís Batlle i Rossell
202ebf794c vm/rpm/rpm-closure.pl: make it deterministic
Some recent perl version introduced "keys" to return the keys
in random order. As some of the packages are solved by "provides" and
based on the order, this randomness affects what packages get into the
closure.

This problem may be in other nix perl scripts.
2016-03-01 11:02:42 +01:00
Cole Mickens
718848d5aa azure: package qemu @ 2.2.0
This commit packages qemu-220. This package is qemu-2.2.0
and is only used with Azure.
2016-02-18 21:08:28 -08:00
Vladimír Čunát
d039c87984 Merge branch 'master' into closure-size 2016-02-14 08:33:51 +01:00
Tuomas Tynkkynen
3cc7fa0be9 treewide: Mass replace 'lzma}/bin' to refer the 'bin' output 2016-02-01 20:46:15 +02:00
Eelco Dolstra
9855cbcbf8 debian: 8.2 -> 8.3 2016-01-30 11:08:37 +01:00
Vladimír Čunát
f9f6f41bff Merge branch 'master' into closure-size
TODO: there was more significant refactoring of qtbase and plasma 5.5
on master, and I'm deferring pointing to correct outputs to later.
2015-12-31 09:53:02 +01:00
aszlig
a5bc11f9eb
nixos/vm-tests: Remove msize mount option
This seems to be the root cause of the random page allocation failures
and @wizeman did a very good job on not only finding the root problem
but also giving a detailed explanation of it in #10828.

Here is an excerpt:

  The problem here is that the kernel is trying to allocate a contiguous
  section of 2^7=128 pages, which is 512 KB. This is way too much:
  kernel pages tend to get fragmented over time and kernel developers
  often go to great lengths to try allocating at most only 1 contiguous
  page at a time whenever they can.

  From the error message, it looks like the culprit is unionfs, but this
  is misleading: unionfs is the name of the userspace process that was
  running when the system ran out of memory, but it wasn't unionfs who
  was allocating the memory: it was the kernel; specifically it was the
  v9fs_dir_readdir_dotl() function, which is the code for handling the
  readdir() function in the 9p filesystem (the filesystem that is used
  to share a directory structure between a qemu host and its VM).

  If you look at the code, here's what it's doing at the moment it tries
  to allocate memory:

    buflen = fid->clnt->msize - P9_IOHDRSZ;

    rdir = v9fs_alloc_rdir_buf(file, buflen);

  If you look into v9fs_alloc_rdir_buf(), you will see that it will try
  to allocate a contiguous buffer of memory (using kzalloc(), which is a
  wrapper around kmalloc()) of size buflen + 8 bytes or so.

  So in reality, this code actually allocates a buffer of size
  proportional to fid->clnt->msize. What is this msize? If you follow
  the definition of the structures, you will see that it's the
  negotiated buffer transfer size between 9p client and 9p server. On
  the client side, it can be controlled with the msize mount option.

  What this all means is that, the reason for running out of memory is
  that the code (which we can't easily change) tries to allocate a
  contiguous buffer of size more or less equal to "negotiated 9p
  protocol buffer size", which seems to be way too big (in our NixOS
  tests, at least).

After that initial finding, @lethalman tested the gnome3 gdm test
without setting the msize parameter at all and it seems to have resolved
the problem.

The reason why I'm committing this without testing against all of the
NixOS VM test is basically that I think we can only go better but not
worse than the current state.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-12-14 17:26:24 +01:00
Luca Bruno
a412927924 Merge remote-tracking branch 'origin/master' into closure-size 2015-11-25 21:37:30 +01:00
Eelco Dolstra
75e41b0210 Add Fedora 23 2015-11-25 16:18:15 +01:00
Eelco Dolstra
e4eee41ad0 Add Ubuntu 15.10 2015-11-25 15:40:08 +01:00
Vladimír Čunát
333d69a5f0 Merge staging into closure-size
The most complex problems were from dealing with switches reverted in
the meantime (gcc5, gmp6, ncurses6).
It's likely that darwin is (still) broken nontrivially.
2015-11-20 14:32:58 +01:00
aszlig
82d88b5bcb
vmTools.commonDebPackages: Add "mawk".
While debugging an issue with running NixOps tests, I found out that the
output from debClosureGenerator is not deterministic.

The reason behind this is the way how Provides and Replaces fields are
handled. I haven't yet found out what's the exact issue, but so far
packages "Provides" are more or less picked at random.

So, running the NixOps Hetzner tests we get either mawk, original-awk or
gawk altering on every invocation.

While for the test it isn't poisionous whether wi have mawk or gawk,
having original-awk certainly is, because live-build only works with
mawk or gawk.

The best solution would obviously be to make debClosureGenerator
deterministic, but in the case of "Provides: awk", we can safely pick
mawk by default, because the latter has a "Priority: required" in its
package description.

This also has the advantage that we can safely cherry-pick this to
release-15.09 because it's very unlikely that we'll break the
debClosureGenerator by adding a dependency to commonDebPackages.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-10-26 20:56:42 +01:00
Eelco Dolstra
7260c04266 debian: Update to 7.9, 8.2 2015-10-08 11:20:26 +02:00
Eelco Dolstra
106738b196 Give more memory for the disk image builder
http://hydra.nixos.org/build/26480662
2015-10-06 15:03:26 +02:00
Vladimír Čunát
5227fb1dd5 Merge commit staging+systemd into closure-size
Many non-conflict problems weren't (fully) resolved in this commit yet.
2015-10-03 13:33:37 +02:00
Eelco Dolstra
69829dd4ff Debian: Update to 8.1 2015-06-12 11:22:59 +02:00
Eelco Dolstra
99d2e80d97 In VM builds, connect stdin to /dev/null
This prevents programs from hanging while trying to read from stdin.
2015-06-04 20:08:56 +02:00
Benjamin Staffin
1db5bafc94 Typo fix! s/wheezy/jessie/ 2015-05-29 11:16:33 +02:00
Benjamin Staffin
c7ac0ed3eb Add Debian 8.0
Had to add file type logic in debClosureGenerator because Packages.bz2
is not available for Jessie - only Packages.xz is.
2015-05-29 10:49:20 +02:00
Eelco Dolstra
178c52e7ad Add Ubuntu 15.04 2015-05-22 13:21:02 +02:00
Eelco Dolstra
fef86de618 Make module loading work in VMs
Also, execute stage 2 as PID 1.
2015-05-13 17:36:58 +02:00
Eelco Dolstra
b97362989f Provide /run in VMs 2015-05-13 17:36:29 +02:00
Eelco Dolstra
c3f6c8c582 Mount /dev/shm in VMs 2015-05-07 15:31:36 +02:00
Vladimír Čunát
3b9ef2c71b fix "libc}/lib" and similar references
Done mostly without any verification.
I didn't bother with libc}/include, as the path is still correct.
2015-05-05 11:52:08 +02:00
Eelco Dolstra
3a31043b43 fillDiskWithRPMs: Pass --extract-over-symlinks to cpio
http://hydra.nixos.org/build/20003320
2015-02-27 14:00:11 +01:00
Eelco Dolstra
fa47d347a4 Fix conflict between fedora-release and generic-release
http://hydra.nixos.org/build/19594340
2015-02-12 17:55:29 +01:00
Eelco Dolstra
ff37dc6fb6 Remove tabs 2015-02-12 17:55:29 +01:00
Eelco Dolstra
28ea9050fa Remove ancient RedHat 9 / SuSE 9 images
These are 12 years old and I don't think anybody is using them.
2015-02-12 17:55:29 +01:00
Eelco Dolstra
da73688e3d fillDiskWithRPMs: Shut up some warnings 2015-02-12 17:55:28 +01:00
Eelco Dolstra
3786f08d25 Add Fedora 21 2015-02-10 11:32:43 +01:00
Eelco Dolstra
1b1e015e6e Update Debian to 7.8 2015-01-15 14:51:55 +01:00
Eelco Dolstra
7358b8dfda Fix a warning in initrd generation
${pkgs.busybox}/bin and ${pkgs.busybox}/sbin are now the same
directory, so don't copy both.
2015-01-12 17:52:30 +01:00
Eelco Dolstra
872f8c0431 Remove RHEL 7 RC
It no longer exists on the RedHat FTP site.
2015-01-12 15:13:21 +01:00
Eelco Dolstra
02df7cdaa0 Fix CentOS 6.5 URLs 2015-01-12 15:13:05 +01:00
Eelco Dolstra
b8e33a5dcb Ubuntu 14.10: Update hashes 2014-12-05 16:02:59 +01:00
Eelco Dolstra
99a2afd784 Update to Debian 7.7 2014-11-20 11:11:21 +01:00
Eelco Dolstra
e5513ac4dc Add Ubuntu 14.10 2014-11-20 11:11:08 +01:00
Eelco Dolstra
fda5c17a6f vmTools: Fix permissions on /tmp and /dev/* 2014-11-18 14:49:39 +01:00
Shea Levy
3fe41ca763 Debain 7.6-amd64 tarball was updated in place 2014-10-20 15:26:51 -04:00