jolheiser/nixpkgs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
ac34922912
nixpkgs/pkgs/build-support/docker/examples.nix

614 lines
17 KiB
Nix
Raw Normal View History

add examples file
2016-10-03 17:07:33 +00:00
# Examples of using the docker tools to build packages.
#
# This file defines several docker images. In order to use an image,
# build its derivation with `nix-build`, and then load the result with
# `docker load`. For example:
#
# $ nix-build '<nixpkgs>' -A dockerTools.examples.redis
# $ docker load < result
dockerTools.fakeNss: init This provides a /etc/passwd and /etc/group that contain root and nobody. Useful when packaging binaries that insist on using nss to look up username/groups (like nginx). The current nginx example used the `runAsRoot` parameter to setup /etc/group and /etc/passwd (which also doesn't exist in buildLayeredImage), so we can now just use fakeNss there and use buildLayeredImage.
2020-12-02 11:16:56 +00:00
{ pkgs, buildImage, buildLayeredImage, fakeNss, pullImage, shadowSetup, buildImageWithNixDb, pkgsCross }:
add examples file
2016-10-03 17:07:33 +00:00
rec {
# 1. basic example
bash = buildImage {
name = "bash";
dockerTools.examples: explicitly set image tag to fix docker-tools tests docker-tools tests load images without specifying any tag value. Docker then uses the image with tag "latest" which doesn't exist anymore since commit 39e678e24e38f1f374eaf5463b424ebdf75df9af.
2018-07-06 14:31:59 +00:00
tag = "latest";
add examples file
2016-10-03 17:07:33 +00:00
contents = pkgs.bashInteractive;
};
# 2. service example, layered on another image
redis = buildImage {
name = "redis";
tag = "latest";
# for example's sake, we can layer redis on top of bash or debian
fromImage = bash;
# fromImage = debian;
contents = pkgs.redis;
runAsRoot = ''
mkdir -p /data
'';
config = {
Cmd = [ "/bin/redis-server" ];
WorkingDir = "/data";
Volumes = {
"/data" = {};
};
};
};
# 3. another service example
nginx = let
nginxPort = "80";
nginxConf = pkgs.writeText "nginx.conf" ''
dockerTools.fakeNss: init This provides a /etc/passwd and /etc/group that contain root and nobody. Useful when packaging binaries that insist on using nss to look up username/groups (like nginx). The current nginx example used the `runAsRoot` parameter to setup /etc/group and /etc/passwd (which also doesn't exist in buildLayeredImage), so we can now just use fakeNss there and use buildLayeredImage.
2020-12-02 11:16:56 +00:00
user nobody nobody;
add examples file
2016-10-03 17:07:33 +00:00
daemon off;
error_log /dev/stdout info;
pid /dev/null;
events {}
http {
access_log /dev/stdout;
server {
listen ${nginxPort};
index index.html;
location / {
root ${nginxWebRoot};
}
}
}
'';
nginxWebRoot = pkgs.writeTextDir "index.html" ''
<html><body><h1>Hello from NGINX</h1></body></html>
'';
in
dockerTools.fakeNss: init This provides a /etc/passwd and /etc/group that contain root and nobody. Useful when packaging binaries that insist on using nss to look up username/groups (like nginx). The current nginx example used the `runAsRoot` parameter to setup /etc/group and /etc/passwd (which also doesn't exist in buildLayeredImage), so we can now just use fakeNss there and use buildLayeredImage.
2020-12-02 11:16:56 +00:00
buildLayeredImage {
add examples file
2016-10-03 17:07:33 +00:00
name = "nginx-container";
dockerTools.examples: explicitly set image tag to fix docker-tools tests docker-tools tests load images without specifying any tag value. Docker then uses the image with tag "latest" which doesn't exist anymore since commit 39e678e24e38f1f374eaf5463b424ebdf75df9af.
2018-07-06 14:31:59 +00:00
tag = "latest";
dockerTools.fakeNss: init This provides a /etc/passwd and /etc/group that contain root and nobody. Useful when packaging binaries that insist on using nss to look up username/groups (like nginx). The current nginx example used the `runAsRoot` parameter to setup /etc/group and /etc/passwd (which also doesn't exist in buildLayeredImage), so we can now just use fakeNss there and use buildLayeredImage.
2020-12-02 11:16:56 +00:00
contents = [
fakeNss
pkgs.nginx
];
add examples file
2016-10-03 17:07:33 +00:00
dockerTools.examples: fix the nginx image Nginx now requires directories `/var/log/nginx` and `/var/cache/nginx` to be existing. The objective is to fix the test `nixosTests.docker-tools`.
2020-05-13 08:24:36 +00:00
extraCommands = ''
# nginx still tries to read this directory even if error_log
# directive is specifying another file :/
mkdir -p var/log/nginx
mkdir -p var/cache/nginx
'';
add examples file
2016-10-03 17:07:33 +00:00
config = {
Cmd = [ "nginx" "-c" nginxConf ];
ExposedPorts = {
"${nginxPort}/tcp" = {};
};
};
};
# 4. example of pulling an image. could be used as a base for other images
dockerTools.buildImageWithNixDb: populate the Nix Db of the image Nix store Currently, the contents closure is copied to the layer but there is no nix database initialization. If pkgs.nix is added in the contents, nix-store doesn't work because there is no nix database. From the contents of the layer, this commit generates and loads the database in the nix store of the container. This only works if there is no parent layer that already have a nix store (to support several nix layers, we would have to merge nix databases of parent layers). We also add an example to play with the nix store inside the container. Note it seems `more` is a missing dependency of the nix package!
2017-08-25 09:47:28 +00:00
nixFromDockerHub = pullImage {
docker tools: fix pull image function
2017-05-26 16:06:24 +00:00
imageName = "nixos/nix";
nixos/test/docker-tools: fix Nix image digest
2019-03-06 08:18:35 +00:00
imageDigest = "sha256:85299d86263a3059cf19f419f9d286cc9f06d3c13146a8ebbb21b3437f598357";
nixosTests.docker-tools: Fix nixFromDockerHub example sha For https://github.com/NixOS/nixpkgs/pull/125211 I tried to test the fetcher with nix-build -A dockerTools.examples.nixFromDockerHub --option substitute false But it failed. I haven't figured out the cause, but the outputs match, so it's probably the hashing method (flat/recursive) that changed at some point. (The names did match.)
2021-06-01 13:10:10 +00:00
sha256 = "19fw0n3wmddahzr20mhdqv6jkjn1kanh6n2mrr08ai53dr8ph5n7";
nixos/test/docker-tools: fix Nix image digest
2019-03-06 08:18:35 +00:00
finalImageTag = "2.2.1";
dockerTools: add finalImageName parameter for pullImage
2019-03-26 09:35:21 +00:00
finalImageName = "nix";
docker tools: fix pull image function
2017-05-26 16:06:24 +00:00
};
dockerTools: Test pullImage fetcher whenever its implementation changes
2021-09-29 12:37:31 +00:00
# Same example, but re-fetches every time the fetcher implementation changes.
# NOTE: Only use this for testing, or you'd be wasting a lot of time, network and space.
testNixFromDockerHub = pkgs.invalidateFetcherByDrvHash pullImage {
imageName = "nixos/nix";
imageDigest = "sha256:85299d86263a3059cf19f419f9d286cc9f06d3c13146a8ebbb21b3437f598357";
sha256 = "19fw0n3wmddahzr20mhdqv6jkjn1kanh6n2mrr08ai53dr8ph5n7";
finalImageTag = "2.2.1";
finalImageName = "nix";
};
add examples file
2016-10-03 17:07:33 +00:00
# 5. example of multiple contents, emacs and vi happily coexisting
editors = buildImage {
name = "editors";
contents = [
pkgs.coreutils
pkgs.bash
pkgs.emacs
pkgs.vim
pkgs.nano
];
};
dockerTools.buildImageWithNixDb: populate the Nix Db of the image Nix store Currently, the contents closure is copied to the layer but there is no nix database initialization. If pkgs.nix is added in the contents, nix-store doesn't work because there is no nix database. From the contents of the layer, this commit generates and loads the database in the nix store of the container. This only works if there is no parent layer that already have a nix store (to support several nix layers, we would have to merge nix databases of parent layers). We also add an example to play with the nix store inside the container. Note it seems `more` is a missing dependency of the nix package!
2017-08-25 09:47:28 +00:00
dockerTools.examples: correct a typo in comments This commit is for correcting a typo in comments.
2018-01-01 08:13:40 +00:00
# 6. nix example to play with the container nix store
dockerTools.buildImageWithNixDb: populate the Nix Db of the image Nix store Currently, the contents closure is copied to the layer but there is no nix database initialization. If pkgs.nix is added in the contents, nix-store doesn't work because there is no nix database. From the contents of the layer, this commit generates and loads the database in the nix store of the container. This only works if there is no parent layer that already have a nix store (to support several nix layers, we would have to merge nix databases of parent layers). We also add an example to play with the nix store inside the container. Note it seems `more` is a missing dependency of the nix package!
2017-08-25 09:47:28 +00:00
# docker run -it --rm nix nix-store -qR $(nix-build '<nixpkgs>' -A nix)
nix = buildImageWithNixDb {
name = "nix";
dockerTools.examples: explicitly set image tag to fix docker-tools tests docker-tools tests load images without specifying any tag value. Docker then uses the image with tag "latest" which doesn't exist anymore since commit 39e678e24e38f1f374eaf5463b424ebdf75df9af.
2018-07-06 14:31:59 +00:00
tag = "latest";
dockerTools.buildImageWithNixDb: populate the Nix Db of the image Nix store Currently, the contents closure is copied to the layer but there is no nix database initialization. If pkgs.nix is added in the contents, nix-store doesn't work because there is no nix database. From the contents of the layer, this commit generates and loads the database in the nix store of the container. This only works if there is no parent layer that already have a nix store (to support several nix layers, we would have to merge nix databases of parent layers). We also add an example to play with the nix store inside the container. Note it seems `more` is a missing dependency of the nix package!
2017-08-25 09:47:28 +00:00
contents = [
dockerTools.examples.nix: set NIX_PAGER=cat environment variable
2017-09-25 07:39:15 +00:00
# nix-store uses cat program to display results as specified by
# the image env variable NIX_PAGER.
pkgs.coreutils
dockerTools.buildImageWithNixDb: populate the Nix Db of the image Nix store Currently, the contents closure is copied to the layer but there is no nix database initialization. If pkgs.nix is added in the contents, nix-store doesn't work because there is no nix database. From the contents of the layer, this commit generates and loads the database in the nix store of the container. This only works if there is no parent layer that already have a nix store (to support several nix layers, we would have to merge nix databases of parent layers). We also add an example to play with the nix store inside the container. Note it seems `more` is a missing dependency of the nix package!
2017-08-25 09:47:28 +00:00
pkgs.nix
dockerTools: test that tar keeps nix binary symlinks intact
2020-06-18 15:29:21 +00:00
pkgs.bash
dockerTools.buildImageWithNixDb: populate the Nix Db of the image Nix store Currently, the contents closure is copied to the layer but there is no nix database initialization. If pkgs.nix is added in the contents, nix-store doesn't work because there is no nix database. From the contents of the layer, this commit generates and loads the database in the nix store of the container. This only works if there is no parent layer that already have a nix store (to support several nix layers, we would have to merge nix databases of parent layers). We also add an example to play with the nix store inside the container. Note it seems `more` is a missing dependency of the nix package!
2017-08-25 09:47:28 +00:00
];
dockerTools.examples.nix: set NIX_PAGER=cat environment variable
2017-09-25 07:39:15 +00:00
config = {
dockerTools.examples.nix: set USER in the container config A USER is required by Nix. See https://github.com/NixOS/nix/blob/9348f9291e5d9e4ba3c4347ea1b235640f54fd79/src/libutil/util.cc#L478.
2019-12-05 08:45:51 +00:00
Env = [
"NIX_PAGER=cat"
# A user is required by nix
# https://github.com/NixOS/nix/blob/9348f9291e5d9e4ba3c4347ea1b235640f54fd79/src/libutil/util.cc#L478
"USER=nobody"
];
dockerTools.examples.nix: set NIX_PAGER=cat environment variable
2017-09-25 07:39:15 +00:00
};
dockerTools.buildImageWithNixDb: populate the Nix Db of the image Nix store Currently, the contents closure is copied to the layer but there is no nix database initialization. If pkgs.nix is added in the contents, nix-store doesn't work because there is no nix database. From the contents of the layer, this commit generates and loads the database in the nix store of the container. This only works if there is no parent layer that already have a nix store (to support several nix layers, we would have to merge nix databases of parent layers). We also add an example to play with the nix store inside the container. Note it seems `more` is a missing dependency of the nix package!
2017-08-25 09:47:28 +00:00
};
dockerTools: add an onTopOfPulledImage example This allows to test if a pulled image can be updated by using our Docker tools.
2018-02-01 15:13:05 +00:00
# 7. example of adding something on top of an image pull by our
# dockerTools chain.
onTopOfPulledImage = buildImage {
name = "onTopOfPulledImage";
dockerTools.examples: explicitly set image tag to fix docker-tools tests docker-tools tests load images without specifying any tag value. Docker then uses the image with tag "latest" which doesn't exist anymore since commit 39e678e24e38f1f374eaf5463b424ebdf75df9af.
2018-07-06 14:31:59 +00:00
tag = "latest";
dockerTools: add an onTopOfPulledImage example This allows to test if a pulled image can be updated by using our Docker tools.
2018-02-01 15:13:05 +00:00
fromImage = nixFromDockerHub;
contents = [ pkgs.hello ];
};
tests/docker-tools: Adds regression test for #34779
2018-05-23 00:25:04 +00:00
# 8. regression test for erroneous use of eval and string expansion.
# See issue #34779 and PR #40947 for details.
runAsRootExtraCommands = pkgs.dockerTools.buildImage {
name = "runAsRootExtraCommands";
dockerTools.examples: explicitly set image tag to fix docker-tools tests docker-tools tests load images without specifying any tag value. Docker then uses the image with tag "latest" which doesn't exist anymore since commit 39e678e24e38f1f374eaf5463b424ebdf75df9af.
2018-07-06 14:31:59 +00:00
tag = "latest";
tests/docker-tools: Adds regression test for #34779
2018-05-23 00:25:04 +00:00
contents = [ pkgs.coreutils ];
# The parens here are to create problematic bash to embed and eval. In case
# this is *embedded* into the script (with nix expansion) the initial quotes
# will close the string and the following parens are unexpected
runAsRoot = ''echo "(runAsRoot)" > runAsRoot'';
extraCommands = ''echo "(extraCommand)" > extraCommands'';
};
dockerTools.buildImage: test that created=now makes an unstable date
2018-09-20 17:02:22 +00:00
# 9. Ensure that setting created to now results in a date which
# isn't the epoch + 1
unstableDate = pkgs.dockerTools.buildImage {
name = "unstable-date";
tag = "latest";
contents = [ pkgs.coreutils ];
created = "now";
};
dockerTools: test buildLayeredImage
2018-09-27 18:16:23 +00:00
# 10. Create a layered image
layered-image = pkgs.dockerTools.buildLayeredImage {
name = "layered-image";
tag = "latest";
dockerTools: allow to pass extraCommands, uid and gid to buildLayeredImage
2018-12-25 22:04:16 +00:00
extraCommands = ''echo "(extraCommand)" > extraCommands'';
dockerTools: test buildLayeredImage
2018-09-27 18:16:23 +00:00
config.Cmd = [ "${pkgs.hello}/bin/hello" ];
dockerTools.buildImage: support using a layered image in fromImage Docker images used to be, essentially, a linked list of layers. Each layer would have a tarball and a json document pointing to its parent, and the image pointed to the top layer: imageA ----> layerA | v layerB | v layerC The current image spec changed this format to where the Image defined the order and set of layers: imageA ---> layerA |--> layerB `--> layerC For backwards compatibility, docker produces images which follow both specs: layers point to parents, and images also point to the entire list: imageA ---> layerA | | | v |--> layerB | | | v `--> layerC This is nice for tooling which supported the older version and never updated to support the newer format. Our `buildImage` code only supported the old version, so in order for `buildImage` to properly generate an image based on another image with `fromImage`, the parent image's layers must fully support the old mechanism. This is not a problem in general, but is a problem with `buildLayeredImage`. `buildLayeredImage` creates images with newer image spec, because individual store paths don't have a guaranteed parent layer. Including a specific parent ID in the layer's json makes the output less likely to cache hit when published or pulled. This means until now, `buildLayeredImage` could not be the input to `buildImage`. The changes in this PR change `buildImage` to only use the layer's manifest when locating parent IDs. This does break buildImage on extremely old Docker images, though I do wonder how many of these exist. This work has been sponsored by Target.
2018-12-04 17:18:06 +00:00
contents = [ pkgs.hello pkgs.bash pkgs.coreutils ];
};
# 11. Create an image on top of a layered image
layered-on-top = pkgs.dockerTools.buildImage {
name = "layered-on-top";
tag = "latest";
fromImage = layered-image;
extraCommands = ''
mkdir ./example-output
chmod 777 ./example-output
'';
config = {
Env = [ "PATH=${pkgs.coreutils}/bin/" ];
WorkingDir = "/example-output";
Cmd = [
"${pkgs.bash}/bin/bash" "-c" "echo hello > foo; cat foo"
];
};
dockerTools: test buildLayeredImage
2018-09-27 18:16:23 +00:00
};
dockerTools.examples.runAsRootParentImage: init Example of running something as root on top of a parent image. This is a regression test related to the PR #52109.
2018-12-27 10:10:53 +00:00
dockerTools.buildLayeredImage: support fromImage It is now possible to pass a `fromImage` to `buildLayeredImage` and `streamLayeredImage`, similar to what `buildImage` currently supports. This will prepend the layers of the given base image to the resulting image, while ensuring that at most `maxLayers` are used. It will also ensure that environment variables from the base image are propagated to the final image.
2021-03-08 20:36:13 +00:00
# 12 Create a layered image on top of a layered image
layered-on-top-layered = pkgs.dockerTools.buildLayeredImage {
name = "layered-on-top-layered";
tag = "latest";
fromImage = layered-image;
extraCommands = ''
mkdir ./example-output
chmod 777 ./example-output
'';
config = {
Env = [ "PATH=${pkgs.coreutils}/bin/" ];
WorkingDir = "/example-output";
Cmd = [
"${pkgs.bash}/bin/bash" "-c" "echo hello > foo; cat foo"
];
};
};
# 13. example of running something as root on top of a parent image
dockerTools.examples.runAsRootParentImage: init Example of running something as root on top of a parent image. This is a regression test related to the PR #52109.
2018-12-27 10:10:53 +00:00
# Regression test related to PR #52109
runAsRootParentImage = buildImage {
name = "runAsRootParentImage";
tag = "latest";
runAsRoot = "touch /example-file";
fromImage = bash;
};
nixos/tests/docker-tools: verify order of layers in stacked images
2019-04-29 22:46:00 +00:00
dockerTools.buildLayeredImage: support fromImage It is now possible to pass a `fromImage` to `buildLayeredImage` and `streamLayeredImage`, similar to what `buildImage` currently supports. This will prepend the layers of the given base image to the resulting image, while ensuring that at most `maxLayers` are used. It will also ensure that environment variables from the base image are propagated to the final image.
2021-03-08 20:36:13 +00:00
# 14. example of 3 layers images This image is used to verify the
nixos/tests/docker-tools: verify order of layers in stacked images
2019-04-29 22:46:00 +00:00
# order of layers is correct.
# It allows to validate
# - the layer of parent are below
# - the order of parent layer is preserved at image build time
# (this is why there are 3 images)
layersOrder = let
l1 = pkgs.dockerTools.buildImage {
name = "l1";
tag = "latest";
extraCommands = ''
mkdir -p tmp
echo layer1 > tmp/layer1
echo layer1 > tmp/layer2
echo layer1 > tmp/layer3
'';
};
l2 = pkgs.dockerTools.buildImage {
name = "l2";
fromImage = l1;
tag = "latest";
extraCommands = ''
mkdir -p tmp
echo layer2 > tmp/layer2
echo layer2 > tmp/layer3
'';
};
in pkgs.dockerTools.buildImage {
name = "l3";
fromImage = l2;
tag = "latest";
contents = [ pkgs.coreutils ];
extraCommands = ''
mkdir -p tmp
echo layer3 > tmp/layer3
'';
};
dockerTools.buildLayeredImage: support fromImage It is now possible to pass a `fromImage` to `buildLayeredImage` and `streamLayeredImage`, similar to what `buildImage` currently supports. This will prepend the layers of the given base image to the resulting image, while ensuring that at most `maxLayers` are used. It will also ensure that environment variables from the base image are propagated to the final image.
2021-03-08 20:36:13 +00:00
# 15. Environment variable inheritance.
dockerTools.buildImage: Preserve environment variables from the parent image
2020-05-08 09:49:16 +00:00
# Child image should inherit parents environment variables,
# optionally overriding them.
dockerTools.buildLayeredImage: support fromImage It is now possible to pass a `fromImage` to `buildLayeredImage` and `streamLayeredImage`, similar to what `buildImage` currently supports. This will prepend the layers of the given base image to the resulting image, while ensuring that at most `maxLayers` are used. It will also ensure that environment variables from the base image are propagated to the final image.
2021-03-08 20:36:13 +00:00
environmentVariablesParent = pkgs.dockerTools.buildImage {
name = "parent";
tag = "latest";
config = {
Env = [
"FROM_PARENT=true"
"LAST_LAYER=parent"
];
dockerTools.buildImage: Preserve environment variables from the parent image
2020-05-08 09:49:16 +00:00
};
dockerTools.buildLayeredImage: support fromImage It is now possible to pass a `fromImage` to `buildLayeredImage` and `streamLayeredImage`, similar to what `buildImage` currently supports. This will prepend the layers of the given base image to the resulting image, while ensuring that at most `maxLayers` are used. It will also ensure that environment variables from the base image are propagated to the final image.
2021-03-08 20:36:13 +00:00
};
environmentVariables = pkgs.dockerTools.buildImage {
name = "child";
fromImage = environmentVariablesParent;
tag = "latest";
contents = [ pkgs.coreutils ];
config = {
Env = [
"FROM_CHILD=true"
"LAST_LAYER=child"
];
};
};
environmentVariablesLayered = pkgs.dockerTools.buildLayeredImage {
dockerTools.buildImage: Preserve environment variables from the parent image
2020-05-08 09:49:16 +00:00
name = "child";
dockerTools.buildLayeredImage: support fromImage It is now possible to pass a `fromImage` to `buildLayeredImage` and `streamLayeredImage`, similar to what `buildImage` currently supports. This will prepend the layers of the given base image to the resulting image, while ensuring that at most `maxLayers` are used. It will also ensure that environment variables from the base image are propagated to the final image.
2021-03-08 20:36:13 +00:00
fromImage = environmentVariablesParent;
dockerTools.buildImage: Preserve environment variables from the parent image
2020-05-08 09:49:16 +00:00
tag = "latest";
contents = [ pkgs.coreutils ];
config = {
Env = [
"FROM_CHILD=true"
"LAST_LAYER=child"
];
};
};
dockerTools.buildLayeredImage: support fromImage It is now possible to pass a `fromImage` to `buildLayeredImage` and `streamLayeredImage`, similar to what `buildImage` currently supports. This will prepend the layers of the given base image to the resulting image, while ensuring that at most `maxLayers` are used. It will also ensure that environment variables from the base image are propagated to the final image.
2021-03-08 20:36:13 +00:00
# 16. Create another layered image, for comparing layers with image 10.
nixos/tests/docker-tools: check layer sharing with buildLayeredImage Adapted from grahamc's blog post on layered Docker images in Nix: https://grahamc.com/blog/nix-and-layered-docker-images
2019-06-06 08:39:50 +00:00
another-layered-image = pkgs.dockerTools.buildLayeredImage {
name = "another-layered-image";
tag = "latest";
config.Cmd = [ "${pkgs.hello}/bin/hello" ];
};
dockerTools.buildLayeredImage: support fromImage It is now possible to pass a `fromImage` to `buildLayeredImage` and `streamLayeredImage`, similar to what `buildImage` currently supports. This will prepend the layers of the given base image to the resulting image, while ensuring that at most `maxLayers` are used. It will also ensure that environment variables from the base image are propagated to the final image.
2021-03-08 20:36:13 +00:00
# 17. Create a layered image with only 2 layers
dockerTools.buildLayeredImage: fix image with only 2 layers A test is also added to ensure an image with 2 layers can be built.
2020-01-30 09:42:13 +00:00
two-layered-image = pkgs.dockerTools.buildLayeredImage {
name = "two-layered-image";
tag = "latest";
config.Cmd = [ "${pkgs.hello}/bin/hello" ];
contents = [ pkgs.bash pkgs.hello ];
maxLayers = 2;
};
dockerTools.buildLayeredImage: store all paths passed in final layer Fixes #78744 My previous change broke when there are more packages than the maximum number of layers. I had assumed that the `store-path-to-layer.sh` was only ever passed a single store path, but that is not the case if there are multiple packages going into the final layer. To fix this, we loop through the paths going into the final layer, appending them to the tar file and making sure they end up at the right path.
2020-01-29 21:56:05 +00:00
dockerTools.buildLayeredImage: support fromImage It is now possible to pass a `fromImage` to `buildLayeredImage` and `streamLayeredImage`, similar to what `buildImage` currently supports. This will prepend the layers of the given base image to the resulting image, while ensuring that at most `maxLayers` are used. It will also ensure that environment variables from the base image are propagated to the final image.
2021-03-08 20:36:13 +00:00
# 18. Create a layered image with more packages than max layers.
nixosTests.docker-tools: add bulk-layer test A regression test for https://github.com/NixOS/nixpkgs/issues/78744.
2020-01-30 21:35:16 +00:00
# coreutils and hello are part of the same layer
bulk-layer = pkgs.dockerTools.buildLayeredImage {
name = "bulk-layer";
tag = "latest";
contents = with pkgs; [
coreutils hello
];
maxLayers = 2;
};
dockerTools.buildLayeredImage: support fromImage It is now possible to pass a `fromImage` to `buildLayeredImage` and `streamLayeredImage`, similar to what `buildImage` currently supports. This will prepend the layers of the given base image to the resulting image, while ensuring that at most `maxLayers` are used. It will also ensure that environment variables from the base image are propagated to the final image.
2021-03-08 20:36:13 +00:00
# 19. Create a layered image with a base image and more packages than max
# layers. coreutils and hello are part of the same layer
layered-bulk-layer = pkgs.dockerTools.buildLayeredImage {
name = "layered-bulk-layer";
tag = "latest";
fromImage = two-layered-image;
contents = with pkgs; [
coreutils hello
];
maxLayers = 4;
};
# 20. Create a "layered" image without nix store layers. This is not
buildLayeredImage: Allow empty store, no paths to add This is useful when buildLayeredImage is called in a generic way that should allow simple (base) images to be built, which may not reference any store paths.
2020-02-23 23:41:55 +00:00
# recommended, but can be useful for base images in rare cases.
no-store-paths = pkgs.dockerTools.buildLayeredImage {
name = "no-store-paths";
tag = "latest";
extraCommands = ''
# This removes sharing of busybox and is not recommended. We do this
# to make the example suitable as a test case with working binaries.
cp -r ${pkgs.pkgsStatic.busybox}/* .
dockerTools: Add store dependencies of the customization layer
2021-10-01 11:47:01 +00:00
# This is a "build" dependency that will not appear in the image
${pkgs.hello}/bin/hello
buildLayeredImage: Allow empty store, no paths to add This is useful when buildLayeredImage is called in a generic way that should allow simple (base) images to be built, which may not reference any store paths.
2020-02-23 23:41:55 +00:00
'';
};
dockerTools: test that tar keeps nix binary symlinks intact
2020-06-18 15:29:21 +00:00
nixLayered = pkgs.dockerTools.buildLayeredImageWithNixDb {
name = "nix-layered";
tag = "latest";
contents = [
# nix-store uses cat program to display results as specified by
# the image env variable NIX_PAGER.
pkgs.coreutils
pkgs.nix
pkgs.bash
];
config = {
Env = [
"NIX_PAGER=cat"
# A user is required by nix
# https://github.com/NixOS/nix/blob/9348f9291e5d9e4ba3c4347ea1b235640f54fd79/src/libutil/util.cc#L478
"USER=nobody"
];
};
};
dockerTools.buildLayeredImage: support fromImage It is now possible to pass a `fromImage` to `buildLayeredImage` and `streamLayeredImage`, similar to what `buildImage` currently supports. This will prepend the layers of the given base image to the resulting image, while ensuring that at most `maxLayers` are used. It will also ensure that environment variables from the base image are propagated to the final image.
2021-03-08 20:36:13 +00:00
# 21. Support files in the store on buildLayeredImage
dockerTools: Support files directly under /nix/store Also makes sure that the files inside a layer added in a sorted order to make the results more deterministic.
2020-07-04 10:00:57 +00:00
# See: https://github.com/NixOS/nixpkgs/pull/91084#issuecomment-653496223
dockerTools: Verify nix-store contents on buildLayeredImage test
2020-07-06 04:59:58 +00:00
filesInStore = pkgs.dockerTools.buildLayeredImageWithNixDb {
dockerTools: Support files directly under /nix/store Also makes sure that the files inside a layer added in a sorted order to make the results more deterministic.
2020-07-04 10:00:57 +00:00
name = "file-in-store";
tag = "latest";
dockerTools: Verify nix-store contents on buildLayeredImage test
2020-07-06 04:59:58 +00:00
contents = [
pkgs.coreutils
pkgs.nix
(pkgs.writeScriptBin "myscript" ''
#!${pkgs.runtimeShell}
cat ${pkgs.writeText "somefile" "some data"}
'')
dockerTools: Support files directly under /nix/store Also makes sure that the files inside a layer added in a sorted order to make the results more deterministic.
2020-07-04 10:00:57 +00:00
];
dockerTools: Verify nix-store contents on buildLayeredImage test
2020-07-06 04:59:58 +00:00
config = {
Cmd = [ "myscript" ];
# For some reason 'nix-store --verify' requires this environment variable
Env = [ "USER=root" ];
};
dockerTools: Support files directly under /nix/store Also makes sure that the files inside a layer added in a sorted order to make the results more deterministic.
2020-07-04 10:00:57 +00:00
};
dockerTools.buildLayeredImage: fix created=now
2020-07-09 07:34:18 +00:00
dockerTools.buildLayeredImage: support fromImage It is now possible to pass a `fromImage` to `buildLayeredImage` and `streamLayeredImage`, similar to what `buildImage` currently supports. This will prepend the layers of the given base image to the resulting image, while ensuring that at most `maxLayers` are used. It will also ensure that environment variables from the base image are propagated to the final image.
2021-03-08 20:36:13 +00:00
# 22. Ensure that setting created to now results in a date which
dockerTools.buildLayeredImage: fix created=now
2020-07-09 07:34:18 +00:00
# isn't the epoch + 1 for layered images.
unstableDateLayered = pkgs.dockerTools.buildLayeredImage {
name = "unstable-date-layered";
tag = "latest";
contents = [ pkgs.coreutils ];
created = "now";
};
nixos/tests/docker-tools: add test for pre-runAsRoot layer unpack order
2021-12-18 01:26:53 +00:00
# 23. Ensure that layers are unpacked in the correct order before the
# runAsRoot script is executed.
layersUnpackOrder =
let
layerOnTopOf = parent: layerName:
pkgs.dockerTools.buildImage {
name = "layers-unpack-order-${layerName}";
tag = "latest";
fromImage = parent;
contents = [ pkgs.coreutils ];
runAsRoot = ''
#!${pkgs.runtimeShell}
echo -n "${layerName}" >> /layer-order
'';
};
# When executing the runAsRoot script when building layer C, if layer B is
# not unpacked on top of layer A, the contents of /layer-order will not be
# "ABC".
layerA = layerOnTopOf null "a";
layerB = layerOnTopOf layerA "b";
layerC = layerOnTopOf layerB "c";
in layerC;
dockerTools: Always set imageTag attribute The image tag can be specified or generated from the output hash. Previously, a generated tag could be recovered from the evaluated image with some string operations. However, with the introduction of streamLayeredImage, it's not feasible to compute the generated tag yourself. With this change, the imageTag attribute is set unconditionally, for the buildImage, buildLayeredImage, streamLayeredImage functions.
2020-07-11 13:51:58 +00:00
# buildImage without explicit tag
bashNoTag = pkgs.dockerTools.buildImage {
name = "bash-no-tag";
contents = pkgs.bashInteractive;
};
# buildLayeredImage without explicit tag
bashNoTagLayered = pkgs.dockerTools.buildLayeredImage {
name = "bash-no-tag-layered";
contents = pkgs.bashInteractive;
};
# buildImage without explicit tag
bashNoTagStreamLayered = pkgs.dockerTools.streamLayeredImage {
name = "bash-no-tag-stream-layered";
contents = pkgs.bashInteractive;
};
nixos/tests/dockerTools: add test for running non-root containers with buildLayeredImage Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2020-07-30 15:18:41 +00:00
# buildLayeredImage with non-root user
bashLayeredWithUser =
let
nonRootShadowSetup = { user, uid, gid ? uid }: with pkgs; [
(
writeTextDir "etc/shadow" ''
root:!x:::::::
${user}:!:::::::
''
)
(
writeTextDir "etc/passwd" ''
root:x:0:0::/root:${runtimeShell}
${user}:x:${toString uid}:${toString gid}::/home/${user}:
''
)
(
writeTextDir "etc/group" ''
root:x:0:
${user}:x:${toString gid}:
''
)
(
writeTextDir "etc/gshadow" ''
root:x::
${user}:x::
''
)
];
in
pkgs.dockerTools.buildLayeredImage {
name = "bash-layered-with-user";
tag = "latest";
dockerTools.streamLayeredImage: Store the customisation layer as a tarball This fixes as issue described here[1], where permissions set by 'extraCommands' were ignored by Nix. [1] https://github.com/NixOS/nixpkgs/pull/91084#issuecomment-669834938
2020-08-14 09:06:00 +00:00
contents = [ pkgs.bash pkgs.coreutils ] ++ nonRootShadowSetup { uid = 999; user = "somebody"; };
nixos/tests/dockerTools: add test for running non-root containers with buildLayeredImage Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2020-07-30 15:18:41 +00:00
};
dockerTools: Add cross example
2020-11-19 16:31:38 +00:00
# basic example, with cross compilation
dockerTools: Always cross compile for another arch in the cross example The example fails to build on aarch64, so lets cross build for gnu64.
2020-11-20 10:57:56 +00:00
cross = let
# Cross compile for x86_64 if on aarch64
crossPkgs =
if pkgs.system == "aarch64-linux" then pkgsCross.gnu64
else pkgsCross.aarch64-multiplatform;
in crossPkgs.dockerTools.buildImage {
dockerTools: Add cross example
2020-11-19 16:31:38 +00:00
name = "hello-cross";
tag = "latest";
dockerTools: Always cross compile for another arch in the cross example The example fails to build on aarch64, so lets cross build for gnu64.
2020-11-20 10:57:56 +00:00
contents = crossPkgs.hello;
dockerTools: Add cross example
2020-11-19 16:31:38 +00:00
};
dockerTools: Test buildLayeredImage with symlinks This exercises layer creation in face of store path symlinks, ensuring they are not dereferenced, which can lead to broken layer tarballs
2021-01-04 20:33:32 +00:00
# layered image where a store path is itself a symlink
layeredStoreSymlink =
let
target = pkgs.writeTextDir "dir/target" "Content doesn't matter.";
treewide: runCommandNoCC -> runCommand This has been synonymous for ~5y.
2021-08-15 15:12:23 +00:00
symlink = pkgs.runCommand "symlink" {} "ln -s ${target} $out";
dockerTools: Test buildLayeredImage with symlinks This exercises layer creation in face of store path symlinks, ensuring they are not dereferenced, which can lead to broken layer tarballs
2021-01-04 20:33:32 +00:00
in
pkgs.dockerTools.buildLayeredImage {
name = "layeredstoresymlink";
tag = "latest";
contents = [ pkgs.bash symlink ];
} // { passthru = { inherit symlink; }; };
dockerTools.buildLayeredImage: image names with registry/ prefix When using `buildLayeredImage`, it is not possible to specify an image name of the form `<registry>/my/image`, although it is a valid name. This is due to derivations under `buildLayeredImage` using that image name as their derivation name, but slashes are not permitted in that context. A while ago, #13099 fixed that exact same problem in `buildImage` by using `baseNameOf name` in derivation names instead of `name`. This change does the same thing for `buildLayeredImage`.
2021-03-09 18:32:54 +00:00
# image with registry/ prefix
prefixedImage = pkgs.dockerTools.buildImage {
name = "registry-1.docker.io/image";
tag = "latest";
config.Cmd = [ "${pkgs.hello}/bin/hello" ];
};
# layered image with registry/ prefix
prefixedLayeredImage = pkgs.dockerTools.buildLayeredImage {
name = "registry-1.docker.io/layered-image";
tag = "latest";
config.Cmd = [ "${pkgs.hello}/bin/hello" ];
};
dockerTools.streamLayeredImage: Add test for fakeRootCommands
2021-04-07 09:11:02 +00:00
# layered image with files owned by a user other than root
layeredImageWithFakeRootCommands = pkgs.dockerTools.buildLayeredImage {
name = "layered-image-with-fake-root-commands";
tag = "latest";
contents = [
pkgs.pkgsStatic.busybox
];
fakeRootCommands = ''
mkdir -p ./home/jane
chown 1000 ./home/jane
nixosTests.docker-tools: Use unique binary in test case
2021-12-02 22:26:05 +00:00
ln -s ${pkgs.hello.overrideAttrs (o: {
# A unique `hello` to make sure that it isn't included via another mechanism by accident.
configureFlags = o.configureFlags or "" + " --program-prefix=layeredImageWithFakeRootCommands-";
doCheck = false;
})} ./hello
dockerTools.streamLayeredImage: Add test for fakeRootCommands
2021-04-07 09:11:02 +00:00
'';
};
dockerTools: add merged example images
2021-03-30 09:37:17 +00:00
# tarball consisting of both bash and redis images
mergedBashAndRedis = pkgs.dockerTools.mergeImages [
bash
redis
];
# tarball consisting of bash (without tag) and redis images
mergedBashNoTagAndRedis = pkgs.dockerTools.mergeImages [
bashNoTag
redis
];
# tarball consisting of bash and layered image with different owner of the
# /home/jane directory
mergedBashFakeRoot = pkgs.dockerTools.mergeImages [
bash
layeredImageWithFakeRootCommands
];
dockerTools: Allow omitting all store paths Adds includeStorePaths, allowing the omission of the store paths. You generally want to leave it on, but tooling may disable this to insert the store paths more efficiently via other means, such as bind mounting the host store.
2021-05-25 13:04:45 +00:00
helloOnRoot = pkgs.dockerTools.streamLayeredImage {
name = "hello";
tag = "latest";
contents = [
(pkgs.buildEnv {
name = "hello-root";
paths = [ pkgs.hello ];
})
];
config.Cmd = [ "hello" ];
};
helloOnRootNoStore = pkgs.dockerTools.streamLayeredImage {
name = "hello";
tag = "latest";
contents = [
(pkgs.buildEnv {
name = "hello-root";
paths = [ pkgs.hello ];
})
];
config.Cmd = [ "hello" ];
includeStorePaths = false;
};
docker-tools: add example for exportImage functionality and test
2021-05-08 11:49:39 +00:00
# Example export of the bash image
exportBash = pkgs.dockerTools.exportImage { fromImage = bash; };
dockerTools: Fix and test #118722 path in contents
2021-10-18 10:39:51 +00:00
dockerTools: Add fakechroot to fakeRootCommands
2021-10-01 13:53:30 +00:00
imageViaFakeChroot = pkgs.dockerTools.streamLayeredImage {
name = "image-via-fake-chroot";
tag = "latest";
config.Cmd = [ "hello" ];
dockerTools: Keep fakechroot disabled by default Avoid risk of breaking existing images by making it opt-in.
2021-12-04 13:17:56 +00:00
enableFakechroot = true;
dockerTools: Add fakechroot to fakeRootCommands
2021-10-01 13:53:30 +00:00
# Crucially, instead of a relative path, this creates /bin, which is
# intercepted by fakechroot.
# This functionality is not available on darwin as of 2021.
fakeRootCommands = ''
mkdir /bin
ln -s ${pkgs.hello}/bin/hello /bin/hello
'';
};
dockerTools: Fix and test #118722 path in contents
2021-10-18 10:39:51 +00:00
build-image-with-path = buildImage {
name = "build-image-with-path";
tag = "latest";
contents = [ pkgs.bashInteractive ./test-dummy ];
};
layered-image-with-path = pkgs.dockerTools.streamLayeredImage {
name = "layered-image-with-path";
tag = "latest";
contents = [ pkgs.bashInteractive ./test-dummy ];
};
add examples file
2016-10-03 17:07:33 +00:00
}
Reference in New Issue Copy Permalink