Readme update
This commit is contained in:
parent
5440bdb82c
commit
4d60430886
203
docker/README.md
203
docker/README.md
@ -13,67 +13,120 @@
|
||||
|
||||
##Using the Dockers##
|
||||
|
||||
### Running Dockers ###
|
||||
|
||||
In order to run the dockers, the docker has to be built, and then run with a
|
||||
lot of arguments to get the mount points right, etc... A convenient script is
|
||||
supplied to make this all easy. By running
|
||||
|
||||
./docker/run_docker.bsh
|
||||
|
||||
All the git-lfs_* images are built automatically, and then run. The default
|
||||
command is the build script. The currently checkout version of git-lfs is
|
||||
copied into the docker, and ```git clean -xdf``` is run to remove any
|
||||
non-tracked files, but non-commited changes are kept. Then git-lfs is built,
|
||||
and a pacakges is created (deb or rpm)
|
||||
|
||||
To only run certain docker images, supply them as arguments, e.g.
|
||||
|
||||
./docker/run_docker.bsh git-lfs_debian_7
|
||||
./docker/run_docker.bsh git-lfs_debian_*
|
||||
./docker/run_docker.bsh git-lfs_*[6-8]
|
||||
|
||||
And only those images will be run.
|
||||
|
||||
There are a few environment variables you can set to easily adjust the behavior
|
||||
of the run_docker.bsh script.
|
||||
|
||||
REPO_HOSTNAME - Override the hostname for all the repos generated/tested (see below)
|
||||
|
||||
BUILD_LOCAL - Set to 1 (default) to use the currently checked out version of
|
||||
the git-lfs to build against. If it's not 1 the released archived is downloaded
|
||||
and built against. Currently only works for RPMs. DEB always builds the currently
|
||||
checkout version. Build local only affect the version of the code used in generating
|
||||
the rpms, not the scripts running to generate the rpms.
|
||||
|
||||
DOCKER_AUTOBUILD - Default 1. run_docker.bsh always calls build_docker.bsh to
|
||||
ensure your docker image is uptodate. Somtimes you may not want this. If set
|
||||
this to 0, it will not build docker images before running
|
||||
|
||||
AUTO_REMOVE - Default 1. Docker instances are automatically deleted on program
|
||||
exit. If set to 0, the docker instance will not be automatically deleted upon
|
||||
exit. This can be useful for a post mortem analysis (using other docker commands
|
||||
not covered here). Just make sure you clean up the docker instances manually.
|
||||
|
||||
###Development with Dockers###
|
||||
|
||||
Sometimes you don't want to just build git-lfs and destroy the container, you
|
||||
want to get in there, run a lot of command, DEVELOP! To do this, the best
|
||||
command to run is bash, and then you have an interactive shell to use. To do this
|
||||
|
||||
./docker/run_docker.bsh {image(s)} -- bash
|
||||
|
||||
After listing the image(s) you want to run, add a double dash (--) and then any
|
||||
command (and arguments) you want executed in the docker. Remember, the command
|
||||
you are executing has to be part of the docker image.
|
||||
|
||||
### Building Dockers (Optional) ###
|
||||
|
||||
```run_dockers.bsh``` now call build_dockers ```build_dockers.bsh```, but you
|
||||
```run_dockers.bsh``` calls build_dockers ```build_dockers.bsh```, but you
|
||||
can still call the script manually to get it all out of the way once while you
|
||||
go make that tea/coffee.
|
||||
go make some tea/coffee.
|
||||
|
||||
In order to use the docker **images**, they have to be built so that they are
|
||||
ready to be used. For OSes like Debian, this is a fairly quick process.
|
||||
However CentOS takes considerably longer time, since it has to build go, ruby,
|
||||
or git from source, depending on the version. Fortunately, you can build the
|
||||
docker images JUST once, and you won't have to build it again (unless something
|
||||
significant changes, which should be fairly uncommon). This means all the
|
||||
compiling, yum/apt-get/custom dependency compiling is done once and saved.
|
||||
(This is done in CentOS by using the already existing ```./rpm/rpm_build.bsh```
|
||||
script to bootstrap the image and saving the image.)
|
||||
or git from source, depending on the distro. Fortunately, you can build the
|
||||
docker images JUST once, and you won't have to build it again (until the version
|
||||
changed.) The build script uses a downloaded release from github of git-lfs to
|
||||
bootstap the centos image and build/install all the neccessary software. Currently
|
||||
the only way to change what version the image is built off of is by changing the
|
||||
url in the Dockerfile for git-lfs_centos_*.
|
||||
|
||||
There is a script to take care of ALL of these details for you. Simply run
|
||||
This means all the compiling, yum/apt-get/custom dependency compiling is done
|
||||
once and saved. (This is done in CentOS by using the already existing
|
||||
```./rpm/rpm_build.bsh``` script to bootstrap the image and saving the image.)
|
||||
|
||||
The script that takes care of ALL of these details for you is
|
||||
|
||||
./docker/build_dockers.bsh
|
||||
|
||||
And all the git-lfs_* images will be built automatically. These are all a
|
||||
developer would need to test the different OSes.
|
||||
All the git-lfs_* images will be built automatically. These are all a
|
||||
developer would need to test the different OSes. And create just git-lfs rpm or
|
||||
deb pacakges.
|
||||
|
||||
If you were more interested in creating all the special packages for CentOS
|
||||
(For example, you may want to hand out the git rpm for CentOS 6, since only
|
||||
git 1.7.1 is available, or else everyone will have to build git from source
|
||||
just to use git-lfs on CentOS 6), then the git-lfs-full-build_* dockers are
|
||||
useful. Instead of building everyone necessary into the image, these contain
|
||||
bare CentOS image, so the ```./rpm/rpm_build.bsh``` script will build all of
|
||||
its dependencies when you run the dockers, making the rpms available. These
|
||||
images can be build by running
|
||||
However, in order to distribute git-lfs or build dependencies, the packages
|
||||
that were installed for you by ```build_docker.bsh``` need to be saved too.
|
||||
This is currently only a CentOS problem. In order to genereate THOSE rpms,
|
||||
the git-lfs-full-build_* will use NON-bootstrapped images to build every pacakge
|
||||
and git-lfs and generate rpms. This takes as long as building the image in the
|
||||
first place, but you don't get the benifit of a saved state. The
|
||||
```./rpm/rpm_build.bsh``` script will build all of its dependencies when you
|
||||
run the dockers, making the rpms available. These images can be built by running
|
||||
|
||||
./docker/docker_build.bsh ./docker/git-lfs-full-build_*
|
||||
|
||||
In fact, any subset of images can be built only, by passing the list of of
|
||||
directories as argument. (Each directory contains a Dockerfile, this is used to
|
||||
tell the .bsh scripts which you want to work on AND names them in
|
||||
```docker images``` based on the directory name.)
|
||||
|
||||
(To manually build a docker, run ```docker build -t $(basename ${DockerDir})
|
||||
-f ${DockerDir}/Dockerfile ./docker```
|
||||
This is most important for CentOS 6 where git 1.8.2 or newer is not available,
|
||||
only git 1.7.1 is available, so every user either has to build git from source,
|
||||
or use the rpms generated by the ```git-lfs-full-build_centos_6``` image.
|
||||
|
||||
### Running Dockers ###
|
||||
(To manually build a docker image, run
|
||||
```docker build -t $(basename ${DockerDir}) -f ${DockerDir}/Dockerfile ./docker```
|
||||
|
||||
After the docker images are build, a lot of arguments need to me added to get
|
||||
the mount points right, etc... again, a convenient script is supplied to make
|
||||
this all easy. By running
|
||||
|
||||
./docker/run_docker.bsh
|
||||
|
||||
All the git-lfs_* images build their packages and tranfer them to the
|
||||
```./docker/repos``` directory.
|
||||
|
||||
|
||||
Copies your current source
|
||||
Cleans the copies, so all untracked files are deleted, but uncommited changes are kept
|
||||
|
||||
###Development with Dockers###
|
||||
|
||||
##Deploying/Building Repositories##
|
||||
|
||||
When ```./docker/run_dockers.bsh``` is done building git-lfs and the rpms/deb,
|
||||
it actually creates a repository for distubution too. Each distro gets a repo
|
||||
gerenated in ```./docker/repos/{DISTRO_NAME}/{VERSION #}```. Just drop the repo
|
||||
directory onto a webserver and you have a fully functioning linux repo. (See
|
||||
Testing the Repositories below for more detail)
|
||||
|
||||
The two major packages included are:
|
||||
git-lfs-....* - the git-lfs package
|
||||
git-lfs-repo-release....* - A package to install the repo.
|
||||
|
||||
When using ```BUILD_LOCAL=1```, all UNTRACKED files are removed during RPM
|
||||
generation, except any stray directories containing a .git folder will not be
|
||||
cleared. This shouldn't be the case, unless you are temporarily storing another
|
||||
@ -82,14 +135,23 @@ if you are producing packages.
|
||||
|
||||
### Setting the website URL ###
|
||||
|
||||
The git-lfs-repo-release must contain the URL where the repo is to be hosted.
|
||||
The current default value is 'git-lfs.github.com' but this can be overridden
|
||||
using the REPO_HOSTNAME env var, e.g.
|
||||
|
||||
REPO_HOSTNAME=www.notgithub.uk.co ./docker/run_dockers.bsh
|
||||
|
||||
Now all the git-lfs-repo-release....* files will point to that url instead
|
||||
|
||||
### GPG signing ###
|
||||
|
||||
For private repo testing, GPG signing can be skipped. apt-get and yum can
|
||||
install .deb/.rpm directly without gpg keys and everything will work. This
|
||||
section is for distribution in a repo. Most if not all this functionality is
|
||||
automatically disabled when there is no signing key present.
|
||||
automatically disabled when there is no signing key present
|
||||
(```./docker/signing.key```).
|
||||
|
||||
Or order to sign packages, you need to place the keys in the right place
|
||||
In order to sign packages, you need to generate and place GPG keys in the right place
|
||||
|
||||
1. gpg --gen-key
|
||||
|
||||
@ -110,12 +172,6 @@ Or order to sign packages, you need to place the keys in the right place
|
||||
|
||||
Keep in mind, signing.key must NEVER be accidentally commited to the repo.
|
||||
|
||||
Signing in CentOS 5 is... problemsome. While it can not handle v4 signatures,
|
||||
it should work with RSA on v3. However, it doesn't. So... The only way around it
|
||||
is to either sign it with the key instructinos above aand install it with the
|
||||
```yum install --nogpgcheck``` OR create a NEW DSA key just for CentOS 5. Even
|
||||
then, the default size of 2048 did not work. A DSA key of 1024 bits does work.
|
||||
|
||||
To prevent MANY passphrase entries at random times, the gpg-agent is used to
|
||||
cache your signing key. This is done by running gpg-agent in the host, and passing
|
||||
the connection to each docker image. This will be done for you automatically by
|
||||
@ -127,10 +183,26 @@ GPG agent ttl set to 5 hours, should be plenty to build everything. If this is
|
||||
not good for you, set the GPG_MAX_CACHE and GPG_DEFAULT_CACHE environment variables
|
||||
(in seconds)
|
||||
|
||||
### GPG capabilited by Distro ###
|
||||
|
||||
Debian WILL work with 4096 bit RSA signing subkeys like [1] suggests, but will
|
||||
also work with 4096 bit RSA signing keys.
|
||||
|
||||
CentOS will NOT work with subkeys. CentOS 6 and 7 will work with 4096 bit RSA
|
||||
signing keys
|
||||
|
||||
CentOS 5 will NOT work with v4 signatures. The rpms will be so unrecognizable
|
||||
that it can't even be installed with --nogpgcheck. It should work with RSA on v3.
|
||||
However, I could not get it to it builds v3 correctly now, but for some reason
|
||||
the GPG check fails. However ```yum install --nogpgcheck``` does work! CentOS 5
|
||||
will NOT work with 2048 bit DSA keys... I suspect 2048 is too big for it to
|
||||
fathom. CentOS 5 WILL work with 1024 bit DSA keys. Either sign it with the key
|
||||
instructinos above and install it with the ```yum install --nogpgcheck``` OR
|
||||
create a NEW DSA key just for CentOS 5.
|
||||
|
||||
[1] https://www.digitalocean.com/community/tutorials/how-to-use-reprepro-for-a-secure-package-repository-on-ubuntu-14-04
|
||||
[2] https://iuscommunity.org/pages/CreatingAGPGKeyandSigningRPMs.html#exporting-the-public-gpg-key
|
||||
[3] http://www.redhat.com/archives/rpm-list/2006-November/msg00105.html
|
||||
- Rpms do NOT SUPPORT subkeys. So don't try
|
||||
|
||||
### Testing the Repositories ###
|
||||
|
||||
@ -141,10 +213,11 @@ again, run
|
||||
|
||||
(which is basically just ```./docker/run_dockers.bsh ./docker/git-lfs-test_*```)
|
||||
|
||||
REPO_HOSTNAME can be used for BOTH ```run_dockers.bsh``` and ```test_dockers.bsh```
|
||||
to run a local test (on ```localhost:{Port Number}```, for example)
|
||||
Remember to set REPO_HOSTNAME if you changed it for ```./docker/build_docker.bsh```
|
||||
This can also be used to run a local test (on ```localhost:{Port Number}```, for
|
||||
example)
|
||||
|
||||
An easy way to test the repositories, is to run host them on a webserver such as
|
||||
An easy way to test the repositories locally, is to run host them on a webserver such as
|
||||
|
||||
cd ./docker/repos
|
||||
python -m SimpleHTTPServer {Port number}
|
||||
@ -158,20 +231,21 @@ or
|
||||
## Adding addition OSes ##
|
||||
|
||||
To add another operating system, simply follow the already existing pattern,
|
||||
and all the script will them them up. A new Dockerfile should go in a directory
|
||||
and all the script will pick them up. A new Dockerfile should go in a directory
|
||||
named
|
||||
|
||||
./docker/git-lfs_{OS NAME}_{OS VERSION}
|
||||
./docker/git-lfs_{OS NAME}_{OS VERSION #}
|
||||
|
||||
where **{OS NAME}** and **{OS VERSION}** should not contain underscores (_).
|
||||
where **{OS NAME}** and **{OS VERSION #}** should not contain underscores (_).
|
||||
Any files that need to be added to the docker image can be dropped in the
|
||||
```./docker``` directory, since that is the root they are built against (not
|
||||
the directory containing the Dockerfile)
|
||||
```./docker/common``` directory, since that is the root they are built against
|
||||
(not the directory containing the Dockerfile like most dockers)
|
||||
|
||||
The docker image should write it's repo files to /repo inside the docker, and
|
||||
they will end up in
|
||||
The docker image should run a script that write it's repo files to the /repo
|
||||
direcrtory inside the docker, and writing to /repo in the docker will cause the
|
||||
files to end up in
|
||||
|
||||
./docker/repos/{OS NAME}/{OS VERSION}/
|
||||
./docker/repos/{OS NAME}/{OS VERSION #}/
|
||||
|
||||
## Docker Cheat sheet ##
|
||||
|
||||
@ -189,10 +263,6 @@ http://docs.docker.com/ Install -> Docker Engine -> Installation on ...
|
||||
|
||||
docker rm $(docker ps --filter=status=exited -q)
|
||||
|
||||
1. How much space are all these Dockers taking up?
|
||||
|
||||
No idea. sudo du /var/lib/docker
|
||||
|
||||
|
||||
# Troubleshooting #
|
||||
|
||||
@ -211,8 +281,9 @@ ignoring many Ctrl+C's
|
||||
|
||||
3. That answer's not good enough. How do I resume a docker?
|
||||
|
||||
Well, first you have to remove the ```--rm``` flag. This will keep the
|
||||
docker around after stopping. Be careful! They multiply like rabbits. Then
|
||||
Well, first you have to set the environment variable ```AUTO_REMOVE=0```
|
||||
before running the image you want to resume. This will keep the docker
|
||||
around after stopping. (Be careful! They multiply like rabbits.) Then
|
||||
|
||||
```docker commit {container name/id} {new_name}```
|
||||
|
||||
|
Loading…
Reference in New Issue
Block a user