Docker image for test and experiment Keras (#3035)
* Docker image for test and experiment Keras - Docker image with CUDA support on ubuntu 14.04 - nvidia-docker script to forward the GPU to the container - MakeFile to simplify docker commands for build, run, test, ..etc - Add useful tools like jupyter notebook, ipdb, sklearn for experiments * update nvidia-docker plugin * use .theanorc in Dockerfile * Add tensorflow to the docker image * update Docker image to cuDNN v5 * test fixes * move docker to sub directory * README for docker * Fix typos * Add visualization to Dockerfile
This commit is contained in:
parent
3726aba2ee
commit
df84c69676
46
docker/Dockerfile
Normal file
46
docker/Dockerfile
Normal file
@ -0,0 +1,46 @@
|
|||||||
|
FROM nvidia/cuda:7.5-cudnn5-devel
|
||||||
|
|
||||||
|
ENV CONDA_DIR /opt/conda
|
||||||
|
ENV PATH $CONDA_DIR/bin:$PATH
|
||||||
|
|
||||||
|
RUN mkdir -p $CONDA_DIR && \
|
||||||
|
echo export PATH=$CONDA_DIR/bin:'$PATH' > /etc/profile.d/conda.sh && \
|
||||||
|
apt-get update && \
|
||||||
|
apt-get install -y wget git libhdf5-dev g++ graphviz && \
|
||||||
|
wget --quiet https://repo.continuum.io/miniconda/Miniconda3-3.9.1-Linux-x86_64.sh && \
|
||||||
|
echo "6c6b44acdd0bc4229377ee10d52c8ac6160c336d9cdd669db7371aa9344e1ac3 *Miniconda3-3.9.1-Linux-x86_64.sh" | sha256sum -c - && \
|
||||||
|
/bin/bash /Miniconda3-3.9.1-Linux-x86_64.sh -f -b -p $CONDA_DIR && \
|
||||||
|
rm Miniconda3-3.9.1-Linux-x86_64.sh
|
||||||
|
|
||||||
|
ENV NB_USER keras
|
||||||
|
ENV NB_UID 1000
|
||||||
|
|
||||||
|
RUN useradd -m -s /bin/bash -N -u $NB_UID $NB_USER && \
|
||||||
|
mkdir -p $CONDA_DIR && \
|
||||||
|
chown keras $CONDA_DIR -R && \
|
||||||
|
mkdir -p /src && \
|
||||||
|
chown keras /src
|
||||||
|
|
||||||
|
USER keras
|
||||||
|
|
||||||
|
# Python
|
||||||
|
ARG python_version=3.5.1
|
||||||
|
ARG tensorflow_version=0.9.0rc0-cp35-cp35m
|
||||||
|
RUN conda install -y python=${python_version} && \
|
||||||
|
pip install https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-${tensorflow_version}-linux_x86_64.whl && \
|
||||||
|
pip install git+git://github.com/Theano/Theano.git && \
|
||||||
|
pip install ipdb pytest pytest-cov python-coveralls coverage==3.7.1 pytest-xdist pep8 pytest-pep8 pydot_ng && \
|
||||||
|
conda install Pillow scikit-learn notebook pandas matplotlib nose pyyaml six h5py && \
|
||||||
|
pip install git+git://github.com/fchollet/keras.git && \
|
||||||
|
conda clean -yt
|
||||||
|
|
||||||
|
ADD theanorc /home/keras/.theanorc
|
||||||
|
|
||||||
|
ENV PYTHONPATH='/src/:$PYTHONPATH'
|
||||||
|
|
||||||
|
WORKDIR /src
|
||||||
|
|
||||||
|
EXPOSE 8888
|
||||||
|
|
||||||
|
CMD jupyter notebook --port=8888 --ip=0.0.0.0
|
||||||
|
|
26
docker/Makefile
Normal file
26
docker/Makefile
Normal file
@ -0,0 +1,26 @@
|
|||||||
|
help:
|
||||||
|
@cat Makefile
|
||||||
|
|
||||||
|
DATA?="${HOME}/Data"
|
||||||
|
GPU?=0
|
||||||
|
DOCKER_FILE=Dockerfile
|
||||||
|
DOCKER=GPU=$(GPU) nvidia-docker
|
||||||
|
BACKEND=tensorflow
|
||||||
|
TEST=tests/
|
||||||
|
SRC=$(shell dirname `pwd`)
|
||||||
|
|
||||||
|
build:
|
||||||
|
docker build -t keras --build-arg python_version=3.5 -f $(DOCKER_FILE) .
|
||||||
|
|
||||||
|
bash: build
|
||||||
|
$(DOCKER) run -it -v $(SRC):/src -v $(DATA):/data --env KERAS_BACKEND=$(BACKEND) keras bash
|
||||||
|
|
||||||
|
ipython: build
|
||||||
|
$(DOCKER) run -it -v $(SRC):/src -v $(DATA):/data --env KERAS_BACKEND=$(BACKEND) keras ipython
|
||||||
|
|
||||||
|
notebook: build
|
||||||
|
$(DOCKER) run -it -v $(SRC):/src -v $(DATA):/data --net=host --env KERAS_BACKEND=$(BACKEND) keras
|
||||||
|
|
||||||
|
test: build
|
||||||
|
$(DOCKER) run -it -v $(SRC):/src -v $(DATA):/data --env KERAS_BACKEND=$(BACKEND) keras py.test $(TEST)
|
||||||
|
|
58
docker/README.md
Normal file
58
docker/README.md
Normal file
@ -0,0 +1,58 @@
|
|||||||
|
# Using Keras via Docker
|
||||||
|
|
||||||
|
This directory contains `Dockerfile` to make it easy to get up and running with
|
||||||
|
Keras via [Docker](http://www.docker.com/).
|
||||||
|
|
||||||
|
## Installing Docker
|
||||||
|
|
||||||
|
General installation instructions are
|
||||||
|
[on the Docker site](https://docs.docker.com/installation/), but we give some
|
||||||
|
quick links here:
|
||||||
|
|
||||||
|
* [OSX](https://docs.docker.com/installation/mac/): [docker toolbox](https://www.docker.com/toolbox)
|
||||||
|
* [ubuntu](https://docs.docker.com/installation/ubuntulinux/)
|
||||||
|
|
||||||
|
## Running the container
|
||||||
|
|
||||||
|
We are using `Makefile` to simplify docker commands within make commands.
|
||||||
|
|
||||||
|
Build the container and start a jupyter notebook
|
||||||
|
|
||||||
|
$ make notebook
|
||||||
|
|
||||||
|
Build the container and start an iPython shell
|
||||||
|
|
||||||
|
$ make ipython
|
||||||
|
|
||||||
|
Build the container and start a bash
|
||||||
|
|
||||||
|
$ make bash
|
||||||
|
|
||||||
|
For GPU support install NVidia drivers (ideally latest) and
|
||||||
|
[nvidia-docker](https://github.com/NVIDIA/nvidia-docker). Run using
|
||||||
|
|
||||||
|
$ make notebook GPU=0 # or [ipython, bash]
|
||||||
|
|
||||||
|
Switch between Theano and TensorFlow
|
||||||
|
|
||||||
|
$ make notebook BACKEND=theano
|
||||||
|
$ make notebook BACKEND=tensorflow
|
||||||
|
|
||||||
|
Mount a volume for external data sets
|
||||||
|
|
||||||
|
$ make DATA=~/mydata
|
||||||
|
|
||||||
|
Prints all make tasks
|
||||||
|
|
||||||
|
$ make help
|
||||||
|
|
||||||
|
You can change Theano parameters by editing `/docker/theanorc`.
|
||||||
|
|
||||||
|
|
||||||
|
Note: If you would have a problem running nvidia-docker you may try the old way
|
||||||
|
we have used. But it is not recommended. If you find a bug in the nvidia-docker report
|
||||||
|
it there please and try using the nvidia-docker as described above.
|
||||||
|
|
||||||
|
$ export CUDA_SO=$(\ls /usr/lib/x86_64-linux-gnu/libcuda.* | xargs -I{} echo '-v {}:{}')
|
||||||
|
$ export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
|
||||||
|
$ docker run -it -p 8888:8888 $CUDA_SO $DEVICES gcr.io/tensorflow/tensorflow:latest-gpu
|
5
docker/theanorc
Normal file
5
docker/theanorc
Normal file
@ -0,0 +1,5 @@
|
|||||||
|
[global]
|
||||||
|
floatX = float32
|
||||||
|
optimizer=None
|
||||||
|
device = gpu
|
||||||
|
|
Loading…
Reference in New Issue
Block a user