Add MkDocs documentation skeleton (spooky!).

docs/README.md

@@ -0,0 +1,9 @@
+# Keras Documentation
+
+The source for Keras documentation is in this directory under `sources/`. 
+Our documentation uses extended Markdown, as implemented by [MkDocs](http://mkdocs.org).
+
+## Building the documentation
+
+- install MkDocs: `sudo pip install mkdocs`
+- `cd` to the `docs/` folder and run: `mkdocs serve`

docs/mkdocs.yml

@@ -0,0 +1,28 @@
+site_name: Keras Documentation
+theme: readthedocs
+docs_dir: sources
+repo_url: https://github.com/fchollet/keras
+site_url: /
+site_description: Documentation for fast and lightweight Keras Deep Learning library.
+
+pages:
+- [about.md, Index]
+- [index.md, Index]
+
+- [examples.md, Examples]
+- [optimizers.md, Optimizers]
+- [objectives.md, Objectives]
+- [activations.md, Activations]
+- [initializations.md, Initializations]
+
+- [datasets.md, Datasets]
+
+- [layers/core.md, Layers, Core Layers]
+- [layers/convolutional.md, Layers, Convolutional Layers]
+- [layers/recurrent.md, Layers, Recurrent Layers]
+- [layers/advanced_activations.md, Layers, Advanced Activations Layers]
+- [layers/normalization.md, Layers, Normalization Layers]
+
+- [preprocessing/sequence.md, Preprocessing, Sequence Preprocessing]
+- [preprocessing/text.md, Preprocessing, Text Preprocessing]
+- [preprocessing/image.md, Preprocessing, Image Preprocessing]

docs/sources/about.md

@@ -0,0 +1,26 @@
+# Keras: Theano-based Deep Learning library
+
+## Overview
+
+Keras is a minimalist, highly modular neural network library in the spirit of Torch, written in Python/Theano. It was developed with a focus on enabling fast experimentation. 
+
+## Guiding principles
+
+- __Modularity.__ A model is understood as a sequence of standalone, fully-configurable modules that can be plugged together with as little restrictions as possible. In particular, neural layers, cost functions, optimizers, initialization schemes, activation functions and dropout are all standalone modules that you can combine to create new models. 
+
+- __Minimalism.__ Each module should be kept short and simple (<100 lines of code). Every piece of code should be transparent upon first reading. No black magic: it hurts iteration speed and ability to innovate. 
+
+- __Easy extensibility.__ A new feature (a new module, per the above definition, or a new way to combine modules together) are dead simple to add (as new classes/functions), and existing modules provide ample examples.
+
+- __Work with Python__. No separate models configuration files in a declarative format (like in Caffe or PyLearn2). Models are described in Python code, which is compact, easier to debug, benefits from syntax highlighting, and most of all, allows for ease of extensibility.
+
+
+## Contribution Guidelines
+
+Keras welcomes all contributions from the community. 
+
+- Keep a pragmatic mindset and avoid bloat. Only add to the source if that is the only path forward. Every additional line of code is a liability.
+- New features should be documented. Make sure you update the documentation along with your Pull Request.
+- The documentation for every new feature should include a usage example in the form of a code snippet. 
+- All changes should be tested. A formal test process will be introduced very soon.
+- Even if you don't contribute to the Keras source code, if you have an application of Keras that is concise and powerful, please consider adding to our collection of [examples](https://github.com/fchollet/keras/tree/master/examples).

docs/sources/examples.md

@@ -0,0 +1,149 @@
+## Examples
+
+### Multilayer Perceptron (MLP):
+
+```python
+from keras.models import Sequential
+from keras.layers.core import Dense, Dropout, Activation
+from keras.optimizers import SGD
+
+model = Sequential()
+model.add(Dense(20, 64, init='uniform'))
+model.add(Activation('tanh'))
+model.add(Dropout(0.5))
+model.add(Dense(64, 64, init='uniform'))
+model.add(Activation('tanh'))
+model.add(Dropout(0.5))
+model.add(Dense(64, 1, init='uniform'))
+model.add(Activation('softmax'))
+
+sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
+model.compile(loss='mean_squared_error', optimizer=sgd)
+
+model.fit(X_train, y_train, nb_epoch=20, batch_size=16)
+score = model.evaluate(X_test, y_test, batch_size=16)
+```
+
+### Alternative implementation of MLP:
+
+```python
+model = Sequential()
+model.add(Dense(20, 64, init='uniform', activation='tanh'))
+model.add(Dropout(0.5))
+model.add(Dense(64, 64, init='uniform', activation='tanh'))
+model.add(Dropout(0.5))
+model.add(Dense(64, 1, init='uniform', activation='softmax')
+
+sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
+model.compile(loss='mean_squared_error', optimizer=sgd)
+```
+
+### VGG-like convnet:
+
+```python
+from keras.models import Sequential
+from keras.layers.core import Dense, Dropout, Activation, Flatten
+from keras.layers.convolutional import Convolution2D, MaxPooling2D
+from keras.optimizers import SGD
+
+model = Sequential()
+model.add(Convolution2D(32, 3, 3, 3, border_mode='full')) 
+model.add(Activation('relu'))
+model.add(Convolution2D(32, 32, 3, 3))
+model.add(Activation('relu'))
+model.add(MaxPooling2D(poolsize=(2, 2)))
+model.add(Dropout(0.25))
+
+model.add(Convolution2D(64, 32, 3, 3, border_mode='full')) 
+model.add(Activation('relu'))
+model.add(Convolution2D(64, 64, 3, 3)) 
+model.add(Activation('relu'))
+model.add(MaxPooling2D(poolsize=(2, 2)))
+model.add(Dropout(0.25))
+
+model.add(Flatten(64*8*8))
+model.add(Dense(64*8*8, 256))
+model.add(Activation('relu'))
+model.add(Dropout(0.5))
+
+model.add(Dense(256, 10))
+model.add(Activation('softmax'))
+
+sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
+model.compile(loss='categorical_crossentropy', optimizer=sgd)
+
+model.fit(X_train, Y_train, batch_size=32, nb_epoch=1)
+
+```
+
+### Sequence classification with LSTM:
+
+```python
+from keras.models import Sequential
+from keras.layers.core import Dense, Dropout, Activation, Embedding
+from keras.layers.recurrent import LSTM
+
+model = Sequential()
+model.add(Embedding(max_features, 256))
+model.add(LSTM(256, 128, activation='sigmoid', inner_activation='hard_sigmoid'))
+model.add(Dropout(0.5))
+model.add(Dense(128, 1))
+model.add(Activation('sigmoid'))
+
+model.compile(loss='binary_crossentropy', optimizer='rmsprop')
+
+model.fit(X_train, Y_train, batch_size=16, nb_epoch=10)
+score = model.evaluate(X_test, Y_test, batch_size=16)
+```
+
+### Architecture for learning image captions with a convnet and a Gated Recurrent Unit:
+(word-level embedding, caption of maximum length 16 words).
+
+Note that getting this to actually "work" will require using a bigger convnet, initialized with pre-trained weights.
+Displaying readable results will also require an embedding decoder.
+
+```python
+max_caption_len = 16
+
+model = Sequential()
+model.add(Convolution2D(32, 3, 3, 3, border_mode='full')) 
+model.add(Activation('relu'))
+model.add(Convolution2D(32, 32, 3, 3))
+model.add(Activation('relu'))
+model.add(MaxPooling2D(poolsize=(2, 2)))
+
+model.add(Convolution2D(64, 32, 3, 3, border_mode='full')) 
+model.add(Activation('relu'))
+model.add(Convolution2D(64, 64, 3, 3)) 
+model.add(Activation('relu'))
+model.add(MaxPooling2D(poolsize=(2, 2)))
+
+model.add(Convolution2D(128, 64, 3, 3, border_mode='full')) 
+model.add(Activation('relu'))
+model.add(Convolution2D(128, 128, 3, 3)) 
+model.add(Activation('relu'))
+model.add(MaxPooling2D(poolsize=(2, 2)))
+
+model.add(Flatten(128*4*4))
+model.add(Dense(128*4*4, 256))
+model.add(Activation('relu'))
+model.add(Dropout(0.5))
+
+model.add(Repeat(max_caption_len)) 
+# the GRU below returns sequences of max_caption_len vectors of size 256 (our word embedding size)
+model.add(GRU(256, 256, return_sequences=True))
+
+model.compile(loss='mean_squared_error', optimizer='rmsprop')
+
+# "images" is a numpy array of shape (nb_samples, nb_channels=3, width, height) 
+# "captions" is a numpy array of shape (nb_samples, max_caption_len=16, embedding_dim=256)
+# captions are supposed already embedded (dense vectors).
+model.fit(images, captions, batch_size=16, nb_epoch=100)
+    
+```
+
+In the [examples folder](https://github.com/fchollet/keras/tree/master/examples), you will find example models for real datasets:
+
+- CIFAR10 small images classification: Convnet with realtime data augmentation
+- IMDB movie review sentiment classification: LSTM over sequences of words
+- Reuters newswires topic classification: Multilayer Perceptron

docs/sources/index.md

@@ -0,0 +1,25 @@
+# Keras Documentation Index
+
+- [About](about.md)
+- [Index](index.md)
+
+- [Examples](examples.md)
+
+- [Optimizers](optimizers.md)
+- [Objectives](objectives.md)
+- [Activations](activations.md)
+- [Initializations](initializations.md)
+
+- [Datasets](datasets.md)
+
+- Layers
+    - [Core](layers/core.md)
+    - [Convolutional](layers/convolutional.md)
+    - [Recurrent](layers/recurrent.md)
+    - [Advanced Activations](layers/advanced_activations.md)
+    - [Normalization](layers/normalization.md)
+
+- Preprocessing
+    - [Sequence](preprocessing/sequence.md)
+    - [Text](preprocessing/text.md)
+    - [Image](preprocessing/image.md)

docs/sources/optimizers.md

@@ -0,0 +1,27 @@
+# Optimizers
+
+## Base class
+
+Optimizer
+
+All optimizers descending from this class support the following keyword arguments:
+- l1
+- l2
+- clipnorm
+- maxnorm
+
+##  SGD
+
+SGD(lr=0.01, momentum=0., decay=0., nesterov=False)
+
+##  Adagrad
+
+Adagrad(lr=0.01, epsilon=1e-6)
+
+##  Adadelta
+
+Adadelta(lr=1.0, rho=0.95, epsilon=1e-6)
+
+##  RMSprop
+
+RMSprop(lr=0.001, rho=0.9, epsilon=1e-6)