Neural Networks with Python


Neural network is one of the current state of the art method for Machine Learning. Python offers several ways to implement a neural network.

A very brief overview of Neural Nets

Neural networks intend to mimic the human brain. The neural network is composed of several layers of artificial neurons, and the different layers are connected with link of variable strength. The network is trained on the training data and the weights are set to minimise the error.
Here are some example of architectures of neural network:

  • Artificial (and classic) neural networks: The neurons are using activation function such as hyperbolic tangent, linear unit, sigmoid, …
  • Convolutional neural network: Mainly used for image recognition, convolutional layers  detect patterns in the images and pooling layers improves generalisation of the detection of these patterns.
  •   Autoencoder: the neural net is trained to learn a representation of the input data. Autoencoders are used for non-linear dimensionality reductions.

Back-end library and Neural Network implementation in Python

Python has several modules that are great to implement neural networks:

  • Scikit-learn, multi-layer perceptron and restricted boltzmann machine can be created and fitted with sklearn.neural_network module. Since scikit-learn is a general machine learning library, you have less control over the neural network than with module that are specialised. This is the easiest way to create simple neural network.
  • Theano is a backend numerical computation library developed by the University of Montreal. Theano offers an interface to build a computational graph and computes gradient automatically, it comes with all the necessary functions to build convolutional net, recurrent net, … Furthermore, the computation can done on both the CPU and the GPU with CUDA. I would advise Theano for learning NN since it is mature and offers access to state-of-the-art Neural Net.
  • Tensorflow is as Theano a computational library and has been developed and supported by Google. It is probably the most popular library for Python and for Deep Learning. As Theano, it uses computational graph and can work on both CPU and GPU.
  • Torch and PyTorch, Torch is a C and LuaJIT deep learning library. The library is more flexible than Tensorflow and Theano and somewhat faster. However PyTorch is more recent than Tensorflow and Theano and has hence a smaller community.
  • FANN and its python wrapper, This library is designed to be fast and can speed up the computation by almost 100 times. However it does not contains fancy layers and the library is limited to MLP with various propagation techniques.

Front-end library for easy Neural Network implementation in Python

The previous libraries may be hard to use when you begin with Neural Nets and can take some times to get the neural network working. Some libraries are build on the top of Theano and Tensorflow for easy prototyping.

  • Keras is build on the top of Tensor Flow and Theano. Which means you can build your model on both backends. Implementation of a ConvNet or a recurrent neural net only takes a few lines. Furthermore, the project has a lot of support and a new major stable version has just been released; To my mind, this is the easiest way to implement complex (and yet classical) neural net and it makes prototyping very easy.
  • Lasagne is build on the top of Theano. While you can write a Keras NN without any knowledge of Theano, Lasagne intends to make it easier to write Theano neural network. So, if you want a very easy way to prototype a Neural Network you should go with Keras. On the other hand, to write Theano efficiently, Lasagne is a great option.
  • And also: Blocks and Pylearn (Which is not maintained anymore)


Please enter your comment!
Please enter your name here