Introduction:
I started working on a new classification problem and the first question in my mind: which framework to use? So decided to explore the pros and cons with the available frameworks before to start building the model. Instead of writing hundreds of lines of code, we can use a suitable framework to help us to build a Deep Learning model quickly. For Example, To code a Convolutional Neural Network(CNN) from scratch is a complicated and time-consuming process. This is where deep learning frameworks have truly changed the landscape. Each framework is built in a different manner for different purposes. Here, we look at some of the top deep learning frameworks in order to get a better idea on which framework will be the perfect fit or come handy in solving a particular challenge.
Some of the popular deep learning frameworks:
!
Theano is where the whole story has begun. It was created in 2007 by Yoshua Bengio and the research team at the University of Montreal and was the first widely used DL (Deep Learning) framework. Theano is a Python library, extremely fast and powerful but criticised for being a low level deep learning framework. The team behind Theano announced in 2017 that after releasing the latest version there will be no further developments.
Computational graph is nice abstraction
RNNs fit nicely in computational graph
Raw Theano is somewhat low-level
High-level wrappers (Keras, Lasagne) ease the pain
Large models can have long compile times
Much “fatter” than Torch
Buggy on AWS
Single GPU
A Python version of Torch, known as Pytorch, was open-sourced by Facebook in January 2017. PyTorch offers dynamic computation graphs, which let you process variable-length inputs and outputs, which is useful when working with RNNs, for example. It has gained immense interest in the last year, becoming a preferred solution for academic research, and applications of deep learning requiring optimizing custom expressions.
PyTorch, is a lower-level API focused on direct work with array expressions.
PyTorch provides a similar pace which is fast and suitable for high performance.
Lots of modular pieces that are easy to combine
Easy to write your own layer types and run on GPU
You usually write your own training code (Less plug and play)
PyTorch are used for high-performance models and large datasets that require fast execution
Lots of pretrained models
No commercial support
PyTorch has a complex architecture and the readability is less when compared to Keras.
Google created TensorFlow to replace Theano. The two libraries are in fact quite similar. TensorFlow is an open-source software library for dataflow programming across a range of tasks. It is a symbolic math library that is used for machine learning applications like neural networks. TensorFlow is written with a Python API over a C/C++ engine that makes it run faster.
TensorFlow is a framework that provides both high and low level APIs.
Computational graph abstraction, like Theano
Faster compile times than Theano
TensorBoard for visualization
Data and model parallelism
TensorFlow is about more than deep learning. TensorFlow actually has tools to support reinforcement learning and other algos.
Computational graph is pure Python, therefore slow
Not many pretrained models
No commercial support
Keras is an open source neural network library written in Python. It is capable of running on top of TensorFlow. It is designed to enable fast experimentation with deep neural networks.
Keras is a high-level API capable of running on top of TensorFlow, Theano, Deeplearning4j and Deeplearning4j and CNTK
Fast growing framework, Likely to become standard Python API for NNs
Keras has a simple architecture. Lightweight, easy to use and really straightforward
Rapid Prototyping
Multiple back-end support
Keras is usually used for small datasets as it is comparatively slower
Deeplearning4j is the first commercial oriented, open source, distributed deep learning library written for Java and Scala. DL4J supports importing neural models from other major libraries and can be run on top of some very popular big data tools such as Apache Spark.
If you use Java as your programming language, DL4J is the framework to go for
DL4J is a JVM-based, industry-focused, commercially supported, distributed deep-learning framework that solves problems involving massive amounts of data in a reasonable amount of time
It integrates with Kafka, Hadoop and Spark using an arbitrary number of GPUs or CPUs
Deeplearning4j comes with deep network support through RBM, DBN, Convolution Neural Networks (CNN), Recurrent Neural Networks (RNN), Recursive Neural Tensor Network (RNTN) and Long Short-Term Memory (LTSM)
DL4J performance is equivalent to Caffe, and faster than Tensorflow or Torch
Deeplearning4j has Java, Scala and Python APIs, the latter using Keras.
Caffe2 is the second deep-learning framework to be backed by Facebook after Torch/PyTorch. The main difference seems to be the claim that Caffe2 is more scalable and light-weight. It purports to be deep learning for production environments. Like Caffe and PyTorch, Caffe2 offers a Python API running on a C++ engine. Caffe2 framework offers users to use pre-trained models to build demo applications without extra hassle.
Popularly known for easy training and combination of popular model types across servers, the Microsoft Cognitive Toolkit (earlier known as CNTK-Computational Network Toolkit) The library includes feed-forward DNNs, convolutional nets, and recurrent networks. CNTK offers a Python API over C++ code. It performs efficient Convolution Neural Networks and training for image, speech, and text-based data. Similar to Caffe, it is supported by interfaces such as Python, C++ and the Command Line Interface.
MxNet is a machine-learning framework with APIs in languages like JavaScript, R and Go as well as Python and C++, which has been adopted by Amazon Web Services.
BigDL, a new deep learning framework with a focus on Apache Spark, only works on Intel chips.