Most recent update: July 15, 2021
You may have read about some of the recent impressive advances in deep-learning applications, and are now eager to build your own neural networks. But which of the many libraries should you use? In this article, we present PyTorch and TensorFlow, the two most commonly used frameworks for deep-learning.
Read on to find out which one offers the best conditions for you to implement your projects.
Recap: Defining deep-learning
Let’s briefly recall what we mean by the term deep-learning. It refers to a group of algorithms from the large family of machine-learning models. Its name comes from the deep interconnected networks, called neural nets, which are the building blocks for deep-learning models.
The recent advances in computational resources and access to large data collections have facilitated a surge in deep-learning applications in many areas, from machine translation to automated driving. If you have a complex task that conventional machine-learning algorithms find hard to solve, chances are that a neural network will improve the performance — provided that you have the data to train it.
PyTorch vs. TensorFlow: The Key Facts
PyTorch was released in 2016 by Facebook’s AI Research lab. As the name implies, it is primarily meant to be used in Python, but it has a C++ interface, too (so it’s available to other languages). Python enthusiasts love it for its imperative programming style, which they see as more “pythonic” than that of other, more declarative frameworks. Given its pythonic nature, PyTorch fits smoothly into the Python machine learning ecosystem.
TensorFlow, on the other hand, has interfaces in many programming languages. But the high-level Keras API for TensorFlow in Python has proven so successful with deep-learning practitioners that the newest TensorFlow version integrates it by default.
The Keras interface offers ready-made building blocks which significantly improve the speed at which even newcomers can implement deep-learning architectures. TensorFlow, which is named after the high-dimensional data frames that “flow” through a neural network, was developed at Google Brain and has been around a little bit longer than PyTorch, since 2015.
A Deeper Dive
Below, we get into the nitty-gritty of PyTorch and TensorFlow and cover their main differences.
Let’s say you have successfully trained your neural network. How do you make it available to other people? The migration of your model to production is referred to as deployment.
Until recently, PyTorch did not have a comparable set of features. But in March 2020, Facebook announced the release of TorchServe, a PyTorch model-serving library. Much like its competitor TensorFlow Serving, TorchServe offers various features such as support for deploying multiple models and exposing RESTful endpoints for integration with your apps. In so doing, PyTorch has now eliminated the need for developers to write custom code.
Today, PyTorch boasts a rich selection of fully-fledged deployment tools. Apart from TorchServe, you can use TorchScript to enable models to be serialized and used in non-Python environments, and PyTorch Mobile to deploy deep-learning models on mobile devices.
We have seen that both PyTorch and TensorFlow are increasingly looking alike, as each framework integrates new functionality implemented by the other. But when it comes to the libraries’ user bases, the divide between the two is still very much alive.
PyTorch has long been the preferred deep-learning library for researchers, while TensorFlow is much more widely used in production.
PyTorch’s ease of use makes it convenient for fast, hacky solutions and smaller-scale models. But TensorFlow’s extensions for deployment on both servers and mobile devices, combined with the lack of Python overhead, make this the preferred option for companies that work with deep-learning models.
Another distinguishing feature between PyTorch and TensorFlow lies in how readily PyTorch facilitates parallelism. In PyTorch, all it takes to enable training across multiple GPUs is to add a single line of code that wraps a model inside PyTorch’s DataParallel class. Of course, TensorFlow also allows for distributed training, but the solution requires writing more code.
So… Which Library Should You Use?
So, for your first deep-learning model, should you use PyTorch or TensorFlow? There is no definitive answer to this question. As a rule of thumb, if you’re already a Python programmer, or want to build a model just for yourself or for use in research, you can follow the general recommendation and use PyTorch.
On the other hand, TensorFlow has long been geared towards production and many developers prefer it for this exact reason. PyTorch is certainly catching up in this regard, and a few years down the line we can expect PyTorch and TensorFlow to continue becoming increasingly more similar to each other.
Developers for both libraries have continually been integrating popular features from their competitor, resulting in a process of gradual convergence. For instance, TensorBoard, the tool that allows you to observe the behavior of your training parameters over time, used to be a feature exclusive only to TensorFlow. Now, PyTorch also offers native support for TensorBoard. Additionally, PyTorch recently released PyTorch Lightning, a high-level interface to PyTorch — just like Keras is to TensorFlow.
In either case, we encourage you to try and understand as much as possible about your neural networks regardless of which framework you choose. In the end, it will just be a tool to help you build your models. Happy learning!
Both TensorFlow and PyTorch have their advantages as starting platforms to get into neural network programming. Traditionally, researchers and Python enthusiasts have preferred PyTorch, while TensorFlow has long been the favored option for building large scale deep-learning models for use in production.
However, the latest releases have seen the two libraries converge towards a more similar profile. As long as you stick to either TensorFlow or PyTorch as your deep-learning framework, you can’t go wrong.
Looking to get started with deep-learning? Enroll in our AI Nanodegree.