How has the landscape changed for the leading deep learning frameworks in the past six months?
To answer that question, I looked at the number of job listings on Indeed, Monster, LinkedIn, and SimplyHired. I also evaluated changes in Google search volume, GitHub activity, Medium articles, ArXiv articles, and Quora topic followers. Overall, these sources paint a comprehensive picture of growth in demand, usage, and interest.
Weâve recently seen several important developments in the TensorFlow and PyTorch frameworks.
PyTorch v1.0 was pre-released in October 2018, at the same time fastai v1.0 was released. Both releases marked major milestones in the maturity of the frameworks.
TensorFlow 2.0 alpha was released March 4, 2019. It added new features and an improved user experience. It more tightly integrates Keras as its high-level API, too.
In this article, I include Keras and fastai in the comparisons because of their tight integrations with TensorFlow and PyTorch. They also provide scale for evaluating TensorFlow and PyTorch.
I wonât be exploring other deep learning frameworks in this article. I expect I will receive feedback that Caffe, Theano, MXNET, CNTK, DeepLearning4J, or Chainer deserve to be discussed. While these frameworks each have their virtues, none appear to be on a growth trajectory likely to put them near TensorFlow or PyTorch. Nor are they tightly coupled with either of those frameworks.
Searches were performed on March 20â21, 2019. Source data is in this Google Sheet.
I used the plotly data visualization library to explore popularity. For the interactive plotly charts, see my Kaggle Kernel here.
Letâs look at the results in each category.
To determine which deep learning libraries are in demand in todayâs job market I searched job listings on Indeed, LinkedIn, Monster, and SimplyHired.
I searched with the term machine learning, followed by the library name. So TensorFlow was evaluated with machine learning TensorFlow. This method was used for historical comparison reasons. Searching without machine learning didnât yield appreciably different results. The search region was the USA.
I subtracted the number of listings six months ago from the number of listings in March 2019. Hereâs what I found:
TensorFlow had a slightly larger increase in listings than PyTorch. Keras also saw listings growth â about half as much as TensorFlow. Fastai still isnât showing in hardly any job listings.
Note that PyTorch saw a larger number of additional listings than TensorFlow on all job search sites other than LinkedIn. Also note that in absolute terms, TensorFlow appears in nearly three times the number of job listings as PyTorch or Keras.
Web searches on the largest search engine are a gauge of popularity. I looked at search history in Google Trends over the past year. I searched for worldwide interest in the Machine Learning and Artificial Intelligence category_._ Google doesnât provide absolute search numbers, but it does provide relative figures.
I took the average interest score of the past six months and the compared it to the average interest score for the prior six months.
In the past six months, the relative search volume for TensorFlow has decreased, while the relative search volume for PyTorch has grown.
The chart from Google directly below shows search interest over the past year.
Medium is a popular location for data science articles and tutorials. I hope youâre enjoying it! đ
I used Google site search of Medium.com over the past six months and found TensorFlow and Keras had similar numbers of articles published. PyTorch had relatively few.
As high level APIs, Keras and fastai are popular with new deep learning practitioners. Medium has many tutorials showing how to use these frameworks.
arXiv is the online repository where most scholarly deep learning articles are published. I searched for new articles mentioning each framework on arXiv using Google site search results for the past six months.
TensorFlow had the most new article appearances by a good margin.
Recent activity on GitHub is another indicator of framework popularity. I broke out stars, forks, watchers, and contributors in the charts below.
TensorFlow had the most GitHub activity in each category. However, PyTorch was quite close in terms of growth in watchers and contributors. Also, Fastai saw many new contributors.
Some contributors to Keras are no doubt working on it in the TensorFlow library. Itâs worth noting that both TensorFlow and Keras are open source products spearheaded by Googlers.
I added the number of new Quora topic followers to the mix â a new category that I didnât have the data for previously.
TensorFlow added the most new topic followers over the past six months. PyTorch and Keras each added far fewer.
Once I had all the data, I consolidated it into one metric.
Hereâs how I created the growth score:
4. Multiplied weighted scores by 100 for comprehensibility.
5. Summed category scores for each framework into a single growth score.
Job listings make up a little over a third of the total score. As the cliche goes, money talks. đ” This split seemed like an appropriate balance of the various categories. Unlike my 2018 power score analysis, I didnât include KDNuggets usage survey (no new data) or books (not many published in six months).
Here are the changes in tabular form.
Here are the category and final scores.
Here are the final growth scores.
TensorFlow is both the most in demand framework and the fastest growing. Itâs not going anywhere anytime soon. đPyTorch is growing rapidly, too. Its large increase in job listings is evidence of its increased usage and demand. Keras has grown a good bit in the past six months, also. Finally, fastai has grown from a low baseline. Itâs worth remembering that itâs the youngest of the lot.
Both TensorFlow and PyTorch are good frameworks to learn.
If youâre looking to learn TensorFlow, I suggest you start with Keras. I recommend Cholletâs Deep Learning with Python and Dan Beckerâs DataCamp course on Keras. Tensorflow 2.0 is using Keras as its high-level API through tf.keras. Hereâs a quick getting started intro to TensorFlow 2.0 by Chollet.
If youâre looking to learn PyTorch, I suggest you start with fast.aiâs MOOC Practical Deep Learning for Coders, v3. Youâll learn deep learning fundamentals, fastai, and PyTorch basics.
Whatâs ahead for TensorFlow and PyTorch?
Iâve consistently heard that folks enjoy using PyTorch more than TensorFlow. PyTorch is more pythonic and has a more consistent API. It also has native ONNX model exports, which can be used to speed up inference. Also, PyTorch shares many commands with numpy, which reduces the barrier to learning it.
However, TensorFlow 2.0 is all about improved UX, as Googleâs Chief Decision Intelligence Engineer, Cassie Kozyrkov, explains here. TensorFlow will now have a more straightforward API, a streamlined Keras integration, and an eager execution option. These changes, and TensorFlowâs broad adoption, should help the framework remain popular for years to come.
TensorFlow recently announced another exciting plan: the development of Swift for TensorFlow. Swift is a programming language originally built by Apple. Swift has a number of advantages over Python in terms of execution and development speed. Fast.ai will be using Swift for TensorFlow in part of its advanced MOOC â see fast.ai cofounder Jeremy Howardâs post on the topic here. The language probably wonât be ready for prime time for a year or two, but it could be an improvement over current deep learning frameworks.
Collaboration and cross-pollination among languages and frameworks is definitely happening. đ đ·
Another advancement that will affect deep learning frameworks is quantum computing. A usable quantum computer is likely a few years off, but Google, IBM, Microsoft, and others are thinking about how to integrate quantum computing with deep learning. Frameworks will need to be adapted to work with this new technology.
Youâve seen that both TensorFlow and PyTorch are growing. Both now have nice high-level APIs â tf.keras and fastai â that have lowered the barriers to getting started with deep learning. Youâve also heard a bit about recent developments and future directions.
To play with the charts in this article interactively or fork the Jupyter Notebook, please head to my Kaggle Kernel.
I hope youâve found this comparison helpful. If you have, please share it on your favorite social media channel so others can find it, too. đ
I write about deep learning, DevOps, data science, and other tech topics. If any of those are of interest to you, check them out and follow me here.
To make sure you donât miss out on great content, join my mailing list below.
30s ad
Criando Modelos de Machine Learning
â http://academy.learnstartup.net/p/Hye8xKzKW
The Complete TensorFlow Masterclass: Machine Learning Models
â http://academy.learnstartup.net/p/8vTep-sSu
Data Science and Machine Learning for Managers and MBAs
â http://academy.learnstartup.net/p/HyOY4p4qW
Machine Learning Masterclass: From Beginner to Advanced
â http://academy.learnstartup.net/p/s0Aen59fj
Machine Learning and Data Science Essentials with Python & R
â http://academy.learnstartup.net/p/yxdg4LEhP
â Python Tutorials for Beginners - Learn Python Online
â What is Python and Why You Must Learn It in [2019]
â Learn Python in 12 Hours | Python Tutorial For Beginners
â Complete Python Tutorial for Beginners (2019)
â Python Programming Tutorial | Full Python Course for Beginners 2019