Deep Learning Frameworks: Difference between revisions

From HPCWIKI
Jump to navigation Jump to search
No edit summary
 
(17 intermediate revisions by the same user not shown)
Line 1: Line 1:
There have been many Deep Learning (DL) frameworks, like Theano, CNTK, Caffe2, and MXNet. Nowadays, they appear to be dead or dying, as just two frameworks heavily dominate the DL scene: Google TensorFlow (TF), which includes Keras and PyTorch from Meta aka FaceBook.
There have been many Deep Learning (DL) frameworks, like Theano, CNTK, Caffe2, and MXNet. Nowadays, some of them appear to be dead or dying, as just two frameworks heavily dominate the DL scene: Google TensorFlow (TF) and PyTorch from Meta aka FaceBook.


However, there is no reason to believe such a duopoly will persist forever. All the time, new DL frameworks are proposed. We have no idea which DL framework will be popular in, say, ten years.
However, there is no reason to believe such a duopoly will persist forever. Cause new DL frameworks are proposed here and there. So it is hard to say which DL framework will be popular in ten years later..., for example, Google has at least two (perhaps more) competing AI teams: Google Brain and DeepMind. Even in the TensorFlow era, DeepMind used their own layer API called Sonnet (instead of the usual Keras)


Google has at least two (perhaps more) competing AI teams: Google Brain and DeepMind. Even in the TensorFlow era, DeepMind used their own layer API called Sonnet (instead of the usual Keras)
 
As of March'23, someone<ref>https://medium.com/@andreiliphd</ref> said that TensorFlow and PyTorch like a wooden bear and teddy bear. There is so much love in teddy bears. PyTorch has logical components that perfectly fit together to form a product.


=== Popular Deep Learning Framework ===
=== Popular Deep Learning Framework ===
{| class="wikitable"
{| class="wikitable sortable mw-collapsible"
!Framework
!Framework
!Issue
!HPCMATE custom build<ref>HPCMATE supports custom build service to optimize binary performance on given system based on customer requirements </ref>
!Developer
!Developer
!Status as of 2023
!Language
!Language
!GPU Support
!GPU [[Support]]
!Distributed Computing
!Distributed Computing
!Auto-differentiation
!Auto-differentiation
Line 17: Line 21:
!Deployment
!Deployment
|-
|-
|TensorFlow
|[https://www.tensorflow.org/?gclid=EAIaIQobChMItN6Kl9f4_QIV1DMrCh0rZwVcEAAYASAAEgIdKPD_BwE&hl=en TensorFlow]
|
|Yes
|Google
|Google
|
|Python
|Python
|Yes
|Yes
Line 27: Line 34:
|Yes
|Yes
|-
|-
|PyTorch
|[https://pytorch.org PyTorch]
|PyTorch issue list<ref>https://github.com/pytorch/pytorch/issues</ref>
|Yes
 
|Facebook
|Facebook
|
* Pytorch version 2.0<ref>https://github.com/pytorch/pytorch/releases</ref> released as of 12/2/22
*Facebook use PyTorch to train the model and Caffe2 to deploy it<ref>https://www.codingninjas.com/codestudio/library/pytorch-vs-caffe2</ref>
* Caffe2 has been merged into PyTorch as of Apr'2018 <ref>https://synced.medium.com/caffe2-merges-with-pytorch-a89c70ad9eb7</ref>
|Python
|Python
|Yes
|Yes
Line 38: Line 52:
|-
|-
|Keras
|Keras
|
|NA
|Google
|Google
|Merged into TensorFlow
|Python
|Python
|Yes
|Yes
Line 48: Line 65:
|-
|-
|MXNet
|MXNet
|
|NA
|Apache
|Apache
|Since late 2022 the code development has mostly halted and community engagement slowed<ref>https://github.com/apache/mxnet/issues/21206</ref>
|Multiple
|Multiple
|Yes
|Yes
Line 58: Line 78:
|-
|-
|Caffe
|Caffe
|
|NA
|Berkeley AI Research
|Berkeley AI Research
|Caffe2 is built on Caffe and is merged into Meta’s PyTorch
Yahoo has also integrated Caffe with Apache Spark to create CaffeOnSpark, which brings deep learning to Hadoop and Spark clusters
|C++
|C++
|Yes
|Yes
Line 68: Line 92:
|-
|-
|Theano
|Theano
|
|NA
|Université de Montréal
|Université de Montréal
|'''Theano is dead now but''' the other open-source deep libraries which are built on top of Theano are still functioning; these include Keras, Lasagne, and Blocks
|Python
|Python
|Yes
|Yes
Line 77: Line 104:
|No
|No
|-
|-
|Torch
|CNTK
|Facebook AI Research
|
|Lua
|NA
|Yes
|
|No
|CNTK is no longer actively developed, latest version is 2.7 which works with Python 3.6<ref>https://learn.microsoft.com/en-us/cognitive-toolkit/</ref>
|Yes
|
|Yes
|
|No
|
|No
|
|
|
|
|}
|}


===Reference===
===Reference===
<reference/>
<nowiki><references/></nowiki>

Latest revision as of 12:03, 29 October 2023

There have been many Deep Learning (DL) frameworks, like Theano, CNTK, Caffe2, and MXNet. Nowadays, some of them appear to be dead or dying, as just two frameworks heavily dominate the DL scene: Google TensorFlow (TF) and PyTorch from Meta aka FaceBook.

However, there is no reason to believe such a duopoly will persist forever. Cause new DL frameworks are proposed here and there. So it is hard to say which DL framework will be popular in ten years later..., for example, Google has at least two (perhaps more) competing AI teams: Google Brain and DeepMind. Even in the TensorFlow era, DeepMind used their own layer API called Sonnet (instead of the usual Keras)


As of March'23, someone[1] said that TensorFlow and PyTorch like a wooden bear and teddy bear. There is so much love in teddy bears. PyTorch has logical components that perfectly fit together to form a product.

Popular Deep Learning Framework

Framework Issue HPCMATE custom build[2] Developer Status as of 2023 Language GPU Support Distributed Computing Auto-differentiation Pre-trained models Visualization Deployment
TensorFlow Yes Google Python Yes Yes Yes Yes Yes Yes
PyTorch PyTorch issue list[3] Yes Facebook
  • Pytorch version 2.0[4] released as of 12/2/22
  • Facebook use PyTorch to train the model and Caffe2 to deploy it[5]
  • Caffe2 has been merged into PyTorch as of Apr'2018 [6]
 Python Yes Yes Yes Yes Yes Yes
Keras NA Google Merged into TensorFlow Python Yes Yes Yes Yes Yes Yes
MXNet NA Apache Since late 2022 the code development has mostly halted and community engagement slowed[7] Multiple Yes Yes Yes Yes Yes Yes
Caffe NA Berkeley AI Research Caffe2 is built on Caffe and is merged into Meta’s PyTorch

Yahoo has also integrated Caffe with Apache Spark to create CaffeOnSpark, which brings deep learning to Hadoop and Spark clusters

C++ Yes No Yes Yes No Yes
Theano NA Université de Montréal Theano is dead now but the other open-source deep libraries which are built on top of Theano are still functioning; these include Keras, Lasagne, and Blocks Python Yes No Yes Yes No No
CNTK NA CNTK is no longer actively developed, latest version is 2.7 which works with Python 3.6[8]

Reference

<references/>

  1. ↑ https://medium.com/@andreiliphd
  2. ↑ HPCMATE supports custom build service to optimize binary performance on given system based on customer requirements
  3. ↑ https://github.com/pytorch/pytorch/issues
  4. ↑ https://github.com/pytorch/pytorch/releases
  5. ↑ https://www.codingninjas.com/codestudio/library/pytorch-vs-caffe2
  6. ↑ https://synced.medium.com/caffe2-merges-with-pytorch-a89c70ad9eb7
  7. ↑ https://github.com/apache/mxnet/issues/21206
  8. ↑ https://learn.microsoft.com/en-us/cognitive-toolkit/
Retrieved from "http://wiki.hpcmate.com/index.php?title=Deep_Learning_Frameworks&oldid=1376"