Deep Learning Frameworks: Difference between revisions

From HPCWIKI
Jump to navigation Jump to search
(Add categories: AI, Hardware, Reference)
(Phase 6.1: LLM-Optimized Wiki Template migration)
Line 1: Line 1:
There have been many Deep Learning (DL) frameworks, like Theano, CNTK, Caffe2, and MXNet. Nowadays, some of them appear to be dead or dying, as just two frameworks heavily dominate the DL scene: Google TensorFlow (TF) and PyTorch from Meta aka FaceBook.
= Deep Learning Frameworks =


However, there is no reason to believe such a duopoly will persist forever.  Cause new DL frameworks are proposed here and there. So it is hard to say which DL framework will be popular in ten years later..., for example, Google has at least two (perhaps more) competing AI teams: Google Brain and DeepMind. Even in the TensorFlow era, DeepMind used their own layer API called Sonnet (instead of the usual Keras)
{{Status
|status=Draft
|owner=Knowledge Agent
|last_update=2026-07-16
|review=Pending
}}


{{TOC}}


As of March'23, someone<ref>https://medium.com/@andreiliphd</ref> said that TensorFlow and PyTorch like a wooden bear and teddy bear. There is so much love in teddy bears. PyTorch has logical components that perfectly fit together to form a product.
== Overview ==


=== Popular Deep Learning Framework ===
딥러닝 모델을 설계, 훈련, 배포하기 위한 소프트웨어 라이브러리 및 프레임워크의 생태계.
{| class="wikitable sortable mw-collapsible"
 
!Framework
=== Summary ===
!Issue
 
!HPCMATE custom build<ref>HPCMATE supports custom build service to optimize binary performance on given system based on customer requirements </ref>
* 무엇인가? TensorFlow, PyTorch, Keras 등 딥러닝 모델 개발을 위한 프레임워크
!Developer
* 왜 필요한가? 복잡한 신경망 연산을 추상화하여 개발자가 모델에 집중할 수 있게 함
!Status as of 2023
* 언제 사용하는가? 이미지 인식, NLP, 추천 시스템 등 모든 딥러닝 프로젝트
!Language
 
!GPU [[Support]]
---
!Distributed Computing
 
!Auto-differentiation
== Purpose ==
!Pre-trained models
 
!Visualization
주요 딥러닝 프레임워크의 특징, 비교, 선택 가이드 제공
!Deployment
 
* Goal: 개발자가 프로젝트에 맞는 프레임워크를 선택할 수 있도록 정보 제공
* Scope: TensorFlow, PyTorch, Keras, MXNet, Caffe, Theano, CNTK 등 주요 프레임워크
* Non-goals: 프레임워크별 상세 API 문서화, 커스텀 프레임워크 개발 가이드
 
---
 
== Key Concepts ==
 
{| class="wikitable"
! Framework
! Developer
! Status
! GPU [[Support]]
! Language
|-
| [[TensorFlow]]
| Google
| Active
| Yes
| Python
|-
| [[PyTorch]]
| Meta (Facebook)
| Active
| Yes
| Python
|-
|-
|[https://www.tensorflow.org/?gclid=EAIaIQobChMItN6Kl9f4_QIV1DMrCh0rZwVcEAAYASAAEgIdKPD_BwE&hl=en TensorFlow]
| Keras
|
| Google (TF 통합)
|Yes
| Merged into TensorFlow
|Google
| Yes
|
| Python
|Python
|Yes
|Yes
|Yes
|Yes
|Yes
|Yes
|-
|-
|[https://pytorch.org PyTorch]
| MXNet
|PyTorch issue list<ref>https://github.com/pytorch/pytorch/issues</ref>
| Apache
|Yes
| Development halted (late 2022)
 
| Yes
|Facebook
| Multiple
|
|-
* Pytorch version 2.0<ref>https://github.com/pytorch/pytorch/releases</ref> released as of 12/2/22
| Caffe
*Facebook use PyTorch to train the model and Caffe2 to deploy it<ref>https://www.codingninjas.com/codestudio/library/pytorch-vs-caffe2</ref>
| Berkeley AI Research
* Caffe2 has been merged into PyTorch as of Apr'2018 <ref>https://synced.medium.com/caffe2-merges-with-pytorch-a89c70ad9eb7</ref>
| Merged into PyTorch (Caffe2)
|Python
| Yes
|Yes
| C++
|Yes
|Yes
|Yes
|Yes
|Yes
|-
|-
|Keras
| Theano
|
| Universite de Montreal
|NA
| Dead (Keras, Lasagne, Blocks 기반)
|Google
| Yes
|Merged into TensorFlow
| Python
|Python
|Yes
|Yes
|Yes
|Yes
|Yes
|Yes
|-
|-
|MXNet
| CNTK
|
| Microsoft
|NA
| No longer actively developed (v2.7)
|Apache
| Yes
|Since late 2022 the code development has mostly halted and community engagement slowed<ref>https://github.com/apache/mxnet/issues/21206</ref>
| Python 3.6
|Multiple
|}
|Yes
 
|Yes
---
|Yes
 
|Yes
== Architecture ==
|Yes
 
|Yes
=== TensorFlow Architecture ===
 
* Graph-based computation (Static computational graph)
* Keras high-level API
* TensorFlow Serving for deployment
 
=== PyTorch Architecture ===
 
* Dynamic computational graph (Define-by-Run)
* TorchScript for deployment
* PyTorch Lightning for structured training
 
---
 
== Workflow ==
 
# 프레임워크 선택 (프로젝트 요구사항 기반)
# 환경 설정 (CUDA, cuDNN, GPU 드라이버)
# 모델 정의 (네트워크 아키텍처)
# 데이터 로딩 및 전처리
# 모델 훈련 (GPU 가속)
# 모델 평가 및 검증
# 모델 저장 및 배포
 
---
 
== Configuration ==
 
<syntaxhighlight lang="bash">
# PyTorch 설치 (CUDA 11.8)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
 
# TensorFlow 설치 (CUDA 12.0)
pip install tensorflow[and-cuda]
</syntaxhighlight>
 
---
 
== Examples ==
 
=== PyTorch 모델 정의 ===
 
<syntaxhighlight lang="python">
import torch
import torch.nn as nn
 
class SimpleNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc = nn.Linear(784, 10)
   
    def forward(self, x):
        return self.fc(x.view(x.size(0), -1))
 
model = SimpleNet()
</syntaxhighlight>
 
=== TensorFlow/Keras 모델 정의 ===
 
<syntaxhighlight lang="python">
import tensorflow as tf
 
model = tf.keras.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dense(10)
])
</syntaxhighlight>
 
---
 
== Best Practices ==
 
* 프로덕션 배포: PyTorch는 TorchScript, TensorFlow는 SavedModel 사용
* GPU 가속: cuDNN 버전 호환성 반드시 확인
* 대규모 분산 훈련: TensorFlow는 tf.distribute, PyTorch는 DDP 사용
* 모델 최적화: ONNX 형식으로 변환하여 다양한 환경에서 배포
 
---
 
== Performance ==
 
{| class="wikitable"
! Framework
| Training Speed
| Inference Speed
| Ecosystem
|-
|-
|Caffe
| TensorFlow
|
| Fast
|NA
| Fast
|Berkeley AI Research
| Mature (TF Serving, TFLite, TF.js)
|Caffe2 is built on Caffe and is merged into Meta’s PyTorch
Yahoo has also integrated Caffe with [[Apache Spark]] to create CaffeOnSpark, which brings deep learning to Hadoop and Spark clusters
|C++
|Yes
|No
|Yes
|Yes
|No
|Yes
|-
|-
|Theano
| PyTorch
|
| Fast
|NA
| Fast
|Université de Montréal
| Growing (TorchServe, TorchScript)
|'''Theano is dead now but''' the other open-source deep libraries which are built on top of Theano are still functioning; these include Keras, Lasagne, and Blocks
|Python
|Yes
|No
|Yes
|Yes
|No
|No
|-
|-
|CNTK
| Keras
|
| Fast (TF 백엔드)
|NA
| Fast (TF 백엔드)
|
| Simple API, limited customization
|CNTK is no longer actively developed, latest version is 2.7 which works with Python 3.6<ref>https://learn.microsoft.com/en-us/cognitive-toolkit/</ref>
|
|
|
|
|
|
|
|}
|}


===Reference===
---
<nowiki><references/></nowiki>
 
[[Category:AI]]
== Limitations ==
 
* TensorFlow: Graph 기반이라 디버깅이 어려움 (TF2에서 개선됨)
* PyTorch: 프로덕션 배포 생태계가 TensorFlow보다 성숙도 낮음
* Theano/CNTK: 더 이상 개발되지 않음 - 新项目 권장 안함
* MXNet: 커뮤니티 활동이 크게 감소
 
---
 
== References ==
 
* [https://www.tensorflow.org/ TensorFlow Official Site]
* [https://pytorch.org PyTorch Official Site]
* [https://keras.io Keras Official Site]
* [https://mxnet.apache.org MXNet Official Site]
* [https://caffe.berkeleyvision.org Caffe Official Site]
 
---
 
== Related Pages ==
 
* [[CUDA]]
* [[cuDNN]]
* [[Deep Learning Workflow]]
* [[GPU Support]]
* [[Training and Inference]]
* [[NVIDIA GPU]]


[[Category:Hardware]]
---


[[Category:Reference]]
[[Category:AI]]
[[Category:Guide]]

Revision as of 14:47, 16 July 2026

Deep Learning Frameworks

Template:Status

Template:TOC

Overview

딥러닝 모델을 설계, 훈련, 배포하기 위한 소프트웨어 라이브러리 및 프레임워크의 생태계.

Summary

  • 무엇인가? TensorFlow, PyTorch, Keras 등 딥러닝 모델 개발을 위한 프레임워크
  • 왜 필요한가? 복잡한 신경망 연산을 추상화하여 개발자가 모델에 집중할 수 있게 함
  • 언제 사용하는가? 이미지 인식, NLP, 추천 시스템 등 모든 딥러닝 프로젝트

---

Purpose

주요 딥러닝 프레임워크의 특징, 비교, 선택 가이드 제공

  • Goal: 개발자가 프로젝트에 맞는 프레임워크를 선택할 수 있도록 정보 제공
  • Scope: TensorFlow, PyTorch, Keras, MXNet, Caffe, Theano, CNTK 등 주요 프레임워크
  • Non-goals: 프레임워크별 상세 API 문서화, 커스텀 프레임워크 개발 가이드

---

Key Concepts

Framework Developer Status GPU Support Language
TensorFlow Google Active Yes Python
PyTorch Meta (Facebook) Active Yes Python
Keras Google (TF 통합) Merged into TensorFlow Yes Python
MXNet Apache Development halted (late 2022) Yes Multiple
Caffe Berkeley AI Research Merged into PyTorch (Caffe2) Yes C++
Theano Universite de Montreal Dead (Keras, Lasagne, Blocks 기반) Yes Python
CNTK Microsoft No longer actively developed (v2.7) Yes Python 3.6

---

Architecture

TensorFlow Architecture

  • Graph-based computation (Static computational graph)
  • Keras high-level API
  • TensorFlow Serving for deployment

PyTorch Architecture

  • Dynamic computational graph (Define-by-Run)
  • TorchScript for deployment
  • PyTorch Lightning for structured training

---

Workflow

  1. 프레임워크 선택 (프로젝트 요구사항 기반)
  2. 환경 설정 (CUDA, cuDNN, GPU 드라이버)
  3. 모델 정의 (네트워크 아키텍처)
  4. 데이터 로딩 및 전처리
  5. 모델 훈련 (GPU 가속)
  6. 모델 평가 및 검증
  7. 모델 저장 및 배포

---

Configuration

# PyTorch 설치 (CUDA 11.8)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

# TensorFlow 설치 (CUDA 12.0)
pip install tensorflow[and-cuda]

---

Examples

PyTorch 모델 정의

import torch
import torch.nn as nn

class SimpleNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc = nn.Linear(784, 10)
    
    def forward(self, x):
        return self.fc(x.view(x.size(0), -1))

model = SimpleNet()

TensorFlow/Keras 모델 정의

import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dense(10)
])

---

Best Practices

  • 프로덕션 배포: PyTorch는 TorchScript, TensorFlow는 SavedModel 사용
  • GPU 가속: cuDNN 버전 호환성 반드시 확인
  • 대규모 분산 훈련: TensorFlow는 tf.distribute, PyTorch는 DDP 사용
  • 모델 최적화: ONNX 형식으로 변환하여 다양한 환경에서 배포

---

Performance

Framework Training Speed Inference Speed Ecosystem
TensorFlow Fast Fast Mature (TF Serving, TFLite, TF.js)
PyTorch Fast Fast Growing (TorchServe, TorchScript)
Keras Fast (TF 백엔드) Fast (TF 백엔드) Simple API, limited customization

---

Limitations

  • TensorFlow: Graph 기반이라 디버깅이 어려움 (TF2에서 개선됨)
  • PyTorch: 프로덕션 배포 생태계가 TensorFlow보다 성숙도 낮음
  • Theano/CNTK: 더 이상 개발되지 않음 - 新项目 권장 안함
  • MXNet: 커뮤니티 활동이 크게 감소

---

References

---

Related Pages

---

Retrieved from "http://wiki.hpcmate.com/index.php?title=Deep_Learning_Frameworks&oldid=2700"