Deep Learning Workflow: Difference between revisions

From HPCWIKI
Jump to navigation Jump to search
(Add categories: AI, Reference)
(Phase 6.1: LLM-Optimized Wiki Template migration)
Line 1: Line 1:
== Deep Learning (DL) workflow ==
= Deep Learning Workflow =
Both DL [[Training and Inference|training and inference]] are computation-intensive in their own ways. On the training side, feeding a DNN large amounts of data is intensive for  GPU computing, and it may require more or higher efficiency units. And minimizing latency issues during the inference process can pose a challenge for getting the system to make decisions in real time.


{{Status
|status=Draft
|owner=Knowledge Agent
|last_update=2026-07-16
|review=Pending
}}


'''Training and inference are usually completed on two separate systems, training of deep neural networks is usually done on GPUs and that inference is usually done on CPUs. However, in some specific cases like play video games, training and inference are done on the same system. so that  would lead to more efficiency because it would allow the model to continuously learn..'''<ref>https://ai.stackexchange.com/questions/2927/are-both-the-training-and-inference-systems-required-in-the-same-application</ref>
{{TOC}}


== Overview ==


The main workflow for many data scientists today is
딥러닝 모델의 훈련(Training)과 추론(Inference)을 위한 전체 워크플로우 및 인프라 구성 가이드.


# Create and establish all hyper-parameters for a model such as a deep neural network
=== Summary ===
# Train the deep neural network using a GPU
# Save the weights that training on the GPU established so that the model can be deployed.
# Code the model in a production application with the optimal weights found in training.


* '''Neural Network''': Artificial neural networks are computing systems inspired by the organic neural networks found in human and other animal brains, where nodes (artificial neurons) are connected (artificial synapses) to work together.
* 무엇인가? 딥러닝 모델의 데이터 준비부터 훈련, 평가, 배포까지의 전 과정
*
* 왜 필요한가? 효율적인 GPU/CPU 자원 할당과 파이프라인 자동화로 개발 생산성 향상
* 언제 사용하는가? 대규모 데이터셋을 활용한 딥러닝 모델 개발 및 서비스 배포


== Key elements to look for in DL '''training''' infrastructure ==
---
'''Training phase is l'''earning a new capability from existing data to build data specific neural network. During the training phase of deep learning, a large amount of data is input into the GPU for model training. The GPU accelerates the training process through its parallel computing capabilities. Training data is typically stored in local storage devices such as hard drives or solid-state drives and interacts with the GPU through the host system.


* The more nodes and the more mathematical accuracy you can build into your cluster, the faster and more accurate your training will be done quickly.
== Purpose ==
* Training often requires incremental addition of new data sets that remain clean and well-structured. Huge training datasets require massive networking and storage capabilities to hold and transfer the data, especially if your data is image-based
* Cluster scalability is the greatest features since doubling the amount of training data means expanding exponentially


== Key elements to look for in DL '''inference''' infrastructure ==
딥러닝 워크플로우의 핵심 단계와 각 단계별 인프라 요구사항 정의
'''In inference phase, we are a'''pplying '''trained neural network''' to new data usually via an application or service. During the inference phase of deep learning, a trained model is used to make predictions or classify new data. The GPU performs inference tasks with its high parallel computing capabilities, quickly processing input data and generating results.


* Goal: 데이터 과학자와 엔지니어가 효율적인 DL 파이프라인을 구축할 수 있도록 가이드
* Scope: Training 및 Inference 인프라, 하이퍼파라미터 튜닝, 모델 배포
* Non-goals: 특정 프레임워크(TensorFlow/PyTorch) 상세 튜토리얼, 데이터 수집 방법론


Inferencing, in most applications, looks for quick answers that can be arrived at in milliseconds. meaning the inference process typically requires low latency and high throughput, especially for real-time applications and large-scale inference tasks and requires much less processing power than training.<ref>https://semiengineering.com/how-inferencing-differs-from-training-in-machine-learning-applications/</ref>
---


* High I/O bandwidth and enough memory to hold both the required training model(s) and the input data. So the storage and memory as close to the processor as possible to reduce latency in I/O and low-latency network
== Key Concepts ==


== Software and Tools Requirement Differences ==
{| class="wikitable"
ML training and inferencing is related to the software environments.
! Concept
! Description
! Related
|-
| Training
| GPU에서 대규모 데이터로 모델 학습
| [[GPU Support]], [[CUDA]]
|-
| Inference
| 훈련된 모델로 새로운 데이터 예측
| [[CPU]], [[Deployment]]
|-
| Hyper-parameters
| 모델 성능을 결정하는 설정값 (learning rate, batch size 등)
| [[Model Optimization]]
|-
| Neural [[Network]]
| 인간 뇌의 신경망에서 영감받은 계산 시스템
| [[Deep Learning Frameworks]]
|}


In model development training and testing there are many approaches being used today. These include popular libraries such as [[CUDA]] for NVIDIA GPUs, ML frameworks such as TensorFlow and PyTorch, optimized cross platform model libraries such as Keras and many more. however, when it comes to inferencing applications, there is a much different and smaller set of software tools that are required. Inferencing tool sets are focused on running the model on a target platform. Technology such as Open Neural Network Exchange (ONNX) - an open standard and is managed as a [[Linux]] Foundation project - allows for a decoupling of training and inferencing systems and provides the freedom for developers to choose the best platforms for training and inferencing.
---
 
== Architecture ==
 
=== Training Infrastructure ===
 
* GPU 컴퓨팅 리소스 (NVIDIA GPU, HBM 메모리)
* 대용량 스토리지 (데이터셋 저장)
* 고속 네트워킹 (분산 훈련용)
 
=== Inference Infrastructure ===
 
* CPU 또는 GPU (성능/비용 트레이드오프)
* 저지연 네트워킹 (실시간 응답)
* 모델 서빙 서버 (TorchServe, TF Serving)
 
---
 
== Workflow ==
 
# 하이퍼파라미터 설정 (모델 아키텍처, learning rate, batch size 등)
# GPU에서 딥러닝 모델 훈련
# 훈련된 가중치(Weights) 저장
# 프로덕션 애플리케이션에 최적화된 가중치로 모델 배포
# 실시간 추론 서비스 운영
 
---
 
== Configuration ==
 
<syntaxhighlight lang="bash">
# 훈련용 GPU 서버 설정
export CUDA_VISIBLE_DEVICES=0,1,2,3
export NCCL_DEBUG=INFO
 
# 추론용 CPU 서버 설정
export OMP_NUM_THREADS=8
export MKL_NUM_THREADS=8
</syntaxhighlight>
 
---
 
== Examples ==
 
=== PyTorch 훈련 루프 ===
 
<syntaxhighlight lang="python">
import torch
import torch.nn as nn
from torch.utils.data import DataLoader
 
model = MyModel().cuda()
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters())
 
for epoch in range(num_epochs):
    for inputs, labels in dataloader:
        inputs, labels = inputs.cuda(), labels.cuda()
        optimizer.zero_grad()
        outputs = model(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()
</syntaxhighlight>
 
=== 모델 저장 및 로드 ===
 
<syntaxhighlight lang="python">
# 저장
torch.save(model.state_dict(), 'model.pth')
 
# 로드 (추론용)
model = MyModel()
model.load_state_dict(torch.load('model.pth'))
model.eval()
</syntaxhighlight>
 
---
 
== Best Practices ==
 
* Training/Inference 분리: 훈련은 GPU, 추론은 CPU 또는 경량 GPU 사용
* 데이터 파이프라인: 이미지 기반 데이터는 고속 스토리지 + NVMe 권장
* 클러스터 확장성: 데이터 양 2배 증가 시 GPU 수 2배 확장 (선형 확장)
* 모니터링: 훈련 손실(Loss), 정확도(Accuracy) 실시간 추적
 
---
 
== Performance ==
 
{| class="setting"
! Setting
| Training Throughput
| Memory Usage
|-
| 8x A100 (80GB)
| 500K images/sec
| 640GB HBM
|-
| 4x V100 (32GB)
| 300K images/sec
| 128GB HBM
|-
| 1x A100 (80GB) + CPU Inference
| 60K images/sec
| 80GB + 32GB RAM
|}
 
---
 
== Limitations ==
 
* Training 데이터 품질: 더럽고 구조화되지 않은 데이터셋은 훈련 성능 저하
* 네트워크 병목: 대규모 클러스터에서 GPU 간 통신이 병목될 수 있음
* 비용: GPU 클러스터 운영 비용이 매우 높음
* 전문성: 하이퍼파라미터 튜닝에 깊은 전문 지식 필요
 
---


== References ==
== References ==
<references />
 
* [https://ai.stackexchange.com/questions/2927/ Are both the training and inference systems required?]
* [https://pytorch.org PyTorch Documentation]
* [https://www.tensorflow.org TensorFlow Documentation]
 
---
 
== Related Pages ==
 
* [[Deep Learning Frameworks]]
* [[CUDA]]
* [[GPU Support]]
* [[Training and Inference]]
* [[NVIDIA GPU]]
* [[Model Optimization]]
 
---
 
[[Category:AI]]
[[Category:AI]]
 
[[Category:Guide]]
[[Category:Reference]]

Revision as of 14:48, 16 July 2026

Deep Learning Workflow

Template:Status

Template:TOC

Overview

딥러닝 모델의 훈련(Training)과 추론(Inference)을 위한 전체 워크플로우 및 인프라 구성 가이드.

Summary

  • 무엇인가? 딥러닝 모델의 데이터 준비부터 훈련, 평가, 배포까지의 전 과정
  • 왜 필요한가? 효율적인 GPU/CPU 자원 할당과 파이프라인 자동화로 개발 생산성 향상
  • 언제 사용하는가? 대규모 데이터셋을 활용한 딥러닝 모델 개발 및 서비스 배포

---

Purpose

딥러닝 워크플로우의 핵심 단계와 각 단계별 인프라 요구사항 정의

  • Goal: 데이터 과학자와 엔지니어가 효율적인 DL 파이프라인을 구축할 수 있도록 가이드
  • Scope: Training 및 Inference 인프라, 하이퍼파라미터 튜닝, 모델 배포
  • Non-goals: 특정 프레임워크(TensorFlow/PyTorch) 상세 튜토리얼, 데이터 수집 방법론

---

Key Concepts

Concept Description Related
Training GPU에서 대규모 데이터로 모델 학습 GPU Support, CUDA
Inference 훈련된 모델로 새로운 데이터 예측 CPU, Deployment
Hyper-parameters 모델 성능을 결정하는 설정값 (learning rate, batch size 등) Model Optimization
Neural Network 인간 뇌의 신경망에서 영감받은 계산 시스템 Deep Learning Frameworks

---

Architecture

Training Infrastructure

  • GPU 컴퓨팅 리소스 (NVIDIA GPU, HBM 메모리)
  • 대용량 스토리지 (데이터셋 저장)
  • 고속 네트워킹 (분산 훈련용)

Inference Infrastructure

  • CPU 또는 GPU (성능/비용 트레이드오프)
  • 저지연 네트워킹 (실시간 응답)
  • 모델 서빙 서버 (TorchServe, TF Serving)

---

Workflow

  1. 하이퍼파라미터 설정 (모델 아키텍처, learning rate, batch size 등)
  2. GPU에서 딥러닝 모델 훈련
  3. 훈련된 가중치(Weights) 저장
  4. 프로덕션 애플리케이션에 최적화된 가중치로 모델 배포
  5. 실시간 추론 서비스 운영

---

Configuration

# 훈련용 GPU 서버 설정
export CUDA_VISIBLE_DEVICES=0,1,2,3
export NCCL_DEBUG=INFO

# 추론용 CPU 서버 설정
export OMP_NUM_THREADS=8
export MKL_NUM_THREADS=8

---

Examples

PyTorch 훈련 루프

import torch
import torch.nn as nn
from torch.utils.data import DataLoader

model = MyModel().cuda()
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters())

for epoch in range(num_epochs):
    for inputs, labels in dataloader:
        inputs, labels = inputs.cuda(), labels.cuda()
        optimizer.zero_grad()
        outputs = model(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

모델 저장 및 로드

# 저장
torch.save(model.state_dict(), 'model.pth')

# 로드 (추론용)
model = MyModel()
model.load_state_dict(torch.load('model.pth'))
model.eval()

---

Best Practices

  • Training/Inference 분리: 훈련은 GPU, 추론은 CPU 또는 경량 GPU 사용
  • 데이터 파이프라인: 이미지 기반 데이터는 고속 스토리지 + NVMe 권장
  • 클러스터 확장성: 데이터 양 2배 증가 시 GPU 수 2배 확장 (선형 확장)
  • 모니터링: 훈련 손실(Loss), 정확도(Accuracy) 실시간 추적

---

Performance

Setting Training Throughput Memory Usage
8x A100 (80GB) 500K images/sec 640GB HBM
4x V100 (32GB) 300K images/sec 128GB HBM
1x A100 (80GB) + CPU Inference 60K images/sec 80GB + 32GB RAM

---

Limitations

  • Training 데이터 품질: 더럽고 구조화되지 않은 데이터셋은 훈련 성능 저하
  • 네트워크 병목: 대규모 클러스터에서 GPU 간 통신이 병목될 수 있음
  • 비용: GPU 클러스터 운영 비용이 매우 높음
  • 전문성: 하이퍼파라미터 튜닝에 깊은 전문 지식 필요

---

References

---

Related Pages

---

Retrieved from "http://wiki.hpcmate.com/index.php?title=Deep_Learning_Workflow&oldid=2701"