NVIDIA GPU: Difference between revisions

From HPCWIKI
Jump to navigation Jump to search
(Phase 0.4: Expand NVIDIA GPU)
 
(37 intermediate revisions by 2 users not shown)
Line 1: Line 1:
= NVIDIA GPU =


=== 7GPU Tenser performance notes for RTX 4090 ===
{{Status
According to [https://www.reddit.com/r/hardware/comments/xt38d9/comment/iqo2bi4/?utm_source=reddit&utm_medium=web2x&context=3 this thread] '''NVIDIA''' '''looks cut the tensor FP16 &  TF32 operation rate in half, resulting in a 4090 with even lower FP16 & TF32 performance than the 4080 16GB.''' This may have been done to prevent the 4090 from cannibalizing the Quadro/Tesla sales. So if you are choosing GPUs, you can choose the 4090 for memory, but lower tensor performance than the 4080 16GB. eventhough 4090 has more than twice the ray tracing performance of the 4080 12GB.
|status=Draft
{| class="wikitable sortable"
|owner=Knowledge Agent
!
|last_update=2026-07-15
!RTX 4090
|review=Pending
!RTX 4080 16GB
}}
!RTX 4080 12GB
 
!RTX 3090 Ti
{{TOC}}
|-
 
|non-tensor FP32 tflops
== Overview ==
|82.6 (206%)
 
|48.7 (122%)
NVIDIA GPU(Graphics Processing Unit)는 병렬 컴퓨팅을 위한 그래픽 처리 장치입니다. 게이밍부터 HPC, AI/ML까지 광범위하게 사용됩니다. Ampere, Hopper, Blackwell 아키텍처를 통해 지속적인 성능 향상을 이루고 있습니다.
|40.1 (100%)
 
|40 (100%)
=== Summary ===
 
* 무엇인가? — NVIDIA의 그래픽 및 컴퓨팅 처리 장치
* 왜 필요한가? — 병렬 컴퓨팅, AI/ML 훈련/추론, 그래픽 렌더링
* 언제 사용하는가? — AI 서버, HPC 클러스터, 워크스테이션
 
 
== Purpose ==
 
이 문서가 존재하는 이유
 
* Goal: NVIDIA GPU 제품군, 아키텍처, MIG, 드라이버, 모니터링 방법 제공
* Scope: GPU 라인업, 아키텍처 비교, MIG, 드라이버 설치, nvidia-smi
* Non-goals: CUDA 프로그래밍, 특정 애플리케이션 설정은 별도 문서
 
 
== Key Concepts ==
 
{| class="wikitable"
! Concept
! Description
! Related
|-
|-
|non-tensor FP16 tflops
| Ampere
|82.6 (206%)
| SM80 아키텍처 — A100/H100 기반
|48.7 (122%)
| [[NVIDIA GPU]]
|40.1 (100%)
|40 (100%)
|-
|-
|Tensor Cores
| Hopper
|512 (152%)
| SM90 아키텍처 — H200/B200 기반
|304 (90%)
| [[NVIDIA GPU]]
|240 (71%)
|336 (100%)
|-
|-
|Optical flow TOPS
| Blackwell
|305 (242%)
| SM100 아키텍처 — B200 기반
|305 (242%)
| [[NVIDIA GPU]]
|305 (242%)
|126 (100%)
|-
|-
|tensor FP16 w/ FP32 accumulate TFLOPS **
| MIG
|165.2 (207%)
| Multi-Instance GPU — GPU 분할
|194.9 (244%)
| [[MIG]]
|160.4 (200%)
|80 (100%)
|-
|-
|tensor TF32 TFLOPS **
| CUDA
|82.6 (207%)
| NVIDIA 병렬 컴퓨팅 플랫폼
|97.5 (244%)
| [[CUDA]]
|80.2 (200%)
|40 (100%)
|-
|-
|Ray trace Cores
| Tensor Core
|128 (152%)
| AI 연산 전용 코어
|76 (90%)
| [[NVIDIA GPU]]
|60 (71%)
|84 (100%)
|-
|-
|Ray trace TFLOPS
| NVLink
|191 (245%)
| GPU 간 고속 연결
|112.7 (144%)
| [[NVIDIA GPU]]
|92.7 (119%)
|78.1 (100%)
|-
|-
|POWER (W)
| nvidia-smi
|450 (100%)
| GPU 모니터링/관리 도구
|320 (71%)
| [[NVIDIA GPU]]
|285 (63%)
|450 (100%)
|}
|}


=== NVIDIA GPU Architecture ===


nvcc sm flags and what they’re used for: When compiling with NVCC<ref>https://arnon.dk/matching-sm-architectures-arch-and-gencode-for-various-nvidia-cards/</ref>,
== Architecture ==
 
NVIDIA GPU 아키텍처 진화:
 
<syntaxhighlight lang="mermaid">
graph LR
    A[Turing] --> B[Ampere]
    B --> C[Hopper]
    C --> D[Blackwell]
    B --> E[A100]
    B --> F[A30]
    C --> G[H100]
    C --> H[H200]
    D --> I[B200]
    D --> J[RTX PRO 6000]
    D --> K[RTX PRO 5000]
</syntaxhighlight>
 


* the arch flag (‘'''<code>-arch</code>'''‘) specifies the name of the NVIDIA GPU architecture that the [[CUDA]] files will be compiled for.
== Workflow ==
* Gencodes (‘'''<code>-gencode</code>'''‘) allows for more PTX generations and can be repeated many times for different architectures.


Matching CUDA arch and CUDA gencode for various NVIDIA architectures
{| class="wikitable"
{| class="wikitable sortable"
! Stage
!Series
! Input
!Architecture
! Output
(--arch)
!CUDA gencode
(--sm)
!Compute Capability
!Notable Models
!Supported CUDA version
!Key Features
|-
|-
|Tesla
| Install Driver
|Tesla
| NVIDIA driver package
|
| nvidia-smi
|1.0, 1.1, 2.0, 2.1
|C1060, M2050, K80, P100, V100, A100
|
|First dedicated GPGPU series
|-
|-
|Fermi
| Check Status
|Fermi
| nvidia-smi
|sm_20
| GPU info
|3.0, 3.1
|-
|GTX 400, GTX 500, Tesla 20-series, Quadro 4000/5000
| Enable MIG
|CUDA 3.2 until CUDA 8
| nvidia-smi -mig 1
|First to feature CUDA cores and [[support]] for ECC memory
| MIG Mode
|-
| Create Instance
| nvidia-smi -cgi
| GPU Instances
|-
| Monitor
| nvidia-smi / dcgmi
| Metrics
|}


* '''SM20 or SM_20, compute_30''' –  GeForce 400, 500, 600, GT-630. '''''Completely dropped from CUDA 10 onwards.'''''


|-
== Detailed Explanation ==
|Kepler
|Kepler
|sm_30
sm_35,
sm_37
|3.2, 3.5, 3.7
|GTX 600, GTX 700, Tesla K-series, Quadro K-series
|CUDA 5 until CUDA 10
|First to feature Dynamic Parallelism and Hyper-Q


* '''''SM30 or <code>SM_30, compute_30</code> –'''''  '''''Kepler architecture (e.g. generic Kepler, GeForce 700, GT-730).'''''  '''''Adds support for unified memory programmingCompletely dropped from CUDA 11 onwards.'''''
=== GPU 제품군 ===
* '''''SM35 or <code>SM_35, compute_35</code> –'''''  '''''Tesla K40.''''' '''''Adds support for dynamic parallelism.'''''  '''''Deprecated from CUDA 11, will be dropped in future versions.'''''
* '''''SM37 or <code>SM_37, compute_37</code> –'''''  '''''Tesla K80.''''' '''''Adds a few more registers.'''''  '''''Deprecated from CUDA 11, will be dropped in future versions, strongly suggest replacing with a 32GB [[PCIe]] Tesla V100.'''''
|-
|Maxwell
|Maxwell
|sm_50,
sm_52,
sm_53
|5.0, 5.2
|GTX 900, GTX 1000, Quadro M-series
|CUDA 6 until CUDA 11
|First to support VR and 4K displays


* '''''SM50 or <code>SM_50, compute_50</code> –'''''  '''''Tesla/Quadro M series.'''''  '''''Deprecated from CUDA 11, will be dropped in future versions, strongly suggest replacing with a Quadro RTX 4000 or A6000.'''''
{| class="wikitable"
* '''''SM52 or <code>SM_52, compute_52</code> –'''''  '''''Quadro M6000 , GeForce 900, GTX-970, GTX-980, GTX Titan X.'''''
! 제품군
* '''''SM53 or <code>SM_53, compute_53</code> –'''''  '''''Tegra (Jetson) TX1 / Tegra X1, Drive CX, Drive PX, Jetson Nano.'''''
! 아키텍처
! 주요 모델
! 용도
|-
|-
|Pascal
| Data Center
|Pascal
| Ampere/Hopper/Blackwell
|sm_60,
| A100, H100, H200, B200
sm_61,
| AI/ML, HPC
sm_62
|6.0, 6.1, 6.2
|GTX 1000, Quadro P-series
|'''CUDA 8 and later'''
|First to support simultaneous multi-projection
 
* '''SM60 or <code>SM_60, compute_60</code>''' –  Quadro GP100, Tesla P100, DGX-1 (Generic Pascal)
* '''SM61 or <code>SM_61, compute_61</code>'''–  GTX 1080, GTX 1070, GTX 1060, GTX 1050, GTX 1030 (GP108), GT 1010 (GP108) Titan Xp, Tesla P40, Tesla P4, Discrete GPU on the NVIDIA Drive PX2
* '''SM62 or <code>SM_62, compute_62</code>''' –  Integrated GPU on the NVIDIA Drive PX2, Tegra (Jetson) TX2
|-
|-
|Volta
| RTX Workstation
|Volta
| Ada/Blackwell
|sm_70,  
| RTX 6000 Ada, RTX PRO 6000
sm_72
| 워크스테이션, VFX
(Xavier)
|7.0, 7.2, 7.5
|Titan V, Tesla V100, Quadro GV100
|CUDA 9 and later
|First to feature Tensor Cores and NVLink 2.0
 
* '''SM70 or <code>SM_70, compute_70</code>''' –  DGX-1 with Volta, Tesla V100, GTX 1180 (GV104), Titan V, Quadro GV100
* '''SM72 or <code>SM_72, compute_72</code>''' – Jetson AGX Xavier, Drive AGX Pegasus, Xavier NX
|-
|-
|Turing
| GeForce
|Turing
| Ada
|sm_75
| RTX 4090, RTX 4080
|7.5, 7.6
| 게이밍, 엔트리 AI
|RTX 2000, GTX 1600, Quadro RTX
|CUDA 10 and later
|First to feature Ray Tracing Cores and RTX technology
 
* '''SM75 or <code>SM_75, compute_75</code>''' – GTX/RTX Turing – GTX 1660 Ti, RTX 2060, RTX 2070, RTX 2080, Titan RTX, Quadro RTX 4000, Quadro RTX 5000, Quadro RTX 6000, Quadro RTX 8000, Quadro T1000/T2000, Tesla T4
* [[Turing GPU]]
|-
|-
|Ampere
| L-Series
|Ampere
| Ada
|sm_80,
| L40S, L4
sm_86,
| 가상 데스크톱, 인퍼런스
sm_87 (Orin)
|}
|8.0, 8.6
|RTX 3000, A-series
|CUDA 11.1 and later
|Features third-generation Tensor Cores and more


* [[Ampere GPU]]
=== 아키텍처 비교 ===
* '''SM80 or <code>SM_80, compute_80</code>''' –  NVIDIA A100 (the name “Tesla” has been dropped – GA100), NVIDIA DGX-A100
* '''SM86 or <code>SM_86, compute_86</code> –''' (from CUDA 11.1 onwards) Tesla GA10x cards, RTX Ampere – RTX 3080, GA102 – RTX 3090, RTX A2000, A3000, RTX A4000, A5000, A6000, NVIDIA A40, GA106 – RTX 3060, GA104 – RTX 3070, GA107 – RTX 3050, RTX A10, RTX A16, RTX A40, A2 Tensor Core GPU


* '''SM87 or <code>SM_87, compute_87</code> –''' (from CUDA 11.4 onwards, introduced with PTX ISA 7.4 / Driver r470 and newer) – for Jetson AGX Orin and Drive AGX Orin only
{| class="wikitable"
 
! 아키텍처
<blockquote>“''Devices of compute capability '''8.6''' have 2x more FP32 operations per cycle per SM than devices of compute capability 8.0. While a binary compiled for 8.0 will run as is on 8.6, it is recommended to compile explicitly for 8.6 to benefit from the increased FP32 throughput.''“</blockquote>
! SM
! Tensor Core
! Memory Type
! 대표 모델
|-
| Ampere (SM80)
| 80
| 3rd Gen
| HBM2e/HBM2
| A100, A30
|-
| Hopper (SM90)
| 90
| 4th Gen
| HBM3/HBM3e
| H100, H200
|-
|-
|Lovelace
| Blackwell (SM100)
|Ada Lovelace<ref>https://en.wikipedia.org/wiki/Ada_Lovelace_(microarchitecture)</ref>
| 100
|sm_89
| 5th Gen
|8.7, 8.9
| HBM3e
|GeForce RTX 4070 Ti (AD104)
| B200, RTX PRO 6000
|}


GeForce RTX 4080 (AD103)
=== MIG (Multi-Instance GPU) ===


GeForce RTX 4090 (AD102)
단일 GPU를 여러 격리된 인스턴스로 분할합니다. Ampere 아키텍처부터 도입되었습니다.


Nvidia RTX 6000 Ada Generation (AD102, formerly Quadro)
* GI(GPU Instance): 완전 격리된 리소스 집합
* CI(Compute Instance): GI 내 부분 격리된 SM 집합
* 18가지 프로파일 조합 가능


Nvidia L40 (AD102, formerly Tesla)
자세한 내용은 [[MIG]] 문서를 참조하세요.
|CUDA 11.8 and later
[[cuDNN]] 8.6 and later
|
* Fourth-Gen Tensor Cores  increasing throughput by up to 5X, to 1.4 Tensor-petaFLOPS using the new FP8 Transformer Engine (like H100 model)
* Third-generation RT Cores have twice the ray-triangle intersection throughput, increasing RT-TFLOP performance by over 2x
* The new RT Cores also include a new Opacity Micromap (OMM) Engine and a new Displaced Micro-Mesh (DMM) Engine. The OMM Engine enables much faster ray tracing of alpha-tested textures often used for foliage, particles, and fences. The DMM Engine delivers up to 10X faster Bounding Volume Hierarchy (BVH) build time with up to 20X less BVH storage space, enabling real-time ray tracing of geometrically complex scenes
* Shader Execution Reordering (SER) technology dynamically reorganizes these previously inefficient workloads into considerably more efficient ones. SER can improve shader performance for ray tracing operations by up to 3X, and in-game frame rates by up to 25%.
* DLSS 3 is a revolutionary breakthrough in AI-powered graphics that massively boosts performance. Powered by the new fourth-gen Tensor Cores and Optical Flow Accelerator on GeForce RTX 40 Series GPUs, DLSS 3 uses AI to create additional high-quality frames
* Graphics cards built upon the Ada architecture feature new eighth generation NVIDIA Encoders (NVENC) with AV1 encoding, enabling a raft of new possibilities for streamers, broadcasters, and video callers.
* It’s 40% more efficient than H.264 and allows users who are streaming at 1080p to increase their stream resolution to 1440p while running at the same bitrate and quality.
* '''SM89 or <code>SM_89, compute_</code>89''' – NVIDIA GeForce RTX 4090, RTX 4080, RTX 6000, Tesla L40
|-
|Hopper<ref>https://www.nvidia.com/en-us/data-center/h100/</ref>
|Hopper
|sm_90, sm_90a(Thor)
|9.0
|
|CUDA 12 and later
|TODO


* '''SM90 or <code>SM_90, compute_90</code>''' – NVIDIA H100 (GH100)
=== NVIDIA 드라이버 ===
* '''SM90a or <code>SM_90a, compute_90a</code>''' – (for PTX ISA version 8.0) – adds acceleration for features like <samp>wgmma</samp> and <samp>setmaxnreg</samp>. This is required for NVIDIA CUTLASS
|}


=== NVIDIA GPU Models ===
{| class="wikitable"
{| class="wikitable sortable"
! GPU
!Model
! CUDA Version
!Architecture
! NVIDIA Driver Version
!CUDA Cores
!Tensor Cores
!RT Cores
!FF
!Memory Size
!MIG<ref>https://docs.nvidia.com/datacenter/tesla/mig-user-guide/</ref>
!Memory Bandwidth
!TDP
!Launch Date
|-
|H100-SXM5
|Hopper
(GH100)
|16896
|4th Gen
528
|No
|SXM5
|80GB HBM3
50 MB L2 cache
|7@10GB
|3.35TB/s
|700W
|2023
|-
|H100-PCIE<ref>https://www.nvidia.com/content/dam/en-zz/Solutions/gtcs22/data-center/h100/PB-11133-001_v01.pdf</ref><ref>https://resources.nvidia.com/en-us-tensor-core/nvidia-tensor-core-gpu-datasheet</ref>
|Hopper
(GH100)
|14592
|4th Gen 456
|No
|PCIe
Gen 5 x16
|80 GB HBM2
50 MB L2 cache
|7@10GB
|2TB/s
|300~350W
|2023
|-
|Tesla C1060
|Tesla
|240
|No
|No
|
|4 GB GDDR3
|
|102 GB/s
|238W
|Dec 2008
|-
|Tesla K10
|Kepler
|3072
|No
|No
|
|8 GB GDDR5
|
|320 GB/s
|225W
|May 2012
|-
|Tesla K20
|Kepler
|2496
|No
|No
|
|5/6 GB GDDR5
|
|208 GB/s
|225W
|Nov 2012
|-
|Tesla K40
|Kepler
|2880
|No
|No
|
|12 GB GDDR5
|
|288 GB/s
|235W
|Nov 2013
|-
|Tesla K80
|Kepler
|4992
|No
|No
|
|24 GB GDDR5
|
|480 GB/s
|300W
|Nov 2014
|-
|Tesla M40
|Maxwell
|3072
|No
|No
|
|12 GB GDDR5
|
|288 GB/s
|250W
|Nov 2015
|-
|Tesla P4
|Pascal
|2560
|No
|No
|
|8 GB GDDR5
|
|192 GB/s
|75W
|Sep 2016
|-
|Tesla P40
|Pascal
|3840
|No
|No
|
|24 GB GDDR5X
|
|480 GB/s
|250W
|Sep 2016
|-
|Tesla V100
|Volta
|5120
|640
|Yes
|
|16/32 GB HBM2
|
|900 GB/s
|300W
|May 2017
|-
|Tesla T4
|Turing
|2560
|320
|No
|
|16 GB
|
|
|
|
|-
|A100 PCIe
|Ampere (GA100)
|6912
|432
|Yes
|
|40 GB HBM2 / 80 GB HBM2
|
|1555 GB/s
|250W
|May 2020
|-
|A100 SXM4
|Ampere
|6912
|432
|Yes
|
|40 GB HBM2 / 80 GB HBM2
|7
|1555 GB/s
|400W
|May 2020
|-
|A30
|Ampere
|7424
|184
|No
|
|24 GB GDDR6
|4
|696 GB/s
|165W
|Apr 2021
|-
|-
|A40
| A100 / A30
|Ampere
| CUDA 11
|10752
| R525 (>= 525.53)
|336
|No
|
|48 GB GDDR6
|
|696 GB/s
|300W
|Apr 2021
|-
|-
|A10
| H100 / H200
|Ampere
| CUDA 12
|10240
| R450 (>= 450.80.02)
|320
|No
|
|24 GB GDDR6
|
|624 GB/s
|150W
|Mar 2021
|-
|-
|A16<ref>https://images.nvidia.com/content/Solutions/data-center/vgpu-a16-datasheet.pdf</ref>
| B200
|Ampere
| CUDA 12
|5120
| R570 (>= 570.133.20)
|3rd Gen 160
|40
|PCIe Gen4 x16
|64 GB GDDR6
|
|800 GB/s
|250W
|Mar 2021
|-
|-
|A100 80GB
| RTX PRO 6000 Blackwell
|Ampere
| CUDA 12
(GA100)
| R575 (>= 575.51.03)
|6912
|432
|Yes
|
|80 GB HBM2
|7@
10GB
|1935GB/s
|300W
|Apr 2021
|-
|A100 40GB
|Ampere
(GA100)
|6912
|432
|Yes
|
|40 GB HBM2
|7@
5GB
|1555 GB/s
|250W
|May 2020
|-
|A200 PCIe
|Ampere
|10752
|672
|Yes
|
|80 GB HBM2 / 160 GB HBM2
|
|2050 GB/s
|400W
|Nov 2021
|-
|A200 SXM4
|Ampere
|10752
|672
|Yes
|
|80 GB HBM2 / 160 GB HBM2
|
|2050 GB/s
|400W
|Nov 2021
|-
|A5000
|Ampere
|8192
|256
|Yes
|
|24 GB GDDR6
|
|768 GB/s
|230W
|Apr 2021
|-
|A4000
|Ampere
|6144
|192
|Yes
|
|16 GB GDDR6
|
|512 GB/s
|140W
|Apr 2021
|-
|A3000
|Ampere
|3584
|112
|Yes
|
|24 GB G
|
|
|
|
|-
|Titan RTX
|Turing
|4608
|576
|Yes
|
|24 GB GDDR6
|
|672 GB/s
|280W
|Dec 2018
|-
|GeForce RTX 3090
|Turing
|10496
|328
|Yes
|
|24 GB GDDR6X
|
|936 GB/s
|350W
|Sep 2020
|-
|GeForce RTX 3080 Ti
|Turing
|10240
|320
|Yes
|
|12 GB GDDR6X
|
|912 GB/s
|350W
|May 2021
|-
|GeForce RTX 3080
|Turing
|8704
|272
|Yes
|
|10 GB GDDR6X
|
|760 GB/s
|320W
|Sep 2020
|-
|GeForce RTX 3070 Ti
|Turing
|6144
|192
|Yes
|
|8 GB GDDR6X
|
|608 GB/s
|290W
|Jun 2021
|-
|GeForce RTX 3070
|Turing
|5888
|184
|Yes
|
|8 GB GDDR6
|
|448 GB/s
|220W
|Oct 2020
|-
|GeForce RTX 3060 Ti
|Turing
|4864
|152
|Yes
|
|8 GB GDDR6
|
|448 GB/s
|200W
|Dec 2020
|-
|GeForce RTX 3060
|Turing
|3584
|112
|No
|
|12 GB GDDR6
|
|360 GB/s
|170W
|Feb 2021
|-
|Quadro RTX 8000
|Turing
|4608
|576
|Yes
|
|48 GB GDDR6
|
|624 GB/s
|295W
|Aug 2018
|-
|Quadro RTX 6000
|Turing
|4608
|576
|Yes
|
|24 GB GDDR6
|
|432 GB/s
|260W
|Aug 2018
|-
|Tesla L40
|Ada Lovelace
|18,176
|4th Gen 568
|3rd Gen
142
|PCIe Gen4x1
|48GB GDDR6 with ECC
|
|864GB/s
|300W
|2023
|-
|Quadro RTX 5000
|Turing
|3072
|384
|Yes
|
|16 GB GDDR6
|
|448 GB/s
|230W
|Nov 2018
|-
|Quadro RTX 4000
|Turing
|2304
|288
|Yes
|
|8 GB GDDR6
|
|416 GB/s
|160W
|Nov 2018
|-
|Titan RTX (T-Rex)
|Turing
|4608
|576
|No
|
|24 GB
|
|672 Gb/s
|280 W
|
|-
|Titan V
|Volta
|5120
|640
|
|
|12 GB HBM2
|
|652.8 GB/s
|250W
|Dec 2017
|-
|Tesla V100 (PCIe)
|Volta
|5120
|640
|No
|
|32/16 GB HBM2
|
|900 GB/s
|250W
|June 2017
|-
|Tesla V100 (SXM2)
|Volta
|5120
|640
|No
|
|32/16 GB HBM2
|
|900 GB/s
|300W
|June 2017
|-
|Quadro GV100
|Volta
|5120
|640
|No
|
|32 GB HBM2
|
|870 GB/s
|250W
|Mar 2018
|-
|Tesla GV100 (SXM2)
|Volta
|5120
|640
|No
|
|32 GB HBM2
|
|900 GB/s
|300W
|Mar 2018
|}
|}


=== NVIDIA Features by Architecture<ref>https://videocardz.com/newz/nvidia-details-ad102-gpu-up-to-18432-cuda-cores-76-3b-transistors-and-608-mm%C2%B2</ref> ===
=== nvidia-smi 기본 명령어 ===
 
<syntaxhighlight lang="bash">
# GPU 상태 확인
$ nvidia-smi
 
# GPU 목록 확인
$ nvidia-smi -L
 
# 모니터링 (1초 간격)
$ nvidia-smi -l 1
 
# MIG 상태 확인
$ nvidia-smi --query-gpu=pci.bus_id,mig.mode.current --format=csv
 
# Persistence Mode 활성화
$ sudo nvidia-smi -pm 1
 
# MIG 활성화
$ sudo nvidia-smi -mig 1
</syntaxhighlight>
 
=== GPU 모니터링 ===
 
{| class="wikitable"
{| class="wikitable"
! colspan="7" |NVIDIA GPU Architectures
! 명령어
!
! 설명
|-
!
!AD102
!GA102
!GA100
!TU102
!GV100
!GP102
!GP100
|-
|-
!Launch Year
| nvidia-smi
|'''2022'''
| 기본 GPU 상태
|2020
|2020
|2018
|2017
|2017
|–
|-
|-
!Architecture
| nvidia-smi -L
|'''Ada Lovelace'''
| GPU UUID 목록
|Ampere
|Ampere
|Turing
|Volta
|Pascal
|Pascal
|-
|-
!Form Factor
| nvidia-smi mig -lgi
|–
| MIG 인스턴스 목록
|–
|SXM4/PCIe
|–
|SXM2/PCIe
|–
|SXM/PCIe
|-
|-
!TDP
| nvidia-smi mig -lgip
|–
| MIG 프로파일 목록
|–
|400W
|–
|300W
|–
|300W
|-
|-
!Node
| dcgmi dmon
|'''TSMC 4N'''
| DCGM 고급 모니터링
|SAMSUNG 8N
|–
|TSMC 12nm
|TSMC 12nm
|TSMC 16nm
|–
|-
|-
!CUDA Cores
| nvtop
|'''18432'''
| 실시간 모니터링 (nvtop 설치 필요)
|10752
|–
|4608
|5120
|3840
|–
|-
!Tensor Cores
|'''576 Gen4'''
|336 Gen3
|–
|576 Gen2
|640
|–
|–
|-
!RT Cores
|'''144 Gen3'''
|84 Gen2
|–
|72 Gen1
|–
|–
|–
|-
!Memory Bus
|'''GDDR6X 384-bit'''
|GDDR6X 384-bit
|–
|GDDR6 384-bit
|HBM2 3072-bit
|GDDR6X 384-bit
|–
|}
|}


=== NVIDIA Grace Architecture ===
 
NVIDIA has announced that they will be partnering with server manufacturers such as HPE, Atos, and Supermicro to create servers that integrate the Grace architecture with ARM-based CPUs. These servers are expected to be available in the second half of 2023
== Configuration ==
 
<syntaxhighlight lang="bash">
# NVIDIA 드라이버 설치
$ sudo apt-get install nvidia-driver-535
 
# Persistence Mode 활성화
$ sudo nvidia-smi -pm 1
 
# MIG 활성화
$ sudo nvidia-smi -mig 1
 
# GPU 상태 확인
$ nvidia-smi
 
# MIG 인스턴스 생성
$ sudo nvidia-smi mig -cgi 9,19,19,19 -C
</syntaxhighlight>
 
 
== Examples ==
 
=== Example 1: GPU 상태 확인 ===
 
<syntaxhighlight lang="bash">
# 전체 GPU 상태
$ nvidia-smi
 
# GPU UUID 확인
$ nvidia-smi -L
 
# MIG 상태 확인
$ nvidia-smi --query-gpu=pci.bus_id,mig.mode.current --format=csv
</syntaxhighlight>
 
=== Example 2: Docker에서 GPU 사용 ===
 
<syntaxhighlight lang="bash">
# 전체 GPU 사용
$ docker run --gpus all --rm nvidia/cuda:12.0 nvidia-smi
 
# 특정 GPU 사용
$ docker run --gpus '"device=0"' --rm nvidia/cuda:12.0 nvidia-smi
 
# MIG 디바이스 사용
$ docker run --gpus '"device=MIG-UUID"' --rm nvidia/cuda:12.0 nvidia-smi
</syntaxhighlight>
 
 
== Best Practices ==
 
* Persistence Mode 필수 활성화
* MIG 활성화 시 nvsm/dcgm 중지 후 진행
* 워크로드에 맞는 MIG 프로파일 선택
* DCGM v2.0.13+ 사용 (MIG 모니터링)
* 정기적인 드라이버 업데이트
* RTX PRO Blackwell은 Display Mode 먼저 Compute로 설정
* SSH 접근 확인 후 Display Mode 변경
 
 
== Limitations ==
 
* MIG 활성화/비활성화는 GPU 리셋 필요
* 그래픽 컨텍스트 미지원 (MIG)
* P2P/NVLink 미지원 (MIG)
* nvidia-smi만으로는 MIG 메트릭 확인 불가
* RTX PRO Blackwell primary display GPU는 Display Mode 변경 시 디스플레이 출력 중단
 
 
== References ==
 
* https://docs.nvidia.com/datacenter/tesla/mig-user-guide/
* https://www.nvidia.com/en-us/data-center/
 
 
== Related Pages ==
 
* [[MIG]]
* [[CUDA]]
* [[NVIDIA driver]]
* [[NVIDIA GPU Cloud (NGC)]]
* [[DCGM]]
 
 
[[Category:GPU]]
== Knowledge Graph ==
 
Related
 
→ [[CUDA]]
→ [[GPU]]
→ [[H100]]
→ [[NVLink]]
→ [[AMD GPUs]]
→ [[A100]]
→ [[MIG]]
→ [[NVIDIA GPUs]]
 
[[Category:Reference]]
 
== Latest GPU Lineup (2024-2025) ==
 
NVIDIA는 2024년 Blackwell 아키텍처와 H200를 출시. AI/ML 워크로드 성능이 크게 향상.
 
=== Blackwell B100/B200 ===
 
* 아키텍처: Blackwell (SM 10.x)
* TFLOPS (FP4): 2,900 TFLOPS
* 메모리: HBM3e, 192GB
* 메모리 대역폭: 8 TB/s
* TDP: 1,000W
* NVLink: 1.8 TB/s (B200)
* 출시: 2024년
 
=== H200 ===
 
* 아키텍처: Hopper (SM 9.0)
* 메모리: HBM3e, 141GB
* 메모리 대역폭: 4.8 TB/s
* TDP: 700W
* NVLink: 900 GB/s
* 출시: 2024년 Q2
* H100 대비 메모리 27% 증가, 대역폭 20% 증가
 
=== H100 (기존) ===
 
* 아키텍처: Hopper (SM 9.0)
* Tensor TFLOPS (FP8): 989 TFLOPS
* 메모리: HBM3, 80GB
* 메모리 대역폭: 3.35 TB/s
* TDP: 700W
* NVLink: 900 GB/s
 
=== Blackwell vs Hopper 비교 ===
 
{| class="wikitable"
{| class="wikitable"
!Architecture
! Feature ! H100 ! H200 ! B100 ! B200
!Key Features
|-
| 아키텍처 | Hopper | Hopper | Blackwell | Blackwell
|-
| Tensor TFLOPS (FP8) | 989 | 989 | ~1500 | ~2000
|-
|-
| rowspan="4" |Grace
| 메모리 | HBM3 80GB | HBM3e 141GB | HBM3e 192GB | HBM3e 288GB
|CPU-GPU integration, ARM Neoverse CPU, HBM2E memory
|-
|-
|900 GB/s memory bandwidth, support for PCIe 5.0 and NVLink
| 대역폭 | 3.35 TB/s | 4.8 TB/s | ~6 TB/s | ~8 TB/s
|-
|-
|10x performance improvement for certain HPC workloads
| TDP | 700W | 700W | 1000W | 1000W
|-
|-
|Energy efficiency improvements through unified memory space
| NVLink | 900 GB/s | 900 GB/s | 1.8 TB/s | 1.8 TB/s
|}
|}
=== Reference ===
<references/>

Latest revision as of 13:37, 17 July 2026

NVIDIA GPU

Template:Status

Template:TOC

Overview

NVIDIA GPU(Graphics Processing Unit)는 병렬 컴퓨팅을 위한 그래픽 처리 장치입니다. 게이밍부터 HPC, AI/ML까지 광범위하게 사용됩니다. Ampere, Hopper, Blackwell 아키텍처를 통해 지속적인 성능 향상을 이루고 있습니다.

Summary

  • 무엇인가? — NVIDIA의 그래픽 및 컴퓨팅 처리 장치
  • 왜 필요한가? — 병렬 컴퓨팅, AI/ML 훈련/추론, 그래픽 렌더링
  • 언제 사용하는가? — AI 서버, HPC 클러스터, 워크스테이션


Purpose

이 문서가 존재하는 이유

  • Goal: NVIDIA GPU 제품군, 아키텍처, MIG, 드라이버, 모니터링 방법 제공
  • Scope: GPU 라인업, 아키텍처 비교, MIG, 드라이버 설치, nvidia-smi
  • Non-goals: CUDA 프로그래밍, 특정 애플리케이션 설정은 별도 문서


Key Concepts

Concept Description Related
Ampere SM80 아키텍처 — A100/H100 기반 NVIDIA GPU
Hopper SM90 아키텍처 — H200/B200 기반 NVIDIA GPU
Blackwell SM100 아키텍처 — B200 기반 NVIDIA GPU
MIG Multi-Instance GPU — GPU 분할 MIG
CUDA NVIDIA 병렬 컴퓨팅 플랫폼 CUDA
Tensor Core AI 연산 전용 코어 NVIDIA GPU
NVLink GPU 간 고속 연결 NVIDIA GPU
nvidia-smi GPU 모니터링/관리 도구 NVIDIA GPU


Architecture

NVIDIA GPU 아키텍처 진화: 

graph LR
    A[Turing] --> B[Ampere]
    B --> C[Hopper]
    C --> D[Blackwell]
    B --> E[A100]
    B --> F[A30]
    C --> G[H100]
    C --> H[H200]
    D --> I[B200]
    D --> J[RTX PRO 6000]
    D --> K[RTX PRO 5000]


Workflow

Stage Input Output
Install Driver NVIDIA driver package nvidia-smi
Check Status nvidia-smi GPU info
Enable MIG nvidia-smi -mig 1 MIG Mode
Create Instance nvidia-smi -cgi GPU Instances
Monitor nvidia-smi / dcgmi Metrics


Detailed Explanation

GPU 제품군

제품군 아키텍처 주요 모델 용도
Data Center Ampere/Hopper/Blackwell A100, H100, H200, B200 AI/ML, HPC
RTX Workstation Ada/Blackwell RTX 6000 Ada, RTX PRO 6000 워크스테이션, VFX
GeForce Ada RTX 4090, RTX 4080 게이밍, 엔트리 AI
L-Series Ada L40S, L4 가상 데스크톱, 인퍼런스

아키텍처 비교

아키텍처 SM Tensor Core Memory Type 대표 모델
Ampere (SM80) 80 3rd Gen HBM2e/HBM2 A100, A30
Hopper (SM90) 90 4th Gen HBM3/HBM3e H100, H200
Blackwell (SM100) 100 5th Gen HBM3e B200, RTX PRO 6000

MIG (Multi-Instance GPU)

단일 GPU를 여러 격리된 인스턴스로 분할합니다. Ampere 아키텍처부터 도입되었습니다.

  • GI(GPU Instance): 완전 격리된 리소스 집합
  • CI(Compute Instance): GI 내 부분 격리된 SM 집합
  • 18가지 프로파일 조합 가능

자세한 내용은 MIG 문서를 참조하세요.

NVIDIA 드라이버

GPU CUDA Version NVIDIA Driver Version
A100 / A30 CUDA 11 R525 (>= 525.53)
H100 / H200 CUDA 12 R450 (>= 450.80.02)
B200 CUDA 12 R570 (>= 570.133.20)
RTX PRO 6000 Blackwell CUDA 12 R575 (>= 575.51.03)

nvidia-smi 기본 명령어

# GPU 상태 확인
$ nvidia-smi

# GPU 목록 확인
$ nvidia-smi -L

# 모니터링 (1초 간격)
$ nvidia-smi -l 1

# MIG 상태 확인
$ nvidia-smi --query-gpu=pci.bus_id,mig.mode.current --format=csv

# Persistence Mode 활성화
$ sudo nvidia-smi -pm 1

# MIG 활성화
$ sudo nvidia-smi -mig 1

GPU 모니터링

명령어 설명
nvidia-smi 기본 GPU 상태
nvidia-smi -L GPU UUID 목록
nvidia-smi mig -lgi MIG 인스턴스 목록
nvidia-smi mig -lgip MIG 프로파일 목록
dcgmi dmon DCGM 고급 모니터링
nvtop 실시간 모니터링 (nvtop 설치 필요)


Configuration

# NVIDIA 드라이버 설치
$ sudo apt-get install nvidia-driver-535

# Persistence Mode 활성화
$ sudo nvidia-smi -pm 1

# MIG 활성화
$ sudo nvidia-smi -mig 1

# GPU 상태 확인
$ nvidia-smi

# MIG 인스턴스 생성
$ sudo nvidia-smi mig -cgi 9,19,19,19 -C


Examples

Example 1: GPU 상태 확인

# 전체 GPU 상태
$ nvidia-smi

# GPU UUID 확인
$ nvidia-smi -L

# MIG 상태 확인
$ nvidia-smi --query-gpu=pci.bus_id,mig.mode.current --format=csv

Example 2: Docker에서 GPU 사용

# 전체 GPU 사용
$ docker run --gpus all --rm nvidia/cuda:12.0 nvidia-smi

# 특정 GPU 사용
$ docker run --gpus '"device=0"' --rm nvidia/cuda:12.0 nvidia-smi

# MIG 디바이스 사용
$ docker run --gpus '"device=MIG-UUID"' --rm nvidia/cuda:12.0 nvidia-smi


Best Practices

  • Persistence Mode 필수 활성화
  • MIG 활성화 시 nvsm/dcgm 중지 후 진행
  • 워크로드에 맞는 MIG 프로파일 선택
  • DCGM v2.0.13+ 사용 (MIG 모니터링)
  • 정기적인 드라이버 업데이트
  • RTX PRO Blackwell은 Display Mode 먼저 Compute로 설정
  • SSH 접근 확인 후 Display Mode 변경


Limitations

  • MIG 활성화/비활성화는 GPU 리셋 필요
  • 그래픽 컨텍스트 미지원 (MIG)
  • P2P/NVLink 미지원 (MIG)
  • nvidia-smi만으로는 MIG 메트릭 확인 불가
  • RTX PRO Blackwell primary display GPU는 Display Mode 변경 시 디스플레이 출력 중단


References


Related Pages

Knowledge Graph

Related

→ CUDA → GPU → H100 → NVLink → AMD GPUs → A100 → MIG → NVIDIA GPUs

Latest GPU Lineup (2024-2025)

NVIDIA는 2024년 Blackwell 아키텍처와 H200를 출시. AI/ML 워크로드 성능이 크게 향상.

Blackwell B100/B200

  • 아키텍처: Blackwell (SM 10.x)
  • TFLOPS (FP4): 2,900 TFLOPS
  • 메모리: HBM3e, 192GB
  • 메모리 대역폭: 8 TB/s
  • TDP: 1,000W
  • NVLink: 1.8 TB/s (B200)
  • 출시: 2024년

H200

  • 아키텍처: Hopper (SM 9.0)
  • 메모리: HBM3e, 141GB
  • 메모리 대역폭: 4.8 TB/s
  • TDP: 700W
  • NVLink: 900 GB/s
  • 출시: 2024년 Q2
  • H100 대비 메모리 27% 증가, 대역폭 20% 증가

H100 (기존)

  • 아키텍처: Hopper (SM 9.0)
  • Tensor TFLOPS (FP8): 989 TFLOPS
  • 메모리: HBM3, 80GB
  • 메모리 대역폭: 3.35 TB/s
  • TDP: 700W
  • NVLink: 900 GB/s

Blackwell vs Hopper 비교

Feature ! H100 ! H200 ! B100 ! B200
Hopper | Hopper | Blackwell | Blackwell
989 | 989 | ~1500 | ~2000
HBM3 80GB | HBM3e 141GB | HBM3e 192GB | HBM3e 288GB
3.35 TB/s | 4.8 TB/s | ~6 TB/s | ~8 TB/s
700W | 700W | 1000W | 1000W
900 GB/s | 900 GB/s | 1.8 TB/s | 1.8 TB/s
Retrieved from "http://wiki.hpcmate.com/index.php?title=NVIDIA_GPU&oldid=3518"