NVIDIA GPU: Difference between revisions

From HPCWIKI
Jump to navigation Jump to search
(Phase 4.4: NVIDIA GPUs 메인 페이지 링크 추가)
(Phase 6.1: LLM-Optimized Wiki Template 적용 (NVIDIA GPU))
Line 1: Line 1:
== See Also ==
= NVIDIA GPU =
* [[NVIDIA GPUs]]


[http://www.hpcmate.com/ HPCMATE]  provides all level of GPU model as air-cooling or liquid-cooling version for any type of server or workstation.
{{Status
|status=Draft
|owner=Knowledge Agent
|last_update=2026-07-15
|review=Pending
}}


== Nvidia GPU tips and tricks ==
{{TOC}}


* TRX - RTX stands for real-time ray tracing
== Overview ==
* GeForce vs Quadro - In general, GeForce cards have less computing power than Quadro cards. Quadro cards simply have more computation and memory than GeForce cards. So if we are training a neural [[network]], rendering an animated film, or running exhaustive CAD/CAE applications , that extra memory is extra welcome


== GPU Tenser performance notes for RTX 4090 ==
NVIDIA GPU(Graphics Processing Unit)는 병렬 컴퓨팅을 위한 그래픽 처리 장치입니다. 게이밍부터 HPC, AI/ML까지 광범위하게 사용됩니다. Ampere, Hopper, Blackwell 아키텍처를 통해 지속적인 성능 향상을 이루고 있습니다.
According to [https://www.reddit.com/r/hardware/comments/xt38d9/comment/iqo2bi4/?utm_source=reddit&utm_medium=web2x&context=3 this thread] '''NVIDIA''' '''looks cut the tensor FP16 &  TF32 operation rate in half, resulting in a 4090 with even lower FP16 & TF32 performance than the 4080 16GB.''' This may have been done to prevent the 4090 from cannibalizing the Quadro/Tesla sales. So if you are choosing GPUs, you can choose the 4090 for memory, but lower tensor performance than the 4080 16GB. eventhough 4090 has more than twice the ray tracing performance of the 4080 12GB.
 
{| class="wikitable sortable"
=== Summary ===
!
 
!RTX 4090
* 무엇인가? — NVIDIA의 그래픽 및 컴퓨팅 처리 장치
!RTX 4080 16GB
* 왜 필요한가? — 병렬 컴퓨팅, AI/ML 훈련/추론, 그래픽 렌더링
!RTX 4080 12GB
* 언제 사용하는가? — AI 서버, HPC 클러스터, 워크스테이션
!RTX 3090 Ti
 
|-
---
|non-tensor FP32 tflops
 
|82.6 (206%)
== Purpose ==
|48.7 (122%)
 
|40.1 (100%)
이 문서가 존재하는 이유
|40 (100%)
 
* Goal: NVIDIA GPU 제품군, 아키텍처, MIG, 드라이버, 모니터링 방법 제공
* Scope: GPU 라인업, 아키텍처 비교, MIG, 드라이버 설치, nvidia-smi
* Non-goals: CUDA 프로그래밍, 특정 애플리케이션 설정은 별도 문서
 
---
 
== Key Concepts ==
 
{| class="wikitable"
! Concept
! Description
! Related
|-
|-
|non-tensor FP16 tflops
| Ampere
|82.6 (206%)
| SM80 아키텍처 — A100/H100 기반
|48.7 (122%)
| [[NVIDIA GPU]]
|40.1 (100%)
|40 (100%)
|-
|-
|Tensor Cores
| Hopper
|512 (152%)
| SM90 아키텍처 — H200/B200 기반
|304 (90%)
| [[NVIDIA GPU]]
|240 (71%)
|336 (100%)
|-
|-
|Optical flow TOPS
| Blackwell
|305 (242%)
| SM100 아키텍처 — B200 기반
|305 (242%)
| [[NVIDIA GPU]]
|305 (242%)
|126 (100%)
|-
|-
|tensor FP16 w/ FP32 accumulate TFLOPS **
| MIG
|165.2 (207%)
| Multi-Instance GPU — GPU 분할
|194.9 (244%)
| [[MIG]]
|160.4 (200%)
|80 (100%)
|-
|-
|tensor TF32 TFLOPS **
| CUDA
|82.6 (207%)
| NVIDIA 병렬 컴퓨팅 플랫폼
|97.5 (244%)
| [[CUDA]]
|80.2 (200%)
|40 (100%)
|-
|-
|Ray trace Cores
| Tensor Core
|128 (152%)
| AI 연산 전용 코어
|76 (90%)
| [[NVIDIA GPU]]
|60 (71%)
|84 (100%)
|-
|-
|Ray trace TFLOPS
| NVLink
|191 (245%)
| GPU 간 고속 연결
|112.7 (144%)
| [[NVIDIA GPU]]
|92.7 (119%)
|78.1 (100%)
|-
|-
|POWER (W)
| nvidia-smi
|450 (100%)
| GPU 모니터링/관리 도구
|320 (71%)
| [[NVIDIA GPU]]
|285 (63%)
|450 (100%)
|}
|}


== NVIDIA GPU Architecture ==
---
nvcc sm flags and what they’re used for: When compiling with NVCC<ref>https://arnon.dk/matching-sm-architectures-arch-and-gencode-for-various-nvidia-cards/</ref>,


* the arch flag (‘'''<code>-arch</code>'''‘) specifies the name of the NVIDIA GPU architecture that the [[CUDA]] files will be compiled for.
== Architecture ==
* Gencodes (‘'''<code>-gencode</code>'''‘) allows for more PTX generations and can be repeated many times for different architectures.


Matching CUDA arch and CUDA gencode for various NVIDIA architectures
NVIDIA GPU 아키텍처 진화:
{| class="wikitable sortable"
!Series
!Architecture
(--arch)
!CUDA gencode
(--sm)
!Compute Capability
!Notable Models
!Supported CUDA version
!Key Features
|-
|Tesla
|Tesla
|
|1.0, 1.1, 2.0, 2.1
|C1060, M2050, K80, P100, V100, A100
|
|First dedicated GPGPU series
|-
|Fermi
|Fermi
|sm_20
|3.0, 3.1
|GTX 400, GTX 500, Tesla 20-series, Quadro 4000/5000
|CUDA 3.2 until CUDA 8
|First to feature CUDA cores and [[support]] for ECC memory


* '''SM20 or SM_20, compute_30''' –  GeForce 400, 500, 600, GT-630. '''''Completely dropped from CUDA 10 onwards.'''''
<syntaxhighlight lang="mermaid">
graph LR
    A[Turing] --> B[Ampere]
    B --> C[Hopper]
    C --> D[Blackwell]
    B --> E[A100]
    B --> F[A30]
    C --> G[H100]
    C --> H[H200]
    D --> I[B200]
    D --> J[RTX PRO 6000]
    D --> K[RTX PRO 5000]
</syntaxhighlight>


|-
---
|Kepler
|Kepler
|sm_30
sm_35,
sm_37
|3.2, 3.5, 3.7
|GTX 600, GTX 700, Tesla K-series, Quadro K-series
|CUDA 5 until CUDA 10
|First to feature Dynamic Parallelism and Hyper-Q


* '''''SM30 or <code>SM_30, compute_30</code> –'''''  '''''Kepler architecture (e.g. generic Kepler, GeForce 700, GT-730).'''''  '''''Adds support for unified memory programmingCompletely dropped from CUDA 11 onwards.'''''
== Workflow ==
* '''''SM35 or <code>SM_35, compute_35</code> –'''''  '''''Tesla K40.''''' '''''Adds support for dynamic parallelism.'''''  '''''Deprecated from CUDA 11, will be dropped in future versions.'''''
* '''''SM37 or <code>SM_37, compute_37</code> –'''''  '''''Tesla K80.''''' '''''Adds a few more registers.'''''  '''''Deprecated from CUDA 11, will be dropped in future versions, strongly suggest replacing with a 32GB [[PCIe]] Tesla V100.'''''
|-
|Maxwell
|Maxwell
|sm_50,
sm_52,
sm_53
|5.0, 5.2
|GTX 900, GTX 1000, Quadro M-series
|CUDA 6 until CUDA 11
|First to support VR and 4K displays


* '''''SM50 or <code>SM_50, compute_50</code> –'''''  '''''Tesla/Quadro M series.'''''  '''''Deprecated from CUDA 11, will be dropped in future versions, strongly suggest replacing with a Quadro RTX 4000 or A6000.'''''
{| class="wikitable"
* '''''SM52 or <code>SM_52, compute_52</code> –'''''  '''''Quadro M6000 , GeForce 900, GTX-970, GTX-980, GTX Titan X.'''''
! Stage
* '''''SM53 or <code>SM_53, compute_53</code> –'''''  '''''Tegra (Jetson) TX1 / Tegra X1, Drive CX, Drive PX, Jetson Nano.'''''
! Input
! Output
|-
|-
|Pascal
| Install Driver
|Pascal
| NVIDIA driver package
|sm_60,
| nvidia-smi
sm_61,
sm_62
|6.0, 6.1, 6.2
|GTX 1000, Quadro P-series
|'''CUDA 8 and later'''
|First to support simultaneous multi-projection
 
* '''SM60 or <code>SM_60, compute_60</code>''' –  Quadro GP100, Tesla P100, DGX-1 (Generic Pascal)
* '''SM61 or <code>SM_61, compute_61</code>'''–  GTX 1080, GTX 1070, GTX 1060, GTX 1050, GTX 1030 (GP108), GT 1010 (GP108) Titan Xp, Tesla P40, Tesla P4, Discrete GPU on the NVIDIA Drive PX2
* '''SM62 or <code>SM_62, compute_62</code>''' –  Integrated GPU on the NVIDIA Drive PX2, Tegra (Jetson) TX2
|-
|-
|Volta
| Check Status
|Volta
| nvidia-smi
|sm_70,
| GPU info
sm_72
(Xavier)
|7.0, 7.2, 7.5
|Titan V, Tesla V100, Quadro GV100
|CUDA 9 and later
|First to feature Tensor Cores and NVLink 2.0
 
* '''SM70 or <code>SM_70, compute_70</code>''' –  DGX-1 with Volta, Tesla V100, GTX 1180 (GV104), Titan V, Quadro GV100
* '''SM72 or <code>SM_72, compute_72</code>''' – Jetson AGX Xavier, Drive AGX Pegasus, Xavier NX
|-
|-
|Turing
| Enable MIG
|Turing
| nvidia-smi -mig 1
|sm_75
| MIG Mode
|7.5, 7.6
|RTX 2000, GTX 1600, Quadro RTX
|CUDA 10 and later
|First to feature Ray Tracing Cores and RTX technology
 
* '''SM75 or <code>SM_75, compute_75</code>''' – GTX/RTX Turing – GTX 1660 Ti, RTX 2060, RTX 2070, RTX 2080, Titan RTX, Quadro RTX 4000, Quadro RTX 5000, Quadro RTX 6000, Quadro RTX 8000, Quadro T1000/T2000, Tesla T4
* [[Turing GPU]]
|-
|-
|Ampere
| Create Instance
|Ampere
| nvidia-smi -cgi
|sm_80,
| GPU Instances
sm_86,
sm_87 (Orin)
|8.0, 8.6
|RTX 3000, A-series
|CUDA 11.1 and later
|Features third-generation Tensor Cores and more
 
* [[Ampere GPU]]
* '''SM80 or <code>SM_80, compute_80</code>''' –  NVIDIA A100 (the name “Tesla” has been dropped – GA100), NVIDIA DGX-A100
* '''SM86 or <code>SM_86, compute_86</code> –''' (from CUDA 11.1 onwards) Tesla GA10x cards, RTX Ampere – RTX 3080, GA102 – RTX 3090, RTX A2000, A3000, RTX A4000, A5000, A6000, NVIDIA A40, GA106 – RTX 3060, GA104 – RTX 3070, GA107 – RTX 3050, RTX A10, RTX A16, RTX A40, A2 Tensor Core GPU
 
* '''SM87 or <code>SM_87, compute_87</code> –''' (from CUDA 11.4 onwards, introduced with PTX ISA 7.4 / Driver r470 and newer) – for Jetson AGX Orin and Drive AGX Orin only
 
<blockquote>“''Devices of compute capability '''8.6''' have 2x more FP32 operations per cycle per SM than devices of compute capability 8.0. While a binary compiled for 8.0 will run as is on 8.6, it is recommended to compile explicitly for 8.6 to benefit from the increased FP32 throughput.''“</blockquote>
|-
|-
|Lovelace
| Monitor
|Ada Lovelace<ref>https://en.wikipedia.org/wiki/Ada_Lovelace_(microarchitecture)</ref>
| nvidia-smi / dcgmi
|sm_89
| Metrics
|8.9
|}
|GeForce RTX 4070 Ti (AD104)


GeForce RTX 4080 (AD103)
---


GeForce RTX 4090 (AD102)
== Detailed Explanation ==


Nvidia RTX 6000 Ada Generation (AD102, formerly Quadro)
=== GPU 제품군 ===


Nvidia L40 (AD102, formerly Tesla)
{| class="wikitable"
|CUDA 11.8 and later
! 제품군
[[cuDNN]] 8.6 and later
! 아키텍처
|
! 주요 모델
* Fourth-Gen Tensor Cores  increasing throughput by up to 5X, to 1.4 Tensor-petaFLOPS using the new FP8 [[Transformer Engine]] (like H100 model)
! 용도
* Third-generation RT Cores have twice the ray-triangle intersection throughput, increasing RT-TFLOP performance by over 2x
* The new RT Cores also include a new Opacity Micromap (OMM) Engine and a new Displaced Micro-Mesh (DMM) Engine. The OMM Engine enables much faster ray tracing of alpha-tested textures often used for foliage, particles, and fences. The DMM Engine delivers up to 10X faster Bounding Volume Hierarchy (BVH) build time with up to 20X less BVH storage space, enabling real-time ray tracing of geometrically complex scenes
* Shader Execution Reordering (SER) technology dynamically reorganizes these previously inefficient workloads into considerably more efficient ones. SER can improve shader performance for ray tracing operations by up to 3X, and in-game frame rates by up to 25%.
* DLSS 3 is a revolutionary breakthrough in AI-powered graphics that massively boosts performance. Powered by the new fourth-gen Tensor Cores and Optical Flow Accelerator on GeForce RTX 40 Series GPUs, DLSS 3 uses AI to create additional high-quality frames
* Graphics cards built upon the Ada architecture feature new eighth generation NVIDIA Encoders (NVENC) with AV1 encoding, enabling a raft of new possibilities for streamers, broadcasters, and video callers.
* It’s 40% more efficient than H.264 and allows users who are streaming at 1080p to increase their stream resolution to 1440p while running at the same bitrate and quality.
* '''SM89 or <code>SM_89, compute_</code>89''' – NVIDIA GeForce RTX 4090, RTX 4080, RTX 6000, Tesla L40
|-
|-
|Hopper<ref>https://www.nvidia.com/en-us/data-center/h100/</ref>
| Data Center
|Hopper
| Ampere/Hopper/Blackwell
|sm_90, sm_90a(Thor)
| A100, H100, H200, B200
|9.0
| AI/ML, HPC
|
|CUDA 12 and later
|TODO
 
* '''SM90 or <code>SM_90, compute_90</code>''' – NVIDIA H100 (GH100)
* '''SM90a or <code>SM_90a, compute_90a</code>''' – (for PTX ISA version 8.0) – adds acceleration for features like <samp>wgmma</samp> and <samp>setmaxnreg</samp>. This is required for NVIDIA CUTLASS
|}
 
== NVIDIA GPU Models<ref>https://www.nvidia.com/content/dam/en-zz/Solutions/gtcs22/design-visualization/quadro-product-literature/rtx-6000-l40-linecard-nvidia-us-2653097-r7-web.pdf</ref> <ref>https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/rtx/quadro-ampere-linecard-us-nvidia.pdf</ref> ==
{| class="wikitable sortable"
!Model
!Architecture
!CUDA Cores
!Tensor Cores
!RT Cores
!NVLink
!FF
!Memory Size
!MIG<ref>https://docs.nvidia.com/datacenter/tesla/mig-user-guide/</ref>
!Memory Bandwidth
!Thermal
!Power connector
!TDP
!Launch Date
|-
|-
|H100-SXM5
| RTX Workstation
|Hopper
| Ada/Blackwell
(GH100)
| RTX 6000 Ada, RTX PRO 6000
|16896
| 워크스테이션, VFX
|4th Gen
528
|No
|
|SXM5
|80GB HBM3
50 MB L2 cache
|7@10GB
|3.35TB/s
|
|
|700W
|Jan 2023
|-
|-
|H100-PCIE<ref>https://www.nvidia.com/content/dam/en-zz/Solutions/gtcs22/data-center/h100/PB-11133-001_v01.pdf</ref><ref>https://resources.nvidia.com/en-us-tensor-core/nvidia-tensor-core-gpu-datasheet</ref><ref>https://www.aspsys.com/wp-content/uploads/2023/09/nvidia-h100-datasheet.pdf</ref>
| GeForce
|Hopper
| Ada
(GH100)
| RTX 4090, RTX 4080
|14592
| 게이밍, 엔트리 AI
|4th Gen 456
|No
|NVLink
(600GB PCIe,
 
128GB Gen5)
|PCIe
Gen 5 x16
|80 GB HBM2
50 MB L2 cache
|7@10GB
|2TB/s
|Passive
|
|300~350W
(configurable)
|Jan 2023
|-
|-
|H100 NVL<ref>https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/h100/PB-11773-001_v01.pdf</ref> <ref>https://www.aspsys.com/wp-content/uploads/2023/09/nvidia-h100-datasheet.pdf</ref>
| L-Series
|Hopper
| Ada
(P1010 SKU 210)
| L40S, L4
| 가상 데스크톱, 인퍼런스
|}


| colspan="3" |2 H100 PCIe boards that come already bridged together<ref>https://www.anandtech.com/show/18780/nvidia-announces-h100-nvl-max-memory-server-card-for-large-language-models</ref>
=== 아키텍처 비교 ===
|NVLink
(600GB PCIe,


128GB Gen5)
{| class="wikitable"
|2 x PCIe
! 아키텍처
Gen 5 x16
! SM
|94 GB HBM3
! Tensor Core
|14@12GB
! Memory Type
|
! 대표 모델
|Passive
|One PCIe 16-pin auxiliary power connector (12v-2x6 auxiliary power
connector)
|2x 300~450W (configurable)
|Sept  2023
|-
|-
|Tesla C1060
| Ampere (SM80)
|Tesla
| 80
|240
| 3rd Gen
|No
| HBM2e/HBM2
|No
| A100, A30
|
|
|4 GB GDDR3
|
|102 GB/s
|
|
|238W
|Dec 2008
|-
|-
|Tesla K10
| Hopper (SM90)
|Kepler
| 90
|3072
| 4th Gen
|No
| HBM3/HBM3e
|No
| H100, H200
|
|
|8 GB GDDR5
|
|320 GB/s
|
|
|225W
|May 2012
|-
|-
|Tesla K20
| Blackwell (SM100)
|Kepler
| 100
|2496
| 5th Gen
|No
| HBM3e
|No
| B200, RTX PRO 6000
|
|}
|
 
|5/6 GB GDDR5
=== MIG (Multi-Instance GPU) ===
|
 
|208 GB/s
단일 GPU를 여러 격리된 인스턴스로 분할합니다. Ampere 아키텍처부터 도입되었습니다.
|
|
|225W
|Nov 2012
|-
|Tesla K40
|Kepler
|2880
|No
|No
|
|
|12 GB GDDR5
|
|288 GB/s
|
|
|235W
|Nov 2013
|-
|Tesla K80
|Kepler
|4992
|No
|No
|
|
|24 GB GDDR5
|
|480 GB/s
|
|
|300W
|Nov 2014
|-
|Tesla M40
|Maxwell
|3072
|No
|No
|
|
|12 GB GDDR5
|
|288 GB/s
|
|
|250W
|Nov 2015
|-
|Tesla P4
|Pascal
|2560
|No
|No
|
|
|8 GB GDDR5
|
|192 GB/s
|
|
|75W
|Sep 2016
|-
|Tesla P40
|Pascal
|3840
|No
|No
|
|
|24 GB GDDR5X
|
|480 GB/s
|
|
|250W
|Sep 2016
|-
|Tesla P100<ref>https://images.nvidia.com/content/tesla/pdf/nvidia-tesla-p100-PCIe-datasheet.pdf</ref><ref>https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/solutions/resources/documents1/NV-tesla-p100-pcie-PB-08248-001-v01.pdf</ref>
|Pascal
|3584
|
|
|
|
|16GB CoWoS HBM2 at
732 GB/s or


12GB CoWoS HBM2 at
* GI(GPU Instance): 완전 격리된 리소스 집합
* CI(Compute Instance): GI 내 부분 격리된 SM 집합
* 18가지 프로파일 조합 가능


549 GB/
자세한 내용은 [[MIG]] 문서를 참조하세요.
|
|732.2GB/s
PCIe 3.0 × 16
|Passive
|
|250 W
|
|-
|Tesla V100
|Volta
|5120
|640
|Yes
|
|
|16/32 GB HBM2
|
|900 GB/s
|
|
|300W
|May 2017
|-
|Tesla T4
|Turing
|2560
|320
|No
|
|
|16 GB
|
|
|
|
|
|
|-
|A100 PCIe
|Ampere (GA100)
|6912
|432
|Yes
|
|
|40 GB HBM2 / 80 GB HBM2
|
|1555 GB/s
|
|
|250W
|May 2020
|-
|A100 SXM4
|Ampere
|6912
|432
|Yes
|
|
|40 GB HBM2 / 80 GB HBM2
|7
|1555 GB/s
|
|
|400W
|May 2020
|-
|A30<ref>https://www.nvidia.com/content/dam/en-zz/Solutions/data-center/products/a30-gpu/pdf/a30-datasheet.pdf</ref>
|Ampere
|7424
|184
|No
|
|
|24GB HBM2
|4 MIGs @ 6GB each
2 MIGs @ 12GB each


1 MIGs @ 24G
=== NVIDIA 드라이버 ===
|933GB/
|
|
|165W
|Apr 2021
|-
|A40<ref>https://images.nvidia.com/content/Solutions/data-center/a40/nvidia-a40-datasheet.pdf</ref>
|Ampere
|10752
|336
|84
|NVIDIA® NVLink® 112.5 GB/s
(bidirectional)3 PCIe Gen4: 64GB/s
|PCI
4.4" (H) x 10.5" (L) dual sl, Passive
|48 GB GDDR6 with ECC
|
|696 GB/s
|
|
|300W
|Apr 2021
|-
|A10<ref>https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/a10/pdf/datasheet-new/nvidia-a10-datasheet.pdf</ref>
|Ampere
|10240
|320
|No
|
|
|24 GB GDDR6 with ECC
|
|600 GB/s
|
|
|150W
|Mar 2021
|-
|A16<ref>https://images.nvidia.com/content/Solutions/data-center/vgpu-a16-datasheet.pdf</ref>
|Ampere
|5120
|3rd Gen 160
|40
|
|PCIe Gen4 x16
|64 GB GDDR6
|
|800 GB/s
|
|
|250W
|Mar 2021
|-
|A100 80GB
|Ampere
(GA100)
|6912
|432
| -
|
|
|80 GB HBM2e
|7@
10GB
|1935GB/s
|
|
|300W
|Apr 2021
|-
|A100 40GB
|Ampere
(GA100)
|6912
|432
|Yes
|
|
|40 GB HBM2
|7@
5GB
|1555 GB/s
|
|
|250W
|May 2020
|-
|A200 PCIe
|Ampere
|10752
|672
|Yes
|
|
|80 GB HBM2 / 160 GB HBM2
|
|2050 GB/s
|
|
|400W
|Nov 2021
|-
|A200 SXM4
|Ampere
|10752
|672
|Yes
|
|
|80 GB HBM2 / 160 GB HBM2
|
|2050 GB/s
|
|
|400W
|Nov 2021
|-
|RTX A4500<ref>https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/rtx/nvidia-rtx-a4500-datasheet.pdf</ref>
|Ampere
|7,16
|224
|56
|2Way (2slots or 3slots)
112.5 GB/s (bidirectional)
|PCI Express Gen 4 x 16
|20 GB GDDR6 with ECC
|
|640 GB/s
|Active
|1x 8-pin PCI
|200W
|2023
|-
|Quadro RTX 6000 Ada<ref>https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/proviz-print-rtx6000-datasheet-web-2504660.pdf</ref>
|Ada Lovelace
|18176
|fourth-generation 568
|third-generation
142
|No NVLink
VR ready
vGPU ready
|PCIe 4.0 x16
|48GB GDDR6 with ECC
|
|960 GB/s
|Active
|1x PCIe CEM5 16-pin
|300 W
|Jan 2023
|-
|A6000<ref>https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/quadro-product-literature/proviz-print-nvidia-rtx-a6000-datasheet-us-nvidia-1454980-r9-web%20(1).pdf</ref>
|Ampere
|10752
|336
|84
|
|
|48 GB GDDR6
|
|768 GB/s
|
|
|300 W
|
|-
|A5000<ref>https://nvdam.widen.net/s/wrqrqt75vh/nvidia-rtx-a5000-datasheet</ref>
|Ampere
|8192
|256
|64
|112.5 GB/s (bidirectional
|PCIe 4.0 x16
|24 GB GDDR6 with ECC
|
|768 GB/s
|
|1x 8-pin PCIe
|230W
|Apr 2021
|-
|'''''RTX 5000 Ada'''''<ref>https://www.nvidia.com/en-us/design-visualization/rtx-5000/</ref><ref>https://resources.nvidia.com/en-us-design-viz-stories-ep/rtx-5000-ada-datasheet?lx=CCKW39&contentType=data-sheet</ref>
|Ada Lovelace
|12800
|400
|100
|
|PCIe 4.0 x16
|32GB GDDR6 with ECC
|
|576GB/s
|
|
|250W
|
|-
|NVIDIA RTX A5500<ref>https://www.nvidia.com/content/dam/en-zz/Solutions/gtcs22/rtx-a5500/nvidia-rtx-a5500-datasheet.pdf</ref>
|Ampere
|10,240
|320 (3rd Gen)
|80 (2nd Gen)
|Low profile bridges connect two
NVIDIA RTX A5500 GPUs,


112.5 GB/s (bidirectional
{| class="wikitable"
|PCI Express 4.0 x16
! GPU
|24 GB GDDR6 with ECC
! CUDA Version
|
! NVIDIA Driver Version
|768 GB/s
|
|1x 8-pin PCIe
|230 W
|
|-
|A4000<ref>https://www.nvidia.com/content/dam/en-zz/Solutions/gtcs21/rtx-a4000/nvidia-rtx-a4000-datasheet.pdf</ref>
|Ampere
|6144
|192
|Yes
|
|
|16 GB GDDR6
|
|512 GB/s
|
|
|140W
|Apr 2021
|-
|NVIDIA RTX 4000 SFF Ada Generation
|Ada Lovelace
|6,14
|192 (4th Gen)
|48 (3rd Gen)
|
|
|20 GB
|
|320 GB/
|
|
|
|
|-
|A3000
|Ampere
|3584
|112
|Yes
|
|
|24 GB G
|
|
|
|
|
|
|-
|Titan RTX
|Turing
|4608
|576
|Yes
|
|
|24 GB GDDR6
|
|672 GB/s
|
|
|280W
|Dec 2018
|-
|GeForce RTX 5090
|Blackwell
|21,760
|318(4th Gen)
|3,352 (5th Gen)
|
|PCIe 5.0 16x
|32GB GDDR7
|
|1,792GB/s
|
|1x 16pin
|575W
|'''Jan, 30, 2025'''
|-
|GeForce RTX 4090
|Ada Lovelace
|16384
|512
|Yes, 128
|
|
|24 GB GDDR6X
|
|21.2Gbps
|
|
|450W
|
|-
|GeForce RTX 3090 Ti
|Turing
|10752
|336
|84
|
|
|24 GB GDDR6X
|
|21.2Gbps
|
|
|450W
|
|-
|GeForce RTX 3090
|Turing
|10496
|328
|Yes
|
|
|24 GB GDDR6X
|
|936 GB/s
|
|
|350W
|Sep 2020
|-
|GeForce RTX 3080 Ti
|Turing
|10240
|320
|Yes
|
|
|12 GB GDDR6X
|
|912 GB/s
|
|
|350W
|May 2021
|-
|GeForce RTX 3080
|Turing
|8704
|272
|Yes
|
|
|10 GB GDDR6X
|
|760 GB/s
|
|
|320W
|Sep 2020
|-
|GeForce RTX 3070 Ti
|Turing
|6144
|192
|Yes
|
|
|8 GB GDDR6X
|
|608 GB/s
|
|
|290W
|Jun 2021
|-
|GeForce RTX 3070
|Turing
|5888
|184
|Yes
|
|
|8 GB GDDR6
|
|448 GB/s
|
|
|220W
|Oct 2020
|-
|GeForce RTX 3060 Ti
|Turing
|4864
|152
|Yes
|
|
|8 GB GDDR6
|
|448 GB/s
|
|
|200W
|Dec 2020
|-
|GeForce RTX 3060
|Turing
|3584
|112
|No
|
|
|12 GB GDDR6
|
|360 GB/s
|
|
|170W
|Feb 2021
|-
|Quadro RTX 8000
|Turing
|4608
|576
|Yes
|
|
|48 GB GDDR6
|
|624 GB/s
|
|
|295W
|Aug 2018
|-
|Quadro RTX 6000
|Turing
|4608
|576
|Yes
|
|
|24 GB GDDR6
|
|432 GB/s
|
|
|260W
|Aug 2018
|-
|Tesla L40<ref>https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/support-guide/NVIDIA-L40-Datasheet-January-2023.pdf</ref>
|Ada Lovelace
|18176
|4th Gen 568
|3rd Gen
142
|No
|PCIe Gen4 x16: 64GB/s bidirectional
|48GB GDDR6 with ECC
|No
|864GB/s
|Passive
|16-pin
|300W
|Jan 2023
|-
|Tesla L40S<ref>https://resources.nvidia.com/en-us-l40s/l40s-datasheet-28413</ref>
|Ada Lovelace
|18176
|4th Gen 568
|3rd Gen
142
|No
|PCIe Gen4 x16: 64GB/s bidirectional
|48GB GDDR6 with ECC
|No
|864GB/s
|Passive
|16-pin
|350W
|2023
|-
|Quadro RTX 5000
|Turing
|3072
|384
|Yes
|
|
|16 GB GDDR6
|
|448 GB/s
|
|
|230W
|Nov 2018
|-
|Quadro RTX 4000
|Turing
|2304
|288
|Yes
|
|
|8 GB GDDR6
|
|416 GB/s
|
|
|160W
|Nov 2018
|-
|Titan RTX (T-Rex)
|Turing
|4608
|576
|No
|
|
|24 GB
|
|672 Gb/s
|
|
|280 W
|
|-
|Titan V
|Volta
|5120
|640
|
|
|
|12 GB HBM2
|
|652.8 GB/s
|
|
|250W
|Dec 2017
|-
|-
|Tesla V100 (PCIe)
| A100 / A30
|Volta
| CUDA 11
|5120
| R525 (>= 525.53)
|640
|No
|
|
|32/16 GB HBM2
|
|900 GB/s
|
|
|250W
|June 2017
|-
|-
|Tesla V100 (SXM2)
| H100 / H200
|Volta
| CUDA 12
|5120
| R450 (>= 450.80.02)
|640
|No
|
|
|32/16 GB HBM2
|
|900 GB/s
|
|
|300W
|June 2017
|-
|-
|Quadro GV100
| B200
|Volta
| CUDA 12
|5120
| R570 (>= 570.133.20)
|640
|No
|
|
|32 GB HBM2
|
|870 GB/s
|
|
|250W
|Mar 2018
|-
|-
|Tesla GV100 (SXM2)
| RTX PRO 6000 Blackwell
|Volta
| CUDA 12
|5120
| R575 (>= 575.51.03)
|640
|No
|
|
|32 GB HBM2
|
|900 GB/s
|
|
|300W
|Mar 2018
|}
|}


{{NVIDIA ARCH TIMELINE}}
=== nvidia-smi 기본 명령어 ===
 
<syntaxhighlight lang="bash">
# GPU 상태 확인
$ nvidia-smi
 
# GPU 목록 확인
$ nvidia-smi -L
 
# 모니터링 (1초 간격)
$ nvidia-smi -l 1
 
# MIG 상태 확인
$ nvidia-smi --query-gpu=pci.bus_id,mig.mode.current --format=csv
 
# Persistence Mode 활성화
$ sudo nvidia-smi -pm 1
 
# MIG 활성화
$ sudo nvidia-smi -mig 1
</syntaxhighlight>


=== NVIDIA Features by Architecture<ref>https://videocardz.com/newz/nvidia-details-ad102-gpu-up-to-18432-cuda-cores-76-3b-transistors-and-608-mm%C2%B2</ref> ===
=== GPU 모니터링 ===


{| class="wikitable"
{| class="wikitable"
! colspan="7" |NVIDIA GPU Architectures
! 명령어
!
! 설명
|-
!
!AD102
!GA102
!GA100
!TU102
!GV100
!GP102
!GP100
|-
!Launch Year
|'''2022'''
|2020
|2020
|2018
|2017
|2017
|–
|-
!Architecture
|'''Ada Lovelace'''
|Ampere
|Ampere
|Turing
|Volta
|Pascal
|Pascal
|-
!Form Factor
|–
|–
|SXM4/PCIe
|–
|SXM2/PCIe
|–
|SXM/PCIe
|-
|-
!TDP
| nvidia-smi
|–
| 기본 GPU 상태
|–
|400W
|–
|300W
|–
|300W
|-
|-
!Node
| nvidia-smi -L
|'''TSMC 4N'''
| GPU UUID 목록
|SAMSUNG 8N
|–
|TSMC 12nm
|TSMC 12nm
|TSMC 16nm
|–
|-
|-
!CUDA Cores
| nvidia-smi mig -lgi
|'''18432'''
| MIG 인스턴스 목록
|10752
|–
|4608
|5120
|3840
|–
|-
|-
!Tensor Cores
| nvidia-smi mig -lgip
|'''576 Gen4'''
| MIG 프로파일 목록
|336 Gen3
|–
|576 Gen2
|640
|–
|–
|-
|-
!RT Cores
| dcgmi dmon
|'''144 Gen3'''
| DCGM 고급 모니터링
|84 Gen2
|–
|72 Gen1
|–
|–
|–
|-
|-
!Memory Bus
| nvtop
|'''GDDR6X 384-bit'''
| 실시간 모니터링 (nvtop 설치 필요)
|GDDR6X 384-bit
|–
|GDDR6 384-bit
|HBM2 3072-bit
|GDDR6X 384-bit
|–
|}
|}


=== NVIDIA Grace Architecture ===
---
NVIDIA has announced that they will be partnering with server manufacturers such as HPE, Atos, and Supermicro to create servers that integrate the Grace architecture with ARM-based CPUs. These servers are expected to be available in the second half of 2023, by then [http://www.hpcmate.com/ HPCMATE] starts to offer those products through local and global partners.
 
{| class="wikitable"
== Configuration ==
!Architecture
 
!Key Features
<syntaxhighlight lang="bash">
|-
# NVIDIA 드라이버 설치
| rowspan="4" |Grace
$ sudo apt-get install nvidia-driver-535
|CPU-GPU integration, ARM Neoverse CPU, HBM2E memory
 
|-
# Persistence Mode 활성화
|900 GB/s memory bandwidth, support for PCIe 5.0 and NVLink
$ sudo nvidia-smi -pm 1
|-
 
|10x performance improvement for certain HPC workloads
# MIG 활성화
|-
$ sudo nvidia-smi -mig 1
|Energy efficiency improvements through unified memory space
 
|}
# GPU 상태 확인
$ nvidia-smi
 
# MIG 인스턴스 생성
$ sudo nvidia-smi mig -cgi 9,19,19,19 -C
</syntaxhighlight>
 
---
 
== Examples ==
 
=== Example 1: GPU 상태 확인 ===
 
<syntaxhighlight lang="bash">
# 전체 GPU 상태
$ nvidia-smi
 
# GPU UUID 확인
$ nvidia-smi -L
 
# MIG 상태 확인
$ nvidia-smi --query-gpu=pci.bus_id,mig.mode.current --format=csv
</syntaxhighlight>
 
=== Example 2: Docker에서 GPU 사용 ===
 
<syntaxhighlight lang="bash">
# 전체 GPU 사용
$ docker run --gpus all --rm nvidia/cuda:12.0 nvidia-smi
 
# 특정 GPU 사용
$ docker run --gpus '"device=0"' --rm nvidia/cuda:12.0 nvidia-smi
 
# MIG 디바이스 사용
$ docker run --gpus '"device=MIG-UUID"' --rm nvidia/cuda:12.0 nvidia-smi
</syntaxhighlight>
 
---
 
== Best Practices ==
 
* Persistence Mode 필수 활성화
* MIG 활성화 시 nvsm/dcgm 중지 후 진행
* 워크로드에 맞는 MIG 프로파일 선택
* DCGM v2.0.13+ 사용 (MIG 모니터링)
* 정기적인 드라이버 업데이트
* RTX PRO Blackwell은 Display Mode 먼저 Compute로 설정
* SSH 접근 확인 후 Display Mode 변경
 
---
 
== Limitations ==
 
* MIG 활성화/비활성화는 GPU 리셋 필요
* 그래픽 컨텍스트 미지원 (MIG)
* P2P/NVLink 미지원 (MIG)
* nvidia-smi만으로는 MIG 메트릭 확인 불가
* RTX PRO Blackwell primary display GPU는 Display Mode 변경 시 디스플레이 출력 중단
 
---
 
== References ==
 
* https://docs.nvidia.com/datacenter/tesla/mig-user-guide/
* https://www.nvidia.com/en-us/data-center/
 
---
 
== Related Pages ==
 
* [[MIG]]
* [[CUDA]]
* [[NVIDIA driver]]
* [[NVIDIA GPU Cloud (NGC)]]
* [[DCGM]]
 
---


=== Reference ===
<references/>
[[Category:GPU]]
[[Category:GPU]]
[[Category:Reference]]
[[Category:Reference]]

Revision as of 13:07, 15 July 2026

NVIDIA GPU

Template:Status

Template:TOC

Overview

NVIDIA GPU(Graphics Processing Unit)는 병렬 컴퓨팅을 위한 그래픽 처리 장치입니다. 게이밍부터 HPC, AI/ML까지 광범위하게 사용됩니다. Ampere, Hopper, Blackwell 아키텍처를 통해 지속적인 성능 향상을 이루고 있습니다.

Summary

  • 무엇인가? — NVIDIA의 그래픽 및 컴퓨팅 처리 장치
  • 왜 필요한가? — 병렬 컴퓨팅, AI/ML 훈련/추론, 그래픽 렌더링
  • 언제 사용하는가? — AI 서버, HPC 클러스터, 워크스테이션

---

Purpose

이 문서가 존재하는 이유

  • Goal: NVIDIA GPU 제품군, 아키텍처, MIG, 드라이버, 모니터링 방법 제공
  • Scope: GPU 라인업, 아키텍처 비교, MIG, 드라이버 설치, nvidia-smi
  • Non-goals: CUDA 프로그래밍, 특정 애플리케이션 설정은 별도 문서

---

Key Concepts

Concept Description Related
Ampere SM80 아키텍처 — A100/H100 기반 NVIDIA GPU
Hopper SM90 아키텍처 — H200/B200 기반 NVIDIA GPU
Blackwell SM100 아키텍처 — B200 기반 NVIDIA GPU
MIG Multi-Instance GPU — GPU 분할 MIG
CUDA NVIDIA 병렬 컴퓨팅 플랫폼 CUDA
Tensor Core AI 연산 전용 코어 NVIDIA GPU
NVLink GPU 간 고속 연결 NVIDIA GPU
nvidia-smi GPU 모니터링/관리 도구 NVIDIA GPU

---

Architecture

NVIDIA GPU 아키텍처 진화: 

graph LR
    A[Turing] --> B[Ampere]
    B --> C[Hopper]
    C --> D[Blackwell]
    B --> E[A100]
    B --> F[A30]
    C --> G[H100]
    C --> H[H200]
    D --> I[B200]
    D --> J[RTX PRO 6000]
    D --> K[RTX PRO 5000]

---

Workflow

Stage Input Output
Install Driver NVIDIA driver package nvidia-smi
Check Status nvidia-smi GPU info
Enable MIG nvidia-smi -mig 1 MIG Mode
Create Instance nvidia-smi -cgi GPU Instances
Monitor nvidia-smi / dcgmi Metrics

---

Detailed Explanation

GPU 제품군

제품군 아키텍처 주요 모델 용도
Data Center Ampere/Hopper/Blackwell A100, H100, H200, B200 AI/ML, HPC
RTX Workstation Ada/Blackwell RTX 6000 Ada, RTX PRO 6000 워크스테이션, VFX
GeForce Ada RTX 4090, RTX 4080 게이밍, 엔트리 AI
L-Series Ada L40S, L4 가상 데스크톱, 인퍼런스

아키텍처 비교

아키텍처 SM Tensor Core Memory Type 대표 모델
Ampere (SM80) 80 3rd Gen HBM2e/HBM2 A100, A30
Hopper (SM90) 90 4th Gen HBM3/HBM3e H100, H200
Blackwell (SM100) 100 5th Gen HBM3e B200, RTX PRO 6000

MIG (Multi-Instance GPU)

단일 GPU를 여러 격리된 인스턴스로 분할합니다. Ampere 아키텍처부터 도입되었습니다.

  • GI(GPU Instance): 완전 격리된 리소스 집합
  • CI(Compute Instance): GI 내 부분 격리된 SM 집합
  • 18가지 프로파일 조합 가능

자세한 내용은 MIG 문서를 참조하세요.

NVIDIA 드라이버

GPU CUDA Version NVIDIA Driver Version
A100 / A30 CUDA 11 R525 (>= 525.53)
H100 / H200 CUDA 12 R450 (>= 450.80.02)
B200 CUDA 12 R570 (>= 570.133.20)
RTX PRO 6000 Blackwell CUDA 12 R575 (>= 575.51.03)

nvidia-smi 기본 명령어

# GPU 상태 확인
$ nvidia-smi

# GPU 목록 확인
$ nvidia-smi -L

# 모니터링 (1초 간격)
$ nvidia-smi -l 1

# MIG 상태 확인
$ nvidia-smi --query-gpu=pci.bus_id,mig.mode.current --format=csv

# Persistence Mode 활성화
$ sudo nvidia-smi -pm 1

# MIG 활성화
$ sudo nvidia-smi -mig 1

GPU 모니터링

명령어 설명
nvidia-smi 기본 GPU 상태
nvidia-smi -L GPU UUID 목록
nvidia-smi mig -lgi MIG 인스턴스 목록
nvidia-smi mig -lgip MIG 프로파일 목록
dcgmi dmon DCGM 고급 모니터링
nvtop 실시간 모니터링 (nvtop 설치 필요)

---

Configuration

# NVIDIA 드라이버 설치
$ sudo apt-get install nvidia-driver-535

# Persistence Mode 활성화
$ sudo nvidia-smi -pm 1

# MIG 활성화
$ sudo nvidia-smi -mig 1

# GPU 상태 확인
$ nvidia-smi

# MIG 인스턴스 생성
$ sudo nvidia-smi mig -cgi 9,19,19,19 -C

---

Examples

Example 1: GPU 상태 확인

# 전체 GPU 상태
$ nvidia-smi

# GPU UUID 확인
$ nvidia-smi -L

# MIG 상태 확인
$ nvidia-smi --query-gpu=pci.bus_id,mig.mode.current --format=csv

Example 2: Docker에서 GPU 사용

# 전체 GPU 사용
$ docker run --gpus all --rm nvidia/cuda:12.0 nvidia-smi

# 특정 GPU 사용
$ docker run --gpus '"device=0"' --rm nvidia/cuda:12.0 nvidia-smi

# MIG 디바이스 사용
$ docker run --gpus '"device=MIG-UUID"' --rm nvidia/cuda:12.0 nvidia-smi

---

Best Practices

  • Persistence Mode 필수 활성화
  • MIG 활성화 시 nvsm/dcgm 중지 후 진행
  • 워크로드에 맞는 MIG 프로파일 선택
  • DCGM v2.0.13+ 사용 (MIG 모니터링)
  • 정기적인 드라이버 업데이트
  • RTX PRO Blackwell은 Display Mode 먼저 Compute로 설정
  • SSH 접근 확인 후 Display Mode 변경

---

Limitations

  • MIG 활성화/비활성화는 GPU 리셋 필요
  • 그래픽 컨텍스트 미지원 (MIG)
  • P2P/NVLink 미지원 (MIG)
  • nvidia-smi만으로는 MIG 메트릭 확인 불가
  • RTX PRO Blackwell primary display GPU는 Display Mode 변경 시 디스플레이 출력 중단

---

References

---

Related Pages

---

Retrieved from "http://wiki.hpcmate.com/index.php?title=NVIDIA_GPU&oldid=2657"