CUDA: Difference between revisions

From HPCWIKI
Jump to navigation Jump to search
(Add categories: GPU, Reference)
(Phase 6.1: LLM-Optimized Wiki Template 적용)
Line 1: Line 1:
== Compile CUDA code ==
{{Status
When you compile CUDA code, you should always compile only one ‘'''<code>-arch</code>'''‘ flag that matches your most used GPU cards. This will <u>enable faster runtime</u>, because code generation will occur during compilation.
|status=Draft
|owner=Knowledge Agent
|last_update=2026-07-16
|review=Pending
}}


If you only mention ‘'''<code>-gencode</code>'''‘, but omit the ‘'''<code>-arch</code>'''‘ flag, the <u>GPU code generation will occur on the '''JIT''' compiler by the CUDA driver.</u>
{{TOC}}


When you want to speed up CUDA compilation, you want to reduce the amount of irrelevant ‘'''<code>-gencode</code>'''‘ flags. However, sometimes you may wish to have better CUDA backwards compatibility by adding more comprehensive ‘'''<code>-gencode</code>'''‘ flags.
== Overview ==


== Sample nvcc gencode and arch Flags in GCC ==
CUDA(NVIDIA Compute Unified Device Architecture)는 NVIDIA GPU에서 일반 연산을 수행할 수 있는 병렬 컴퓨팅 플랫폼과 프로그래밍 모델입니다.
According to NVIDIA:<blockquote>The <code>arch=</code> clause of the <code>-gencode=</code> command-line option to <code>nvcc</code> specifies the front-end compilation target and must always be a PTX version. The <code>code=</code> clause specifies the back-end compilation target and can either be cubin or PTX or both. Only the back-end target version(s) specified by the <code>code=</code> clause will be retained in the resulting binary; at least one must be PTX to provide Ampere compatibility.</blockquote>


* GCC generation on CUDA 7.0 for maximum compatibility with all cards from the era
=== Summary ===


-arch=sm_30 \
* 무엇인가? NVIDIA GPU 병렬 컴퓨팅 플랫폼 및 프로그래밍 모델
  -gencode=arch=compute_20,code=sm_20 \
* 왜 필요한가? GPU의 대규모 병렬 처리 능력을 활용하여 AI/ML, 과학 계산, 영상 처리 가속
  -gencode=arch=compute_30,code=sm_30 \
* 언제 사용하는가? 딥러닝 훈련/추론, 과학 시뮬레이션, 영상 처리, 암호화 등
  -gencode=arch=compute_50,code=sm_50 \
  -gencode=arch=compute_52,code=sm_52 \
  -gencode=arch=compute_52,code=compute_52


* generation on CUDA 8.1 for maximum compatibility with cards predating Volta:
---


-arch=sm_30 \
== Purpose ==
  -gencode=arch=compute_20,code=sm_20 \
  -gencode=arch=compute_30,code=sm_30 \
  -gencode=arch=compute_50,code=sm_50 \
  -gencode=arch=compute_52,code=sm_52 \
  -gencode=arch=compute_60,code=sm_60 \
  -gencode=arch=compute_61,code=sm_61 \
  -gencode=arch=compute_61,code=compute_61


* generation on CUDA 9.2 for maximum '''compatibility''' with Volta cards
이 문서가 존재하는 이유


  -arch=sm_50 \
* Goal: CUDA 컴파일 방법, nvcc 플래그, PyTorch/TensorRT 설정, 호환성 정보 제공
* Scope: CUDA 컴파일 플래그, TORCH_CUDA_ARCH_LIST, CMake 설정, CUDA 호환성
* Non-goals: CUDA C++ 프로그래밍 상세, 커널 최적화 기법
 
---
 
== Key Concepts ==
 
{| class="wikitable"
! Concept
! Description
! Related
|-
| CUDA
| NVIDIA Compute Unified Device Architecture — GPU 병렬 컴퓨팅 플랫폼
| [[NVIDIA GPU]]
|-
| nvcc
| NVIDIA CUDA 컴파일러 — CUDA 코드를 GPU 명령어로 컴파일
| [[CUDA]]
|-
| -arch
| 컴파일 대상 GPU 아키텍처 지정 플래그
| [[CUDA]]
|-
| -gencode
| 생성할 GPU 코드 지정 플래그
| [[CUDA]]
|-
| Compute Capability
| GPU 아키텍처 버전 (sm_XX, compute_XX)
| [[NVIDIA GPU]]
|-
| TORCH_CUDA_ARCH_LIST
| PyTorch CUDA 아키텍처 환경 변수
| [[PyTorch]]
|-
| TensorRT
| NVIDIA 추론 최적화 라이브러리
| [[TensorRT]]
|-
| CUTLASS
| CUDA 템플릿 라이브러리 for Linear Algebra
| [[CUDA]]
|}
 
---
 
== Architecture ==
 
CUDA 컴파일 프로세스:
 
<syntaxhighlight lang="mermaid">
graph TD
    A[CUDA 소스 코드] --> B[nvcc 컴파일러]
    B --> C{arch 플래그}
    C -->|지정됨| D[컴파일 시간 코드 생성]
    C -->|미지정| E[JIT 컴파일러]
    D --> F[최적화된 GPU 바이너리]
    E --> F
    F --> G[GPU 실행]
</syntaxhighlight>
 
---
 
== Workflow ==
 
CUDA 컴파일 워크플로우:
 
# CUDA 코드 컴파일
# nvcc -arch=sm_XX -gencode=arch=compute_XX,code=sm_XX source.cu -o output
 
# PyTorch 설치 시 아키텍처 지정
# TORCH_CUDA_ARCH_LIST="7.0 7.5 8.0 8.6" python3 setup.py install
 
# TensorRT CMake 설정
# cmake <...> -DGPU_ARCHS="70"
 
---
 
== Detailed Explanation ==
 
=== CUDA 컴파일 기본 ===
 
CUDA 코드를 컴파일할 때 가장 많이 사용하는 GPU 카드와 일치하는 '''<code>-arch</code>''' 플래그 하나만 컴파일해야 합니다. 이렇게 하면 컴파일 시 코드 생성이 발생하여 '''런타임이 빨라집니다'''.
 
<code>-gencode</code>만 지정하고 <code>-arch</code> 플래그를 생략하면 '''CUDA 드라이버에 의해 JIT 컴파일러에서 GPU 코드 생성이 발생합니다'''.
 
CUDA 컴파일을 빠르게 하려면 관련 없는 <code>-gencode</code> 플래그의 양을 줄여야 합니다. 그러나 때로는 더 포괄적인 <code>-gencode</code> 플래그를 추가하여 더 나은 CUDA 후방 호환성을 원할 수 있습니다.
 
=== NVIDIA 설명: arch= 와 code= ===
 
NVIDIA에 따르면:
 
<blockquote>The <code>arch=</code> clause of the <code>-gencode=</code> command-line option to <code>nvcc</code> specifies the front-end compilation target and must always be a PTX version. The <code>code=</code> clause specifies the back-end compilation target and can either be cubin or PTX or both. Only the back-end target version(s) specified by the <code>code=</code> clause will be retained in the resulting binary; at least one must be PTX to provide Ampere compatibility.</blockquote>
 
=== nvcc gencode 및 arch 플래그 예제 ===
 
==== CUDA 7.0 — 모든 카드 호환 ====
 
<syntaxhighlight lang="bash">
-arch=sm_30 \
-gencode=arch=compute_20,code=sm_20 \
  -gencode=arch=compute_30,code=sm_30 \
-gencode=arch=compute_50,code=sm_50 \
-gencode=arch=compute_52,code=sm_52 \
-gencode=arch=compute_52,code=compute_52
</syntaxhighlight>
 
==== CUDA 8.1 — Volta 이전 카드 호환 ====
 
<syntaxhighlight lang="bash">
-arch=sm_30 \
-gencode=arch=compute_20,code=sm_20 \
-gencode=arch=compute_30,code=sm_30 \
  -gencode=arch=compute_50,code=sm_50 \
  -gencode=arch=compute_50,code=sm_50 \
  -gencode=arch=compute_52,code=sm_52 \
  -gencode=arch=compute_52,code=sm_52 \
  -gencode=arch=compute_60,code=sm_60 \
  -gencode=arch=compute_60,code=sm_60 \
  -gencode=arch=compute_61,code=sm_61 \
  -gencode=arch=compute_61,code=sm_61 \
  -gencode=arch=compute_70,code=sm_70 \  
-gencode=arch=compute_61,code=compute_61
</syntaxhighlight>
 
==== CUDA 9.2 — Volta 카드 호환 ====
 
<syntaxhighlight lang="bash">
-arch=sm_50 \
-gencode=arch=compute_50,code=sm_50 \
-gencode=arch=compute_52,code=sm_52 \
-gencode=arch=compute_60,code=sm_60 \
-gencode=arch=compute_61,code=sm_61 \
  -gencode=arch=compute_70,code=sm_70 \
  -gencode=arch=compute_70,code=compute_70
  -gencode=arch=compute_70,code=compute_70
</syntaxhighlight>


* Sample flags for generation on '''CUDA 10.1''' for maximum '''compatibility''' with V100 and T4 Turing cards:
==== CUDA 10.1 — V100 및 T4 Turing 카드 호환 ====


-arch=sm_50 \  
<syntaxhighlight lang="bash">
  -gencode=arch=compute_50,code=sm_50 \  
-arch=sm_50 \
  -gencode=arch=compute_52,code=sm_52 \  
  -gencode=arch=compute_50,code=sm_50 \
  -gencode=arch=compute_60,code=sm_60 \  
  -gencode=arch=compute_52,code=sm_52 \
  -gencode=arch=compute_61,code=sm_61 \  
  -gencode=arch=compute_60,code=sm_60 \
  -gencode=arch=compute_70,code=sm_70 \  
  -gencode=arch=compute_61,code=sm_61 \
  -gencode=arch=compute_70,code=sm_70 \
  -gencode=arch=compute_75,code=sm_75 \
  -gencode=arch=compute_75,code=sm_75 \
  -gencode=arch=compute_75,code=compute_75  
  -gencode=arch=compute_75,code=compute_75
</syntaxhighlight>


* Sample flags for generation on '''CUDA 11.0''' for maximum '''compatibility''' with V100 and T4 Turing cards:
==== CUDA 11.0 — V100 및 T4 Turing 카드 호환 ====


-arch=sm_52 \  
<syntaxhighlight lang="bash">
  -gencode=arch=compute_52,code=sm_52 \  
-arch=sm_52 \
  -gencode=arch=compute_60,code=sm_60 \  
  -gencode=arch=compute_52,code=sm_52 \
  -gencode=arch=compute_61,code=sm_61 \  
  -gencode=arch=compute_60,code=sm_60 \
  -gencode=arch=compute_70,code=sm_70 \  
  -gencode=arch=compute_61,code=sm_61 \
  -gencode=arch=compute_70,code=sm_70 \
  -gencode=arch=compute_75,code=sm_75 \
  -gencode=arch=compute_75,code=sm_75 \
  -gencode=arch=compute_80,code=sm_80 \
  -gencode=arch=compute_80,code=sm_80 \
  -gencode=arch=compute_80,code=compute_80  
  -gencode=arch=compute_80,code=compute_80
</syntaxhighlight>


* Sample flags for generation on '''CUDA 11.7''' for maximum '''compatibility''' with V100 and T4 Turing cards, but also [[support]] newer RTX 3080, and Drive AGX Orin:
==== CUDA 11.7 — V100, T4, RTX 3080, Drive AGX Orin 호환 ====


-arch=sm_52 \  
<syntaxhighlight lang="bash">
  -gencode=arch=compute_52,code=sm_52 \  
-arch=sm_52 \
  -gencode=arch=compute_60,code=sm_60 \  
  -gencode=arch=compute_52,code=sm_52 \
  -gencode=arch=compute_61,code=sm_61 \  
  -gencode=arch=compute_60,code=sm_60 \
  -gencode=arch=compute_70,code=sm_70 \  
  -gencode=arch=compute_61,code=sm_61 \
  -gencode=arch=compute_70,code=sm_70 \
  -gencode=arch=compute_75,code=sm_75 \
  -gencode=arch=compute_75,code=sm_75 \
  -gencode=arch=compute_80,code=sm_80 \
  -gencode=arch=compute_80,code=sm_80 \
  -gencode=arch=compute_86,code=sm_86 \
  -gencode=arch=compute_86,code=sm_86 \
  -gencode=arch=compute_87,code=sm_87
  -gencode=arch=compute_87,code=sm_87 \
  -gencode=arch=compute_86,code=compute_86
  -gencode=arch=compute_86,code=compute_86
</syntaxhighlight>


* Sample flags for generation on '''CUDA 11.4''' for best performance with RTX 3080 cards:
==== CUDA 11.4 — RTX 3080 최적 성능 ====


-arch=sm_80 \  
<syntaxhighlight lang="bash">
-arch=sm_80 \
  -gencode=arch=compute_80,code=sm_80 \
  -gencode=arch=compute_80,code=sm_80 \
  -gencode=arch=compute_86,code=sm_86 \
  -gencode=arch=compute_86,code=sm_86 \
  -gencode=arch=compute_87,code=sm_87 \
  -gencode=arch=compute_87,code=sm_87 \
  -gencode=arch=compute_86,code=compute_86
  -gencode=arch=compute_86,code=compute_86
</syntaxhighlight>


* Sample flags for generation on '''CUDA 12''' for best performance with GeForce RTX 4080 cards:
==== CUDA 12 — RTX 4080 최적 성능 ====


-arch=sm_89 \  
<syntaxhighlight lang="bash">
-arch=sm_89 \
  -gencode=arch=compute_89,code=sm_89 \
  -gencode=arch=compute_89,code=sm_89 \
  -gencode=arch=compute_89,code=compute_89
  -gencode=arch=compute_89,code=compute_89
</syntaxhighlight>


* Sample flags for generation on '''CUDA 12''' (PTX ISA version 8.0) for best performance with NVIDIA H100 (Hopper) GPUs, and no backwards compatibility for previous generations:
==== CUDA 12 (PTX ISA version 8.0) — H100 (Hopper) 최적 성능 ====


-arch=sm_90 \  
<syntaxhighlight lang="bash">
-arch=sm_90 \
  -gencode=arch=compute_90,code=sm_90 \
  -gencode=arch=compute_90,code=sm_90 \
  -gencode=arch=compute_90a,code=sm_90a \
  -gencode=arch=compute_90a,code=sm_90a \
  -gencode=arch=compute_90a,code=compute_90a
  -gencode=arch=compute_90a,code=compute_90a
</syntaxhighlight>


* To add more compatibility for Hopper GPUs and some backwards compatibility:
==== Hopper GPU 후방 호환성 추가 ====


-arch=sm_52 \
<syntaxhighlight lang="bash">
  -gencode=arch=compute_52,code=sm_52 \
-arch=sm_52 \
  -gencode=arch=compute_60,code=sm_60 \
  -gencode=arch=compute_52,code=sm_52 \
  -gencode=arch=compute_61,code=sm_61 \
  -gencode=arch=compute_60,code=sm_60 \
  -gencode=arch=compute_70,code=sm_70 \
  -gencode=arch=compute_61,code=sm_61 \
  -gencode=arch=compute_75,code=sm_75 \  
  -gencode=arch=compute_70,code=sm_70 \
  -gencode=arch=compute_80,code=sm_80 \  
  -gencode=arch=compute_75,code=sm_75 \
  -gencode=arch=compute_86,code=sm_86 \  
  -gencode=arch=compute_80,code=sm_80 \
  -gencode=arch=compute_86,code=sm_86 \
  -gencode=arch=compute_87,code=sm_87 \
  -gencode=arch=compute_87,code=sm_87 \
  -gencode=arch=compute_90,code=sm_90 \  
  -gencode=arch=compute_90,code=sm_90 \
  -gencode=arch=compute_90,code=compute_90
  -gencode=arch=compute_90,code=compute_90
</syntaxhighlight>
=== PyTorch에서 TORCH_CUDA_ARCH_LIST 사용 ===
PyTorch를 사용하는 경우 설치 중 <code>TORCH_CUDA_ARCH_LIST</code> 환경 변수를 사용하여 아키텍처를 설정할 수 있습니다:
<syntaxhighlight lang="bash">
$ TORCH_CUDA_ARCH_LIST="7.0 7.5 8.0 8.6" python3 setup.py install
</syntaxhighlight>
이 변수에 모든 아키텍처를 지정할 수 있지만, 각 아키텍처마다 커널을 컴파일해야 하므로 빌드 시간이 길어집니다.


== Using TORCH_CUDA_ARCH_LIST for PyTorch ==
가장 최근 아키텍처에 <code>+PTX</code> 접미사를 추가하여 새 카드와 전방 호환인 PTX 코드를 생성할 수도 있습니다:
If you’re using PyTorch you can set the architectures using the <code>TORCH_CUDA_ARCH_LIST</code> env variable during installation like this:


<code>$ TORCH_CUDA_ARCH_LIST="7.0 7.5 8.0 8.6" python3 setup.py install</code>
<syntaxhighlight lang="bash">
$ TORCH_CUDA_ARCH_LIST="7.0 7.5 8.0 8.6+PTX" python3 build_my_extension.py
</syntaxhighlight>


Note that while you can specify every single arch in this variable, each one will prolong the build time as kernels will have to compiled for every architecture.
=== TensorRT CMake 사용 ===


You can also tell PyTorch to generate PTX code that is forward compatible by newer cards by adding a <code>+PTX</code> suffix to the most recent architecture you specify:
CMAKE로 TensorRT를 컴파일하는 경우 <code>sm_</code> 및 <code>compute_</code> 접두사를 제거하고 계산 능력만 참조합니다.


<code>$ TORCH_CUDA_ARCH_LIST="7.0 7.5 8.0 8.6+PTX" python3 build_my_extension.py</code>
Tesla V100 및 Volta 카드:


== Using Cmake for TensorRT ==
<syntaxhighlight lang="bash">
If you’re compiling TensorRT with CMAKE, drop the <code>sm_</code> and <code>compute_</code> prefixes, refer only to the compute capabilities instead.
cmake <...> -DGPU_ARCHS="70"
</syntaxhighlight>


Example for Tesla V100 and Volta cards in general:
NVIDIA RTX 2070 및 Tesla T4:


<code>cmake <...> -DGPU_ARCHS="70"</code>
<syntaxhighlight lang="bash">
cmake <...> -DGPU_ARCHS="75"
</syntaxhighlight>


Example for NVIDIA RTX 2070 and Tesla T4:
NVIDIA A100:


<code>cmake <...> -DGPU_ARCHS="75"</code>
<syntaxhighlight lang="bash">
cmake <...> -DGPU_ARCHS="80"
</syntaxhighlight>


Example for NVIDIA A100:
NVIDIA RTX 3080 및 A100 함께:


<code>cmake <...> -DGPU_ARCHS="80"</code>
<syntaxhighlight lang="bash">
cmake <...> -DGPU_ARCHS="80 86"
</syntaxhighlight>


Example for NVIDIA RTX 3080 and A100 together:
NVIDIA H100:


<code>cmake <...> -DGPU_ARCHS="80 86"</code>
<syntaxhighlight lang="bash">
cmake <...> -DGPU_ARCHS="90"
</syntaxhighlight>


Example for NVIDIA H100:
=== CUTLASS with Hopper GH100 CMake ===


<code>cmake <...> -DGPU_ARCHS="90"</code>
<syntaxhighlight lang="bash">
cmake .. -DCUTLASS_NVCC_ARCHS=90a
</syntaxhighlight>


== Using Cmake for CUTLASS with Hopper GH100 ==
=== "Value 'sm_86' is not defined for option 'gpu-architecture'" 오류 ===
<code>cmake .. -DCUTLASS_NVCC_ARCHS=90a</code>


== What does <code>"Value 'sm_86' is not defined for option 'gpu-architecture'"</code> mean? ==
이러한 오류가 발생하는 경우:
If you get an error that looks like this:
 
<code>nvcc fatal : Value 'sm_86' is not defined for option 'gpu-architecture'</code>
<syntaxhighlight lang="bash">
You probably have an '''older version of CUDA and/or the driver installed'''. Upgrade to a more recent driver, at least '''450.36.06''' or higher, to support sm_8x cards like the A100, RTX 3080.
nvcc fatal : Value 'sm_86' is not defined for option 'gpu-architecture'
</syntaxhighlight>
 
'''더 오래된 버전의 CUDA 및/또는 드라이버가 설치되어 있을 가능성이 높습니다'''. A100, RTX 3080과 같은 sm_8x 카드를 지원하려면 '''450.36.06''' 이상의 최신 드라이버로 업그레이드하세요.
 
=== CUDA 호환성 ===


=== CUDA Compatibility ===
{| class="wikitable"
{| class="wikitable"
!CUDA Version
! CUDA Version
![[cuDNN]] Version
! cuDNN Version
!NCCL Version
! NCCL Version
![[NVIDIA GPU]] Driver Version
! NVIDIA GPU Driver Version
!Compute Capability Support
! Compute Capability [[Support]]
|-
|-
|CUDA 12.0
| CUDA 12.0
|
| —
|
| —
|
| —
|
| —
|-
|-
|CUDA 11.8
| CUDA 11.8
|
| —
|
| —
|
| —
|
| —
|-
|-
|CUDA 11.7
| CUDA 11.7
|
| —
|
| —
|
| —
|
| —
|-
|-
|CUDA 11.5
| CUDA 11.5
|8.3.x
| 8.3.x
|2.10.x
| 2.10.x
|510.39 or later
| 510.39 or later
|Compute Capability 3.0 to 8.6
| Compute Capability 3.0 to 8.6
|-
|-
|CUDA 11.4
| CUDA 11.4
|8.2.x
| 8.2.x
|2.10.x
| 2.10.x
|470.42.01 or later
| 470.42.01 or later
|Compute Capability 3.0 to 8.6
| Compute Capability 3.0 to 8.6
|-
|-
|CUDA 11.3
| CUDA 11.3
|8.2.x
| 8.2.x
|2.10.x
| 2.10.x
|465.19.01 or later
| 465.19.01 or later
|Compute Capability 3.0 to 8.6
| Compute Capability 3.0 to 8.6
|-
|-
|CUDA 11.2
| CUDA 11.2
|8.1.x
| 8.1.x
|2.9.x
| 2.9.x
|460.32.03 or later
| 460.32.03 or later
|Compute Capability 3.0 to 8.6
| Compute Capability 3.0 to 8.6
|-
|-
|CUDA 11.1
| CUDA 11.1
|8.0.x
| 8.0.x
|2.9.x
| 2.9.x
|455.23.04 or later
| 455.23.04 or later
|Compute Capability 3.0 to 8.6
| Compute Capability 3.0 to 8.6
|-
|-
|CUDA 11.0
| CUDA 11.0
|7.6.x
| 7.6.x
|2.8.x
| 2.8.x
|450.36.06 or later
| 450.36.06 or later
|Compute Capability 3.0 to 8.6
| Compute Capability 3.0 to 8.6
|-
|-
|CUDA 10.2
| CUDA 10.2
|7.6.x
| 7.6.x
|2.7.x
| 2.7.x
|440.33 or later
| 440.33 or later
|Compute Capability 3.0 to 7.5
| Compute Capability 3.0 to 7.5
|-
|-
|CUDA 10.1
| CUDA 10.1
|7.6.x
| 7.6.x
|2.4.x
| 2.4.x
|418.39 or later
| 418.39 or later
|Compute Capability 3.0 to 7.5
| Compute Capability 3.0 to 7.5
|-
|-
|CUDA 10.0
| CUDA 10.0
|7.4.x
| 7.4.x
|2.2.x
| 2.2.x
|410.48 or later
| 410.48 or later
|Compute Capability 3.0 to 7.5
| Compute Capability 3.0 to 7.5
|-
|-
|CUDA 9.2
| CUDA 9.2
|7.2.x
| 7.2.x
|2.1.x
| 2.1.x
|396.26 or later
| 396.26 or later
|Compute Capability 3.0 to 7.5
| Compute Capability 3.0 to 7.5
|-
|-
|CUDA 9.1
| CUDA 9.1
|7.1.x
| 7.1.x
|2.0.x
| 2.0.x
|390.46 or later
| 390.46 or later
|Compute Capability 3.0 to 7.5
| Compute Capability 3.0 to 7.5
|-
|-
|CUDA 9.0
| CUDA 9.0
|7.0.x
| 7.0.x
|1.3.x
| 1.3.x
|384.81 or later
| 384.81 or later
|Compute Capability 3.0 to 7.5
| Compute Capability 3.0 to 7.5
|-
|-
|CUDA 8.0
| CUDA 8.0
|6.0.x
| 6.0.x
|1.3.x
| 1.3.x
|375.26 or later
| 375.26 or later
|Compute Capability 2.0 to 6.2
| Compute Capability 2.0 to 6.2
|-
|-
|CUDA 7.5
| CUDA 7.5
|5.1.x
| 5.1.x
|1.3.x
| 1.3.x
|352.31 or later
| 352.31 or later
|Compute Capability 2.0 to 5.2
| Compute Capability 2.0 to 5.0
|}
|}


===Reference===
---
<references/>
 
== Configuration ==
 
=== nvcc 컴파일 플래그 ===
 
<syntaxhighlight lang="bash">
# 기본 컴파일
nvcc -arch=sm_XX -gencode=arch=compute_XX,code=sm_XX source.cu -o output
 
# PyTorch 설치
TORCH_CUDA_ARCH_LIST="7.0 7.5 8.0 8.6" python3 setup.py install
 
# TensorRT CMake
cmake <...> -DGPU_ARCHS="70"
 
# CUTLASS CMake
cmake .. -DCUTLASS_NVCC_ARCHS=90a
</syntaxhighlight>
 
---
 
== Examples ==
 
=== Example 1: CUDA 11.7로 RTX 3080 컴파일 ===
 
입력:
 
<syntaxhighlight lang="bash">
nvcc -arch=sm_80 \
-gencode=arch=compute_80,code=sm_80 \
-gencode=arch=compute_86,code=sm_86 \
-gencode=arch=compute_87,code=sm_87 \
-gencode=arch=compute_86,code=compute_86 \
source.cu -o output
</syntaxhighlight>
 
=== Example 2: PyTorch 설치 ===
 
입력:
 
<syntaxhighlight lang="bash">
TORCH_CUDA_ARCH_LIST="7.0 7.5 8.0 8.6+PTX" python3 setup.py install
</syntaxhighlight>
 
---
 
== Best Practices ==
 
* '''가장 많이 사용하는 GPU 아키텍처만 -arch에 지정''' — 불필요한 -gencode 플래그 제거로 컴파일 시간 단축
* '''PyTorch 설치 시 TORCH_CUDA_ARCH_LIST 사용''' — 빌드 시간 단축
* '''+PTX 접미사 사용''' — 새 카드와의 전방 호환성 확보
* '''CUDA 드라이버 최신 유지''' — sm_8x 카드 지원에는 450.36.06 이상 필요
* '''TensorRT CMake 시 sm_/compute_ 접두사 제거''' — 계산 능력만 지정
 
---
 
== Limitations ==
 
* 각 아키텍처마다 커널을 컴파일하면 빌드 시간이 길어짐
* 구형 CUDA 버전은 새 GPU 아키텍처를 지원하지 않음
* CUDA 호환성은 드라이버 버전과 cuDNN/NCCL 버전과 밀접한 관계가 있음
* Hopper GPU(sm_90)는 CUDA 12 이상 필요
 
---
 
== References ==
 
* NVIDIA nvcc 문서
 
---
 
== Related Pages ==
 
* [[NVIDIA GPU]]
* [[cuDNN]]
* [[NCCL]]
* [[PyTorch]]
* [[TensorRT]]
* [[CUTLASS]]
* [[TORCH_CUDA_ARCH_LIST]]
 
---
 
[[Category:GPU]]
[[Category:GPU]]
[[Category:Reference]]
[[Category:Reference]]

Revision as of 14:30, 16 July 2026

Template:Status

Template:TOC

Overview

CUDA(NVIDIA Compute Unified Device Architecture)는 NVIDIA GPU에서 일반 연산을 수행할 수 있는 병렬 컴퓨팅 플랫폼과 프로그래밍 모델입니다.

Summary

  • 무엇인가? NVIDIA GPU 병렬 컴퓨팅 플랫폼 및 프로그래밍 모델
  • 왜 필요한가? GPU의 대규모 병렬 처리 능력을 활용하여 AI/ML, 과학 계산, 영상 처리 가속
  • 언제 사용하는가? 딥러닝 훈련/추론, 과학 시뮬레이션, 영상 처리, 암호화 등

---

Purpose

이 문서가 존재하는 이유

  • Goal: CUDA 컴파일 방법, nvcc 플래그, PyTorch/TensorRT 설정, 호환성 정보 제공
  • Scope: CUDA 컴파일 플래그, TORCH_CUDA_ARCH_LIST, CMake 설정, CUDA 호환성
  • Non-goals: CUDA C++ 프로그래밍 상세, 커널 최적화 기법

---

Key Concepts

Concept Description Related
CUDA NVIDIA Compute Unified Device Architecture — GPU 병렬 컴퓨팅 플랫폼 NVIDIA GPU
nvcc NVIDIA CUDA 컴파일러 — CUDA 코드를 GPU 명령어로 컴파일 CUDA
-arch 컴파일 대상 GPU 아키텍처 지정 플래그 CUDA
-gencode 생성할 GPU 코드 지정 플래그 CUDA
Compute Capability GPU 아키텍처 버전 (sm_XX, compute_XX) NVIDIA GPU
TORCH_CUDA_ARCH_LIST PyTorch CUDA 아키텍처 환경 변수 PyTorch
TensorRT NVIDIA 추론 최적화 라이브러리 TensorRT
CUTLASS CUDA 템플릿 라이브러리 for Linear Algebra CUDA

---

Architecture

CUDA 컴파일 프로세스: 

graph TD
    A[CUDA 소스 코드] --> B[nvcc 컴파일러]
    B --> C{arch 플래그}
    C -->|지정됨| D[컴파일 시간 코드 생성]
    C -->|미지정| E[JIT 컴파일러]
    D --> F[최적화된 GPU 바이너리]
    E --> F
    F --> G[GPU 실행]

---

Workflow

CUDA 컴파일 워크플로우:

  1. CUDA 코드 컴파일
  2. nvcc -arch=sm_XX -gencode=arch=compute_XX,code=sm_XX source.cu -o output
  1. PyTorch 설치 시 아키텍처 지정
  2. TORCH_CUDA_ARCH_LIST="7.0 7.5 8.0 8.6" python3 setup.py install
  1. TensorRT CMake 설정
  2. cmake <...> -DGPU_ARCHS="70"

---

Detailed Explanation

CUDA 컴파일 기본

CUDA 코드를 컴파일할 때 가장 많이 사용하는 GPU 카드와 일치하는 -arch 플래그 하나만 컴파일해야 합니다. 이렇게 하면 컴파일 시 코드 생성이 발생하여 런타임이 빨라집니다.

-gencode만 지정하고 -arch 플래그를 생략하면 CUDA 드라이버에 의해 JIT 컴파일러에서 GPU 코드 생성이 발생합니다.

CUDA 컴파일을 빠르게 하려면 관련 없는 -gencode 플래그의 양을 줄여야 합니다. 그러나 때로는 더 포괄적인 -gencode 플래그를 추가하여 더 나은 CUDA 후방 호환성을 원할 수 있습니다.

NVIDIA 설명: arch= 와 code=

NVIDIA에 따르면:

The arch= clause of the -gencode= command-line option to nvcc specifies the front-end compilation target and must always be a PTX version. The code= clause specifies the back-end compilation target and can either be cubin or PTX or both. Only the back-end target version(s) specified by the code= clause will be retained in the resulting binary; at least one must be PTX to provide Ampere compatibility.

nvcc gencode 및 arch 플래그 예제

CUDA 7.0 — 모든 카드 호환

-arch=sm_30 \
 -gencode=arch=compute_20,code=sm_20 \
 -gencode=arch=compute_30,code=sm_30 \
 -gencode=arch=compute_50,code=sm_50 \
 -gencode=arch=compute_52,code=sm_52 \
 -gencode=arch=compute_52,code=compute_52

CUDA 8.1 — Volta 이전 카드 호환

-arch=sm_30 \
 -gencode=arch=compute_20,code=sm_20 \
 -gencode=arch=compute_30,code=sm_30 \
 -gencode=arch=compute_50,code=sm_50 \
 -gencode=arch=compute_52,code=sm_52 \
 -gencode=arch=compute_60,code=sm_60 \
 -gencode=arch=compute_61,code=sm_61 \
 -gencode=arch=compute_61,code=compute_61

CUDA 9.2 — Volta 카드 호환

-arch=sm_50 \
 -gencode=arch=compute_50,code=sm_50 \
 -gencode=arch=compute_52,code=sm_52 \
 -gencode=arch=compute_60,code=sm_60 \
 -gencode=arch=compute_61,code=sm_61 \
 -gencode=arch=compute_70,code=sm_70 \
 -gencode=arch=compute_70,code=compute_70

CUDA 10.1 — V100 및 T4 Turing 카드 호환

-arch=sm_50 \
 -gencode=arch=compute_50,code=sm_50 \
 -gencode=arch=compute_52,code=sm_52 \
 -gencode=arch=compute_60,code=sm_60 \
 -gencode=arch=compute_61,code=sm_61 \
 -gencode=arch=compute_70,code=sm_70 \
 -gencode=arch=compute_75,code=sm_75 \
 -gencode=arch=compute_75,code=compute_75

CUDA 11.0 — V100 및 T4 Turing 카드 호환

-arch=sm_52 \
 -gencode=arch=compute_52,code=sm_52 \
 -gencode=arch=compute_60,code=sm_60 \
 -gencode=arch=compute_61,code=sm_61 \
 -gencode=arch=compute_70,code=sm_70 \
 -gencode=arch=compute_75,code=sm_75 \
 -gencode=arch=compute_80,code=sm_80 \
 -gencode=arch=compute_80,code=compute_80

CUDA 11.7 — V100, T4, RTX 3080, Drive AGX Orin 호환

-arch=sm_52 \
 -gencode=arch=compute_52,code=sm_52 \
 -gencode=arch=compute_60,code=sm_60 \
 -gencode=arch=compute_61,code=sm_61 \
 -gencode=arch=compute_70,code=sm_70 \
 -gencode=arch=compute_75,code=sm_75 \
 -gencode=arch=compute_80,code=sm_80 \
 -gencode=arch=compute_86,code=sm_86 \
 -gencode=arch=compute_87,code=sm_87 \
 -gencode=arch=compute_86,code=compute_86

CUDA 11.4 — RTX 3080 최적 성능

-arch=sm_80 \
 -gencode=arch=compute_80,code=sm_80 \
 -gencode=arch=compute_86,code=sm_86 \
 -gencode=arch=compute_87,code=sm_87 \
 -gencode=arch=compute_86,code=compute_86

CUDA 12 — RTX 4080 최적 성능

-arch=sm_89 \
 -gencode=arch=compute_89,code=sm_89 \
 -gencode=arch=compute_89,code=compute_89

CUDA 12 (PTX ISA version 8.0) — H100 (Hopper) 최적 성능

-arch=sm_90 \
 -gencode=arch=compute_90,code=sm_90 \
 -gencode=arch=compute_90a,code=sm_90a \
 -gencode=arch=compute_90a,code=compute_90a

Hopper GPU 후방 호환성 추가

-arch=sm_52 \
 -gencode=arch=compute_52,code=sm_52 \
 -gencode=arch=compute_60,code=sm_60 \
 -gencode=arch=compute_61,code=sm_61 \
 -gencode=arch=compute_70,code=sm_70 \
 -gencode=arch=compute_75,code=sm_75 \
 -gencode=arch=compute_80,code=sm_80 \
 -gencode=arch=compute_86,code=sm_86 \
 -gencode=arch=compute_87,code=sm_87 \
 -gencode=arch=compute_90,code=sm_90 \
 -gencode=arch=compute_90,code=compute_90

PyTorch에서 TORCH_CUDA_ARCH_LIST 사용

PyTorch를 사용하는 경우 설치 중 TORCH_CUDA_ARCH_LIST 환경 변수를 사용하여 아키텍처를 설정할 수 있습니다: 

$ TORCH_CUDA_ARCH_LIST="7.0 7.5 8.0 8.6" python3 setup.py install

이 변수에 모든 아키텍처를 지정할 수 있지만, 각 아키텍처마다 커널을 컴파일해야 하므로 빌드 시간이 길어집니다.

가장 최근 아키텍처에 +PTX 접미사를 추가하여 새 카드와 전방 호환인 PTX 코드를 생성할 수도 있습니다: 

$ TORCH_CUDA_ARCH_LIST="7.0 7.5 8.0 8.6+PTX" python3 build_my_extension.py

TensorRT CMake 사용

CMAKE로 TensorRT를 컴파일하는 경우 sm_ 및 compute_ 접두사를 제거하고 계산 능력만 참조합니다.

Tesla V100 및 Volta 카드: 

cmake <...> -DGPU_ARCHS="70"

NVIDIA RTX 2070 및 Tesla T4: 

cmake <...> -DGPU_ARCHS="75"

NVIDIA A100: 

cmake <...> -DGPU_ARCHS="80"

NVIDIA RTX 3080 및 A100 함께: 

cmake <...> -DGPU_ARCHS="80 86"

NVIDIA H100: 

cmake <...> -DGPU_ARCHS="90"

CUTLASS with Hopper GH100 CMake

cmake .. -DCUTLASS_NVCC_ARCHS=90a

"Value 'sm_86' is not defined for option 'gpu-architecture'" 오류

이러한 오류가 발생하는 경우: 

nvcc fatal : Value 'sm_86' is not defined for option 'gpu-architecture'

더 오래된 버전의 CUDA 및/또는 드라이버가 설치되어 있을 가능성이 높습니다. A100, RTX 3080과 같은 sm_8x 카드를 지원하려면 450.36.06 이상의 최신 드라이버로 업그레이드하세요.

CUDA 호환성

CUDA Version cuDNN Version NCCL Version NVIDIA GPU Driver Version Compute Capability Support
CUDA 12.0 — — — —
CUDA 11.8 — — — —
CUDA 11.7 — — — —
CUDA 11.5 8.3.x 2.10.x 510.39 or later Compute Capability 3.0 to 8.6
CUDA 11.4 8.2.x 2.10.x 470.42.01 or later Compute Capability 3.0 to 8.6
CUDA 11.3 8.2.x 2.10.x 465.19.01 or later Compute Capability 3.0 to 8.6
CUDA 11.2 8.1.x 2.9.x 460.32.03 or later Compute Capability 3.0 to 8.6
CUDA 11.1 8.0.x 2.9.x 455.23.04 or later Compute Capability 3.0 to 8.6
CUDA 11.0 7.6.x 2.8.x 450.36.06 or later Compute Capability 3.0 to 8.6
CUDA 10.2 7.6.x 2.7.x 440.33 or later Compute Capability 3.0 to 7.5
CUDA 10.1 7.6.x 2.4.x 418.39 or later Compute Capability 3.0 to 7.5
CUDA 10.0 7.4.x 2.2.x 410.48 or later Compute Capability 3.0 to 7.5
CUDA 9.2 7.2.x 2.1.x 396.26 or later Compute Capability 3.0 to 7.5
CUDA 9.1 7.1.x 2.0.x 390.46 or later Compute Capability 3.0 to 7.5
CUDA 9.0 7.0.x 1.3.x 384.81 or later Compute Capability 3.0 to 7.5
CUDA 8.0 6.0.x 1.3.x 375.26 or later Compute Capability 2.0 to 6.2
CUDA 7.5 5.1.x 1.3.x 352.31 or later Compute Capability 2.0 to 5.0

---

Configuration

nvcc 컴파일 플래그

# 기본 컴파일
nvcc -arch=sm_XX -gencode=arch=compute_XX,code=sm_XX source.cu -o output

# PyTorch 설치
TORCH_CUDA_ARCH_LIST="7.0 7.5 8.0 8.6" python3 setup.py install

# TensorRT CMake
cmake <...> -DGPU_ARCHS="70"

# CUTLASS CMake
cmake .. -DCUTLASS_NVCC_ARCHS=90a

---

Examples

Example 1: CUDA 11.7로 RTX 3080 컴파일

입력: 

nvcc -arch=sm_80 \
 -gencode=arch=compute_80,code=sm_80 \
 -gencode=arch=compute_86,code=sm_86 \
 -gencode=arch=compute_87,code=sm_87 \
 -gencode=arch=compute_86,code=compute_86 \
 source.cu -o output

Example 2: PyTorch 설치

입력: 

TORCH_CUDA_ARCH_LIST="7.0 7.5 8.0 8.6+PTX" python3 setup.py install

---

Best Practices

  • 가장 많이 사용하는 GPU 아키텍처만 -arch에 지정 — 불필요한 -gencode 플래그 제거로 컴파일 시간 단축
  • PyTorch 설치 시 TORCH_CUDA_ARCH_LIST 사용 — 빌드 시간 단축
  • +PTX 접미사 사용 — 새 카드와의 전방 호환성 확보
  • CUDA 드라이버 최신 유지 — sm_8x 카드 지원에는 450.36.06 이상 필요
  • TensorRT CMake 시 sm_/compute_ 접두사 제거 — 계산 능력만 지정

---

Limitations

  • 각 아키텍처마다 커널을 컴파일하면 빌드 시간이 길어짐
  • 구형 CUDA 버전은 새 GPU 아키텍처를 지원하지 않음
  • CUDA 호환성은 드라이버 버전과 cuDNN/NCCL 버전과 밀접한 관계가 있음
  • Hopper GPU(sm_90)는 CUDA 12 이상 필요

---

References

  • NVIDIA nvcc 문서

---

Related Pages

---

Retrieved from "http://wiki.hpcmate.com/index.php?title=CUDA&oldid=2690"