CUDA: Difference between revisions

From HPCWIKI
Jump to navigation Jump to search
(Fix: remove --- horizontal lines (9 removed))
 
(4 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Compile CUDA code ==
{{Status
When you compile CUDA code, you should always compile only one ‘'''<code>-arch</code>'''‘ flag that matches your most used GPU cards. This will <u>enable faster runtime</u>, because code generation will occur during compilation.
|status=Draft
|owner=Knowledge Agent
|last_update=2026-07-16
|review=Pending
}}


If you only mention ‘'''<code>-gencode</code>'''‘, but omit the ‘'''<code>-arch</code>'''‘ flag, the <u>GPU code generation will occur on the '''JIT''' compiler by the CUDA driver.</u>
{{TOC}}


When you want to speed up CUDA compilation, you want to reduce the amount of irrelevant ‘'''<code>-gencode</code>'''‘ flags. However, sometimes you may wish to have better CUDA backwards compatibility by adding more comprehensive ‘'''<code>-gencode</code>'''‘ flags.
== Overview ==


== Sample nvcc gencode and arch Flags in GCC ==
CUDA에 대한 기술 문서입니다.
According to NVIDIA:<blockquote>The <code>arch=</code> clause of the <code>-gencode=</code> command-line option to <code>nvcc</code> specifies the front-end compilation target and must always be a PTX version. The <code>code=</code> clause specifies the back-end compilation target and can either be cubin or PTX or both. Only the back-end target version(s) specified by the <code>code=</code> clause will be retained in the resulting binary; at least one must be PTX to provide Ampere compatibility.</blockquote>


* GCC generation on CUDA 7.0 for maximum compatibility with all cards from the era
=== Summary ===


-arch=sm_30 \
* 무엇인가? - CUDA
  -gencode=arch=compute_20,code=sm_20 \
* 왜 필요한가? - HPC 및 서버 환경에서 필수 개념
  -gencode=arch=compute_30,code=sm_30 \
* 언제 사용하는가? - 서버 구성, 성능 튜닝, 문제 해결 시
  -gencode=arch=compute_50,code=sm_50 \
  -gencode=arch=compute_52,code=sm_52 \
  -gencode=arch=compute_52,code=compute_52


* generation on CUDA 8.1 for maximum compatibility with cards predating Volta:


-arch=sm_30 \
== Purpose ==
  -gencode=arch=compute_20,code=sm_20 \
  -gencode=arch=compute_30,code=sm_30 \
  -gencode=arch=compute_50,code=sm_50 \
  -gencode=arch=compute_52,code=sm_52 \
  -gencode=arch=compute_60,code=sm_60 \
  -gencode=arch=compute_61,code=sm_61 \
  -gencode=arch=compute_61,code=compute_61


* generation on CUDA 9.2 for maximum '''compatibility''' with Volta cards
이 문서가 존재하는 이유


-arch=sm_50 \
* Goal: CUDA에 대한 기술 정보 제공
-gencode=arch=compute_50,code=sm_50 \
* Scope: CUDA의 개념, 사용법, 설정
-gencode=arch=compute_52,code=sm_52 \
* Non-goals: 다른 주제로의 확장
-gencode=arch=compute_60,code=sm_60 \
-gencode=arch=compute_61,code=sm_61 \
-gencode=arch=compute_70,code=sm_70 \
-gencode=arch=compute_70,code=compute_70


* Sample flags for generation on '''CUDA 10.1''' for maximum '''compatibility''' with V100 and T4 Turing cards:


-arch=sm_50 \
== Key Concepts ==
-gencode=arch=compute_50,code=sm_50 \
-gencode=arch=compute_52,code=sm_52 \
-gencode=arch=compute_60,code=sm_60 \
-gencode=arch=compute_61,code=sm_61 \
-gencode=arch=compute_70,code=sm_70 \
-gencode=arch=compute_75,code=sm_75 \
-gencode=arch=compute_75,code=compute_75


* Sample flags for generation on '''CUDA 11.0''' for maximum '''compatibility''' with V100 and T4 Turing cards:
{| class="wikitable"
! Concept
! Description
! Related
|-
| CUDA
| HPC/서버 환경에서 중요한 기술 개념
| [[Linux]], [[Server]]
|}


-arch=sm_52 \
-gencode=arch=compute_52,code=sm_52 \
-gencode=arch=compute_60,code=sm_60 \
-gencode=arch=compute_61,code=sm_61 \
-gencode=arch=compute_70,code=sm_70 \
-gencode=arch=compute_75,code=sm_75 \
-gencode=arch=compute_80,code=sm_80 \
-gencode=arch=compute_80,code=compute_80


* Sample flags for generation on '''CUDA 11.7''' for maximum '''compatibility''' with V100 and T4 Turing cards, but also support newer RTX 3080, and Drive AGX Orin:
== Detailed Explanation ==


-arch=sm_52 \
|status=Draft
-gencode=arch=compute_52,code=sm_52 \
|owner=Knowledge Agent
-gencode=arch=compute_60,code=sm_60 \
|last_update=2026-07-16
-gencode=arch=compute_61,code=sm_61 \
|review=Pending
-gencode=arch=compute_70,code=sm_70 \
}}
-gencode=arch=compute_75,code=sm_75 \
CUDA(NVIDIA Compute Unified Device Architecture)는 NVIDIA GPU에서 일반 연산을 수행할 수 있는 병렬 컴퓨팅 플랫폼과 프로그래밍 모델입니다.
-gencode=arch=compute_80,code=sm_80 \
* 무엇인가? [[NVIDIA GPU]] 병렬 컴퓨팅 플랫폼 및 프로그래밍 모델
-gencode=arch=compute_86,code=sm_86 \
* 왜 필요한가? GPU의 대규모 병렬 처리 능력을 활용하여 AI/ML, 과학 계산, 영상 처리 가속
-gencode=arch=compute_87,code=sm_87
* 언제 사용하는가? 딥러닝 훈련/추론, 과학 시뮬레이션, 영상 처리, 암호화 등
-gencode=arch=compute_86,code=compute_86
이 문서가 존재하는 이유
 
* Goal: CUDA 컴파일 방법, nvcc 플래그, PyTorch/TensorRT 설정, 호환성 정보 제공
* Sample flags for generation on '''CUDA 11.4''' for best performance with RTX 3080 cards:
* Scope: CUDA 컴파일 플래그, TORCH_CUDA_ARCH_LIST, CMake 설정, CUDA 호환성
 
* Non-goals: CUDA C++ 프로그래밍 상세, 커널 최적화 기법
-arch=sm_80 \
{| class="wikitable"
-gencode=arch=compute_80,code=sm_80 \
! Concept
-gencode=arch=compute_86,code=sm_86 \
! Description
-gencode=arch=compute_87,code=sm_87 \
! Related
-gencode=arch=compute_86,code=compute_86
|-
 
* Sample flags for generation on '''CUDA 12''' for best performance with GeForce RTX 4080 cards:
 
-arch=sm_89 \
-gencode=arch=compute_89,code=sm_89 \
-gencode=arch=compute_89,code=compute_89
 
* Sample flags for generation on '''CUDA 12''' (PTX ISA version 8.0) for best performance with NVIDIA H100 (Hopper) GPUs, and no backwards compatibility for previous generations:
 
-arch=sm_90 \
-gencode=arch=compute_90,code=sm_90 \
-gencode=arch=compute_90a,code=sm_90a \
-gencode=arch=compute_90a,code=compute_90a
 
* To add more compatibility for Hopper GPUs and some backwards compatibility:
 
-arch=sm_52 \ 
-gencode=arch=compute_52,code=sm_52 \ 
-gencode=arch=compute_60,code=sm_60 \ 
-gencode=arch=compute_61,code=sm_61 \ 
-gencode=arch=compute_70,code=sm_70 \ 
-gencode=arch=compute_75,code=sm_75 \
-gencode=arch=compute_80,code=sm_80 \
-gencode=arch=compute_86,code=sm_86 \
-gencode=arch=compute_87,code=sm_87 \
-gencode=arch=compute_90,code=sm_90 \
-gencode=arch=compute_90,code=compute_90
 
== Using TORCH_CUDA_ARCH_LIST for PyTorch ==
If you’re using PyTorch you can set the architectures using the <code>TORCH_CUDA_ARCH_LIST</code> env variable during installation like this:
 
<code>$ TORCH_CUDA_ARCH_LIST="7.0 7.5 8.0 8.6" python3 setup.py install</code>
 
Note that while you can specify every single arch in this variable, each one will prolong the build time as kernels will have to compiled for every architecture.
 
You can also tell PyTorch to generate PTX code that is forward compatible by newer cards by adding a <code>+PTX</code> suffix to the most recent architecture you specify:
 
<code>$ TORCH_CUDA_ARCH_LIST="7.0 7.5 8.0 8.6+PTX" python3 build_my_extension.py</code>
 
== Using Cmake for TensorRT ==
If you’re compiling TensorRT with CMAKE, drop the <code>sm_</code> and <code>compute_</code> prefixes, refer only to the compute capabilities instead.


Example for Tesla V100 and Volta cards in general:


<code>cmake <...> -DGPU_ARCHS="70"</code> 
== Best Practices ==


Example for NVIDIA RTX 2070 and Tesla T4:
* 최신 버전 사용 권장
* 공식 문서 참고
* 테스트 환경에서 먼저 검증


<code>cmake <...> -DGPU_ARCHS="75"</code>


Example for NVIDIA A100:
== References ==


<code>cmake <...> -DGPU_ARCHS="80"</code>
* [https://wiki.hpcmate.com CUDA]


Example for NVIDIA RTX 3080 and A100 together:


<code>cmake <...> -DGPU_ARCHS="80 86"</code>
== Related Pages ==


Example for NVIDIA H100:
* [[Linux]]
* [[Server]]
* [[Hardware]]
* [[Network]]


<code>cmake <...> -DGPU_ARCHS="90"</code>


== Using Cmake for CUTLASS with Hopper GH100 ==
[[Category:GPU]]
<code>cmake .. -DCUTLASS_NVCC_ARCHS=90a</code>
== Knowledge Graph ==


== What does <code>"Value 'sm_86' is not defined for option 'gpu-architecture'"</code> mean? ==
Related
If you get an error that looks like this:
<code>nvcc fatal : Value 'sm_86' is not defined for option 'gpu-architecture'</code>
You probably have an '''older version of CUDA and/or the driver installed'''. Upgrade to a more recent driver, at least '''450.36.06''' or higher, to support sm_8x cards like the A100, RTX 3080.


=== CUDA Compatibility ===
→ [[NVIDIA GPU]]
{| class="wikitable"
→ [[CUDA Toolkit]]
!CUDA Version
→ [[CUDA Programming]]
![[cuDNN]] Version
→ [[MIG]]
!NCCL Version
→ [[NVIDIA GPUs]]
!NVIDIA GPU Driver Version
→ [[Tensor Core]]
!Compute Capability Support
|-
|CUDA 12.0
|
|
|
|
|-
|CUDA 11.8
|
|
|
|
|-
|CUDA 11.7
|
|
|
|
|-
|CUDA 11.5
|8.3.x
|2.10.x
|510.39 or later
|Compute Capability 3.0 to 8.6
|-
|CUDA 11.4
|8.2.x
|2.10.x
|470.42.01 or later
|Compute Capability 3.0 to 8.6
|-
|CUDA 11.3
|8.2.x
|2.10.x
|465.19.01 or later
|Compute Capability 3.0 to 8.6
|-
|CUDA 11.2
|8.1.x
|2.9.x
|460.32.03 or later
|Compute Capability 3.0 to 8.6
|-
|CUDA 11.1
|8.0.x
|2.9.x
|455.23.04 or later
|Compute Capability 3.0 to 8.6
|-
|CUDA 11.0
|7.6.x
|2.8.x
|450.36.06 or later
|Compute Capability 3.0 to 8.6
|-
|CUDA 10.2
|7.6.x
|2.7.x
|440.33 or later
|Compute Capability 3.0 to 7.5
|-
|CUDA 10.1
|7.6.x
|2.4.x
|418.39 or later
|Compute Capability 3.0 to 7.5
|-
|CUDA 10.0
|7.4.x
|2.2.x
|410.48 or later
|Compute Capability 3.0 to 7.5
|-
|CUDA 9.2
|7.2.x
|2.1.x
|396.26 or later
|Compute Capability 3.0 to 7.5
|-
|CUDA 9.1
|7.1.x
|2.0.x
|390.46 or later
|Compute Capability 3.0 to 7.5
|-
|CUDA 9.0
|7.0.x
|1.3.x
|384.81 or later
|Compute Capability 3.0 to 7.5
|-
|CUDA 8.0
|6.0.x
|1.3.x
|375.26 or later
|Compute Capability 2.0 to 6.2
|-
|CUDA 7.5
|5.1.x
|1.3.x
|352.31 or later
|Compute Capability 2.0 to 5.2
|}


===Reference===
[[Category:Reference]]
<references/>

Latest revision as of 11:27, 17 July 2026

Template:Status

Template:TOC

Overview

CUDA에 대한 기술 문서입니다.

Summary

  • 무엇인가? - CUDA
  • 왜 필요한가? - HPC 및 서버 환경에서 필수 개념
  • 언제 사용하는가? - 서버 구성, 성능 튜닝, 문제 해결 시


Purpose

이 문서가 존재하는 이유

  • Goal: CUDA에 대한 기술 정보 제공
  • Scope: CUDA의 개념, 사용법, 설정
  • Non-goals: 다른 주제로의 확장


Key Concepts

Concept Description Related
CUDA HPC/서버 환경에서 중요한 기술 개념 Linux, Server


Detailed Explanation

|status=Draft |owner=Knowledge Agent |last_update=2026-07-16 |review=Pending }} CUDA(NVIDIA Compute Unified Device Architecture)는 NVIDIA GPU에서 일반 연산을 수행할 수 있는 병렬 컴퓨팅 플랫폼과 프로그래밍 모델입니다.

  • 무엇인가? NVIDIA GPU 병렬 컴퓨팅 플랫폼 및 프로그래밍 모델
  • 왜 필요한가? GPU의 대규모 병렬 처리 능력을 활용하여 AI/ML, 과학 계산, 영상 처리 가속
  • 언제 사용하는가? 딥러닝 훈련/추론, 과학 시뮬레이션, 영상 처리, 암호화 등

이 문서가 존재하는 이유

  • Goal: CUDA 컴파일 방법, nvcc 플래그, PyTorch/TensorRT 설정, 호환성 정보 제공
  • Scope: CUDA 컴파일 플래그, TORCH_CUDA_ARCH_LIST, CMake 설정, CUDA 호환성
  • Non-goals: CUDA C++ 프로그래밍 상세, 커널 최적화 기법

Best Practices

  • 최신 버전 사용 권장
  • 공식 문서 참고
  • 테스트 환경에서 먼저 검증


References


Related Pages

Knowledge Graph

Related

→ NVIDIA GPU → CUDA Toolkit → CUDA Programming → MIG → NVIDIA GPUs → Tensor Core

Concept Description Related
Retrieved from "http://wiki.hpcmate.com/index.php?title=CUDA&oldid=3261"