CUDA: Difference between revisions

From HPCWIKI
Jump to navigation Jump to search
(Add categories: GPU, Reference)
(Fix: remove --- horizontal lines (9 removed))
 
(3 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Compile CUDA code ==
{{Status
When you compile CUDA code, you should always compile only one ‘'''<code>-arch</code>'''‘ flag that matches your most used GPU cards. This will <u>enable faster runtime</u>, because code generation will occur during compilation.
|status=Draft
|owner=Knowledge Agent
|last_update=2026-07-16
|review=Pending
}}


If you only mention ‘'''<code>-gencode</code>'''‘, but omit the ‘'''<code>-arch</code>'''‘ flag, the <u>GPU code generation will occur on the '''JIT''' compiler by the CUDA driver.</u>
{{TOC}}


When you want to speed up CUDA compilation, you want to reduce the amount of irrelevant ‘'''<code>-gencode</code>'''‘ flags. However, sometimes you may wish to have better CUDA backwards compatibility by adding more comprehensive ‘'''<code>-gencode</code>'''‘ flags.
== Overview ==


== Sample nvcc gencode and arch Flags in GCC ==
CUDA에 대한 기술 문서입니다.
According to NVIDIA:<blockquote>The <code>arch=</code> clause of the <code>-gencode=</code> command-line option to <code>nvcc</code> specifies the front-end compilation target and must always be a PTX version. The <code>code=</code> clause specifies the back-end compilation target and can either be cubin or PTX or both. Only the back-end target version(s) specified by the <code>code=</code> clause will be retained in the resulting binary; at least one must be PTX to provide Ampere compatibility.</blockquote>


* GCC generation on CUDA 7.0 for maximum compatibility with all cards from the era
=== Summary ===


-arch=sm_30 \
* 무엇인가? - CUDA
  -gencode=arch=compute_20,code=sm_20 \
* 왜 필요한가? - HPC 및 서버 환경에서 필수 개념
  -gencode=arch=compute_30,code=sm_30 \
* 언제 사용하는가? - 서버 구성, 성능 튜닝, 문제 해결 시
  -gencode=arch=compute_50,code=sm_50 \
  -gencode=arch=compute_52,code=sm_52 \
  -gencode=arch=compute_52,code=compute_52


* generation on CUDA 8.1 for maximum compatibility with cards predating Volta:


-arch=sm_30 \
== Purpose ==
  -gencode=arch=compute_20,code=sm_20 \
  -gencode=arch=compute_30,code=sm_30 \
  -gencode=arch=compute_50,code=sm_50 \
  -gencode=arch=compute_52,code=sm_52 \
  -gencode=arch=compute_60,code=sm_60 \
  -gencode=arch=compute_61,code=sm_61 \
  -gencode=arch=compute_61,code=compute_61


* generation on CUDA 9.2 for maximum '''compatibility''' with Volta cards
이 문서가 존재하는 이유


-arch=sm_50 \
* Goal: CUDA에 대한 기술 정보 제공
-gencode=arch=compute_50,code=sm_50 \
* Scope: CUDA의 개념, 사용법, 설정
-gencode=arch=compute_52,code=sm_52 \
* Non-goals: 다른 주제로의 확장
-gencode=arch=compute_60,code=sm_60 \
-gencode=arch=compute_61,code=sm_61 \
-gencode=arch=compute_70,code=sm_70 \
-gencode=arch=compute_70,code=compute_70


* Sample flags for generation on '''CUDA 10.1''' for maximum '''compatibility''' with V100 and T4 Turing cards:


-arch=sm_50 \
== Key Concepts ==
-gencode=arch=compute_50,code=sm_50 \
-gencode=arch=compute_52,code=sm_52 \
-gencode=arch=compute_60,code=sm_60 \
-gencode=arch=compute_61,code=sm_61 \
-gencode=arch=compute_70,code=sm_70 \
-gencode=arch=compute_75,code=sm_75 \
-gencode=arch=compute_75,code=compute_75


* Sample flags for generation on '''CUDA 11.0''' for maximum '''compatibility''' with V100 and T4 Turing cards:
{| class="wikitable"
! Concept
! Description
! Related
|-
| CUDA
| HPC/서버 환경에서 중요한 기술 개념
| [[Linux]], [[Server]]
|}


-arch=sm_52 \
-gencode=arch=compute_52,code=sm_52 \
-gencode=arch=compute_60,code=sm_60 \
-gencode=arch=compute_61,code=sm_61 \
-gencode=arch=compute_70,code=sm_70 \
-gencode=arch=compute_75,code=sm_75 \
-gencode=arch=compute_80,code=sm_80 \
-gencode=arch=compute_80,code=compute_80


* Sample flags for generation on '''CUDA 11.7''' for maximum '''compatibility''' with V100 and T4 Turing cards, but also [[support]] newer RTX 3080, and Drive AGX Orin:
== Detailed Explanation ==


-arch=sm_52 \
|status=Draft
-gencode=arch=compute_52,code=sm_52 \
|owner=Knowledge Agent
-gencode=arch=compute_60,code=sm_60 \
|last_update=2026-07-16
-gencode=arch=compute_61,code=sm_61 \
|review=Pending
-gencode=arch=compute_70,code=sm_70 \
}}
-gencode=arch=compute_75,code=sm_75 \
CUDA(NVIDIA Compute Unified Device Architecture)는 NVIDIA GPU에서 일반 연산을 수행할 수 있는 병렬 컴퓨팅 플랫폼과 프로그래밍 모델입니다.
-gencode=arch=compute_80,code=sm_80 \
* 무엇인가? [[NVIDIA GPU]] 병렬 컴퓨팅 플랫폼 및 프로그래밍 모델
-gencode=arch=compute_86,code=sm_86 \
* 왜 필요한가? GPU의 대규모 병렬 처리 능력을 활용하여 AI/ML, 과학 계산, 영상 처리 가속
-gencode=arch=compute_87,code=sm_87
* 언제 사용하는가? 딥러닝 훈련/추론, 과학 시뮬레이션, 영상 처리, 암호화 등
-gencode=arch=compute_86,code=compute_86
이 문서가 존재하는 이유
* Goal: CUDA 컴파일 방법, nvcc 플래그, PyTorch/TensorRT 설정, 호환성 정보 제공
* Scope: CUDA 컴파일 플래그, TORCH_CUDA_ARCH_LIST, CMake 설정, CUDA 호환성
* Non-goals: CUDA C++ 프로그래밍 상세, 커널 최적화 기법
{| class="wikitable"
! Concept
! Description
! Related
|-


* Sample flags for generation on '''CUDA 11.4''' for best performance with RTX 3080 cards:


-arch=sm_80 \
== Best Practices ==
-gencode=arch=compute_80,code=sm_80 \
-gencode=arch=compute_86,code=sm_86 \
-gencode=arch=compute_87,code=sm_87 \
-gencode=arch=compute_86,code=compute_86


* Sample flags for generation on '''CUDA 12''' for best performance with GeForce RTX 4080 cards:
* 최신 버전 사용 권장
* 공식 문서 참고
* 테스트 환경에서 먼저 검증


-arch=sm_89 \
-gencode=arch=compute_89,code=sm_89 \
-gencode=arch=compute_89,code=compute_89


* Sample flags for generation on '''CUDA 12''' (PTX ISA version 8.0) for best performance with NVIDIA H100 (Hopper) GPUs, and no backwards compatibility for previous generations:
== References ==


-arch=sm_90 \
* [https://wiki.hpcmate.com CUDA]
-gencode=arch=compute_90,code=sm_90 \
-gencode=arch=compute_90a,code=sm_90a \
-gencode=arch=compute_90a,code=compute_90a


* To add more compatibility for Hopper GPUs and some backwards compatibility:


-arch=sm_52 \ 
== Related Pages ==
-gencode=arch=compute_52,code=sm_52 \ 
-gencode=arch=compute_60,code=sm_60 \ 
-gencode=arch=compute_61,code=sm_61 \ 
-gencode=arch=compute_70,code=sm_70 \ 
-gencode=arch=compute_75,code=sm_75 \
-gencode=arch=compute_80,code=sm_80 \
-gencode=arch=compute_86,code=sm_86 \
-gencode=arch=compute_87,code=sm_87 \
-gencode=arch=compute_90,code=sm_90 \
-gencode=arch=compute_90,code=compute_90


== Using TORCH_CUDA_ARCH_LIST for PyTorch ==
* [[Linux]]
If you’re using PyTorch you can set the architectures using the <code>TORCH_CUDA_ARCH_LIST</code> env variable during installation like this:
* [[Server]]
* [[Hardware]]
* [[Network]]


<code>$ TORCH_CUDA_ARCH_LIST="7.0 7.5 8.0 8.6" python3 setup.py install</code>


Note that while you can specify every single arch in this variable, each one will prolong the build time as kernels will have to compiled for every architecture.
[[Category:GPU]]
== Knowledge Graph ==


You can also tell PyTorch to generate PTX code that is forward compatible by newer cards by adding a <code>+PTX</code> suffix to the most recent architecture you specify:
Related


<code>$ TORCH_CUDA_ARCH_LIST="7.0 7.5 8.0 8.6+PTX" python3 build_my_extension.py</code>
→ [[NVIDIA GPU]]
 
→ [[CUDA Toolkit]]
== Using Cmake for TensorRT ==
→ [[CUDA Programming]]
If you’re compiling TensorRT with CMAKE, drop the <code>sm_</code> and <code>compute_</code> prefixes, refer only to the compute capabilities instead.
→ [[MIG]]
 
→ [[NVIDIA GPUs]]
Example for Tesla V100 and Volta cards in general:
→ [[Tensor Core]]
 
<code>cmake <...> -DGPU_ARCHS="70"</code> 
 
Example for NVIDIA RTX 2070 and Tesla T4:
 
<code>cmake <...> -DGPU_ARCHS="75"</code>
 
Example for NVIDIA A100:
 
<code>cmake <...> -DGPU_ARCHS="80"</code>
 
Example for NVIDIA RTX 3080 and A100 together:
 
<code>cmake <...> -DGPU_ARCHS="80 86"</code>
 
Example for NVIDIA H100:
 
<code>cmake <...> -DGPU_ARCHS="90"</code>
 
== Using Cmake for CUTLASS with Hopper GH100 ==
<code>cmake .. -DCUTLASS_NVCC_ARCHS=90a</code>
 
== What does <code>"Value 'sm_86' is not defined for option 'gpu-architecture'"</code> mean? ==
If you get an error that looks like this:
<code>nvcc fatal : Value 'sm_86' is not defined for option 'gpu-architecture'</code>
You probably have an '''older version of CUDA and/or the driver installed'''. Upgrade to a more recent driver, at least '''450.36.06''' or higher, to support sm_8x cards like the A100, RTX 3080.
 
=== CUDA Compatibility ===
{| class="wikitable"
!CUDA Version
![[cuDNN]] Version
!NCCL Version
![[NVIDIA GPU]] Driver Version
!Compute Capability Support
|-
|CUDA 12.0
|
|
|
|
|-
|CUDA 11.8
|
|
|
|
|-
|CUDA 11.7
|
|
|
|
|-
|CUDA 11.5
|8.3.x
|2.10.x
|510.39 or later
|Compute Capability 3.0 to 8.6
|-
|CUDA 11.4
|8.2.x
|2.10.x
|470.42.01 or later
|Compute Capability 3.0 to 8.6
|-
|CUDA 11.3
|8.2.x
|2.10.x
|465.19.01 or later
|Compute Capability 3.0 to 8.6
|-
|CUDA 11.2
|8.1.x
|2.9.x
|460.32.03 or later
|Compute Capability 3.0 to 8.6
|-
|CUDA 11.1
|8.0.x
|2.9.x
|455.23.04 or later
|Compute Capability 3.0 to 8.6
|-
|CUDA 11.0
|7.6.x
|2.8.x
|450.36.06 or later
|Compute Capability 3.0 to 8.6
|-
|CUDA 10.2
|7.6.x
|2.7.x
|440.33 or later
|Compute Capability 3.0 to 7.5
|-
|CUDA 10.1
|7.6.x
|2.4.x
|418.39 or later
|Compute Capability 3.0 to 7.5
|-
|CUDA 10.0
|7.4.x
|2.2.x
|410.48 or later
|Compute Capability 3.0 to 7.5
|-
|CUDA 9.2
|7.2.x
|2.1.x
|396.26 or later
|Compute Capability 3.0 to 7.5
|-
|CUDA 9.1
|7.1.x
|2.0.x
|390.46 or later
|Compute Capability 3.0 to 7.5
|-
|CUDA 9.0
|7.0.x
|1.3.x
|384.81 or later
|Compute Capability 3.0 to 7.5
|-
|CUDA 8.0
|6.0.x
|1.3.x
|375.26 or later
|Compute Capability 2.0 to 6.2
|-
|CUDA 7.5
|5.1.x
|1.3.x
|352.31 or later
|Compute Capability 2.0 to 5.2
|}
 
===Reference===
<references/>
[[Category:GPU]]


[[Category:Reference]]
[[Category:Reference]]

Latest revision as of 11:27, 17 July 2026

Template:Status

Template:TOC

Overview

CUDA에 대한 기술 문서입니다.

Summary

  • 무엇인가? - CUDA
  • 왜 필요한가? - HPC 및 서버 환경에서 필수 개념
  • 언제 사용하는가? - 서버 구성, 성능 튜닝, 문제 해결 시


Purpose

이 문서가 존재하는 이유

  • Goal: CUDA에 대한 기술 정보 제공
  • Scope: CUDA의 개념, 사용법, 설정
  • Non-goals: 다른 주제로의 확장


Key Concepts

Concept Description Related
CUDA HPC/서버 환경에서 중요한 기술 개념 Linux, Server


Detailed Explanation

|status=Draft |owner=Knowledge Agent |last_update=2026-07-16 |review=Pending }} CUDA(NVIDIA Compute Unified Device Architecture)는 NVIDIA GPU에서 일반 연산을 수행할 수 있는 병렬 컴퓨팅 플랫폼과 프로그래밍 모델입니다.

  • 무엇인가? NVIDIA GPU 병렬 컴퓨팅 플랫폼 및 프로그래밍 모델
  • 왜 필요한가? GPU의 대규모 병렬 처리 능력을 활용하여 AI/ML, 과학 계산, 영상 처리 가속
  • 언제 사용하는가? 딥러닝 훈련/추론, 과학 시뮬레이션, 영상 처리, 암호화 등

이 문서가 존재하는 이유

  • Goal: CUDA 컴파일 방법, nvcc 플래그, PyTorch/TensorRT 설정, 호환성 정보 제공
  • Scope: CUDA 컴파일 플래그, TORCH_CUDA_ARCH_LIST, CMake 설정, CUDA 호환성
  • Non-goals: CUDA C++ 프로그래밍 상세, 커널 최적화 기법

Best Practices

  • 최신 버전 사용 권장
  • 공식 문서 참고
  • 테스트 환경에서 먼저 검증


References


Related Pages

Knowledge Graph

Related

→ NVIDIA GPU → CUDA Toolkit → CUDA Programming → MIG → NVIDIA GPUs → Tensor Core

Concept Description Related
Retrieved from "http://wiki.hpcmate.com/index.php?title=CUDA&oldid=3261"