NUMA
Overview
NUMA에 대한 기술 문서입니다.
Summary
- 무엇인가? - NUMA
- 왜 필요한가? - HPC 및 서버 환경에서 필수 개념
- 언제 사용하는가? - 서버 구성, 성능 튜닝, 문제 해결 시
Purpose
이 문서가 존재하는 이유
- Goal: NUMA에 대한 기술 정보 제공
- Scope: NUMA의 개념, 사용법, 설정
- Non-goals: 다른 주제로의 확장
Key Concepts
| Concept | Description | Related |
|---|---|---|
| NUMA | HPC/서버 환경에서 중요한 기술 개념 | Linux, Server |
Detailed Explanation
In the past, processors had been designed as Symmetric Multi-processing or Uniform Memory Architecture (UMA) machines, which mean that all processors shared the access to all memory available in the system over the single bus. in UMA machines, due to accessing the memory by multiple CPUs over a single bus, there is more load on the shared bus due to limited bus bandwidth and it cause latency and collisions between multiple CPUs.[1] Non-Uniform Memory Access (NUMA) - more correctly Cache-Coherent Numa (ccNUMA) - describes a shared memory architecture used in contemporary multiprocessing systems. NUMA technology allow all CPUs to access the entire memory directly. So NUMA systems are quite attractive for server-side applications, such as data mining and decision support systems. Furthermore, writing applications for gaming and high-performance software becomes much easier with this architecture.
NUMA is a clever system used for connecting multiple central processing units (CPU) to any amount of computer memory available on the computer. The single NUMA nodes are connected over a scalable network (I/O bus) such that a CPU can systematically access memory associated with other NUMA nodes.
Local memory is the memory that the CPU is using in a particular NUMA node. Foreign or remote memory is the memory that a CPU is taking from another NUMA node. The term NUMA ratio describes the ratio of the cost of accessing foreign memory to the cost of accessing local memory. The greater the ratio, the greater the cost, and thus the longer it takes to access the memory. Local memory access is a major advantage, as it combines low latency with high bandwidth. In contrast, accessing memory belonging to any other CPU has higher latency and lower bandwidth performance. This leads to more efficient use of the 64-bit addressing scheme, resulting in faster movement of data, less replication of data, and easier programming UNMA nodes are a small group of CPUs accessing their local memory all together. So there can be several such groups of CPUs and memory. NUMA architecture supports
- To compensate for memory latency effects due to some memory being closer to the processor than others.
- Useful to scale the VM for large amounts of memory
- I/O optimizations due to device NUMA effects
The Linux kernel has supported NUMA since version 2.5 for process optimization with the two software packages numactl[2] and numad[3].
$numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 64 65 66 67 68 69 70 71 7... [내용 계속]
== Best Practices ==
* 최신 버전 사용 권장
* 공식 문서 참고
* 테스트 환경에서 먼저 검증
== References ==
* [https://wiki.hpcmate.com NUMA]
== Related Pages ==
* [[Linux]]
* [[Server]]
* [[Hardware]]
* [[Network]]
[[Category:Server]]
== Knowledge Graph ==
Related
→ [[Linux]]
→ [[Server]]
→ [[Hardware]]
→ [[Network]]
[[Category:Reference]]
== numactl 사용법 ==
numactl은 NUMA 아키텍처에서 프로세스 메모리/CPU 바인딩을 제어.
<syntaxhighlight lang="bash">
# NUMA 노드 확인
numactl --hardware
# 특정 NUMA 노드에 프로세스 바인딩
numactl --cpunodebind=0 --membind=0 ./my_app
# 특정 CPU 코어에 바인딩
numactl --cpunodebind=0-7 --membind=0 ./my_app
# NUMA 노드별 메모리 사용량 확인
numastat
hwloc 사용법
hwloc은 시스템 토폴로지 감지 도구.
# 시스템 토폴로지 시각화
lstopo-no-graphics
# CPU 정보
lstopo --of xml
hwloc-calc --allbits package:0
# GPU 토폴로지 확인
lstopo --of png gpu-topology.png
# 특정 리소스에 프로세스 바인딩
hwloc-bind package:0 ./my_app
NUMA 최적화 팁
- MPI 애플리케이션: numactl로 노드별 바인딩
- CUDA 애플리케이션: CUDA_VISIBLE_DEVICES로 GPU 노드 지정
- Java 애플리케이션: -XX:ActiveProcessorCount로 코어 수 제한
- Python: taskset으로 CPU 바인딩