Types of VRAM

From HPCWIKI
Revision as of 19:45, 22 October 2023 by Admin (talk | contribs) (Created page with "When discussing AI using GPU, training is very compute-intensive and requires the highest system bandwidth possible. while inference is more common and not quite as bandwidth hungry as training. in both case, VRAM ('''V'''ideo '''R'''andom '''A'''ccess '''M'''emory) is one of the key component along with GPU processor for better performance. == VRAM Performance == Artificial intelligence (AI), machine learning (ML), deep learning (DL), autonomous driving, high-perfo...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

When discussing AI using GPU, training is very compute-intensive and requires the highest system bandwidth possible. while inference is more common and not quite as bandwidth hungry as training.

in both case, VRAM (Video Random Access Memory) is one of the key component along with GPU processor for better performance.

VRAM Performance

Artificial intelligence (AI), machine learning (ML), deep learning (DL), autonomous driving, high-performance computing (HPC), virtual reality (VR), augmented reality (AR) require ultra-bandwidth solutions memory performance.[1]

GDDR5 GDDR5X GDDR6 GDDR6X[2] HBM2 HBM2E
Application Type (Example) Graphics Graphics Graphics

AI Inference Accelerator

Graphics

AI Inference Accelerator

AI Training

Accelerator

AI Training

Accelerator

Frame Buffer of Typical System 8GB 12GB 12GB 12GB 16-32GB 32-96GB
Gb/s/pin 8 11.4 14-16 19-21 1.75-2 3.2-3.6
Signaling Binary Binary Binary PAM4 Binary
DRAM Voltage 1.35 V 1.35 V 1.35 V 1.35 V 1.2 V 1.2 V
Bus Width 352-bit 384-bit 384-bit 3072-bit
Total B/W 548 GB/s 672 GB/s 1008 GB/s 652.8 GB/s
Data Rate DDR QDR QDR DDR

References