Types of VRAM

When discussing AI using GPU, training is very compute-intensive and requires the highest system bandwidth possible. while inference is more common and not quite as bandwidth hungry as training.

in both case, VRAM (Video Random Access Memory) is one of the key component along with GPU processor for better performance.

the amount of VRAM would be need largely depends on what it is being used for.

VRAM Performance

Artificial intelligence (AI), machine learning (ML), deep learning (DL), autonomous driving, high-performance computing (HPC), virtual reality (VR), augmented reality (AR) require ultra-bandwidth solutions memory.

	GDDR5	GDDR5X	GDDR6	GDDR6X	HBM2	HBM2E
Application Type (Example)	Graphics	Graphics	Graphics AI Inference Accelerator	Graphics AI Inference Accelerator	AI Training Accelerator	AI Training Accelerator
Frame Buffer of Typical System	8GB	12GB	12GB	12GB	16-32GB	32-96GB
Gb/s/pin	8	11.4	14-16	19-21	1.75-2	3.2-3.6
Signaling	Binary	Binary	Binary	PAM4	Binary
DRAM Voltage	1.35 V	1.35 V	1.35 V	1.35 V	1.2 V	1.2 V
Bus Width		352-bit	384-bit	384-bit	3072-bit
Total B/W		548 GB/s	672 GB/s	1008 GB/s	652.8 GB/s
Data Rate		DDR	QDR	QDR	DDR