MLPerf

MLPerf is a consortium of key contributors from the AI/ML (Artificial Intelligence and Machine Learning) community that provides unbiased AI/ML performance evaluations of hardware, software, and services.

MLPerf Submission Categories

version	Description	URL
MLPerf Training v2.1	The seventh instantiation for training and consists of eight different workloads covering a broad diversity of use cases, including vision, language, recommenders, and reinforcement learning	https://mlcommons.org/en/training-normal-21/
MLPerf Inference v3.0	The seventh instantiation for inference and tested seven different use cases across seven different kinds of neural networks. Three of these use cases were for computer vision, one was for recommender systems, two were for language processing, and one was for medical imaging.	https://mlcommons.org/en/inference-edge-30/
MLPerf HPC v2.0	The third iteration for HPC and tested three different scientific computing use cases, including climate atmospheric river identification, cosmology parameter prediction, and quantum molecular modeling.	https://mlcommons.org/en/training-hpc-20/

Benchmark Script

#MIG slice benchmark
trap "date; echo failed :(; exit 1" ERR # catch execution failures
ALL_GPUS=$(nvidia-smi -L | grep "UUID: MIG-GPU" | cut -d" " -f8 | cut -d')' -f1)
for gpu in $(echo "$ALL_GPUS"); do
    export CUDA_VISIBLE_DEVICES=$gpu
    $MLPERF_BENCHMARK & # launch workload in background
done

wait # wait for the completion of all the background processes