Training and Inference

Overview

Training and Inference에 대한 기술 문서입니다.

Summary

• 무엇인가? - Training and Inference
• 왜 필요한가? - HPC 및 서버 환경에서 필수 개념
• 언제 사용하는가? - 서버 구성, 성능 튜닝, 문제 해결 시

Purpose

이 문서가 존재하는 이유

• Goal: Training and Inference에 대한 기술 정보 제공
• Scope: Training and Inference의 개념, 사용법, 설정
• Non-goals: 다른 주제로의 확장

Key Concepts

Concept	Description	Related
Training and Inference	HPC/서버 환경에서 중요한 기술 개념	Linux, Server

Detailed Explanation

In AI we must first draw a distinction between training an AI model and using the trained model for inference on a real world data stream. Training stage requires magnitude amount of compute intensive while inference stage requires minimum reasonable amount of compute resources to provide service response time effectivly. The three major categories of AI architectures and the challenges

• Convolutional Neural Networks (CNNs)
  • CNNs then benefit from processing architectures that can expose parallel processing capability such as FPGAs and GPUs rather than sequential CPU and MCU architectures
• Recursive Neural Networks (RNNs)
  • RNN appications like language translation, mathematical biology and financial analysis need to include some form of “memory” in the AI algorithm so that past data can be used to gain insight from subsequent readings by using the "Memory"
  • The memory capability of RNNs makes them good at processing sequential architectures such as CPUs and MCUs than GPU based parallelism
• Transformers
  • Transformers contain massive parallelism. Transformers are more complex and have higher compute requirements than their CNN predecessors and benefit from the massive compute and parallel architectures of FPGAs and GPUs

Inference stage solutions must have low power consumption, be cost effective and be able to operate in harsh environments. GPUs and traditional FPGAs offer a parallel compute capability but consume too much power and are too expensive. Microcontrollers and microprocessors are sequential in nature and lack the performance. The open source community has put a lot of work into developing tools that can quantize AI models and reduce their complexity to a point where running on a microcontroller can, in some cases, deliver the desired performance. example, TensorFlow Lite can create models that, with the aid of runtime libraries, can execute on microcontroller architectures. High performance light weight programable FPGA would be the next blue ocean .

Best Practices

• 최신 버전 사용 권장
• 공식 문서 참고
• 테스트 환경에서 먼저 검증

References

• Training and Inference

Related Pages

• Linux
• Server
• Hardware
• Network

Knowledge Graph

Related

→ Linux → Server → Hardware → Network