Kernel tips and tricks

Overview

Kernel tips and tricks에 대한 기술 문서입니다.

Summary

• 무엇인가? - Kernel tips and tricks
• 왜 필요한가? - HPC 및 서버 환경에서 필수 개념
• 언제 사용하는가? - 서버 구성, 성능 튜닝, 문제 해결 시

Purpose

이 문서가 존재하는 이유

• Goal: Kernel tips and tricks에 대한 기술 정보 제공
• Scope: Kernel tips and tricks의 개념, 사용법, 설정
• Non-goals: 다른 주제로의 확장

Key Concepts

Concept	Description	Related
Kernel tips and tricks	HPC/서버 환경에서 중요한 기술 개념	Linux, Server

Detailed Explanation

• Kernel

In the ARM world from ARMv4 to ARMv7 floating-point support is called VFP, To determine whether you have appropriate floating point support, look for vfpv3 in the “Features” line in /proc/cpuinfo.

1. Identify the specific hardware error source using mcelog or the APEI kernel log (/var/log/kern.log.x)
2. Update System Firmware - Manufacturers often release firmware updates to address hardware error-related issues. Applying the latest firmware (BIOS/UEFI/device firmware) can resolve known APEI generic hardware error problems
3. Kernel Parameter Tuning - although this is not recommended as it may compromise system stability. “apei=off” kernel parameter can be used to disable APEI error handling altogether
4. Hardware Diagnostics - memtest86 for memory testing or CPU stress tests for CPU-related issues

Symptoms The "RT throttling activated" is a message from the operating system scheduler has identified some Real-Time threads that are hogging the CPU and starving other threads. The operating system does this in an attempt to throttle those real-time tasks and keep the operating system from being unresponsive. From the kernel point of view, when a RT thread occupied the cpu by 950ms/1s (defined by /proc/sys/kernel/sched_rt_period_us and /proc/sys/kernel/sched_rt_runtime_us), whether the RT thread maybe is a business thread or other unknown thread. Current Linux kernel only outputs the print "sched: RT throttling activated" when RT throttling happen and it is hard to know what is the RT thread. Simply, a normal thread can't get cpu, And at this moment, Kernel prints 'sched: RT throttling activated' log Troubleshooting

1. Linux patch to print more infor : For further analysis A Linux Kernel patch is available to print current RT task when RT throttling activated that help us to know what is the RT thread in the first time.
2. To reach a only 50% CPU usage for real-time tasks and a larger period the values can be changed with the following commands

   # echo 2000000 > /proc/sys/kernel/sched_rt_period_us 
   # echo 1000000 > /proc/sys/kernel/sched_rt_runtime_us
   

3. Real-time throttling is disabled in case the real-time task runtime has the same length than the the period. This is done automatically by writing `-1` into `sched_rt_runtime_us`:

   # echo -1 > /proc/sys/kernel/sched_rt_runtime_us
   

This mechanism is already implemented in mainline Linux.

Best Practices

• 최신 버전 사용 권장
• 공식 문서 참고
• 테스트 환경에서 먼저 검증

References

• Kernel tips and tricks

Related Pages

• Linux
• Server
• Hardware
• Network

Knowledge Graph

Related

→ Linux Kernel Tuning → THP (Transparent Huge Pages) → Kernel boot param → Linux Kernel → Tunable kernel params