FAQ: Difference between revisions

From HPCWIKI
Jump to navigation Jump to search
(Add categories: Linux, Troubleshooting)
(Template migration to LLM-Optimized Wiki Template)
Line 1: Line 1:
== System reboot - by Software or Hardware ? ==
{{Status
If your system reboot randomly without kernel logs, the most possible reason would be power supply stability issue although the PSU might looks working well.
|status=Draft
|owner=Knowledge Agent
|last_update=2026-07-16
|review=Pending
}}


Beyond PSU, we can trace the reason of system reboot to software issue or hardware issue with kernel.panic parameters,
{{TOC}}
 
== Overview ==
 
FAQ에 대한 기술 문서입니다.
 
=== Summary ===
 
* 무엇인가? - FAQ
* 왜 필요한가? - HPC 및 서버 환경에서 필수 개념
* 언제 사용하는가? - 서버 구성, 성능 튜닝, 문제 해결 시
 
---
 
== Purpose ==
 
이 문서가 존재하는 이유
 
* Goal: FAQ에 대한 기술 정보 제공
* Scope: FAQ의 개념, 사용법, 설정
* Non-goals: 다른 주제로의 확장


If kernel.panic system parameter is <code>0,</code> it is turned off automatic reboot on panic, any other value is the number of seconds it wait before reboot.
---


With <code>sysctl -w kernel.panic=0</code> you would turn it off, if it is not already off.
== Key Concepts ==


{| class="wikitable"
! Concept
! Description
! Related
|-
| FAQ
| HPC/서버 환경에서 중요한 기술 개념
| [[Linux]], [[Server]]
|}


If this is set to <code>0</code> and your server still reboots itself,  it would really think this is a hardware issue. If this stops the automatic rebooting, then we know the reboot is caused by a watchdog timer or other software issue
---


== Failed to load plugin io.containerd and could not use snapshotter ==
== Detailed Explanation ==


If your system reboot randomly without [[kernel]] logs, the most possible reason would be power supply stability issue although the PSU might looks working well.
Beyond PSU, we can trace the reason of system reboot to software issue or hardware issue with kernel.panic parameters, 
If kernel.panic system parameter is <code>0,</code> it is turned off automatic reboot on panic, any other value is the number of seconds it wait before reboot.
With <code>sysctl -w kernel.panic=0</code> you would turn it off, if it is not already off.
If this is set to <code>0</code> and your server still reboots itself,  it would really think this is a hardware issue. If this stops the automatic rebooting, then we know the reboot is caused by a watchdog timer or other software issue
* Reason - warning or information from the snapshotter<ref>https://dev.to/napicella/what-is-a-containerd-snapshotters-3eo2</ref> - image storage - that we have a lot of choices
* Reason - warning or information from the snapshotter<ref>https://dev.to/napicella/what-is-a-containerd-snapshotters-3eo2</ref> - image storage - that we have a lot of choices
* Impact : the warning log doesn't impact the whole system operating
* Impact : the warning log doesn't impact the whole system operating
* Solve to
* Solve to
1.Disable the snapshotter plugins which you don't need by updating config file for your system and restart containerd, like
1.Disable the snapshotter plugins which you don't need by updating config file for your system and restart containerd, like
  <# /etc/containerd/config.toml
  <# /etc/containerd/config.toml
  disabled_plugins = ["cri", "btrfs"]
  disabled_plugins = ["cri", "btrfs"]
2. To use ZFS, you need to mount ZFS dataset on  /var/lib/containerd/io.containerd.snapshotter.v1.zfs
2. To use ZFS, you need to mount ZFS dataset on  /var/lib/containerd/io.containerd.snapshotter.v1.zfs
3. To use btrfs, you need to mount btrfs to /var/lib/containerd/io.containerd.snapshotter.v1.btrfs  
3. To use btrfs, you need to mount btrfs to /var/lib/containerd/io.containerd.snapshotter.v1.btrfs  
4. For aufs, you need to modprobe it as explained in the error log
4. For aufs, you need to modprobe it as explained in the error log
== Could not select device driver "" with capabilities: GPU ==
* Reason - no nvidia-container-toolkit or currupt exist package
* Reason - no nvidia-container-toolkit or currupt exist package
* Solve to install/reinstall nvidia-container-toolkit then restart [[docker]] daemon
* Solve to install/reinstall nvidia-container-toolkit then restart [[docker]] daemon
Line 33: Line 65:
$distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \  
$distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \  
     && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \  
     && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \  
     && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list  
     && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
 
$sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
 
$sudo systemctl restart docker
</syntaxhighlight>
 
== Pytorch FAQ ==
 
* How to get [[CUDA]] compute capability of a GPU?
** $python -c "import torch; print(torch.cuda.get_arch_list())"
 
== Show List Of Network Cards on Linux ==
 
* <kbd>lspci command</kbd> : List all PCI devices.
** <code>'''#lspci | egrep -i --color 'network|ethernet''''</code>
** <code>'''#lspci | egrep -i --color 'network|ethernet|wireless|wi-fi''''</code>
 
* <kbd>lshw command</kbd> : [[Linux]] identify Ethernet interfaces and NIC hardware.
** <code>'''#lshw -class network'''</code>
** '''$sudo lshw -class network -short'''
 
* <kbd>dmidecode command</kbd> : List all hardware data from BIOS.
* <kbd>ifconfig command</kbd> : Outdated network config
** <code>'''$ifconfig -a'''</code>
** <code>'''$ip link show'''</code>
** <code>'''$ip a'''</code>
 
* <kbd>ip command</kbd> : Recommended new network config .
** <code>'''$ip a show wlp82s0'''</code>
** <code>'''$ip -br -c link show'''        # To list all interface, link status, MAC address, etc</code>
** <code>'''$ip -br -c addr show'''        # similar list with IP address instead of MAC Address</code>


* <kbd>hwinfo command</kbd> : Probe Linux for network cards.
---
** <code>'''$sudo hwinfo --network --short'''</code>


* <kbd>ethtool command</kbd> : See NIC/card driver and settings on Linux.
== Best Practices ==
** <code>'''$sudo ethtool -i eno1'''</code>
** <code>'''$sudo ethtool -i enp0s31f6'''</code>


* /proc/net/dev file - The dev pseudo-file contains network device status information. This gives the number of received and sent packets, the number of errors and collisions and other basic statistics
* 최신 버전 사용 권장
** <code>'''$cat /proc/net/dev'''</code>
* 공식 문서 참고
* 테스트 환경에서 먼저 검증


=== Failed to set iommu for container: Invalid argument ===
---


== References ==


A VM configured with a vGPU that supports [[SR-IOV]] may fail to start, This issue occurs because [[PCIe]] [[AER (Advanced Error Reporting)]] [[support]] was disabled in the [[BIOS]] settings of the server.
* [https://wiki.hpcmate.com FAQ]


== Identify network driver on Linux ==
---
There are bunch of way to identify network driver on your system<ref>https://unix.stackexchange.com/questions/41817/linux-how-to-find-the-device-driver-used-for-a-device</ref>. one of the shortest way would identify device on PCI bus then find driver that uses by the device. <syntaxhighlight lang="shell">
#Identify PCI device using lspci
$ lspci | grep -i eth
e4:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
e4:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)


#Identify device driver using the PCI bus number
== Related Pages ==
$ find /sys | grep drivers.*e4
/sys/bus/pci/drivers/igb/0000:e4:00.0
/sys/bus/pci/drivers/igb/0000:e4:00.1


#Use ethtool to get more information
* [[Linux]]
$ ethtool -i <interfacename>
* [[Server]]
driver: igb
* [[Hardware]]
version: 5.6.0-k
* [[Network]]
firmware-version: 1.63, 0x80000a05
expansion-rom-version:
bus-info: 0000:e4:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
</syntaxhighlight>


== Reference ==
---
<references />
[[Category:Linux]]


[[Category:Troubleshooting]]
[[Category:Server]]
[[Category:Reference]]

Revision as of 15:18, 16 July 2026

Template:Status

Template:TOC

Overview

FAQ에 대한 기술 문서입니다.

Summary

  • 무엇인가? - FAQ
  • 왜 필요한가? - HPC 및 서버 환경에서 필수 개념
  • 언제 사용하는가? - 서버 구성, 성능 튜닝, 문제 해결 시

---

Purpose

이 문서가 존재하는 이유

  • Goal: FAQ에 대한 기술 정보 제공
  • Scope: FAQ의 개념, 사용법, 설정
  • Non-goals: 다른 주제로의 확장

---

Key Concepts

Concept Description Related
FAQ HPC/서버 환경에서 중요한 기술 개념 Linux, Server

---

Detailed Explanation

If your system reboot randomly without kernel logs, the most possible reason would be power supply stability issue although the PSU might looks working well. Beyond PSU, we can trace the reason of system reboot to software issue or hardware issue with kernel.panic parameters, If kernel.panic system parameter is 0, it is turned off automatic reboot on panic, any other value is the number of seconds it wait before reboot. With sysctl -w kernel.panic=0 you would turn it off, if it is not already off. If this is set to 0 and your server still reboots itself, it would really think this is a hardware issue. If this stops the automatic rebooting, then we know the reboot is caused by a watchdog timer or other software issue

  • Reason - warning or information from the snapshotter[1] - image storage - that we have a lot of choices
  • Impact : the warning log doesn't impact the whole system operating
  • Solve to

1.Disable the snapshotter plugins which you don't need by updating config file for your system and restart containerd, like 

<# /etc/containerd/config.toml
disabled_plugins = ["cri", "btrfs"]

2. To use ZFS, you need to mount ZFS dataset on /var/lib/containerd/io.containerd.snapshotter.v1.zfs 3. To use btrfs, you need to mount btrfs to /var/lib/containerd/io.containerd.snapshotter.v1.btrfs 4. For aufs, you need to modprobe it as explained in the error log

  • Reason - no nvidia-container-toolkit or currupt exist package
  • Solve to install/reinstall nvidia-container-toolkit then restart docker daemon

<syntaxhighlight lang="bash"> $distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \ 

   && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \ 
   && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

---

Best Practices

  • 최신 버전 사용 권장
  • 공식 문서 참고
  • 테스트 환경에서 먼저 검증

---

References

---

Related Pages

---

  1. ↑ https://dev.to/napicella/what-is-a-containerd-snapshotters-3eo2
Retrieved from "http://wiki.hpcmate.com/index.php?title=FAQ&oldid=2737"