FAQ: Difference between revisions

From HPCWIKI
Jump to navigation Jump to search
(Fix: remove --- horizontal lines (7 removed))
 
(3 intermediate revisions by the same user not shown)
Line 1: Line 1:
== System reboot - by Software or Hardware ? ==
{{Status
If your system reboot randomly without kernel logs, the most possible reason would be power supply stability issue although the PSU might looks working well.
|status=Draft
|owner=Knowledge Agent
|last_update=2026-07-16
|review=Pending
}}


Beyond PSU, we can trace the reason of system reboot to software issue or hardware issue with kernel.panic parameters,
{{TOC}}
 
== Overview ==
 
FAQ에 대한 기술 문서입니다.
 
=== Summary ===
 
* 무엇인가? - FAQ
* 왜 필요한가? - HPC 및 서버 환경에서 필수 개념
* 언제 사용하는가? - 서버 구성, 성능 튜닝, 문제 해결 시
 
 
== Purpose ==
 
이 문서가 존재하는 이유
 
* Goal: FAQ에 대한 기술 정보 제공
* Scope: FAQ의 개념, 사용법, 설정
* Non-goals: 다른 주제로의 확장


If kernel.panic system parameter is <code>0,</code> it is turned off automatic reboot on panic, any other value is the number of seconds it wait before reboot.


With <code>sysctl -w kernel.panic=0</code> you would turn it off, if it is not already off.
== Key Concepts ==


{| class="wikitable"
! Concept
! Description
! Related
|-
| FAQ
| HPC/서버 환경에서 중요한 기술 개념
| [[Linux]], [[Server]]
|}


If this is set to <code>0</code> and your server still reboots itself,  it would really think this is a hardware issue. If this stops the automatic rebooting, then we know the reboot is caused by a watchdog timer or other software issue


== Failed to load plugin io.containerd and could not use snapshotter ==
== Detailed Explanation ==


If your system reboot randomly without [[kernel]] logs, the most possible reason would be power supply stability issue although the PSU might looks working well.
Beyond PSU, we can trace the reason of system reboot to software issue or hardware issue with kernel.panic parameters, 
If kernel.panic system parameter is <code>0,</code> it is turned off automatic reboot on panic, any other value is the number of seconds it wait before reboot.
With <code>sysctl -w kernel.panic=0</code> you would turn it off, if it is not already off.
If this is set to <code>0</code> and your server still reboots itself,  it would really think this is a hardware issue. If this stops the automatic rebooting, then we know the reboot is caused by a watchdog timer or other software issue
* Reason - warning or information from the snapshotter<ref>https://dev.to/napicella/what-is-a-containerd-snapshotters-3eo2</ref> - image storage - that we have a lot of choices
* Reason - warning or information from the snapshotter<ref>https://dev.to/napicella/what-is-a-containerd-snapshotters-3eo2</ref> - image storage - that we have a lot of choices
* Impact : the warning log doesn't impact the whole system operating
* Impact : the warning log doesn't impact the whole system operating
* Solve to
* Solve to
1.Disable the snapshotter plugins which you don't need by updating config file for your system and restart containerd, like
1.Disable the snapshotter plugins which you don't need by updating config file for your system and restart containerd, like
  <# /etc/containerd/config.toml
  <# /etc/containerd/config.toml
  disabled_plugins = ["cri", "btrfs"]
  disabled_plugins = ["cri", "btrfs"]
2. To use ZFS, you need to mount ZFS dataset on  /var/lib/containerd/io.containerd.snapshotter.v1.zfs
2. To use ZFS, you need to mount ZFS dataset on  /var/lib/containerd/io.containerd.snapshotter.v1.zfs
3. To use btrfs, you need to mount btrfs to /var/lib/containerd/io.containerd.snapshotter.v1.btrfs  
3. To use btrfs, you need to mount btrfs to /var/lib/containerd/io.containerd.snapshotter.v1.btrfs  
4. For aufs, you need to modprobe it as explained in the error log
4. For aufs, you need to modprobe it as explained in the error log
== Could not select device driver "" with capabilities: GPU ==
* Reason - no nvidia-container-toolkit or currupt exist package
* Reason - no nvidia-container-toolkit or currupt exist package
* Solve to install/reinstall nvidia-container-toolkit then restart [[docker]] daemon
* Solve to install/reinstall nvidia-container-toolkit then restart [[docker]] daemon
Line 33: Line 62:
$distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \  
$distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \  
     && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \  
     && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \  
     && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list  
     && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
 
$sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
 
$sudo systemctl restart docker
</syntaxhighlight>
 
== Pytorch FAQ ==
 
* How to get [[CUDA]] compute capability of a GPU?
** $python -c "import torch; print(torch.cuda.get_arch_list())"
 
== Show List Of Network Cards on Linux ==


* <kbd>lspci command</kbd> : List all PCI devices.
** <code>'''#lspci | egrep -i --color 'network|ethernet''''</code>
** <code>'''#lspci | egrep -i --color 'network|ethernet|wireless|wi-fi''''</code>


* <kbd>lshw command</kbd> : [[Linux]] identify Ethernet interfaces and NIC hardware.
== Best Practices ==
** <code>'''#lshw -class network'''</code>
** '''$sudo lshw -class network -short'''


* <kbd>dmidecode command</kbd> : List all hardware data from BIOS.
* 최신 버전 사용 권장
* <kbd>ifconfig command</kbd> : Outdated network config
* 공식 문서 참고
** <code>'''$ifconfig -a'''</code>
* 테스트 환경에서 먼저 검증
** <code>'''$ip link show'''</code>
** <code>'''$ip a'''</code>


* <kbd>ip command</kbd> : Recommended new network config .
** <code>'''$ip a show wlp82s0'''</code>
** <code>'''$ip -br -c link show'''        # To list all interface, link status, MAC address, etc</code>
** <code>'''$ip -br -c addr show'''        # similar list with IP address instead of MAC Address</code>


* <kbd>hwinfo command</kbd> : Probe Linux for network cards.
== References ==
** <code>'''$sudo hwinfo --network --short'''</code>


* <kbd>ethtool command</kbd> : See NIC/card driver and settings on Linux.
* [https://wiki.hpcmate.com FAQ]
** <code>'''$sudo ethtool -i eno1'''</code>
** <code>'''$sudo ethtool -i enp0s31f6'''</code>


* /proc/net/dev file - The dev pseudo-file contains network device status information. This gives the number of received and sent packets, the number of errors and collisions and other basic statistics
** <code>'''$cat /proc/net/dev'''</code>


=== Failed to set iommu for container: Invalid argument ===
== Related Pages ==


* [[Linux]]
* [[Server]]
* [[Hardware]]
* [[Network]]


A VM configured with a vGPU that supports SR-IOV may fail to start, This issue occurs because [[PCIe]] [[AER (Advanced Error Reporting)]] [[support]] was disabled in the [[BIOS]] settings of the server.


== Identify network driver on Linux ==
[[Category:Server]]
There are bunch of way to identify network driver on your system<ref>https://unix.stackexchange.com/questions/41817/linux-how-to-find-the-device-driver-used-for-a-device</ref>. one of the shortest way would identify device on PCI bus then find driver that uses by the device. <syntaxhighlight lang="shell">
== Knowledge Graph ==
#Identify PCI device using lspci
$ lspci | grep -i eth
e4:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
e4:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)


#Identify device driver using the PCI bus number
Related
$ find /sys | grep drivers.*e4
/sys/bus/pci/drivers/igb/0000:e4:00.0
/sys/bus/pci/drivers/igb/0000:e4:00.1


#Use ethtool to get more information
→ [[Linux]]
$ ethtool -i <interfacename>
→ [[Server]]
driver: igb
→ [[Troubleshooting]]
version: 5.6.0-k
→ [[Terms]]
firmware-version: 1.63, 0x80000a05
expansion-rom-version:
bus-info: 0000:e4:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
</syntaxhighlight>


== Reference ==
[[Category:Reference]]
<references />

Latest revision as of 11:28, 17 July 2026

Template:Status

Template:TOC

Overview

FAQ에 대한 기술 문서입니다.

Summary

  • 무엇인가? - FAQ
  • 왜 필요한가? - HPC 및 서버 환경에서 필수 개념
  • 언제 사용하는가? - 서버 구성, 성능 튜닝, 문제 해결 시


Purpose

이 문서가 존재하는 이유

  • Goal: FAQ에 대한 기술 정보 제공
  • Scope: FAQ의 개념, 사용법, 설정
  • Non-goals: 다른 주제로의 확장


Key Concepts

Concept Description Related
FAQ HPC/서버 환경에서 중요한 기술 개념 Linux, Server


Detailed Explanation

If your system reboot randomly without kernel logs, the most possible reason would be power supply stability issue although the PSU might looks working well. Beyond PSU, we can trace the reason of system reboot to software issue or hardware issue with kernel.panic parameters, If kernel.panic system parameter is 0, it is turned off automatic reboot on panic, any other value is the number of seconds it wait before reboot. With sysctl -w kernel.panic=0 you would turn it off, if it is not already off. If this is set to 0 and your server still reboots itself, it would really think this is a hardware issue. If this stops the automatic rebooting, then we know the reboot is caused by a watchdog timer or other software issue

  • Reason - warning or information from the snapshotter[1] - image storage - that we have a lot of choices
  • Impact : the warning log doesn't impact the whole system operating
  • Solve to

1.Disable the snapshotter plugins which you don't need by updating config file for your system and restart containerd, like 

<# /etc/containerd/config.toml
disabled_plugins = ["cri", "btrfs"]

2. To use ZFS, you need to mount ZFS dataset on /var/lib/containerd/io.containerd.snapshotter.v1.zfs 3. To use btrfs, you need to mount btrfs to /var/lib/containerd/io.containerd.snapshotter.v1.btrfs 4. For aufs, you need to modprobe it as explained in the error log

  • Reason - no nvidia-container-toolkit or currupt exist package
  • Solve to install/reinstall nvidia-container-toolkit then restart docker daemon

<syntaxhighlight lang="bash"> $distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \ 

   && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \ 
   && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list


Best Practices

  • 최신 버전 사용 권장
  • 공식 문서 참고
  • 테스트 환경에서 먼저 검증


References


Related Pages

Knowledge Graph

Related

→ Linux → Server → Troubleshooting → Terms

  1. ↑ https://dev.to/napicella/what-is-a-containerd-snapshotters-3eo2
Retrieved from "http://wiki.hpcmate.com/index.php?title=FAQ&oldid=3307"