Dockerfile tips and tricks: Difference between revisions

From HPCWIKI
Jump to navigation Jump to search
(Add categories: Network, Reference)
(Phase 6.1: LLM-Optimized Wiki Template migration)
Line 1: Line 1:
== Docker complains to version of requests ==
= Dockerfile tips and tricks =
ERROR: docker 7.0.0 has requirement requests>=2.26.0, but you'll have requests 2.22.0 which is incompatible


install right version of requests packages
{{Status
|status=Draft
|owner=Knowledge Agent
|last_update=2026-07-16
|review=Pending
}}


pip install -U urllib3 requests==<right version>
{{TOC}}


== Set up OpenCL for GPUs on Docker ==
== Overview ==
Following is essential components to setup OpenCL inside Docker<ref>https://linuxhandbook.com/setup-opencl-linux-docker/</ref>  <syntaxhighlight lang="bash">
 
[[Docker]] 이미지 빌드 및 컨테이너 운영 시 유용한 팁과 트릭 모음. 버전 충돌 해결, GPU 설정, 성능 튜닝, 시그널 처리 등 실용적인 문제 해결 가이드.
 
=== Summary ===
 
* 무엇인가? Dockerfile 작성 및 Docker 컨테이너 운영 시 발생하는常见问题 해결 방법
* 왜 필요한가? Docker 환경 설정 오류, 버전 충돌, GPU 연산 문제 등을 빠르게 해결
* 언제 사용하는가? Docker 이미지 빌드, 컨테이너 실행, GPU/OpenCL 설정, 리소스 튜닝 시
 
---
 
== Purpose ==
 
Dockerfile 작성 시 자주 발생하는 오류와 해결책, GPU/OpenCL 설정, 성능 튜닝, 컨테이너 종료 시그널 처리 등 실용적인 팁 제공
 
* Goal: Docker 환경 구축 및 운영 시 시간 절약 및 문제 해결 속도 향상
* Scope: Dockerfile 빌드 팁, GPU/OpenCL 설정, 리소스 튜닝, 시그널 처리
* Non-goals: Docker 기본 사용법 튜토리얼, Docker Compose 상세 설정
 
---
 
== Key Concepts ==
 
{| class="wikitable"
! Topic
! Description
! Related
|-
| Version Conflict
| Docker 패키지 간 의존성 충돌 해결
| [[Docker command workflow]]
|-
| OpenCL in Docker
| 컨테이너 내 GPU/OpenCL 설정
| [[GPU Support]]
|-
| Resource Tuning
| Docker 리소스 및 성능 최적화
| [[Docker container resource]]
|-
| Container Stop Signal
| 컨테이너 종료 시 SIGTERM/SIGKILL 처리
| [[Docker command workflow]]
|}
 
---
 
== Architecture ==
 
Docker 컨테이너는 호스트 OS의 커널을 공유하는 경량 가상화 환경. GPU/OpenCL 연산을 컨테이너 내에서 사용하려면 호스트의 NVIDIA 드라이버와 OpenCL 라이브러리를 컨테이너에 노출해야 함.
 
---
 
== Workflow ==
 
# Dockerfile 기반 이미지 빌드
# 의존성 버전 충돌 시 pip install -U 로 해결
# GPU/OpenCL 설정 시 nvidia-container-toolkit 및 ICD 파일 구성
# 컨테이너 종료 시 SIGTERM → 10초 후 SIGKILL 기본 동작 확인
# ENTRYPOINT/STOPSIGNAL/docker-compose.yml stop_signal 설정으로 동작 커스터마이징
 
---
 
== Configuration ==
 
=== OpenCL 설정 (Docker 내 GPU) ===
 
<syntaxhighlight lang="bash">
FROM ubuntu:20.04
FROM ubuntu:20.04
ARG DEBIAN_FRONTEND=noninteractive
ARG DEBIAN_FRONTEND=noninteractive
Line 21: Line 92:
ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_DRIVER_CAPABILITIES compute,utility
ENV NVIDIA_DRIVER_CAPABILITIES compute,utility
</syntaxhighlight>
</syntaxhighlight>


== Docker Performance and Resource Tuning ==
=== Root 비밀번호 설정 ===
[https://www.pugetsystems.com/labs/hpc/How-To-Setup-NVIDIA-Docker-and-NGC-Registry-on-your-Workstation---Part-5-Docker-Performance-and-Resource-Tuning-1119/ This page] would be good resource for Docker resource/performance tuning.
 
== Root password inside a Docker container ==


* Set container root password, when docker image is built by yourse
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
# 이미지 빌드 시
RUN echo 'root:Docker!' | chpasswd
RUN echo 'root:Docker!' | chpasswd
or
 
RUN echo 'Docker!' | passwd --stdin root  
# 또는
RUN echo 'Docker!' | passwd --stdin root
 
# 실행 중인 컨테이너에서
docker exec -itu 0 {container} passwd
 
# 3rd party 이미지에서 root로 로그인
docker container exec -u 0 -it mycontainer bash
</syntaxhighlight>
</syntaxhighlight>


* To create/change a root password in a running container,
=== Docker requests 버전 충돌 해결 ===


<code>docker exec -itu 0 {container} passwd</code>
<syntaxhighlight lang="bash">
pip install -U urllib3 requests==<right_version>
</syntaxhighlight>


* use -u 0 to login to overriding the USER setting as root, when docker image from 3rd party,
=== NAT chain 오류 해결 ===
 
<syntaxhighlight lang="bash">
sudo apt update && sudo apt upgrade && sudo systemctl restart docker.service
</syntaxhighlight>


<code>docker container exec -u 0 -it mycontainer bash</code>
---


== Failed to create NAT chain Docker as ... ==
== Examples ==
Reason - Docker package looks broken


Solution - $sudo apt update && sudo apt upgrade && sudo systemctl restart docker.service<ref>https://stackoverflow.com/questions/75713844/how-to-resolve-failed-to-create-nat-chain-docker-as-reboot-not-working</ref> 
=== 컨테이너 종료 시그널 처리 ===


== Run script at Container stop<ref>https://kmg.group/posts/2022-05-23-docker-stop-containers-with-signals/</ref> ==
기본 동작:
By default [[docker]] stops your container by sending the <code>'''SIGTERM'''</code> signal to the entry point process (normally with process id 1 in the container). If the container is still running after 10 seconds, <code>docker stop</code> and <code>docker-compose down</code> will send the <code>'''SIGKILL'''</code> signal, which will remove the process from the OS scheduler.
* `docker stop` → ENTRYPOINT 프로세스에 SIGTERM 전송
* 10초 후 → SIGKILL 전송


This can be overridden depending on,
커스터마이징:
* Dockerfile의 `STOPSIGNAL` 지시자 변경
* docker-compose.yml의 `stop_signal` 설정
* docker-compose.yml의 `stop_grace_period` 설정


* The <code>ENTRYPOINT</code> in your Dockerfile, and how it behaves when receiving a signal
예시 (php-fpm/nginx): SIGQUIT 사용 (SIGTERM 대신)
* The <code>STOPSIGNAL</code> in your Dockerfile (default: <code>SIGTERM</code>, but this is not always used in all base containers, see php:8.0-fpm and nginx).
* The <code>stop_signal</code> in your docker-compose.yml file
* The <code>stop_grace_period</code> in your docker-compose.yml file


---


For example, the php-fpm and nginx containers use <code>SIGQUIT</code>instead of <code>SIGTERM</code> as stop signa to graceful shutdown process so that user will not affected from the shutdown.
== Best Practices ==
$ docker inspect nginx:latest | jq '.[].Config.StopSignal'
"SIGQUIT"
$ docker inspect php:7.4-fpm | jq '.[].Config.StopSignal'
"SIGQUIT"


=== Container stop after 10s ===
* 의존성 충돌 시 `pip install -U`로 최신 버전으로 업그레이드
#--- create the init.sh script
* GPU/OpenCL 사용 시 `NVIDIA_VISIBLE_DEVICES`와 `NVIDIA_DRIVER_CAPABILITIES` 환경변수 필수 설정
cat<<EOT > init.sh
* 컨테이너 종료 시 graceful shutdown을 위해 ENTRYPOINT에서 시그널 핸들러 구현
#!/bin/bash
* Dockerfile에서 `ARG DEBIAN_FRONTEND=noninteractive`로 대시로그 상호작용 방지
#We don’t trap the signal, so that we can handle SIGTEM to exit the script
echo "'''This container will not stop immediately after SIGTERM, it uses SIGQUIT'''"
sleep infinity #'''sleep infinity in a way so that our bash script can’t trap the signal'''
EOT
chmod 755 init.sh
#--- create the Dockerfile
cat<<EOT > Dockerfile
from php:8.0-fpm
COPY . /
ENTRYPOINT ["/init.sh"]
EOT


=== Container stop as soon as SIGTERM ~ $docker stop <container> ===
---
cat<<EOT > init.sh
#!/bin/bash
#--- add a function to exit nicely (perhaps kill a few processes and remove some temp files)
function '''exit_container_SIGTERM'''(){
  echo "Caught SIGTERM"
  exit 0
}
#--- trap the SIGTERM signal
'''trap exit_container_SIGTERM SIGTERM'''
echo "This container will stop immediately after SIGTERM"
sleep infinity &
wait
EOT


=== Select which signal to use with the STOPSIGNAL  keyward in Dockerfile ===
== Performance ==
cat<<EOT > Dockerfile
from php:8.0-fpm
COPY . /
'''#--- override the SIGQUIT used in php:8.0-fpm
STOPSIGNAL '''SIGTERM'''
ENTRYPOINT ["/init.sh"]
EOT


=== Handle signal correctly in the bash script  ===
if you don’t take care of how you <code>sleep</code> at the end of the script (bash), the script will not catch any signals sent to it, even if you have a <code>trap</code> in your [[bash script|script]].
{| class="wikitable"
{| class="wikitable"
|+
! Setting
|style="width: 50%"| This does not work
| Impact
|style="width: 50%"| This works
| Notes
|-
| OpenCL ICD 설정
| GPU/OpenCL 연산 가능
| /etc/OpenCL/vendors/nvidia.icd 필수
|-
| NVIDIA_DRIVER_CAPABILITIES
| 컨테이너 내 GPU 접근
| compute,utility 권장
|-
|-
|
| stop_signal 설정
function exit_script(){
| graceful shutdown
  echo "Caught SIGTERM"
| php-fpm/nginx는 SIGQUIT 권장
  exit 0
}
trap exit_script SIGTERM
#--- my init.sh script
./start/my/program &
sleep infinity
|
function exit_script(){
  echo "Caught SIGTERM"
  exit 0
}
trap exit_script SIGTERM
#--- my init.sh script
./start/my/program &
#--- send sleep into the background, then wait for it.
sleep infinity &
#--- "wait" will wait until the command you sent to the background terminates, which will be never.
#--- "wait" is a bash built-in, so bash can now handle the signals sent by "docker stop"
wait
|}
|}


== Install nvm in Dockerfile<ref>https://hub.docker.com/r/nuccess/nuclos/dockerfile</ref> ==
---
RUN mkdir -p $NVM_DIR && \
    curl <nowiki>https://raw.githubusercontent.com/creationix/nvm/v0.36.0/install.sh</nowiki> | bash && \
    . $NVM_DIR/nvm.sh && \
    nvm install $NODE_VERSION
 
== Docker cleanup ==
When working with Docker, you can end up piling up unused images, containers, and datasets that clutter the output and take up disk space.  beyond docker images, disk space can be took up with unused containers, volumes, networks. these objects are generally not removed unless you explicitly ask Docker to do so. otherwise unused objects can cause Docker to use extra disk space.
 
 
 
Although each type of object, Docker provides a <code>prune</code> command, Docker has a single command that cleans up all '''<u>dangling resources</u>''', such as images, containers, volumes, and networks, not tagged or connected to a container.
 
Following commands prunes images, containers and networks only. Volumes are not pruned by default until you speficy the --volume flags in this
 
<code>#docker  system prune</code>
 
 
There are also may possible filter<ref>https://docs.docker.com/engine/reference/commandline/system_prune/#filter</ref> options that you can check on Docker site<ref>https://docs.docker.com/config/pruning/</ref><ref>https://docs.docker.com/engine/reference/commandline/system_prune/</ref>
 
== Dangling image vs unused image ==
'''Dangling images''' are images which do not have a tag, and do not have a '''child image which displays "<code><none></code>" on its name when you run <code>docker images</code>comand.  The main reason to keep them around is for <u>build caching purposes</u> in case you need to build multiple different top images from some common Docker image layers. because they can be used a independent layers that have no relationship to any tagged images.'''
 
 
However, An '''unused image''' is an image that has tags but currently not being used as a container.
 
Ofcourse it is safes to delete them when you build final Docker image and ready to use.
 
* List Dangling images


<code>docker images -f dangling=true</code>
== Limitations ==


* Remove Dangling Images
* Docker 7.0.0+는 requests>=2.26.0 필수
* NAT chain 오류 시 재부팅 대신 docker.service 재시작 필요
* 3rd party 이미지는 USER 설정이 override 될 수 있음 (-u 0 사용)


<code>docker rmi $(docker images -f dangling=true -q)</code>
---
Or
<code>docker images --quiet --filter=dangling=true | xargs --no-run-if-empty docker rmi</code>


Or
== References ==
docker images -a | grep none | awk '{ print $3; }' | xargs docker rmi --force
For easy life, we can add alias command in .bashrc file like


<code>alias docker_clean_images='docker rmi $(docker images -a --filter=dangling=true -q)'</code>
* [https://stackoverflow.com/questions/75713844/ Failed to create NAT chain Docker]
* [https://kmg.group/posts/2022-05-23-docker-stop-containers-with-signals/ Container stop signals]
* [https://linuxhandbook.com/setup-opencl-linux-docker/ OpenCL in Docker]
* [https://www.pugetsystems.com/labs/hpc/ How to Setup NVIDIA Docker and NGC Registry - Part 5]


<code>alias docker_clean_ps='docker rm $(docker ps --filter=status=exited --filter=status=created -q)'</code>
---


== Adding a path to container ==
== Related Pages ==
The docker client does not know about environment variables that are present inside the container, so prepending this at the command-line won't work indeed.


If this is just for an interactive session,  this should probably work<ref>https://github.com/moby/moby/issues/35435</ref>
* [[Docker command workflow]]
docker run -it <container> bash -c 'exec env PATH=/home/app:$PATH bash'
* [[Docker container resource]]
* [[Docker library sharing from host to container]]
or
* [[GPU Support]]
* [[CUDA]]
docker run -it <container> /path-to-script/entrypoint.sh bash


== Reference ==
---
<references />
[[Category:Network]]


[[Category:Reference]]
[[Category:Linux]]
[[Category:Guide]]

Revision as of 14:49, 16 July 2026

Dockerfile tips and tricks

Template:Status

Template:TOC

Overview

Docker 이미지 빌드 및 컨테이너 운영 시 유용한 팁과 트릭 모음. 버전 충돌 해결, GPU 설정, 성능 튜닝, 시그널 처리 등 실용적인 문제 해결 가이드.

Summary

  • 무엇인가? Dockerfile 작성 및 Docker 컨테이너 운영 시 발생하는常见问题 해결 방법
  • 왜 필요한가? Docker 환경 설정 오류, 버전 충돌, GPU 연산 문제 등을 빠르게 해결
  • 언제 사용하는가? Docker 이미지 빌드, 컨테이너 실행, GPU/OpenCL 설정, 리소스 튜닝 시

---

Purpose

Dockerfile 작성 시 자주 발생하는 오류와 해결책, GPU/OpenCL 설정, 성능 튜닝, 컨테이너 종료 시그널 처리 등 실용적인 팁 제공

  • Goal: Docker 환경 구축 및 운영 시 시간 절약 및 문제 해결 속도 향상
  • Scope: Dockerfile 빌드 팁, GPU/OpenCL 설정, 리소스 튜닝, 시그널 처리
  • Non-goals: Docker 기본 사용법 튜토리얼, Docker Compose 상세 설정

---

Key Concepts

Topic Description Related
Version Conflict Docker 패키지 간 의존성 충돌 해결 Docker command workflow
OpenCL in Docker 컨테이너 내 GPU/OpenCL 설정 GPU Support
Resource Tuning Docker 리소스 및 성능 최적화 Docker container resource
Container Stop Signal 컨테이너 종료 시 SIGTERM/SIGKILL 처리 Docker command workflow

---

Architecture

Docker 컨테이너는 호스트 OS의 커널을 공유하는 경량 가상화 환경. GPU/OpenCL 연산을 컨테이너 내에서 사용하려면 호스트의 NVIDIA 드라이버와 OpenCL 라이브러리를 컨테이너에 노출해야 함.

---

Workflow

  1. Dockerfile 기반 이미지 빌드
  2. 의존성 버전 충돌 시 pip install -U 로 해결
  3. GPU/OpenCL 설정 시 nvidia-container-toolkit 및 ICD 파일 구성
  4. 컨테이너 종료 시 SIGTERM → 10초 후 SIGKILL 기본 동작 확인
  5. ENTRYPOINT/STOPSIGNAL/docker-compose.yml stop_signal 설정으로 동작 커스터마이징

---

Configuration

OpenCL 설정 (Docker 내 GPU)

FROM ubuntu:20.04
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get -y upgrade \
  && apt-get install -y \
    ocl-icd-libopencl1 \
    opencl-headers \
    clinfo \
    ;

RUN mkdir -p /etc/OpenCL/vendors && \
    echo "libnvidia-opencl.so.1" > /etc/OpenCL/vendors/nvidia.icd
ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_DRIVER_CAPABILITIES compute,utility

Root 비밀번호 설정

# 이미지 빌드 시
RUN echo 'root:Docker!' | chpasswd

# 또는
RUN echo 'Docker!' | passwd --stdin root

# 실행 중인 컨테이너에서
docker exec -itu 0 {container} passwd

# 3rd party 이미지에서 root로 로그인
docker container exec -u 0 -it mycontainer bash

Docker requests 버전 충돌 해결

pip install -U urllib3 requests==<right_version>

NAT chain 오류 해결

sudo apt update && sudo apt upgrade && sudo systemctl restart docker.service

---

Examples

컨테이너 종료 시그널 처리

기본 동작:

  • `docker stop` → ENTRYPOINT 프로세스에 SIGTERM 전송
  • 10초 후 → SIGKILL 전송

커스터마이징:

  • Dockerfile의 `STOPSIGNAL` 지시자 변경
  • docker-compose.yml의 `stop_signal` 설정
  • docker-compose.yml의 `stop_grace_period` 설정

예시 (php-fpm/nginx): SIGQUIT 사용 (SIGTERM 대신)

---

Best Practices

  • 의존성 충돌 시 `pip install -U`로 최신 버전으로 업그레이드
  • GPU/OpenCL 사용 시 `NVIDIA_VISIBLE_DEVICES`와 `NVIDIA_DRIVER_CAPABILITIES` 환경변수 필수 설정
  • 컨테이너 종료 시 graceful shutdown을 위해 ENTRYPOINT에서 시그널 핸들러 구현
  • Dockerfile에서 `ARG DEBIAN_FRONTEND=noninteractive`로 대시로그 상호작용 방지

---

Performance

Setting Impact Notes
OpenCL ICD 설정 GPU/OpenCL 연산 가능 /etc/OpenCL/vendors/nvidia.icd 필수
NVIDIA_DRIVER_CAPABILITIES 컨테이너 내 GPU 접근 compute,utility 권장
stop_signal 설정 graceful shutdown php-fpm/nginx는 SIGQUIT 권장

---

Limitations

  • Docker 7.0.0+는 requests>=2.26.0 필수
  • NAT chain 오류 시 재부팅 대신 docker.service 재시작 필요
  • 3rd party 이미지는 USER 설정이 override 될 수 있음 (-u 0 사용)

---

References

---

Related Pages

---