Ulimit: Difference between revisions

From HPCWIKI
Jump to navigation Jump to search
(Created page with "'''<code>ulimit</code>''' is a built-in Linux shell command that allows '''viewing or limiting system resource amounts''' that individual users consume. Limiting resource usage is valuable in environments with multiple users and system performance issues. ''limits.conf'' file is a configuration file that defines the system resource allocation settings '''<code>ulimit</code>''' uses. The full path to the configuration file is ''/etc/security/limits.conf''. Changing th...")
 
No edit summary
 
(4 intermediate revisions by the same user not shown)
Line 2: Line 2:




''limits.conf'' file is a configuration file that defines the system resource allocation settings '''<code>ulimit</code>''' uses. The full path to the configuration file is ''/etc/security/limits.conf''.  Changing the values in the file persist after system reboot.




'''To specify the ''type'' with ''ulimit'', we use ''-H'' (hard) and ''-S'' (soft), where''' ''-H'' sets hard limits, while ''-S'' sets soft limits. By default, without either flag, ''-S'' is presumed
''/etc/security/limits.conf'' file is a configuration file that defines the system resource allocation settings '''<code>ulimit</code>''' uses.  Changing the values in the file persist after system reboot.
 
 
'''To specify the ''type'' with ''ulimit'', we use ''-H'' (hard) and ''-S'' (soft), where''' ''-H'' sets hard limits, while ''-S'' sets soft limits. By default, without either flag, ''-S'' is presumed<ref>https://linuxhint.com/linux_ulimit_command/</ref>


== Soft limit ==
== Soft limit ==
Line 21: Line 23:


<code>ulimit [flags][limit]</code>
<code>ulimit [flags][limit]</code>
{| class="wikitable"
{| class="wikitable sortable"
!Flag
!Flag
!Effect to
!Description
!Description
|-
|-
|'''<code>-H</code>'''
|'''<code>-H</code>'''
|
|The '''hard limit''' for the given resource. Only root users can raise the hard limit, and any process can lower it.
|The '''hard limit''' for the given resource. Only root users can raise the hard limit, and any process can lower it.
|-
|-
|'''<code>-S</code>'''
|'''<code>-S</code>'''
|
|The '''soft limit''' for the given resource. Any process can change the soft limit.
|The '''soft limit''' for the given resource. Any process can change the soft limit.
|-
|-
|'''<code>-a</code>'''
|'''<code>-a</code>'''
|
|Lists all current resource limits.
|Lists all current resource limits.
|-
|-
|'''<code>-b</code>'''
|'''<code>-b</code>'''
|socket
|The maximum socket buffer size.
|The maximum socket buffer size.
|-
|-
|'''<code>-c</code>'''
|'''<code>-c</code>'''
|core dump
|The core dump size, expressed in the number of 512-byte blocks.
|The core dump size, expressed in the number of 512-byte blocks.
|-
|-
|'''<code>-d</code>'''
|'''<code>-d</code>'''
|file
|The data area size, in kilobytes.
|The data area size, in kilobytes.
|-
|-
|'''<code>-e</code>'''
|'''<code>-e</code>'''
|process
|The highest process scheduling priority (nice).
|The highest process scheduling priority (nice).
|-
|-
|'''<code>-f</code>'''
|'''<code>-f</code>'''
|file
|The file size limit in blocks when using the '''<code>[limit]</code>''' parameter. Not specifying a '''<code>[limit]</code>''' instructs the command to report the file size limit.
|The file size limit in blocks when using the '''<code>[limit]</code>''' parameter. Not specifying a '''<code>[limit]</code>''' instructs the command to report the file size limit.
|-
|-
|'''<code>-i</code>'''
|'''<code>-i</code>'''
|signal
|The pending signal number limit.
|The pending signal number limit.
|-
|-
|'''<code>-k</code>'''
|'''<code>-k</code>'''
|socket
|The queue allocation number limit.
|The queue allocation number limit.
|-
|-
|'''<code>-l</code>'''
|'''<code>-l</code>'''
|memory
|The maximum size allowed for locking in memory.
|The maximum size allowed for locking in memory.
|-
|-
|'''<code>-m</code>'''
|'''<code>-m</code>'''
|memory
|The physical memory size, in kilobytes.
|The physical memory size, in kilobytes.
|-
|-
|'''<code>-n</code>'''
|'''<code>-n</code>'''
|process
|The maximum number of file descriptors that a process can have.
|The maximum number of file descriptors that a process can have.
|-
|-
|'''<code>-p</code>'''
|'''<code>-p</code>'''
|pipe
|The pipe buffer size.
|The pipe buffer size.
|-
|-
|'''<code>-P</code>'''
|'''<code>-P</code>'''
|The maximum number of pseudoterminals.
|terminals
|The maximum number of pseudo terminals.
|-
|-
|'''<code>-q</code>'''
|'''<code>-q</code>'''
|message queues
|The maximum number of bytes in POSIX message queues.
|The maximum number of bytes in POSIX message queues.
|-
|-
|'''<code>-r</code>'''
|'''<code>-r</code>'''
|thread
|The maximum number of threads a process can have.
|The maximum number of threads a process can have.
|-
|-
|'''<code>-R</code>'''
|'''<code>-R</code>'''
|process
|The maximum process running time, expressed in microseconds.
|The maximum process running time, expressed in microseconds.
|-
|-
|'''<code>-s</code>'''
|'''<code>-s</code>'''
|memory
|The stack size, in kilobytes.
|The stack size, in kilobytes.
|-
|-
|'''<code>-t</code>'''
|'''<code>-t</code>'''
|process
|Specifies a process' maximum running time, in seconds.
|Specifies a process' maximum running time, in seconds.
|-
|-
|'''<code>-T</code>'''
|'''<code>-T</code>'''
|thread
|The thread number limit.
|The thread number limit.
|-
|-
|'''<code>-u</code>'''
|'''<code>-u</code>'''
|process
|Specifies how many processes a user can create.
|Specifies how many processes a user can create.
|-
|-
|'''<code>-v</code>'''
|'''<code>-v</code>'''
|memory
|The maximum virtual memory available for processes.
|The maximum virtual memory available for processes.
|-
|-
|'''<code>-x</code>'''
|'''<code>-x</code>'''
|file
|The maximum number of file locks.
|The maximum number of file locks.
|}
|}


== Example ==
== Example ==
<syntaxhighlight lang="bash">
Resource limits set with ulimit are not system-wide and only apply to processes started in the same shell session and their descendants.<syntaxhighlight lang="bash">
#limit the process number to 10
#limit the process number to 10
ulimit -u 10
ulimit -u 10
Line 112: Line 139:
ulimit -n 5
ulimit -n 5
</syntaxhighlight>
</syntaxhighlight>
== limit keywords ==
== limit keywords ==


Line 134: Line 160:
* rtprio – max realtime priority
* rtprio – max realtime priority
* chroot – change root to directory (Debian-specific)
* chroot – change root to directory (Debian-specific)
== Tuning ulimit ==
=== File descriptor limit ===
On Ubuntu, the default file descriptor limit is 1024 which can be checked by $ulimit -n.
To increase this value if you are running an application server which receive thousands or millions of request each second <syntaxhighlight lang="bash">
Add those lines to file /etc/security/limits.conf
*    - nofile 1048576
then relogin and check $ulimit -n
</syntaxhighlight>system wide file descriptor limit can be done through fs.file-max<ref>https://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/</ref> <syntaxhighlight lang="bash">
Use the following command command to display maximum number of open file descriptors:
cat /proc/sys/fs/file-max
The number of concurrently open file descriptors throughout the system can be changed
via /etc/sysctl.conf file under Linux operating systems.
for temperary change,
# sysctl -w fs.file-max=100000
for permenant change, add following line in /etc/sysctl.conf
fs.file-max = 100000
to changes take effect just type and check /proc/sys/fs/file-max
# sysctl -p
</syntaxhighlight>
=== In order to allow allocation of large amount of pinned memory ===
<syntaxhighlight lang="bash">
#ulimit -v unlimited
#ulimit -m unlimited
#ulimit -l unlimited
</syntaxhighlight>
RHEL/CentOS/Fedora/Scientific Linux users need additional configuration to edit /etc/pam.d/common-session file and add/modify the following line
session required pam_limits.so


== Reference ==
== Reference ==
<references />
<references />
[[Category: Terminology]]
[[Category: Tuning]]

Latest revision as of 12:59, 3 January 2024

ulimit is a built-in Linux shell command that allows viewing or limiting system resource amounts that individual users consume. Limiting resource usage is valuable in environments with multiple users and system performance issues.



/etc/security/limits.conf file is a configuration file that defines the system resource allocation settings ulimit uses. Changing the values in the file persist after system reboot.


To specify the type with ulimit, we use -H (hard) and -S (soft), where -H sets hard limits, while -S sets soft limits. By default, without either flag, -S is presumed[1]

Soft limit

The soft limit is manageable by any user, and its maximum value cannot exceed the hard limit. The hard limit acts as a ceiling for the soft limit.

ulimit -Sa

Hard Limit

The hard resource limit defines physical resource limit for a user. At the same time, the hard limit is the maximum value for soft limit. Only root users are allowed to change the hard limit.

ulimit -Ha

Flag

The ulimit command takes the following general syntax

ulimit [flags][limit]

Flag Effect to Description
-H The hard limit for the given resource. Only root users can raise the hard limit, and any process can lower it.
-S The soft limit for the given resource. Any process can change the soft limit.
-a Lists all current resource limits.
-b socket The maximum socket buffer size.
-c core dump The core dump size, expressed in the number of 512-byte blocks.
-d file The data area size, in kilobytes.
-e process The highest process scheduling priority (nice).
-f file The file size limit in blocks when using the [limit] parameter. Not specifying a [limit] instructs the command to report the file size limit.
-i signal The pending signal number limit.
-k socket The queue allocation number limit.
-l memory The maximum size allowed for locking in memory.
-m memory The physical memory size, in kilobytes.
-n process The maximum number of file descriptors that a process can have.
-p pipe The pipe buffer size.
-P terminals The maximum number of pseudo terminals.
-q message queues The maximum number of bytes in POSIX message queues.
-r thread The maximum number of threads a process can have.
-R process The maximum process running time, expressed in microseconds.
-s memory The stack size, in kilobytes.
-t process Specifies a process' maximum running time, in seconds.
-T thread The thread number limit.
-u process Specifies how many processes a user can create.
-v memory The maximum virtual memory available for processes.
-x file The maximum number of file locks.

Example

Resource limits set with ulimit are not system-wide and only apply to processes started in the same shell session and their descendants.

#limit the process number to 10
ulimit -u 10

#limits the file size to 50KB
ulimit -f 50

#limits the virtual memory available for a process to 1000KB
ulimit -v 1000

#limits the number of open files to five
ulimit -n 5

limit keywords

  • core – limits the core file size (KB)
  • data – max data size (KB)
  • fsize – maximum filesize (KB)
  • memlock – max locked-in-memory address space (KB)
  • nofile – max number of open files
  • rss – max resident set size (KB)
  • stack – max stack size (KB)
  • cpu – max CPU time (MIN)
  • nproc – max number of processes
  • as – address space limit (KB)
  • maxlogins – max number of logins for this user
  • maxsyslogins – max number of logins on the system
  • priority – the priority to run user process with
  • locks – max number of file locks the user can hold
  • sigpending – max number of pending signals
  • msgqueue – max memory used by POSIX message queues (bytes)
  • nice – max nice priority allowed to raise to values: [-20, 19]
  • rtprio – max realtime priority
  • chroot – change root to directory (Debian-specific)

Tuning ulimit

File descriptor limit

On Ubuntu, the default file descriptor limit is 1024 which can be checked by $ulimit -n.

To increase this value if you are running an application server which receive thousands or millions of request each second

Add those lines to file /etc/security/limits.conf

*    - nofile 1048576

then relogin and check $ulimit -n

system wide file descriptor limit can be done through fs.file-max[2]

Use the following command command to display maximum number of open file descriptors:

cat /proc/sys/fs/file-max

The number of concurrently open file descriptors throughout the system can be changed 
via /etc/sysctl.conf file under Linux operating systems.

for temperary change, 
# sysctl -w fs.file-max=100000

for permenant change, add following line in /etc/sysctl.conf
fs.file-max = 100000

to changes take effect just type and check /proc/sys/fs/file-max
# sysctl -p

In order to allow allocation of large amount of pinned memory

#ulimit -v unlimited
#ulimit -m unlimited
#ulimit -l unlimited


RHEL/CentOS/Fedora/Scientific Linux users need additional configuration to edit /etc/pam.d/common-session file and add/modify the following line

session required pam_limits.so

Reference