Hugepage: Difference between revisions

From HPCWIKI
Jump to navigation Jump to search
No edit summary
 
(2 intermediate revisions by the same user not shown)
Line 2: Line 2:




In Database, Huge pages is especially useful on systems like database servers. Processes like MySQL and PostgreSQL can make use of huge pages if they are enabled, and will put less strain on your RAM cache
A huge page is a memory page that is larger than 4K, On x86_64 architectures, there are two common huge page sizes: 2Mi and 1Gi. Sizes vary on other architectures.
 
 
 
In Database, Huge pages is especially useful on systems like database servers. Processes like MySQL and PostgreSQL can make use of huge pages if they are enabled, and will put less strain on your RAM cache<ref>https://linuxconfig.org/how-to-enable-hugepages-on-linux</ref>
 




Line 8: Line 13:




most standard on modern systems would be 2 MB<syntaxhighlight lang="bash">
most standard on modern systems would be 2 MB while  for 64-bit applications, it is recommended to use 1 GB hugepages if the platform supports them.<syntaxhighlight lang="bash">
$ grep -i huge /proc/meminfo
$ grep -i huge /proc/meminfo
AnonHugePages:        0 kB
AnonHugePages:        0 kB
Line 20: Line 25:
Hugetlb:              0 kB
Hugetlb:              0 kB
</syntaxhighlight>we can change  Hugepagesize <syntaxhighlight lang="bash">
</syntaxhighlight>we can change  Hugepagesize <syntaxhighlight lang="bash">
$ sudo sysctl -w vm.nr_hugepages=102400 # for example 100MB
$ sudo sysctl -w vm.nr_hugepages=1024000 # for example 1G
[sudo] password for hpcmate:  
[sudo] password for hpcmate:  
vm.nr_hugepages = 102400
vm.nr_hugepages = 102400
hpcmate@VPNGW:~/setup/vpn (master)$ grep -i huge /proc/meminfo          
$ grep -i huge /proc/meminfo      
AnonHugePages:        0 kB
AnonHugePages:        0 kB
ShmemHugePages:        0 kB
ShmemHugePages:        0 kB
FileHugePages:        0 kB
FileHugePages:        0 kB
HugePages_Total:  102400
HugePages_Total:  127542
HugePages_Free:    102400
HugePages_Free:    127542
HugePages_Rsvd:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
HugePages_Surp:        0
Hugepagesize:      2048 kB
Hugepagesize:      2048 kB
Hugetlb:        209715200 kB
Hugetlb:        261206016 kB
</syntaxhighlight>To make it persistent across futher reboot, add following line in /etc/sysctl.conf and reboot<syntaxhighlight lang="bash">
</syntaxhighlight>To make it persistent across futher reboot, add following line in /etc/sysctl.conf and reboot<syntaxhighlight lang="bash">
vm.nr_hugepages = 102400
vm.nr_hugepages = 1024000
</syntaxhighlight><syntaxhighlight lang="bash">
</syntaxhighlight><syntaxhighlight lang="bash">
#Check kernel support HugeTLB
#Check kernel support HugeTLB
Line 51: Line 56:
CONFIG_HUGETLBFS=y
CONFIG_HUGETLBFS=y
CONFIG_HUGETLB_PAGE=y
CONFIG_HUGETLB_PAGE=y
#HugeTBL can be setup through /etc/sysctl.conf by adding
vm.nr_hugepages=256
# also need to edit the /etc/fstab file to mount the hugepages at startup
$ sudo vi /etc/fstab by adding
huge /mnt/huge hugetlbfs defaults 0 0
$ sudo mkdir /mnt/huge
$ sudo chmod 777 /mnt/huge
#also reboot your machine as the huge pages must be setup just after boot to make
#sure there is enough contiguous memory for the 2MB pages
</syntaxhighlight>
<syntaxhighlight lang="bash">
#For a single-node system, the command to use is as follows (assuming that 1024 of 2MB pages are required)
echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
# On a NUMA node system
echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
</syntaxhighlight>
</syntaxhighlight>


Some kernel versions may not allow reserving 1 GB hugepages at run time, so reserving them at boot time may be the only option by pass the hugepages option to the kernel.<syntaxhighlight lang="bash">
Some kernel versions may not allow reserving 1 GB hugepages at run time, so reserving them at boot time may be the only option by pass the hugepages option to the kernel.<syntaxhighlight lang="bash">
Line 82: Line 62:
</syntaxhighlight>
</syntaxhighlight>


== In gaming mode ==
The following tweaks can improve gaming performance, but might not be optimal for other workloads/normal desktop usage.<ref>https://linux-gaming.kwindu.eu/index.php?title=Improving_performance</ref>


For 64-bit applications, it is recommended to use 1 GB hugepages if the platform supports them.
* Use Transparent Hugepages: <code>echo always | sudo tee /sys/kernel/mm/transparent_hugepage/enabled</code>
** With shared memory: <code>echo advise | sudo tee /sys/kernel/mm/transparent_hugepage/shmem_enabled</code>
* Use Compaction Proactiveness: <code>echo 0 | sudo tee /proc/sys/vm/compaction_proactiveness</code>
* Disable Hugepage Defragmentation: <code>echo 0 | sudo tee /sys/kernel/mm/transparent_hugepage/khugepaged/defrag</code>
* Lower Page Lock Unfairness: <code>echo 1 | sudo tee /proc/sys/vm/page_lock_unfairness</code>


== Reference ==
== Reference ==
<references />
<references />
[[Category: Terminology]]
[[Category: Tuning]]

Latest revision as of 17:41, 16 June 2023

Computer memory is allocated to processes as pages. Usually these pages are rather small (4K), meaning that a process consuming a lot of memory will also be consuming a lot of pages. Searching through a multitude of pages can result in system slow downs, which is why some servers can benefit from enabling huge pages.


A huge page is a memory page that is larger than 4K, On x86_64 architectures, there are two common huge page sizes: 2Mi and 1Gi. Sizes vary on other architectures.


In Database, Huge pages is especially useful on systems like database servers. Processes like MySQL and PostgreSQL can make use of huge pages if they are enabled, and will put less strain on your RAM cache[1]


In Network stack, Hugepage support is required for the large memory pool allocation used for packet buffers (the HUGETLBFS option must be enabled in the running kernel as indicated the previous section). By using hugepage allocations, performance is increased since fewer pages are needed, and therefore less Translation Lookaside Buffers (TLBs, high speed translation caches), which reduce the time it takes to translate a virtual page address to a physical page address. Without hugepages, high TLB miss rates would occur with the standard 4k page size, slowing performance.[2]


most standard on modern systems would be 2 MB while for 64-bit applications, it is recommended to use 1 GB hugepages if the platform supports them.

$ grep -i huge /proc/meminfo
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
FileHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB

we can change Hugepagesize 

$ sudo sysctl -w vm.nr_hugepages=1024000  # for example 1G
[sudo] password for hpcmate: 
vm.nr_hugepages = 102400
$ grep -i huge /proc/meminfo       
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
FileHugePages:         0 kB
HugePages_Total:   127542
HugePages_Free:    127542
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:        261206016 kB

To make it persistent across futher reboot, add following line in /etc/sysctl.conf and reboot

vm.nr_hugepages = 1024000

#Check kernel support HugeTLB
$grep -i huge /boot/config-5.4.0-150-generic
CONFIG_CGROUP_HUGETLB=y
CONFIG_ARCH_WANT_GENERAL_HUGETLB=y
CONFIG_ARCH_ENABLE_HUGEPAGE_MIGRATION=y
CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE=y
CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD=y
CONFIG_HAVE_ARCH_HUGE_VMAP=y
CONFIG_ARCH_WANT_HUGE_PMD_SHARE=y
CONFIG_TRANSPARENT_HUGEPAGE=y
# CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS is not set
CONFIG_TRANSPARENT_HUGEPAGE_MADVISE=y
CONFIG_TRANSPARENT_HUGE_PAGECACHE=y
CONFIG_HUGETLBFS=y
CONFIG_HUGETLB_PAGE=y

Some kernel versions may not allow reserving 1 GB hugepages at run time, so reserving them at boot time may be the only option by pass the hugepages option to the kernel.

default_hugepagesz=1G hugepagesz=1G hugepages=4

In gaming mode

The following tweaks can improve gaming performance, but might not be optimal for other workloads/normal desktop usage.[3]

  • Use Transparent Hugepages: echo always | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
    • With shared memory: echo advise | sudo tee /sys/kernel/mm/transparent_hugepage/shmem_enabled
  • Use Compaction Proactiveness: echo 0 | sudo tee /proc/sys/vm/compaction_proactiveness
  • Disable Hugepage Defragmentation: echo 0 | sudo tee /sys/kernel/mm/transparent_hugepage/khugepaged/defrag
  • Lower Page Lock Unfairness: echo 1 | sudo tee /proc/sys/vm/page_lock_unfairness

Reference

  1. ↑ https://linuxconfig.org/how-to-enable-hugepages-on-linux
  2. ↑ https://doc.dpdk.org/guides/linux_gsg/sys_reqs.html#use-of-hugepages-in-the-linux-environment
  3. ↑ https://linux-gaming.kwindu.eu/index.php?title=Improving_performance
Retrieved from "http://wiki.hpcmate.com/index.php?title=Hugepage&oldid=1091"