What is transparent hugepage (THP) and how to check THP usage per process in Linux (Explained)

by
Hugepages is a feature that allows the Linux kernel to utilise the multiple page size capabilities of modern hardware architectures.

Memory is managed in blocks known as pages. A page is 4096 bytes. 1MB of memory is equal to 256 pages; 1GB of memory is equal to 256,000 pages, etc.
To get the page size use below commands

# getconf PAGE_SIZE
4096

# getconf PAGESIZE
4096

CPUs have a built-in memory management unit that contains a list of these pages, with each page referenced through a page table entry

There are two ways to enable the system to manage large amounts of memory:

  • Increase the number of page table entries in the hardware memory management unit
  • Increase the page size

The first method is expensive, since the hardware memory management unit in a modern processor only supports hundreds or thousands of page table entries.

In Linux we use hugepages for this purpose where huge pages are blocks of memory that come in 2MB and 1GB sizes. The page tables used by the 2MB pages are suitable for managing multiple gigabytes of memory, whereas the page tables of 1GB pages are best for scaling to terabytes of memory

    A side effect of creating Huge Pages is that the physical memory that is mapped to a Huge Page is no longer subject to normal memory allocations or managed by the kernel virtual memory manager, so Huge Pages are essentially 'protected' and are available only to applications that request them.

    Huge Pages are 'pinned' to physical RAM and cannot be swapped/paged out.

    The kernel will always attempt to satisfy a memory allocation using hugepages. If no hugepages are available (due to non availability of physically continuous memory for example) the kernel will fall back to the regular 4KB pages.

    But to use hugepages effectively, the kernel must find physically continuous areas of memory big enough to satisfy the request, and also properly aligned.

    A typical purpose for allocating Huge Pages is for an application that has characteristic high memory use, and you wish to ensure that the pages it uses are never swapped out when the system is under memory pressure.

    Systems with large amount of memory can be configured to utilize the memory more efficiently by setting aside a portion dedicated for hugepages. The actual size of the page is dependent on the system architecture.

    A typical x86 system will have a Huge Page Size of 2048 kBytes.

    The huge page size may be found by looking at /proc/meminfo

    # cat /proc/meminfo |grep Hugepagesize
    Hugepagesize: 2048 kB

    How to tell if HugePages is enabled or disabled

    In the above steps we disabled the THP to make sure kernel doesnot allocates or reserves any hugepage.

    If the value of HugePages_Total is "0" it means HugePages is disabled on the system.

    # grep -i HugePages_Total /proc/meminfo
    HugePages_Total:       0

    If the value of HugePages_Total is greater than "0", it means HugePages is enabled on the system

    # grep -i HugePages_Total /proc/meminfo
    HugePages_Total:   23090

    Similarly, if the value in /proc/sys/vm/nr_hugepages file or vm.nr_hugepages, sysctl parameter is "0" it means HugePages is disabled on the system

    # cat /proc/sys/vm/nr_hugepages
    0

    # sysctl vm.nr_hugepages
    vm.nr_hugepages = 0

    If the value in /proc/sys/vm/nr_hugepages file or vm.nr_hugepages sysctl parameter is greater than "0", it means HugePages is enabled on the system

    # cat /proc/sys/vm/nr_hugepages
    1024

    # sysctl vm.nr_hugepages
    vm.nr_hugepages = 1024

    How to check THP usage per process

    The number of anonymous transparent huge pages currently used by the system is available by reading the AnonHugePages field in /proc/meminfo

    # grep -i AnonHugePages /proc/meminfo
    AnonHugePages:   1216512 kB

    To identify what applications are using anonymous transparent huge pages, it is necessary to read /proc/PID/smaps and count the AnonHugePages fields for each mapping.

    # grep -e AnonHugePages  /proc/$(pgrep test.sh)/smaps | awk  '{ if($2>0) print $0} '
    AnonHugePages:    120832 kB

    NOTE: There are a number of counters in /proc/vmstat that may be used to monitor how successfully the system is providing huge pages for use.

    I hope the article was useful.