Configure 1G Hugepages in OpenShift 4.2

ocp-4.2-hugepages-1g.md

Configure 1G Hugepages in OpenShift 4.2

Update Node Kernel Parameters

Supporting 1G hugepages requires the Kernel to support 1G Hugepages. This is done by applying a boot Kernel parameter.

• Create a MachineConfig (MC) to apply to the Node type that will be using the 1G hugepages. The following example set 1G hugepages for all worker nodes 50-kargs-1g-hugepages.yaml:

  apiVersion: machineconfiguration.openshift.io/v1
  kind: MachineConfig
  metadata:
      labels:
          machineconfiguration.openshift.io/role: worker
      name: 50-kargs-1g-hugepages
  spec:
      kernelArguments:
          - default_hugepagesz=1G
          - hugepagesz=1G
      #    - hugepages=32 
  

• Apply the MC to the cluster

  oc create -f 50-kargs-1g-hugepages.yaml
  

  This exaple will update the Kernel parameters for all the Worker Nodes

Create Tuned profile

Create hugepages-1g_tuning.yaml with the desired configuration. This is a way to pass Tuned parameters that are handled by the Node Tuning Operator.

Huge Pages are allocated explicitly by the vm.nr_hugepages sysctl parameter.

The following example enables 1G hugepages for any Node with a label of hugepages-1g:

apiVersion: tuned.openshift.io/v1
kind: Tuned
metadata:
    name: hugepages-1g 
    namespace: openshift-cluster-node-tuning-operator
spec:
    profile: 
    - data: |
        [main]
        summary=Configuration for hugepages
        include=openshift-node

        [vm]
        transparent_hugepages=never

        [sysctl]
        # The Default Hugepage size 2MB (1024*2)
        # Alternate   Hugepage size 1G  (1024*1024) -- need to be set with MachineConfig kernelArguments
        # vm.nr_hugepages is the Number of Hugepages to allocate
        # Examples:
        #   If need 1G pool from 2MB hugepagesize then (1024*1024)/2048 = 512
        #   If need 2G pool from 1GB hugepagesize then (1024*1024*2)/(1024*1024)=2
        vm.nr_hugepages=2
        name: node-hugepages-1g
    recommend:
    - match: 
        - label: hugepages-1g
        priority: 30
        profile: node-hugepages-1g

Create the new Tuned profile

oc create -f hugepages-1g_tuning.yaml

Apply the hugepages-1g label to the Nodes to be used with the 1G Hugepage

oc label node <node_using_hugepages> hugepages-1g=true

Validations

After the MC applies the config and reboots the node, it should report the new parameters

[root@worker-0 ~]# cat /proc/cmdline
BOOT_IMAGE=/ostree/rhcos-ee73255863c1e88baaa06eac3fff2d753f20da9a49139224bb88dbfe2a7ba712/vmlinuz-4.18.0-80.11.2.el8_0.x86_64 console=tty0 console=ttyS0,115200n8 rootflags=defaults,prjquota rw root=UUID=6101ba79-239a-439e-b891-6315c6c4b7bd ostree=/ostree/boot.1/rhcos/ee73255863c1e88baaa06eac3fff2d753f20da9a49139224bb88dbfe2a7ba712/0 coreos.oem.id=metal ignition.platform.id=metal default_hugepagesz=1G hugepagesz=1G
[root@worker-0 ~]#

The /proc/meminfo should report the available pages and page size

# grep -i huge /proc/meminfo
AnonHugePages:    ###### ##
ShmemHugePages:        0 kB
HugePages_Total:       2
HugePages_Free:        2
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       #### ##
Hugetlb:            #### ##

The Node description should report the new size

$ oc describe node worker-0.ocp4poc.example.com | grep -i huge
                    hugepages-1g=true
 hugepages-###:  ###
 hugepages-###:  ###