Linux 关闭 Transparent HugePages

1. Transparent HugePages 的概念

 

Transparent Huge Pages的一些官方介绍资料:
	Transparent Huge Pages (THP) are enabled by default in RHEL 6 for all applications. The kernel attempts 
to allocate hugepages whenever possible and any Linux process will receive 2MB pages if the mmap region 
is 2MB naturally aligned. The main kernel address space itself is mapped with hugepages, reducing TLB 
pressure from kernel code. For general information on Hugepages, see: What are Huge Pages and what are 
the advantages of using them? 
	The kernel will always attempt to satisfy a memory allocation using hugepages. If no hugepages are available
(due to non availability of physically continuous memory for example) the kernel will fall back to the regular 
4KB pages. THP are also swappable (unlike hugetlbfs). This is achieved by breaking the huge page to smaller 
4KB pages, which are then swapped out normally. 
	But to use hugepages effectively, the kernel must find physically continuous areas of memory big enough to 
satisfy the request, and also properly aligned. For this, a khugepaged kernel thread has been added. This 
thread will occasionally attempt to substitute smaller pages being used currently with a hugepage allocation,
thus maximizing THP usage. 
	In userland, no modifications to the applications are necessary (hence transparent). But there are ways to 
optimize its use. For applications that want to use hugepages, use of posix_memalign() can also help ensure 
that large allocations are aligned to huge page (2MB) boundaries. 
	Also, THP is only enabled for anonymous memory regions. There are plans to add support for tmpfs and 
page cache. THP tunables are found in the /sys tree under /sys/kernel/mm/redhat_transparent_hugepage. 
	从RedHat 6, OEL 6, SLES 11 and UEK2 kernels 开始，系统缺省会启用 Transparent HugePages ：用来提高内存管理
	的性能透明大页（Transparent HugePages ）和之前版本中的大页功能上类似。主要的区别是：Transparent HugePages 
	可以实时配置，不需要重启才能生效配置

2. Oracle 为什么不建议开启

Transparent HugePages memory is enabled by default with Red Hat Enterprise Linux 6, SUSE Linux Enterprise Server 11, 
and Oracle Linux 6 with earlier releases of Oracle Linux Unbreakable Enterprise Kernel 2 (UEK2) kernels. Transparent 
HugePages memory is disabled in later releases of Oracle Linux UEK2 kernels.

Transparent HugePages can cause memory allocation delays during runtime. To avoid performance issues, Oracle recommends 
that you disable Transparent HugePages on all Oracle Database servers. Oracle recommends that you instead use standard 
HugePages for enhanced performance.

Oracle recommends that you disable Transparent HugePages, because they may causes delays in accessing memory that can 
result in node restarts in Oracle RAC environments, or performance issues or delays for Oracle Database single instances. 
Oracle continues to recommend using standard HugePages for Linux.

Transparent HugePages memory differs from standard HugePages memory because the kernel khugepaged thread allocates memory 
dynamically during runtime. Standard HugePages memory is pre-allocated at startup, and does not change during runtime.

3. 关闭THP

Oracle建议在开始安装之前禁用透明的HugePages。

透明的HugePages内存与标准的HugePages内存不同，因为内核khugep老化线程在运行时动态分配内存。标准的HugePages内存是在启动时

预先分配的，在运行时不会改变。

查看是否启用透明大页

3.1：命令cat /sys/kernel/mm/redhat_transparent_hugepage/enabled 该命令适用于Red Hat Enterprise Linux,CentoS系统 [root@storage2 mm]# more /etc/issue
CentOS release 6.7 (Final)
Kernel /r on an /m
[root@storage2 mm]# cat /sys/kernel/mm/redhat_transparent_hugepage/enabled
[always] madvise never
3.2：命令cat /sys/kernel/mm/transparent_hugepage/enabled 该命令适用于其它Linux系统 [root@storage2 ~]# cat /sys/kernel/mm/transparent_hugepage/enabled 
[always] madvise never
使用命令查看时，如果输出结果为[always]表示透明大页启用了。[never]表示透明大页禁用

3.3 禁用透明大页功能

CentOS6.7中禁用Transparent Huge Pages
[root@dg1 ~]# more /etc/issue
CentOS release 6.7 (Final)
Kernel /r on an /m
方法1：设置/etc/grub.conf文件 kernel 选项后 增加 transparent_hugepage=never 然后重启生效
[root@dg1 ~]# cat /etc/grub.conf
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,1)
#          kernel /vmlinuz-version ro root=/dev/sda4
#          initrd /initrd-[generic-]version.img
#boot=/dev/sda1
device (hd0) HD(1,800,3e8000,05914f6f-3172-42ca-88e9-f0eccea595fe)
default=0
timeout=5
splashimage=(hd0,1)/grub/splash.xpm.gz
hiddenmenu
title CentOS 6 (2.6.32-573.el6.x86_64)
	root (hd0,1)
	kernel /vmlinuz-2.6.32-573.el6.x86_64 ro root=UUID=a6d1da03-604a-4b1a-8f54-594f09e27b22 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet transparent_hugepage=never
	initrd /initramfs-2.6.32-573.el6.x86_64.img
	numa=off
重启之后生效
[root@dg1 ~]# cat /sys/kernel/mm/redhat_transparent_hugepage/enabled
always madvise [never]
方法2：设置/etc/rc.local或者 /etc/rc.d/rc.local文件
增加以下内容：
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
   echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi

if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
   echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi

保存退出，然后赋予/etc/rc.d/rc.local 文件执行权限：
[root@storage2 /]# chmod +x /etc/rc.d/rc.local
重启系统，以后再检查THP就是被禁用了
[root@storage2 /]# cat /sys/kernel/mm/transparent_hugepage/defrag
always madvise [never]
[root@storage2 /]# cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]

redhat 7.3 禁用 Transparent Huge Pages

t HugePages内存在Red Hat企业Linux 6,SUSE 11和Oracle Linux 6的Oracle Linux Unbreakable Enterprise Kernel2(UEK2)早期版本中默认是启用的。Transparent HugePages内存在Oracle Linux Unbreakable Enterprise Kernel2(UEK2)内核中默认情况下是禁用的。
"tuned.service"在Oracle Linux7中默认是将transparent_hugepage设置为always。即使在grub内核命令行中它是禁用的，在boot时tuned service将被设置为'always'。在UEK3中默认情况下是禁用的。

[root@dg1 etc]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.3 (Maipo)
[root@dg1 etc]#  uname -r 
3.10.0-514.el7.x86_64
[root@dg1 etc]# cd /etc/default/
[root@dg1 default]#  cp grub grub.bak --备份grub文件 
修改/etc/default/grub文件,如下是修改之后的效果：
[root@dg1 default]# cat  /etc/default/grub
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=rhel_dg2/root rd.lvm.lv=rhel_dg2/swap rhgb quiet transparent_hugepage=never"
GRUB_DISABLE_RECOVERY="true"
[root@dg1 default]# grub2-mkconfig -o /boot/grub2/grub.cfg ---执行该命令
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-514.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-514.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-a0187aef89d64fa7bbae3e88263260b8
Found initrd image: /boot/initramfs-0-rescue-a0187aef89d64fa7bbae3e88263260b8.img
done
[root@dg1 default]# init 6  ----重启生效 
os重启之后，查看效果
[root@dg1 ~]# cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-3.10.0-514.el7.x86_64 root=/dev/mapper/rhel_dg2-root ro crashkernel=auto rd.lvm.lv=rhel_dg2/root rd.lvm.lv=rhel_dg2/swap rhgb quiet transparent_hugepage=never
[root@dg1 ~]# cat /sys/kernel/mm/transparent_hugepage/enabled 
always madvise [never]
[root@dg1 ~]# cd /sys/kernel/mm/
[root@dg1 mm]# ls  -lrt
total 0
drwxr-xr-x. 3 root root 0 Jul 28 18:17 hugepages
drwxr-xr-x. 2 root root 0 Jul 28 18:17 ksm
drwxr-xr-x. 3 root root 0 Jul 28 18:18 transparent_hugepage
[root@dg1 mm]# cd transparent_hugepage/
[root@dg1 transparent_hugepage]# ls  -lrt
total 0
-rw-r--r--. 1 root root 4096 Jul 28 18:18 enabled
-rw-r--r--. 1 root root 4096 Jul 28 18:19 use_zero_page
drwxr-xr-x. 2 root root    0 Jul 28 18:19 khugepaged
-rw-r--r--. 1 root root 4096 Jul 28 18:19 defrag
[root@dg1 transparent_hugepage]# grep AnonHugePages /proc/meminfo
AnonHugePages:         0 kB   返回值若是零，代表成功禁用THP

Oracle Linux 7 和 redhat 7.3 可以使用以下方法 禁用 Transparent HugePages

对正使用的调整概要文件禁用Transparent HugePage
先识别正使用的调整概要文件
[root@dg1 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.3 (Maipo)
[root@dg1 ~]#  uname -r
3.10.0-514.el7.x86_64
[root@dg1 ~]# cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never
[root@dg1 ~]# tuned-adm active
Current active profile: virtual-guest --这说明当前正使用的概要文件是Virtual-guest
Virtual-guest profile是使用文件/usr/lib/tuned/virtual-guest/tuned.conf，并且它包含了来自文件
/usr/lib/tuned/throughput-performance/tuned.conf中的"throughput-performance"。首先对文件/usr/lib/tuned/virtual-guest/tuned.conf进行备份
[root@dg1 ~]#  cp /usr/lib/tuned/virtual-guest/tuned.conf /usr/lib/tuned/virtual-guest/tuned.conf.bkp
[root@dg1 ~]#  ls  -lrt /usr/lib/tuned/virtual-guest/tuned.conf*
-rw-r--r--. 1 root root 750 Jun 17  2016 /usr/lib/tuned/virtual-guest/tuned.conf
-rw-r--r--. 1 root root 750 Jul 28 18:33 /usr/lib/tuned/virtual-guest/tuned.conf.bkp
向文件/usr/lib/tuned/virtual-guest/tuned.conf中增加"transparent_hugepages=never"来禁用
[root@dg1 ~]# more /usr/lib/tuned/virtual-guest/tuned.conf
#
# tuned configuration
#

[main]
summary=Optimize for running inside a virtual guest.
include=throughput-performance

[vm]
transparent_hugepages=never

[sysctl]
# If a workload mostly uses anonymous memory and it hits this limit, the entire
# working set is buffered for I/O, and any more write buffering would require
# swapping, so it's time to throttle writes until I/O can catch up.  Workloads
# that mostly use file mappings may be able to use even higher values.
#
# The generator of dirty data starts writeback at this percentage (system default
# is 20%)
vm.dirty_ratio = 30

# Filesystem I/O is usually much more efficient than swapping, so try to keep
# swapping low.  It's usually safe to go even lower than this on systems with
# server-grade storage.
vm.swappiness = 30
重启之后验证
[root@dg1 ~]# init 6
[root@dg1 ~]#  cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]

参考链接：

http://docs.oracle.com/database/122/LADBI/disabling-transparent-hugepages.htm#LADBI-GUID-02E9147D-D565-4AF8-B12A-8E6E9F74BEEA