centos7 hugepages的设置
当前系统centos7
[root@qht116 ~]# uname -a
Linux qht116 3.10.0-862.el7.x86_64 #1 SMP Fri Apr 20 16:44:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
oracle 18c安装好后，数据库启动时alert log报以下提醒，是没有配置hugepage的原因
系统默认的pagsize只有4k，对于大sga来说，pagesize对数据库和系统的交换性能起到很大的影响，centos7已自动开启2M的pagesize，不过需要手动指定pagesize的数量。
**********************************************************************
Dump of system resources acquired for SHARED GLOBAL AREA (SGA)
Per process system memlock (soft) limit = 64K
Expected per process system memlock (soft) limit to lock
SHARED GLOBAL AREA (SGA) into memory: 100G
Available system pagesizes:
4K, 2048K
Supported system pagesize(s):
PAGESIZE AVAILABLE_PAGES EXPECTED_PAGES ALLOCATED_PAGES ERROR(s)
4K Configured 8 26214408 NONE
2048K 0 51201 0 NONE
RECOMMENDATION:
1. For optimal performance, configure system with expected number
of pages for every supported system pagesize prior to the next
instance restart operation.
2. Increase per process memlock (soft) limit to at least 100GB
to lock 100% of SHARED GLOBAL AREA (SGA) pages into physical memory
**********************************************************************
对于设置多少pagesize，Oracle在MOS上发布出一个Shell脚本hugepages_settings.sh，可以动态计算出推荐的HugePage数量。
#!/bin/bash
#
# hugepages_setting.sh
#
# Linux bash script to compute values for the
# recommended HugePages/HugeTLB configuration
#
# Note: This script does calculation for all shared memory
# segments available when the script is run, no matter it
# is an Oracle RDBMS shared memory segment or not.
# Check for the kernel version
KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'`
# Find out the HugePage size
HPG_SZ=`grep Hugepagesize /proc/meminfo | awk {'print $2'}`
# Start from 1 pages to be on the safe side and guarantee 1 free HugePage
NUM_PG=1
# Cumulative number of pages required to handle the running shared memory segments
for SEG_BYTES in `ipcs -m | awk {'print $5'} | grep "[0-9][0-9]*"`
do
MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q`
if [ $MIN_PG -gt 0 ]; then
NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q`
fi
done
# Finish with results
case $KERN in
'2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`;
echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;;
'2.6' | '3.8' | '3.10' | '4.1' | '4.14' ) echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
*) echo "Unrecognized kernel version $KERN. Exiting." ;;
esac
# End我计算一下当前系统的情况：
建议的数量51203个,每2M一个，正好是sga的大小(sag=100G)
[root@qht116 ~]# chmod u+x hugepages_setting.sh
[root@qht116 ~]# ./hugepages_setting.sh
Recommended setting: vm.nr_hugepages = 51203
根据建议的数据修改/etc/sysctl.con
[root@qht116 ~]# echo "vm.nr_hugepages = 51203" >> /etc/sysctl.conf
[root@qht116 ~]# cat /etc/sysctl.conf
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
fs.file-max = 6815744
kernel.sem = 250 32000 100 128
kernel.shmmni = 4096
kernel.shmall = 1073741824
kernel.shmmax = 6597069766656
kernel.panic_on_oops = 1
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
#net.ipv4.conf.eth3.rp_filter = 2
#net.ipv4.conf.eth2.rp_filter = 2
#net.ipv4.conf.eth0.rp_filter = 1
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500
vm.nr_hugepages = 51203
[root@qht116 ~]# sysctl -p
接着还要修改/etc/security/limits.conf,开启用户对内存的限制
51203*2048=104,863,744
将下面两行加入/etc/security/limits.conf
* soft memlock 104863744
* hard memlock 104863744
重新Log In后检查内存限制生效了
[root@qht116 ~]# ulimit -l
104863744
[root@qht116 ~]# cat /etc/security/limits.conf
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 65536
oracle hard nofile 65536
* soft memlock 104863744
* hard memlock 104863744
# End of file
重启了系统后，HugePages_Total这个值才对。
[oracle@qht116 ~]$ grep HugePages /proc/meminfo
AnonHugePages: 26624 kB
HugePages_Total: 51203
HugePages_Free: 51203
HugePages_Rsvd: 0
HugePages_Surp: 0
重启数据库后就正常了
**********************************************************************
Dump of system resources acquired for SHARED GLOBAL AREA (SGA)
Per process system memlock (soft) limit = 100G
Expected per process system memlock (soft) limit to lock
SHARED GLOBAL AREA (SGA) into memory: 100G
Available system pagesizes:
4K, 2048K
Supported system pagesize(s):
PAGESIZE AVAILABLE_PAGES EXPECTED_PAGES ALLOCATED_PAGES ERROR(s)
4K Configured 8 20571010 NONE
2048K 51203 51201 51201 NONE
RECOMMENDATION:
1. For optimal performance, configure system with expected number
of pages for every supported system pagesize prior to the next
instance restart operation.
**********************************************************************
参考：
ORACLE-BASE - Configuring HugePages for Oracle on Linux (x86-64)
为Linux版本Oracle 11gR2配置HugePage - 张冲andy - 博客园