linux 采用hugepage提升oracle11g性能
1，关于Hugepage
HugePage广泛启用开始于Kernal 2.6,一些版本下2.4内核也可以是用。在操作系统Linux环境中，内存是以页Page的方式进行分配，默认大小为4K。如果需要比较大的内存空间，则需要进行频繁的页分配和管理寻址动作。
HugePage是传统4K Page的替代方案。顾名思义，是用HugePage可以让我们有更大的内存分页大小。无论是HugePage还是传统的正常Page，这个过程都涉及到OS内存寻址过程。
当一个进程访问内存的时候，并不是直接进行内存位置访问，是需要通过Page Table进行转移变换。在使用HugePage的情况下，PageTable具有了额外的属性，就是判断该页记录是HugePage还是Regular Page。
在Oracle运行环境中开启HugePage是有很多好处的。具体如下：
1）非Swap内存：当开启HugePage的时候，HugePage是不会Swap的；
2）减少TLB（TranslationLook aside Buffer）负担：TBL是在CPU里面的一块缓冲区域，其中包括了部分Page Table内容。使用HugePage可以减少TLB工作负载；
3）减少Page Table空间负载：在PageTable管理中，每条Page记录是要占据64byte的空间的。也就是说，如果一块50G的RAM，4k大小的PageTable要有80MB左右；
4）减少PageTable检索负载：更小的PageTable意味着更快的检索定位能力；
5）内存性能提升：Page数量减少、大小的增加，减少了管理过程的复杂性，进一步减少了瓶颈出现的概率；
2，oracle设置hugepage
对于Oracle而言，实例运行环境（Database和ASM）都面对一个HugePage优化的问题。
如果是使用11g，Oracle版本，一定需要进行额外的配置，就是将使用的AMM退化为ASMM。在早期的11.2.0.1版本中，这个步骤很重要。因为AMM是不支持HugePage的，如果强在AMM+HugePage模式下打开数据库，是会遇到失败信息。
在最新的11.2.0.2版本以及之后，引入了参数use_large_pages，避免了这样的问题。但是AMM与HugePage不兼容的情况，还是存在。所以，需要将ASM切换成AMM，AMM、ASMM切换参考：http://blog.itpub.net/17203031/viewspace-774928/。
假如当前是使用AMM的数据库，HugePages_Total,HugePages_Free均为零，表示没有生效，如下所示：
可以看到，HugePages_Total,HugePages_Free均为零，证明没有使用，但可以配置。如果上面什么都没有显示，证明不能配置。
[root@crmtest ~]# cat /proc/meminfo | grep -i huge AnonHugePages: 28672 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB [root@crmtest ~]# |
blog原文地址：http://blog.csdn.net/mchdba/article/details/51030668，谢绝转载。
如果碰到这种free比total小，证明已经生效，如下所示：
[root@crmtest oracle]# cat /proc/meminfo | grep -i huge AnonHugePages: 20480 kB HugePages_Total: 2588 HugePages_Free: 2449 HugePages_Rsvd: 2421 HugePages_Surp: 0 Hugepagesize: 2048 kB [root@crmtest oracle]# [root@crmtest oracle]# grep Huge /proc/meminfo AnonHugePages: 20480 kB HugePages_Total: 2588 HugePages_Free: 2449 HugePages_Rsvd: 2421 HugePages_Surp: 0 Hugepagesize: 2048 kB [root@crmtest oracle]# |
大内存技术，不支持内存自动管理，必须关闭AMM(自动内存管理）特性才能使用hugepage，调整方法如下：
对于OLTP系统： SGA_TARGET=(物理内存 x 80%) x 80% SGA_MAX_SIZE=(物理内存 x 80%) x 80% PGA_AGGREGATE_TARGET=(物理内存 x 80%) x 20%
ALTER SYSTEM SET sga_max_size = 103079215104 SCOPE=SPFILE; ALTER SYSTEM SET sga_target = 75591424409 SCOPE=SPFILE; ALTER SYSTEM SET PGA_AGGREGATE_TARGET = 18897856102 SCOPE=SPFILE; ALTER SYSTEM SET memory_target = 0 SCOPE=SPFILE; ALTER SYSTEM SET memory_max_target = 0 SCOPE=SPFILE;
|
执行后，保存到spfile中，重启oracle实例：
SQL> shutdown immediate Database closed. Database dismounted. ORACLE instance shut down. SQL> SQL> SQL> startup ORA-00843: Parameter not taking MEMORY_MAX_TARGET into account ORA-00849: SGA_TARGET 103079215104 cannot be set to more than MEMORY_MAX_TARGET 0. SQL>
|
这个问题的原因是Oracle启动过程中对于参数的内部检查。因为MEMORY_MAX_TARGET被“显示”的赋值，与SGA_TARGET赋值相冲突。
解决的方法就是使用参数默认值。创建出pfile之后，将显示赋值为0的MEMORY_TARGET和MEMORY_MAX_TARGET记录行删除掉。再利用pfile启动数据库，重建spfile，如下所示：
SQL> create pfile='/tmp/pfile6.ora' from spfile;
File created.
SQL> exit Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options [oracle@crmtest ~]$ vim /tmp/pfile6.ora [oracle@crmtest ~]$ SQL> startup nomount pfile='/tmp/pfile6.ora'; ORA-32006: LOG_ARCHIVE_START initialization parameter has been deprecated ORA-27102: out of memory Linux-x86_64 Error: 28: No space left on device SQL> exit Disconnected
|
3，设置用户内存配置
3.1，修改/etc/security/limits.conf参数文件，添加数据库实例用户的memlock限制
如果不配置锁定内存，在后alert_sid.log中会有类似如下的建议：
RECOMMENDATION:
Total System Global Area sizeis 5 GB. For optimal performance,
prior to the next instancerestart:
1. Large pages areautomatically locked into physical memory.
Increase the per process memlock(soft) limit to at least 5 GB to lock
100% System Global Area's largepages into physical memory
并且，hugepage并没有真正使用起来！主要是配置如下2个设置：
oracle soft memlock
oracle hard memlock
注意，这里设置的值均以kb为单位的！
设置用户内存配置，有个计算规则是：实际物理内存 > 锁定内存 >=HugePages_Total*Hugepagesize；
[root@crmtest oracle]# free -t
total used free shared buffers cached
Mem: 115742368 76624776 39117592 0 133644 68080568
-/+ buffers/cache: 8410564 107331804
Swap: 0 0 0
Total: 115742368 76624776 39117592
[root@crmtest oracle]#
看到free -t，把total的值取出来，修改/etc/security/limits.conf参数文件，添加数据库实例用户的memlock限制，限制的值就是total值115742368。启用HugePage的第一步就是进行用户参数限制打通，当前内存大小如下：
[root@crmtest oracle]# vim/etc/security/limits.conf
oracle soft memlock 115742368
oracle hard memlock 115742368
注意，这里设置的值均以kb为单位的！
遵循原则是：实际物理内存 > 锁定内存 >=HugePages_Total*Hugepagesize，Hugepagesize值从哪里看呢？可以从meminfo里面找到，如下所示：
[root@pldb1 ~]# cat /proc/meminfo | grep -ihuge
AnonHugePages: 30720 kB
HugePages_Total: 49346
HugePages_Free: 29238
HugePages_Rsvd: 29045
HugePages_Surp: 0
Hugepagesize: 2048 kB
[root@pldb1 ~]#
3.2，oracle中大内存技术的配置建议，oracle针对大内存技术，提供了一个脚本如下：
[root@crmtest oracle]# sh /oracle/hugepages_set.sh
This script is provided by Doc ID 401749.1 from My Oracle Support (http://support.oracle.com) where it is intended to compute values for the recommended HugePages/HugeTLB configuration for the current shared memory segments. Before proceeding with the execution please make sure that: * Oracle Database instance(s) are up and running * Oracle Database 11g Automatic Memory Management (AMM) is not setup (See Doc ID 749851.1) * The shared memory segments can be listed by command: # ipcs -m
Press Enter to proceed...
Recommended setting: vm.nr_hugepages = 49346 [root@crmtest oracle]# [root@crmtest oracle]# [root@crmtest oracle]# more /oracle/hugepages_set.sh #!/bin/bash # # hugepages_settings.sh # # Linux bash script to compute values for the # recommended HugePages/HugeTLB configuration # # Note: This script does calculation for all shared memory # segments available when the script is run, no matter it # is an Oracle RDBMS shared memory segment or not. # # This script is provided by Doc ID 401749.1 from My Oracle Support # http://support.oracle.com
# Welcome text echo " This script is provided by Doc ID 401749.1 from My Oracle Support (http://support.oracle.com) where it is intended to compute values for the recommended HugePages/HugeTLB configuration for the current shared memory segments. Before proceeding with the execution please make sure that: * Oracle Database instance(s) are up and running * Oracle Database 11g Automatic Memory Management (AMM) is not setup (See Doc ID 749851.1) * The shared memory segments can be listed by command: # ipcs -m
Press Enter to proceed..."
read
# Check for the kernel version KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'`
# Find out the HugePage size HPG_SZ=`grep Hugepagesize /proc/meminfo | awk '{print $2}'`
# Initialize the counter NUM_PG=0
# Cumulative number of pages required to handle the running shared memory segments for SEG_BYTES in `ipcs -m | awk '{print $5}' | grep "[0-9][0-9]*"` do MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q` if [ $MIN_PG -gt 0 ]; then NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q` fi done
RES_BYTES=`echo "$NUM_PG * $HPG_SZ * 1024" | bc -q`
# An SGA less than 100MB does not make sense # Bail out if that is the case if [ $RES_BYTES -lt 100000000 ]; then echo "***********" echo "** ERROR **" echo "***********" echo "Sorry! There are not enough total of shared memory segments allocated for HugePages configuration. HugePages can only be used for shared memory segments that you can list by command:
# ipcs -m
of a size that can match an Oracle Database SGA. Please make sure that: * Oracle Database instance is up and running * Oracle Database 11g Automatic Memory Management (AMM) is not configured" exit 1 fi
# Finish with results case $KERN in '2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`; echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;; '2.6') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;; *) echo "Unrecognized kernel version $KERN. Exiting." ;; esac
# End [root@crmtest oracle]#
|
3.3，然后开始运行脚本
[root@crmtest oracle]# sh /oracle/hugepages_set.sh
This script is provided by Doc ID 401749.1 from My Oracle Support (http://support.oracle.com) where it is intended to compute values for the recommended HugePages/HugeTLB configuration for the current shared memory segments. Before proceeding with the execution please make sure that: * Oracle Database instance(s) are up and running * Oracle Database 11g Automatic Memory Management (AMM) is not setup (See Doc ID 749851.1) * The shared memory segments can be listed by command: # ipcs -m
Press Enter to proceed...
Recommended setting: vm.nr_hugepages = 49346 [root@crmtest oracle]#
|
可以看到，运行脚本显示，将vm.nr_hugepages设置为49346
3.3，设置/etc/sysctl.conf
sga_target < = sga_max_size
pga_aggregate_target
[root@crmtest oracle]# vim /etc/sysctl.conf
[root@crmtest oracle]#
[root@crmtest oracle]# sysctl -p
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route =0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
error:"net.bridge.bridge-nf-call-ip6tables" is an unknown key
error:"net.bridge.bridge-nf-call-iptables" is an unknown key
error:"net.bridge.bridge-nf-call-arptables" is an unknown key
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 536870912
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586
vm.nr_hugepages = 49346
[root@crmtest oracle]#
使用sysctl –p生效设置。
4，查看hugepage是否生效
设置好后，最好能shutdown –r now重启下服务器，在meminfo文件中，可以查到HugePages的信息，说明启用成功，如下所示：
[root@crmtest oracle]# grep Huge/proc/meminfo
AnonHugePages: 20480 kB
HugePages_Total: 2588
HugePages_Free: 2449
HugePages_Rsvd: 2421
HugePages_Surp: 0
Hugepagesize: 2048 kB
[root@crmtest oracle]#
现在互联网发展太快，pc server也越来越多，而且内存普遍在64G以上，所以开启hugepage提升oracle性能非常有必要。
AMM、ASMM切换参考：http://blog.itpub.net/17203031/viewspace-774928/
hugepage参考：http://blog.itpub.net/17203031/viewspace-774843/
linux下hugepage性能：http://blog.itpub.net/29371470/viewspace-1063046/