如何有效调整HiveServer2进程的JVM堆内存大小？
我的运行环境：JDK1.8， HIVE1.2 ，HADOOP2.7
HiveServer2在以下简称HS2
[hadoop@tony_hdp_01 bin]$ pwd
${HIVE_HOME}/bin
[hadoop@tony_hdp_01 bin]$ ll
total 40
-rwxr-xr-x 1 hadoop hadoop 881 Nov 10 2017 beeline
drwxrwxr-x 3 hadoop hadoop 4096 Mar 6 2018 ext
-rwxr-xrwx 1 hadoop hadoop 9838 Nov 10 2017 hive
-rwxr-xr-x 1 hadoop hadoop 1900 Nov 10 2017 hive-config.sh
-rwxr-xr-x 1 hadoop hadoop 885 Nov 10 2017 hiveserver2
-rwxr-xr-x 1 hadoop hadoop 880 Nov 10 2017 hplsql
-rwxr-xr-x 1 hadoop hadoop 832 Nov 10 2017 metatool
-rwxr-xr-x 1 hadoop hadoop 884 Nov 10 2017 schematool
// hive这个脚本运行时的参数实际上调用的是 hive-config.sh
[hadoop@tony_hdp_01 bin]$ grep --color -C 3 -iE "hive-config.sh" hive
bin=`dirname "$0"`
bin=`cd "$bin"; pwd`
. "$bin"/hive-config.sh
TMP_USER_DIR="/tmp/${USER}"
STDERR="${TMP_USER_DIR}/stderr"
// 修改HADOOP_HEAPSIZE为12288M即12G
[hadoop@tony_hdp_01 bin]$ cp hive-config.sh hive-config.sh_20180927
[hadoop@tony_hdp_01 bin]$ vim hive-config.sh
export HADOOP_HEAPSIZE=${HADOOP_HEAPSIZE:-12288}
// 改完以后kill掉HS2：后台我的自定义守护hiveServer2的脚本auto_restart_hive2.sh会自动启动HS2
// 但是第二个 -Xmx仍然是 -Xmx2g
[hadoop@tony_hdp_01 hadoop]$ ps aux | grep -v grep | grep --color -iE "auto_clear|auto_restart"
hadoop 912 0.0 0.0 113252 1628 ? S Aug31 0:05 /bin/bash ./auto_clear_hung_jobs.sh
hadoop 3421 0.1 0.0 113636 1924 ? S Sep26 2:57 /bin/sh /home/hadoop/wps_cloud/admin_op/auto/auto_restart_hive2.sh
[hadoop@tony_hdp_01 hadoop]$ ps aux|grep -v grep | grep --color "HiveServer2" | grep --color Xmx
hadoop 7217 19.3 0.7 4184168 938496 ? Sl 14:26 1:11 /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-7.b10.el7.x86_64/bin/java -Xmx12288m -Djava.library.path=${HADOOP_HOME}/lib:${HADOOP_HOME}/lib/native -Djava.library.path=${HADOOP_HOME}/lib:${HADOOP_HOME}/lib/native -Dhadoop.log.dir=${HADOOP_HOME}/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=${HADOOP_HOME} -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console -Djava.library.path=${HADOOP_HOME}/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx2g -Dproc_hiveserver2 -Dlog4j.configurationFile=hive-log4j2.properties -Djava.util.logging.config.file=${HIVE_HOME}/conf/parquet-logging.properties -Djline.terminal=jline.UnsupportedTerminal -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar ${HIVE_HOME}/lib/hive-service-2.3.2.jar org.apache.hive.service.server.HiveServer2
// 联想到hive命令是通过hadoop jar命令加上hive有关的jar包执行的，那么就去从hadoop有关的配置文件中找到 -Xmx=2g出现的配置文件，再修改那个配置文件即可：
[hadoop@tony_hdp_01 hadoop]$ pwd
${HADOOP_HOME}/etc/hadoop
[hadoop@tony_hdp_01 hadoop]$ grep -r --color '\-Xmx2g' ${HADOOP_HOME}/etc/hadoop/
${HADOOP_HOME}/etc/hadoop/hadoop-env.sh:export HADOOP_CLIENT_OPTS="-Xmx2g $HADOOP_CLIENT_OPTS"
// 修改 ${HADOOP_HOME}/etc/hadoop/hadoop-env.sh 中的HADOOP_CLIENT_OPTS值为12g
[hadoop@tony_hdp_01 hadoop]$ cp hadoop-env.sh hadoop-env.sh_20180927
[hadoop@tony_hdp_01 hadoop]$ vim hadoop-env.sh # 将HADOOP_CLIENT_OPTS=-Xmx12g 即可
[hadoop@tony_hdp_01 hadoop]$ ps aux|grep -v grep | grep --color "HiveServer2" | grep --color Xmx
hadoop 7217 19.3 0.7 4184168 938496 ? Sl 14:26 1:11 /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-7.b10.el7.x86_64/bin/java -Xmx12288m -Djava.library.path=${HADOOP_HOME}/lib:${HADOOP_HOME}/lib/native -Djava.library.path=${HADOOP_HOME}/lib:${HADOOP_HOME}/lib/native -Dhadoop.log.dir=${HADOOP_HOME}/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=${HADOOP_HOME} -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console -Djava.library.path=${HADOOP_HOME}/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx2g -Dproc_hiveserver2 -Dlog4j.configurationFile=hive-log4j2.properties -Djava.util.logging.config.file=${HIVE_HOME}/conf/parquet-logging.properties -Djline.terminal=jline.UnsupportedTerminal -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar ${HIVE_HOME}/lib/hive-service-2.3.2.jar org.apache.hive.service.server.HiveServer2
// kill掉HS2：后台我的自定义守护hiveServer2的脚本auto_restart_hive2.sh会自动启动HiveServer2
[hadoop@tony_hdp_01 hadoop]$ kill -9 7217
[hadoop@tony_hdp_01 hadoop]$ ps aux|grep -v grep | grep --color "HiveServer2" | grep --color Xmx
hadoop 13601 109 0.1 14863400 175580 ? Sl 14:33 0:04 /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-7.b10.el7.x86_64/bin/java -Xmx12288m -Djava.library.path=${HADOOP_HOME}/lib:${HADOOP_HOME}/lib/native -Djava.library.path=${HADOOP_HOME}/lib:${HADOOP_HOME}/lib/native -Dhadoop.log.dir=${HADOOP_HOME}/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=${HADOOP_HOME} -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console -Djava.library.path=${HADOOP_HOME}/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx12g -Dproc_hiveserver2 -Dlog4j.configurationFile=hive-log4j2.properties -Djava.util.logging.config.file=${HIVE_HOME}/conf/parquet-logging.properties -Djline.terminal=jline.UnsupportedTerminal -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar ${HIVE_HOME}/lib/hive-service-2.3.2.jar org.apache.hive.service.server.HiveServer2
// 结果：JVM -Xmx12g 生效！那么也需要在提供HiveServer2服务的另外一台主机上做同样的修改、重启HS2
[hadoop@tony_hdp_02 bin]$ cd ${HADOOP_HOME}/etc/hadoop
[hadoop@tony_hdp_02 hadoop]$ cp hadoop-env.sh hadoop-env.sh_20180927
[hadoop@tony_hdp_02 hadoop]$ ll hadoop-env.sh*
-rw-rw-r-- 1 hadoop hadoop 16255 May 18 09:38 hadoop-env.sh
-rw-rw-r-- 1 hadoop hadoop 16255 Sep 27 14:40 hadoop-env.sh_20180927
[hadoop@tony_hdp_02 hadoop]$ vim hadoop-env.sh
[hadoop@tony_hdp_02 hadoop]$ ps aux|grep -v grep |grep --color -iE "auto_clear|auto_restart"
hadoop 3769 0.1 0.0 113380 1780 ? S Sep26 2:28 /bin/sh /home/hadoop/wps_cloud/admin_op/auto/auto_restart_hive2.sh
hadoop 18840 0.0 0.0 113252 1648 ? S Aug28 0:06 /bin/bash auto_clear_hung_jobs.sh
[hadoop@tony_hdp_02 hadoop]$ ps aux|grep -v grep|grep --color "HiveServer2" | grep --color Xmx12
hadoop 32461 35.9 2.0 6558060 2661848 ? Sl 14:21 7:08 /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-7.b10.el7.x86_64/bin/java -Xmx12288m -Djava.library.path=${HADOOP_HOME}/lib:${HADOOP_HOME}/lib/native -Dhadoop.log.dir=${HADOOP_HOME}/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=${HADOOP_HOME} -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console -Djava.library.path=${HADOOP_HOME}/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx12g -Dproc_hiveserver2 -Dlog4j.configurationFile=hive-log4j2.properties -Djava.util.logging.config.file=${HIVE_HOME}/conf/parquet-logging.properties -Djline.terminal=jline.UnsupportedTerminal -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar ${HIVE_HOME}/lib/hive-service-2.3.2.jar org.apache.hive.service.server.HiveServer2
// 重启hs2，可以看到hiveServer2使用的JVM堆大小(12G)已生效！
[hadoop@tony_hdp_02 hadoop]$ kill -9 32461
[hadoop@tony_hdp_02 hadoop]$ ps aux|grep -v grep|grep --color "HiveServer2" | grep --color Xmx12
hadoop 23756 170 0.7 14996400 1023800 ? Sl 14:41 0:44 /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-7.b10.el7.x86_64/bin/java -Xmx12288m -Djava.library.path=${HADOOP_HOME}/lib:${HADOOP_HOME}/lib/native -Dhadoop.log.dir=${HADOOP_HOME}/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=${HADOOP_HOME} -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console -Djava.library.path=${HADOOP_HOME}/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx12g -Dproc_hiveserver2 -Dlog4j.configurationFile=hive-log4j2.properties -Djava.util.logging.config.file=${HIVE_HOME}/conf/parquet-logging.properties -Djline.terminal=jline.UnsupportedTerminal -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar ${HIVE_HOME}/lib/hive-service-2.3.2.jar org.apache.hive.service.server.HiveServer2