Shell脚本一次性安装JDK&MySql&Hadoop&Hive&Zeepelin&Zookeeper&Hbase&Sqoop

---

前言

        在使用计算机的时候，我们的程序越来越多，服务也越来越多，所以我们在拿到一台新机器的时候总是习惯性的去安装那些我们常用的程序&服务；使用虚拟机时也是一样的，所以为了方便以后的操作，我们可以尝试写shell脚本来一次性完成我们需要的所有安装&配置操作。

•         MySql服务用wget获取MySql安装包安装的，这一部分时间较长，所以我单独列出来安装MySql的脚本，其他的服务通过本地安装包解压之后配置安装的所以统一放在一起（如果还是想一次性执行脚本安装，不想分两次执行脚本安装，可以考虑在大的脚本里的最前面先执行MySQL的脚本，这样就不用执行两次脚本了）；
•         由于hive初始化时后台操作进入MySql并创建了数据库，所以我推荐先安装MySql之后再运行配置剩余全部的shell脚本；
•         如果在MySql服务未安装时先执行整体配置的脚本则会报hive初始化失败的报错；
•         如果安装一半提示Hive初始化失败，可能原因是Hive初始化需要后台进入MySQL中创建的数据库已经存在了，所以在安装前务必确认hive配置文档中的数据库名在MySQL中不存在才可进行安装！

---

       代码提取： 百度云链接
       提  取  码：gg4x

---

一、安装前的准备

• 一台网络通畅的虚拟机
• 所有服务的安装包(上传至虚拟机)

        这里列出本文安装的所有服务的安装包信息

• jdk-8u111-linux-x64.tar.gz
• mysql-community-release-el7-5.noarch.rpm
• hadoop-2.6.0-cdh5.14.2.tar.gz
• mysql-connector-java-5.1.32.jar
• hadoop-native-64-2.6.0.tar
• hive-1.1.0-cdh5.14.2.tar.gz
• hbase-1.2.0-cdh5.14.2.tar.gz
• zeppelin-0.8.2-bin-all.tgz
• zookeeper-3.4.5-cdh5.14.2.tar.gz
• sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz
• scala-2.12.10.tgz

---

二、shell安装MySQL

        这一部分在安装前为检测是否有上一次安装过的MySQL服务的痕迹，后面将补上这一部分内容（已经加入自动删除旧版本的校验并增加修改root用户密码的功能）；

• 请在/root目录下运行安装mysql的脚本
• UNINSTALL MARIADB =>卸载mariadb
• UNINSTALL PAST MYSQL =>卸载旧版本MySQL
• yum update =>更新yum
• SEARCH WGET AND ISTALL WGET =>安装wget
• WGET MYSQL PACKAGE =>wget获取MySQL安装包
• RMP INSTALL MySQL =>安装MySQL
• INSTALL MYSQL-SERVER I=>安装MySQL服务
• MODIFY PASSWORD =>修改root用户密码
• CHINESE GARBLED PROBLEM =>修改MySQL字符集，解决中文乱码问题

#!/bin/bash
#------------------UNINSTALL MARIADB------------------------
echo "First,we need to unistall mariadb"
var=`rpm -qa | grep mariadb`
if [[ $var =~ ^mariadb ]]
then
        echo -e Find "\e[1;31m $var \e[0m"
        `rpm -e --nodeps $var`
        rst=`rpm -qa | grep mariadb`
        if [[ $rst =~ "" ]]
        then
                echo "Mariadb uninstall successfully"
        fi
else
        echo "Not find mariadb,just direct install mysql"
fi

#-------------------------UNINSTALL PAST MYSQL---------------------------
echo
echo "Second,we need to check that you pc has past mysql and uninstall"
echo "Start to uninstall mysql server"
VAR=`rpm -qa|grep -i mysql`
VAR=($VAR)
for server in ${VAR[@]}
do
    echo -e uninstall: "\e[1;31m $VAR \e[0m"
    `rpm -e --nodeps $server`
done
VAR=`rpm -qa|grep -i mysql`
if [ ${#VAR} -eq 0 ]
then
    echo "All mysql server were uninstall!"
else
    echo "Find remains mysql server,please reuninstall remain mysql server"
    exit 1
fi
echo
echo "Next,we need to remove remain mysql properties and denpencies"
VAR=`rm -rf /var/lib/mysql`
echo "Mysql propertied and denpencies were all droped"
echo
echo -e Last,we had to remove: "\e[1;31m my.cnf \e[0m"
VAR=`rm -rf /etc/my.cnf`
echo -e File: "\e[1;31m $VAR \e[0m"  has removed!
echo "Your pc with all about mysql has remove, we start to install mysql"

#-------------------------YUM UPDATE---------------------------------------
echo
echo "start to update yum ..."
`yum -y update>/root/test.log 2>&1`
VAR=`awk -F '!'  'END{print $1}' /root/test.log`
if [[ $VAR =~ ^Complete ]]
then
    echo "update yum success!"
elif [[ $VAR =~ ^No ]]
then
    echo "yum has been the newest!"
else
    echo "update yum failed,please reinstall..."
        exit 1
fi

#------------------SEARCH WGET AND INSTALL WGET--------------------
echo
VAR=`yum search wget|grep wget.|awk -F' ' '{print $1}'`
VAR=${VAR##*=}
echo -e yum search wget find: "\e[1;31m $VAR \e[0m"
echo -e start to install: "\e[1;31m $VAR \e[0m"
VAR="yum -y install $VAR>/root/test.log 2>&1"
`eval $VAR`
#验证wget是否安装成功
VAR=`awk -F '!'  'END{print $1}' /root/test.log`
VAR2=`cat /root/test.log |grep wget.|awk -F' ' '{print $0}'`
VAR2=`echo $var|grep "already installed"`
if [[ $VAR =~ ^Complete ]]
then
    echo "install wget success"
elif [[ $VAR2 != NULL ]]
then
    echo "wget already install"
else
    echo "install wget failed,please reinstall..."
        exit 1
fi

#------------------WGET MYSQL PACKAGE-----------------------
echo
echo "Checking past version of mysql package"
VAR=`ls|grep mysql.*`
VAR=($VAR)
for package in ${VAR[@]}
do
    `rm -rf $package`
done
echo "Old version of package, they all been removed"
echo
echo "Start to download mysql package"
VAR=http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm
`eval "wget $VAR>/root/test.log 2>&1"`
#验证wget获取MySQL package结果
ls>test.log
VAR=`cat test.log`
VAR=($VAR)
COUNT=0
for var in ${VAR[@]}
do
    if [[ $var =~ ^mysql ]]
    then
        echo "Download package $var success!"
        COUNT=0
        break
    else
        ((COUNT++))
    fi
done

if [ $COUNT -ne 0 ]
then
    echo "Download package failed .... please redownload package!"
    exit 1
fi
echo

#----------------RMP INSTALL-----------------------
echo "Start to install mysql"
VAR=`ls ./|grep mysql*`
OPER=`eval "rpm -ivh $VAR>/root/test.log 2>&1"`
#验证MySQL安装是否成功
VAR=`awk -F' ' 'END{print $1}' /root/test.log`
if [[ $VAR =~ ^mysql ]]
then
    echo "Install mysql successfully"
else
    echo "Install mysql failed,please reinstall..."
    echo "Install log in /root/test.log,please check it and find reason"
        exit 1
fi
echo

#---------------MYSQL-SERVER INSTALL-----------------
echo "start to install mysql server..."
VAR="yum -y install mysql-server>/root/test.log 2>&1"
`eval $VAR`
#验证MySQL服务是否安装成功
VAR=`rpm -qa|grep -i mysql`
VAR=($VAR)
if [[ ${#VAR[*]} -ge 4 ]]
then
    echo "mysql server install success"
else
    echo "mysql server install fail"
    echo "Install log in /root/test.log,please check it and find reason"
        exit 1
fi

#------------------MODIFY PASSWORD------------------------------
echo
echo "Start to modify password"
#通过配置文件的方法跳过MySQL的登录检查，修改后要删掉次配置
`sed -i '/sql_mode=/i\\skip-grant-tables' /etc/my.cnf`
`systemctl restart mysql`
#modify password
`mysql -uroot -e "use mysql;update user set password=password('root') where user='root';flush privileges;grant all on *.* to root@'%' identified by 'root';"`
echo "User of 'root' password has changed into 'root'"
`sed -i '/skip-grant-tables/d' /etc/my.cnf`
echo "Modify password successfully"

#------------------CHINESE GARBLED PROBLEM------------------------
echo
echo "Start to modify mysql-my.cnf"
`echo "#mysql-setup" > /etc/my.cnf`
sen="[client]
default-character-set=utf8
[mysqld]
default-storage-engine=INNODB
character-set-server=utf8
collation-server=utf8_general_ci
[mysql]
default-character-set=utf8
"
lines=`echo $sen`
for line in $lines
do
    VAR=`eval "sed -i '/#mysql-setup/i\$line' /etc/my.cnf"`
done
#验证修改是否完成
VAR=`awk -F' ' 'END{print $1}' /etc/my.cnf`
if [[ $VAR =~ ^#mysql ]]
then
        echo "modify my.cnf success"
else
        echo "modify my.cnf failed,please check shell..."
        exit 1
fi
echo
echo "Restart mysql"
`systemctl restart mysql`
echo "Install mysql finished!"

---

三、shell安装&配置

        先将整体的安装包解压到/opt/download文件夹下(若无此文件夹，需要手动创建),图中我解压到hadoop文件夹内;接下来就可以启用shell脚本了；

---

1.删除重复文件&配置

• 删除服务所在的目录文件
          本文默认所有服务安装在/opt/software目录下，所以只要递归删除目录software即可；

• 剔除配置的环境变量
          修改环境变量在/etc/profile下，因为本文插入的环境变量都在done 这一行和unset i这一行之间，所以将这两行中间的内容删除即可；

#!/bin/bash
#------------------REMOVE ALL RELATIVE RESOURCES---------------------
echo "##### start to remove relative resources,included env-vars and software-installation ..."
#获取配置环境变量的开始行和结束行
BEGIN=`cat -n /etc/profile|grep done|awk '{print $1}'`
END=`cat -n /etc/profile|grep "unset i"|awk '{print $1}'`
RST=$(((--END)-(++BEGIN)))
if [ $RST -gt 0 ]
then
	RST="'"$BEGIN,$END" d'"
	RST="sed -i $RST /etc/profile"
	RST=`eval $RST`
	echo "... /etc/profile cleared"
else
	echo "... /etc/profile empty"
fi
#删除服务所在目录下的所有文件
if [ -e /opt/software ]
then
	rm -rf /opt/software
	echo "... /etc/profile delete"
else
	echo "... /etc/profile not existed"
fi
echo

---

2.解压安装包&重命名&授权

• 验证安装包是否缺失
          这里防止安装一半时缺失安装包的情况，先验证安装包是否完整；

• 安装目录是否存在
          统一解压到/opt/software目录下，若没有此目录则新建此目录；

• 解压&重命名&授权
          存在.tar和.gz的文件，所以解压方式也不同；解压完之后根据安装包的前半部分和重命名集合里的是否一致，一致则替换名称，最后将权限修改；

#------------------NAME LIST FOR INSTALLATION------------------------
echo "----------------------------------------------"
echo
echo "#### start to unpack all resources ..."
INSTALL_LIST="jdk-8u111-linux-x64.tar.gz scala-2.12.10.tgz hadoop-2.6.0-cdh5.14.2.tar.gz hive-1.1.0-cdh5.14.2.tar.gz zeppelin-0.8.2-bin-all.tgz zookeeper-3.4.5-cdh5.14.2.tar.gz hbase-1.2.0-cdh5.14.2.tar.gz sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz"

#---------------------SOURCE PACKAGE VAILIDATION---------------------
echo "start to check whether all the resource needed exists?"
LIST=`ls /opt/download/hadoop`


for item in $INSTALL_LIST
do
	if [[ $LIST =~ $item ]]
	then
		continue
	else
		echo "ERROR and will exit for the lack of $item"
		exit 1
	fi
done
echo "unpack all finish."
echo "start to installing..."
echo "first,we'll unpack all the items ..."

#---------------------ENSURE THE UNPACK DESTINATION DIR-----------------------
echo -n "start to check destination directory ..."
DEST_DIR="/opt/software/"

if [ ! -e $DEST_DIR ]
then
	RST=`mkdir $DEST_DIR`
	echo "not exist and has been created"
else
	echo "ok"
fi

cd /opt/software
echo "                                  ---------------"
echo "current dir has been changed into |/opt/software|"
echo "                                  ---------------"

#--------------------UNPACK AND RENAME AND CHOWN---------------------------------------
RENAME_LIST="java180 scala21210 hadoop260 hive110 zeppelin082 zookeeper345 hbase120 sqoop146"
RENAME_LIST=($RENAME_LIST)
IX=0

for item in $INSTALL_LIST
do
	if [[ $item =~ tar|gz$ ]]
	then
		echo "start to unpack $item ..."
		OPT=${item##*.tar}
		OPT="-"$([ $OPT ] && echo "z" || echo "" )"xf"
		RST=`eval "tar $OPT /opt/download/hadoop/$item"`
		RST=`eval "mv ${item%%-*}* ${RENAME_LIST[$IX]}"`
		echo "unpack $item and rename ${RENAME_LIST[$IX]} ok!"
		`chown -R root:root ${RENAME_LIST[$IX]}`
	fi
	((IX++))
done
echo

---

3.配置环境变量

• 目录名转大写并判断配置path时是否包含sbin
• 插入时以unset i 这一行为基准，一直往这一行上方插入即可
• CLASSPATH和LOGDIR单独插入

#---------------------ENV_VAR : ALL_XXX_HOME--------------------------------
echo "----------------------------------------------"
echo
echo "#### start to config environment variables ..."
RENAME_ITEMS="java180_bin scala21210_bin hadoop260_bin_sbin hive110_bin zeppelin082_bin zookeeper345_sbin hbase120_bin sqoop146_bin"
ROAD="export PATH="
for item in $RENAME_ITEMS
do
	HOME=${item%%[0-9]*}
	#拼出export对应的字符串
	HOME=${HOME^^}"_HOME"
	#名称转大写
	T="${item//_/ }"
	FIRST=1
	for var in $T
	do
		if [ $FIRST -eq 1 ]
		then
			FIRST=0
		else
			#插入path前拼出字符串
			ROAD+="$"$HOME"/"$var":"
		fi
	done
	echo " .... $HOME finished"
	HOME="export $HOME=$DEST_DIR${item%%_*}"
	#往unset i 上方插入
	RST=`eval "sed -i '/unset i/i\$HOME' /etc/profile"`
done
ROAD+="$""PATH"
RST=`eval "sed -i '/unset i/i\$ROAD' /etc/profile"`
echo " .... PATH finished"
#--------------------------CLASS_PATH---------------------
ROAD="export CLASSPATH=.:$""JAVA_HOME/lib/tools.jar:$""JAVA_HOME/lib/dt.jar"
RST=`eval "sed -i '/unset i/i\$ROAD' /etc/profile"`
echo " .... JAVA_CLASSPATH finished"
#---------------------ZOOKEEPER LOGS---------------------
ROAD="export LOGDIR=$""SQOOP_HOME/mylog/"
RST=`eval "sed -i '/unset i/i\$ROAD' /etc/profile"`
echo " .... ZOOLEEPER LOGDIR finished"

#---------------------ACTIVATE ENV_VAR-------------------
#激活环境变量
source /etc/profile
echo "environment variables have been written into /etc/profile and activated"
echo
#获取虚拟机ip地址
HOST_IP=`ip addr|awk -v FS='/' '/noprefixroute ens33/{print $1}'|awk -F' ' '{print $2}'`
echo
echo "----------------------------------------------"
echo "$HOSTNAME IP is $HOST_IP"
echo "----------------------------------------------"

---

4.Hadoop配置

• hadoop-native-64-2.6.0.tar解压至hadoop根目录的lib目录下的native文件夹
• 修改hadoop-env.sh；
• core-site.xml hdfs-site.xml mapred-site.xml.template yarn-site.xml在同一目录下，使用循环修改；
• 格式化Namenode并验证

#---------------------hadoop-----------------------------
echo
echo "Starting to modify hadoop..."
cd /opt/download/hadoop
VAR=`ls -F | grep "^hadoop-native"`
TMP=`eval "tar -xvf $VAR -C ${HADOOP_HOME}/lib/native"` 
DEST_DIR=$HADOOP_HOME"/etc/hadoop/"
REP="export JAVA_HOME=${JAVA_HOME}"
`sed -i "/^export JAVA_HOME=/c$REP" ${HADOOP_HOME}/etc/hadoop/hadoop-env.sh`
CONFS="core-site.xml hdfs-site.xml mapred-site.xml.template yarn-site.xml"

readonly CONFI1="

hadoop.tmp.dir
$HADOOP_HOME


fs.defaultFS
hdfs://${HOST_IP}:9000

"

RST=`mkdir $HADOOP_HOME"/tmp"`
RST=`mkdir $HADOOP_HOME"/tmp/name" $HADOOP_HOME"/tmp/data"`

readonly CONFI2="

dfs.replication
1


dfs.namenode.name.dir
$HADOOP_HOME/tmp/name


dfs.datanode.data.dir
$HADOOP_HOME/tmp/data

"

readonly CONFI3="

mapreduce.framework.name
yarn

"

readonly CINFI4="

yarn.nodemanager.aux-services
mapreduce_shuffle

"

IX=1
for file in $CONFS
do
	if [[ $file =~ .template ]]
	then
		CMD="mv $DEST_DIR$file $DEST_DIR${file%.*}"
		RST=`eval $CMD`
		file=${file%.*}
	fi
	CMD="sed -i '/<\/configuration>/i\REPL\' $DEST_DIR$file"
	V="CONFI"$IX
	LINES=`eval echo "$"$V`
	echo -n " ---- config $file ..."
	for line in $LINES
	do
		RST=${CMD/REPL/$line}
		RST=`eval $RST`	
	done
	echo " ok"
	((IX++)) 
done

#hadoop namenode format
echo -n " .... start to format namenode ..."
RST=`eval "cd $HADOOP_HOME/tmp/name"`
hdfs namenode -format>/root/nnfmt.log 2>&1
RST=`cat /root/nnfmt.log|grep util.ExitUtil`
RST=${RST##* }
echo $([ $RST -eq 0 ] && echo "ok" || "error")
if [ $RST -ne 0 ]
then
	exit 1
fi
echo
echo "Hadoop modify finish."
echo

---

5.Hive配置

• 创建目录warehouse
• 创建并修改hive-site.xml
• 拷贝mysql的jar包
• 初始化hive

#-------------------------hive-----------------------
echo "----------------------------------------------"
echo
echo "Let's modify hive..."
#-------------create warehouse----------------------
cd $HIVE_HOME
#验证路径是否正确
`eval "pwd>/root/test.log"`
VAR=`cat /root/test.log`
if [[ $VAR =~ hive110$ ]]
then
    echo -n ""
else
    echo "path error,please check curren path!"
    exit 1
fi
echo "currentpath: $VAR"
#创建目录warehouse
echo "first,we need to create directory:warehouse..."
mkdir warehouse/
echo "Directory warehouse create finish!"
echo

#-------------hive-site.xml-------------------------
echo "second,we'll modify hive-site.xml...."
cd ./conf
#验证路径是否正确
`eval "pwd>/root/test.log"`
VAR=`cat /root/test.log`
if [[ $VAR =~ conf$ ]]
then
    echo -n ""
else
    echo "path error,please check curren path!"
    exit 1
fi
echo "currentpath: $VAR"
TEXT="1.0\"#encoding=\"UTF-8\"#standalone=\"no\"?>
text/xsl\"#href=\"configuration.xsl\"?>


hive.metastore.warehouse.dir
$HIVE_HOME/warehouse


javax.jdo.option.ConnectionURL
jdbc:mysql://${HOST_IP}:3306/hive110?createDatabaseIfNotExist=true


javax.jdo.option.ConnectionDriverName
com.mysql.jdbc.Driver


javax.jdo.option.ConnectionUserName
root


javax.jdo.option.ConnectionPassword
root


"
#hive-site.xml不存在，创建并插入；考虑到sed无法插入空文件，先echo一行进入再sed插入，最后将那一行删除
touch hive-site.xml
`echo "#setup">./hive-site.xml`
lines=`echo $TEXT`
for line in $lines
do
    VAR=`eval "sed -i '/#setup/i\${line//#/ }' hive-site.xml"`
done
VAR=`awk -F'#' 'END{print $2}' hive-site.xml`
if [[ $VAR =~ ^setup ]]
then
    echo "modify hive-site.xml success"
else
    echo "modify hive-site.xml failed,please check shell..."
    exit 1
fi
sed -i '$d' hive-site.xml
echo
#---------------------------copy mysql-connection---------------
echo "Third,we should copy drive to hive..."
cd ../lib
#验证路径是否正确
`eval "pwd>/root/test.log"`
VAR=`cat /root/test.log`
if [[ $VAR =~ lib$ ]]
then
    echo -n ""
else
    echo "path error,please check curren path!"
    exit 1
fi
echo "currentpath: $VAR"
#拷贝jar包
cp /opt/download/hadoop/mysql*.jar ./
echo "copy  drive ok!"
echo

#-----------------------Initialize the environment---------------------------
echo "last,we must initialize the environment...."
cd ../bin
#验证路径是否正确
`eval "pwd>/root/test.log"`
VAR=`cat /root/test.log`
if [[ $VAR =~ bin$ ]]
then
    echo -n ""
else
    echo "path error,please check curren path!"
    exit 1
fi
echo "currentpath: $VAR"
echo "start to initialize..."
#初始化hive
VAR="./schematool -dbType mysql -initSchema>/root/test.log 2>&1"
`eval $VAR`
#验证初始化结果
VAR=`awk -F' ' 'END{print $2}' /root/test.log`
if [[ $VAR =~ ^completed ]]
then
    echo "initialize successfully"
else
    echo "initalize failed,please reinitalize..."
    exit 1
fi
echo
echo "hive Configurate finish!"
echo

---

6.Zeppelin配置

• 配置zeppelin-site.xml
• 配置zeppelin-env.xml
• 拷贝hive-site.xml&jar包

#---------------------zeppelin---------------------------
#--------------------authorize---------------------
echo "----------------------------------------------"
echo
echo "Next,we start to deploy zeppelin..."
#授权和上面重复，但不影响程序，可以删掉这一部分
echo "First,we need to authorize directory with its files...."
cd $ZEPPELIN_HOME
cd ../
`eval "pwd>/root/test.log"`
VAR=`cat /root/test.log`
if [[ $VAR =~ software$ ]]
then
    echo -n ""
else
    echo "path error,please check curren path!"
    exit 1
fi
echo "currentpath: $VAR"
VAR=`ls|grep zeppe*`
`chown -R root:root $VAR`
echo "authorized success!"
echo
#-----------modify zeppelin-site.xml---------
echo "Second,we need to modify zeppelin-site.xml..."
cd $ZEPPELIN_HOME
cd ./conf
`eval "pwd>/root/test.log"`
VAR=`cat /root/test.log`
if [[ $VAR =~ conf$ ]]
then
    echo -n ""
else
    echo "path error,please check curren path!"
    exit 1
fi
echo "currentpath: $VAR"
VAR=`ls|grep zeppelin-site*`
echo "We need to copy $VAR one more to modify"
cp $VAR ${VAR%.*}
echo "Copy success,next to modify zeppelin-site.xml..."
VAR="sed -i '24s/127.0.0.1/$HOST_IP/' zeppelin-site.xml"
`eval $VAR`
VAR="sed -i '30s/8080/8000/' zeppelin-site.xml"
`eval $VAR`
VAR=`awk 'NR==30{print $0}' zeppelin-site.xml`
VAR=`echo $VAR | tr -cd "[0-9]"`
if [[ $VAR =~ 8000 ]]
then
    echo "modify zeppelin-site.xml success!"
else
    echo  "modify zeppelin-site.xml failed,please check out..."
    exit 1
fi
echo

#-----------modify zeppelin-env.xml---------
echo "Then,let's modify zeppelin-env.sh..."
VAR=`ls|grep zeppelin-env.sh*`
echo "We need to copy $VAR one more to modify"
cp $VAR ${VAR%.*}
echo "Copy success,next to modify zeppelin-env.sh..."
#TEMP=`grep -n "# export JAVA_HOME=" zeppelin-env.sh`
#TEMP=`echo $a | tr -cd "[0-9]"`
#VAR="sed -i '${TEMP}s/#/ /' zeppelin-env.sh"
#`eval $VAR`
#VAR="sed -i '${TEMP}s/$""/${JAVA_HOME//\//\\/}' zeppelin-env.sh"
#`eval $VAR`
REP="export JAVA_HOME=${JAVA_HOME}"
`sed -i "/^# export JAVA_HOME=/c$REP" zeppelin-env.sh`
#TEMP=`grep -n "# export HADOOP_CONF_DIR.*" zeppelin-env.sh`
#TEMP=`echo $a | tr -cd "[0-9]"`
#VAR="sed -i '${TEMP}s/#/ /' zeppelin-env.sh"
#`eval $VAR`
#VAR="sed -i '${TEMP}s/"="/'='${HADOOP_HOME//\//\\/}' zeppelin-env.sh"
#`eval $VAR`
REP="export HADOOP_CONF_DIR=${HADOOP_HOME}"
`sed -i "/^# export HADOOP_CONF_DIR/c$REP" zeppelin-env.sh`
echo "modify zeppelin-env.sh success!"
echo
#--------------cp jar-----------------------
echo "Next,we need to copy .jar and .xml file..."
#拷贝xml和jar包
`eval "pwd>/root/test.log"`
VAR=`cat /root/test.log`
if [[ $VAR =~ conf$ ]]
then
    echo -n ""
else
    echo "path error,please check curren path!"
    exit 1
fi
echo "currentpath: $VAR"
`cp ${HIVE_HOME}/conf/hive-site.xml ./`
echo "copy hive-site.xml to current path success!"
`cp ${HADOOP_HOME}/share/hadoop/common/hadoop-common-2.6.0-cdh5.14.2.jar ./`
echo "echo hadoop.jar to current path success!"
`cp ${HIVE_HOME}/lib/hive-jdbc-1.1.0-cdh5.14.2-standalone.jar ./`
echo "copy hive.jar to current path success!"
echo
#-----------------start to switch on service-------------
echo "Finally,we'll start to switch on service..."
cd ../bin
`eval "pwd>/root/test.log"`
VAR=`cat /root/test.log`
if [[ $VAR =~ bin$ ]]
then
    echo -n ""
else
    echo "path error,please check curren path!"
    exit 1
fi
echo "currentpath: $VAR"
#启动服务
./zeppelin-daemon.sh start
echo
echo "Zeppelin modify finish."
echo

---

7.Zookeeper配置

• 创建directory mydata&myid
• 修改 zoo.cfg文件

#---------------------zookeeper--------------------------
echo "----------------------------------------------"
echo
echo "After,we start to modify zookeeper..."
echo "First,we need to create directory mydata and myid..."
cd $ZOOKEEPER_HOME
`eval "pwd>/root/test.log"`
VAR=`cat /root/test.log`
echo "currentpath: $VAR"
`mkdir mydata myid`
`echo "1">./myid`
echo "create directory finish!"
echo

echo "Second,we'll modify zoo.cfg..."
cd ./conf
`eval "pwd>/root/test.log"`
VAR=`cat /root/test.log`
if [[ $VAR =~ conf$ ]]
then
    echo -n ""
else
    echo "path error,please check curren path!"
    exit 1
fi
echo "currentpath: $VAR"
VAR=`ls|grep zoo.sample*`
echo "We need to copy $VAR one more to modify"
cp $VAR zoo.cfg
VAR="zoo.cfg"
echo "Copy success,next to modify $VAR..."
REP="dataDir=${ZOOKEEPER_HOME}/mydata"
`sed -i "/^dataDir=/c$REP" $VAR`
TEMP=`grep -n "dataDir=" zoo.cfg`
TEMP=`echo $a | tr -cd "[0-9]"`
SEN="sed -i 'N;${TEMP}aserver.1=master:2888:3888' $VAR"
`eval $SEN`
echo "Modify zoo.cfg success"
echo
echo "Zookeeper modify finish!"
echo

---

8.Hbase配置

• 修改hbase.env.sh
• 修改hbase-site.xml

#---------------------hbase------------------------------
echo "----------------------------------------------"
echo
echo "Afterwards,we start to deploy hbase"
echo "First,we'll modify hbase.env.sh..."
cd ${HBASE_HOME}/conf
`eval "pwd>/root/test.log"`
VAR=`cat /root/test.log`
if [[ $VAR =~ conf$ ]]
then
    echo -n ""
else
    echo "path error,please check current path!"
    exit 1
fi
echo "currentpath: $VAR"
REP="export JAVA_HOME=${JAVA_HOME}"
`sed -i "/^# export JAVA_HOME=/c$REP" hbase-env.sh`
REP="export HBASE_MANAGES_ZK=false"
`sed -i "/^# export HBASE_MANAGES_ZK/c$REP" hbase-env.sh`
echo "hbase-env.sh has been modified."
echo
echo "Second,we should modify hbase-site.xml..."
TEXT="
hbase.rootdir
hdfs://${HOST_IP}:9000/hbase


hbase.cluster.distributed
true


hbase.zookeeper.property.clientport
2181


hbase.zookeeper.property.dataDir
${ZOOKEEPER_HOME}/mydata

"
lines=`echo $TEXT`
for line in $lines
do
    VAR=`eval "sed -i '/<\/configuration>/i\$line' hbase-site.xml"`
done
echo "hbase-site.xml has been modified."
echo
echo "Hbase  deploy finish!"

---

9.Sqoop配置

• 拷贝jar包
• *配置sqoop-env.sh

#----------------------sqoop-------------------------
echo
echo "----------------------------------------"
echo "Let's start to deploy sqoop..."
echo "First,we need to copy more .jar to path of lib..."
cd ${SQOOP_HOME}/lib
`eval "pwd>/root/test.log"`
VAR=`cat /root/test.log`
if [[ $VAR =~ lib$ ]]
then
    echo -n ""
else
    echo "path error,please check current path!"
    exit 1
fi
echo "currentpath: $VAR"
`cp /opt/download/hadoop/mysql-connector-java-5.1.32.jar ./`
echo "copy mysql-connector-java-5.1.32.jar to current path success!"
`cp ${HADOOP_HOME}/share/hadoop/common/hadoop-common-2.6.0-cdh5.14.2.jar ./`
echo "copy hadoop-common-2.6.0-cdh5.14.2.jar to current path success!"
`cp ${HADOOP_HOME}/share/hadoop/hdfs/hadoop-hdfs-2.6.0-cdh5.14.2.jar ./`
echo "copy hadoop-hdfs-2.6.0-cdh5.14.2.jar to current path success!"
`cp ${HADOOP_HOME}/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.0-cdh5.14.2.jar ./`
echo "copy hadoop-mapreduce-client-core-2.6.0-cdh5.14.2.jar to current path success!"
echo
echo "Second ,we start to modify sqoop.env.sh..."
cd ../conf
`eval "pwd>/root/test.log"`
VAR=`cat /root/test.log`
if [[ $VAR =~ conf$ ]]
then
    echo -n ""
else
    echo "path error,please check current path!"
    exit 1
fi
echo "currentpath: $VAR"
VAR=`ls|grep sqoop-env-template.sh`
echo "We need to copy $VAR one more to modify"
`cp $VAR sqoop-env.sh`
echo "Copy success,next to modify sqoop-env.sh..."
REP="export HADOOP_COMMON_HOME=$HADOOP_HOME"
`sed -i "/^#export HADOOP_COMMON_HOME=/c$REP" sqoop-env.sh`
REP="export HADOOP_MAPRED_HOME=${HADOOP_HOME}/share/common/mapreuce"
`sed -i "/^#export HADOOP_MAPRED_HOME=/c$REP" sqoop-env.sh`
REP="export HBASE_HOME=$HBASE_HOME"
`sed -i "/^#export HBASE_HOME=/c$REP" sqoop-env.sh`
REP="export HIVE_HOME=$HIVE_HOME"
`sed -i "/^#export HIVE_HOME=/c$REP" sqoop-env.sh`
REP="export ZOOCFGDIR=$ZOOKEEPER_HOME"
`sed -i "/^#export ZOOCFGDIR=/c$REP" sqoop-env.sh`
echo "sqoop-env.sh has been modified."
echo
echo "sqoop deploy finish!"
echo
echo "----------------------------------------------"

---

PS：如果有写错或者写的不好的地方，欢迎各位大佬在评论区留下宝贵的意见或者建议，敬上！如果这篇博客对您有帮助，希望您可以顺手帮我点个赞！不胜感谢！

原创作者：wsjslient

作者主页：https://blog.csdn.net/wsjslient

---