大数据之---CDH集群离线部署全网终极篇--更新中

1、软件环境和IP规划

RHEL6	角色	jdk-8u45apache-maven-3.3.9hive-1.1.0-cdh5.7.1-src.tar.gz
hadoop-2.8.1.tar.gz	mysql-connector-java-6.0.6.tar.gz
apache-maven-3.3.9	cloudera-manager-el6-cm5.9.3_x86_64.tar	mysql-5.1
CDH-5.9.3-1.cdh5.9.3.p0.4-el6
172.16.18.133	NN && SN && Jobtrack	hadoop01
172.16.18.134	DN && tasktrack	hadoop02
172.16.18.136	DN && tasktrack	hadoop03
172.16.18.143	DN && tasktrack	hadoop04
172.16.18.145	DN && tasktrack	hadoop05

NN=namenode    SN=secondarynamenode  DN=datanode

 

集群介绍：

不收费的Hadoop版本主要有三个（均是国外厂商），分别是：Apache（最原始的版本，所有发行版均基于这个版本进行改进）、Cloudera版本（Cloudera’sDistribution Including Apache Hadoop，简称CDH）、Hortonworks版本(HortonworksData Platform，简称“HDP”），对于国内而言，绝大多数选择CDH版本。

CDH(Cloudera's Distribution, including Apache Hadoop)，是Hadoop众多分支中的一种，由Cloudera维护，基于稳定版本的ApacheHadoop构建，并集成了很多补丁，可直接用于生产环境。

ClouderaManager则是为了便于在集群中进行Hadoop等大数据处理相关的服务安装和监控管理的组件，对集群中主机、Hadoop、Hive、Spark等服务的安装配置管理做了极大简化。

 

 

集群安装，本文采用选择 离线安装CDH

https://www.cloudera.com/downloads/cdh/5-9-0.html 

官网对CDH的描述，CHD对systemJDK database 等版本支持列表

官网看支持jdk1.8但是部分1.8版本会报错，所以我们选择jdk1.7

2、软件包装备

ClouderaManager软件包

http://archive.cloudera.com/cm5/cm/5/cloudera-manager-el6-cm5.9.3_x86_64.tar.gz

CDH软件包 （下载对应Linux版本包）

http://archive.cloudera.com/cdh5/parcels/5.9.3/CDH-5.9.3-1.cdh5.9.3.p0.4-el6.parcel

http://archive.cloudera.com/cdh5/parcels/5.9.3/CDH-5.9.3-1.cdh5.9.3.p0.4-el6.parcel.sha1

mysqljdbc驱动版本是：

http://download.softagency.net/MySQL/Downloads/Connector-J/mysql-connector-java-6.0.6.tar.gz

 3、系统相关配置

所有主机相同   安装JDK关闭selinux  iptables 配置/etc/hosts 配置yum

 [root@hadoop01 ~]# vim /etc/profile

[root@hadoop01 ~]# vim /etc/profile
exportJAVA_HOME=/usr/java/jdk1.7.0_79
exportPATH=$JAVA_HOME/bin:$ORACLE_HOME/bin:$R_HOME/bin:$PATH
[root@hadoop01~]# getenforce
Disabled
[root@hadoop01~]# iptables -L
ChainINPUT (policy ACCEPT)
target     prot opt source               destination         
ChainFORWARD (policy ACCEPT)
target     prot opt source               destination        
ChainOUTPUT (policy ACCEPT)
target     prot opt source               destination        
[root@hadoop01~]# cat /etc/hosts
127.0.0.1   localhost
172.16.18.133    hadoop01 
172.16.18.134    hadoop02
172.16.18.136    hadoop03
172.16.18.143    hadoop04
172.16.18.145    hadoop05
[root@hadoop01~]# hostname

hadoop01

exportJAVA_HOME=/usr/java/jdk1.7.0_79

exportPATH=$JAVA_HOME/bin:$ORACLE_HOME/bin:$R_HOME/bin:$PATH

[root@hadoop01~]# getenforce

Disabled

[root@hadoop01~]# iptables -L

ChainINPUT (policy ACCEPT)

target     prot opt source               destination         

ChainFORWARD (policy ACCEPT)

target     prot opt source               destination        

ChainOUTPUT (policy ACCEPT)

target     prot opt source               destination        

[root@hadoop01~]# cat /etc/hosts

127.0.0.1   localhost

172.16.18.133   hadoop01 

172.16.18.134    hadoop02

172.16.18.136    hadoop03

172.16.18.143    hadoop04

172.16.18.145    hadoop05

[root@hadoop01~]# hostname

hadoop01

4、配置ssh自动登录互信

参考伪分布式ssh互信配置

每个节点验证不需要进行交互输入yes

useraddhadoop   ----建立用户

sshhadoop01 date

sshhadoop02 date

sshhadoop03 date

sshhadoop04 date

sshhadoop05 date

 

5、修改swap空间的swappiness=0

cat  /proc/sys/vm/swappiness

sysctlvm.swappiness=0

echo0 > /proc/sys/vm/swappiness

关闭告警：echonever > /sys/kernel/mm/transparent_hugepage/defrag

 

6、配置NTP服务器

先选定主服务器,其他服务器都同步这台主服务器的时间

#hwclock -w

配置开机启动

[root@hadoop01~]# chkconfig ntpd on
[root@hadoop01~]# chkconfig --list ntpd
[root@hadoop01~]#vi /etc/ntp.conf
    (找到这一行,放开restrict的注释,并且修改ip地址)
    # Hosts on local network are lessrestricted.
    restrict 192.168.128.1 mask 255.255.255.0nomodify notrap
    (找到这一行,注释下面的server)
 vi /etc/ntp.confdriftfile /var/lib/ntp/drift
restrict default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery
restrict 127.0.0.1 
restrict -6 ::1
restrict 172.16.18.133 mask 255.255.255.0 nomodify notrap noquery
server	hadoop01	# local clock
fudge	127.127.1.0 stratum 10	
includefile /etc/ntp/crypto/pw
keys /etc/ntp/keys
[root@hadoop01 ~]# vim /etc/profile
exportJAVA_HOME=/usr/java/jdk1.7.0_79
exportPATH=$JAVA_HOME/bin:$ORACLE_HOME/bin:$R_HOME/bin:$PATH
[root@hadoop01~]# getenforce
Disabled
[root@hadoop01~]# iptables -F
[root@hadoop01~]# /etc/init.d/iptables stop

所有重启ntp服务
[root@hadoop01~]# service ntpd restart
[root@hadoop01~]# ntpstat 
synchronisedto NTP server (172.16.18.33) at stratum 12
timecorrect to within 18 ms
pollingserver every 64 s
[root@hadoop01~]# date

7、禁用ipv6和“透明大页面”

[root@hadoop01~]# echo "alias ipv6 off" >> /etc/modprobe.d/dist.conf 
[root@hadoop01~]# echo "alias net-pf-10 off" >> /etc/modprobe.d/dist.conf
[root@hadoop01~]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@hadoop01~]# echo 'echo never > /sys/kernel/mm/transparent_hugepage/defrag' >>/etc/rc.local
[root@hadoop01~]#

8、准备好mysql数据库

修改mysql 权限：
GRANTALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123' WITH GRANT OPTION;
flushprivileges;
delete from user where host !='%';
[root@hadoop01software]# mysql -h 172.16.18.133  -uroot -p
 

 

###############################准备工作########################################

hadoop01        Server || Agent    
hadoop02        Agent     
hadoop03        Agent  

CDH采用3台服务器，剩下2台做集群添加节点使用

######################################################################

10.CM安装

安装clouderaManager Server、Agent

cdh集群节点都要安装软件准备账号建立

 

主节点：
[root@hadoop01software]# ls cloudera-manager-el6-cm5.9.3_x86_64.tar.gz
cloudera-manager-el6-cm5.9.3_x86_64.tar.gz
[root@hadoop01software]# pwd
/opt/software
[root@hadoop01software]# ls cloudera-manager-el6-cm5.9.3_x86_64.tar.gz
cloudera-manager-el6-cm5.9.3_x86_64.tar.gz
[root@hadoop01software]# mkdir /opt/cloudera-manager
[root@hadoop01software]# tar zxvf cloudera-manager-el6-cm5.9.3_x86_64.tar.gz -C/opt/cloudera-manager/
客户端配置
/opt/cloudera-manager/cm-5.9.3/etc/cloudera-scm-agent/config.ini
server_host=hadoop01   ---在cmserver主机名
[root@hadoop01software]# useradd --system --no-create-home --shell=/bin/false --comment"Cloudera SCM User" cloudera-scm
[root@hadoop01software]# id cloudera-scm
uid=495(cloudera-scm)gid=492(cloudera-scm) 组=492(cloudera-scm)

所有从节点
useradd--system --no-create-home --shell=/bin/false --comment "Cloudera SCMUser" cloudera-scm
mkdir /opt/cloudera-manager
[root@hadoop01opt]# scp -r /opt/cloudera-manager/cm-5.9.3 hadoop02:/opt/cloudera-manager/
[root@hadoop01opt]# scp -r /opt/cloudera-manager/cm-5.9.3 hadoop03:/opt/cloudera-manager/

11、配置CMServer数据库

我们开始准备mysql数据库建立

[root@hadoop01~]# mysql -h172.16.18.133 -uroot -p
mysql>
GRANTALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123' WITH GRANT OPTION;
flushprivileges;
mysql>flush privileges;
[root@hadoop01schema]# pwd
/opt/cloudera-manager/cm-5.9.3/share/cmf/schema
[root@hadoop01schema]# ./scm_prepare_database.sh  mysql-hhadoop01 -uroot -p123 --scm-host hadoop01 cmdb root 123
JAVA_HOME=/usr/java/jdk1.7.0_79
Verifyingthat we can write to /opt/cloudera-manager/cm-5.9.3/etc/cloudera-scm-server
SatApr 28 14:20:38 CST 2018 WARN: Establishing SSL connection without server'sidentity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+and 5.7.6+ requirements SSL connection must be established by default ifexplicit option isn't set. For compliance with existing applications not usingSSL the verifyServerCertificate property is set to 'false'. \nYou need either toexplicitly disable SSL by setting useSSL=false, or set useSSL=true and providetruststore for server certificate verification.
CreatingSCM configuration file in/opt/cloudera-manager/cm-5.9.3/etc/cloudera-scm-server
Executing:  /usr/java/jdk1.7.0_79/bin/java -cp/usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/opt/cloudera-manager/cm-5.9.3/share/cmf/schema/../lib/*com.cloudera.enterprise.dbutil.DbCommandExecutor/opt/cloudera-manager/cm-5.9.3/etc/cloudera-scm-server/db.propertiescom.cloudera.cmf.db.
SatApr 28 14:20:39 CST 2018 WARN: Establishing SSL connection without server'sidentity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+and 5.7.6+ requirements SSL connection must be established by default ifexplicit option isn't set. For compliance with existing applications not usingSSL the verifyServerCertificate property is set to 'false'. You need either toexplicitly disable SSL by setting useSSL=false, or set useSSL=true and providetruststore for server certificate verification.
[   main] DbCommandExecutor              INFO  Successfully connected to database.
Alldone, your SCM database is configured correctly!

说明：这个脚本就是用来创建和配置CMS需要的数据库的脚本。各参数是指：

mysql：数据库用的是mysql，如果安装过程中用的oracle，那么该参数就应该改为oracle。

-hadoop01：数据库建立在hadoop01主机上面,也就是主节点上面。

-uroot：root身份运行mysql。-123：mysql的root密码是

--scm-hosthadoop01 ：CMS的主机，一般是和mysql安装的主机是在同一个主机上,

最后三个参数是：数据库名，数据库用户名，数据库密码。

 

12、制作CDH本地源

Server节点
mkdir-p /opt/cloudera/parcel-repo
chowncloudera-scm:cloudera-scm /opt/cloudera/parcel-repo
Agent节点
mkdir-p /opt/cloudera/parcels
chowncloudera-scm:cloudera-scm /opt/cloudera/parcels

上传到CDH-5.9.3-1.cdh5.9.3.p0.4-el6.parcel   manifest.json主节点/opt/cloudera/parcel-repo/路径
[root@hadoop01CDH]# cd /opt/cloudera/parcel-repo/
[root@hadoop01parcel-repo]# ls
CDH-5.9.3-1.cdh5.9.3.p0.4-el6.parcel   manifest.json
[root@hadoop01parcel-repo]# ls
CDH-5.9.3-1.cdh5.9.3.p0.4-el6.parcel  manifest.json
[root@hadoop01parcel-repo]# mv manifest.json CDH-5.9.3-1.cdh5.9.3.p0.4-el6.parcel.sha
manifest.json改名文件名与你的parel包名一致，并加上.sha后缀

 

 

13、启动

确保mysql先启动
server：hadoop01
[[email protected]]# pwd
/opt/cloudera-manager/cm-5.9.3/etc/init.d
[[email protected]]# ./cloudera-scm-server start
Startingcloudera-scm-server:
agent:hadoop01hadoop02 hadoop02
/opt/cloudera-manager/cm-5.9.3/etc/init.d
./cloudera-scm-agent  start
正在启动cloudera-scm-agent：                              [确定]

2018-04-2814:43:37,022 INFO WebServerImpl:org.mortbay.log: jetty-6.1.26.cloudera.4

2018-04-2814:43:37,024 INFO WebServerImpl:org.mortbay.log: [email protected]:7180

2018-04-2814:43:37,024 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: StartedJetty server.

出现下面内容表示启动成功

14、图形访问进行安装 http://ip:7180 

CM版本太新显示不出来

 

解决里面提示的任何警告和错误，全部都正常进行下一步

 

 

 

这里所有服务器全部安装好

 

 

 

根据提前规划设置好角色分配

配置数据库界面，我们的数据没有，建立几个数据库

--hive数据库
create database hive2 DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
--oozie数据库
create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
---Activity Monitor数据库
create database amoni DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
--hue数据库
create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

 

检查没有通过，因为jdbc驱动问题
hadoop02 hadoop03没有mysql连接驱动，解决完成mysql连接驱动安装，可以直接
yum install *mysql-connect* -y

全部测试成功继续

 

 

 

 

 

遇到失败，重新调整分配角色到不同的主机上面

全部正常进行下一步继续

全部完成安装 进入集群监控管理页面

 

错误大全：

问题1：JDBCdriver驱动

[root@hadoop01schema]# ./scm_prepare_database.sh mysql cmdb -h hadoop01 -uroot -p123456--scm-host hadoop01 scm scm scm
JAVA_HOME=/usr/java/jdk1.7.0_79
Verifyingthat we can write to /opt/cloudera-manager/cm-5.9.3/etc/cloudera-scm-server
[              main] DbProvisioner                  ERROR Unable to find theMySQL JDBC driver. Please make sure that you have installed it as perinstruction in the installation guide.
[             main]DbProvisioner                  ERRORStack Trace:
java.lang.ClassNotFoundException:com.mysql.jdbc.Driver
 atjava.net.URLClassLoader$1.run(URLClassLoader.java:366)[:1.7.0_79]
 atjava.net.URLClassLoader$1.run(URLClassLoader.java:355)[:1.7.0_79]
 at java.security.AccessController.doPrivileged(NativeMethod)[:1.7.0_79]
 atjava.net.URLClassLoader.findClass(URLClassLoader.java:354)[:1.7.0_79]

解决方法下载连接
[root@hadoop01software]# ls mysql-connector-java-5.1.46.zip
mysql-connector-java-5.1.46.zip
[root@hadoop01software]# unzip  mysql-connector-java-5.1.46.zip^C
[root@hadoop01software]# cp mysql-connector-java-5.1.46/
build.xml                           mysql-connector-java-5.1.46-bin.jar README.txt
CHANGES                             mysql-connector-java-5.1.46.jar     src/
COPYING                              README                              
[root@hadoop01software]# cp mysql-connector-java-5.1.46/mysql-connector-java-5.1.46.jar  /usr/share/java/
[root@hadoop01software]# mv /usr/share/java/mysql-connector-java-5.1.46.jar /usr/share/java/mysql-connector-java.jar

问题2：mysql 权限限制
dbcurl 'jdbc:mysql://hadoop01/?useUnicode=true&characterEncoding=UTF-8'
java.sql.SQLException:Access denied for user 'root'@'hadoop01' (using password: YES)
 atcom.mysql.jdbc.SQLError.createSQLException(SQLError.java:965)[mysql-connector-java.jar:5.1.46]

重新进行grant授权

问题3：空白页面点击无响应
登陆页面 输入admin /admin完全无反应
更换浏览器重新访问