Centos7 安装 CDH6.3.2

1. 安装包

1.1 CM

https://archive.cloudera.com/cm6/6.3.1/repo-as-tarball/cm6.3.1-redhat7.tar.gz.md5
https://archive.cloudera.com/cm6/6.3.1/allkeys.asc

1.2 CDH

https://archive.cloudera.com/cdh6/6.3.2/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel
https://archive.cloudera.com/cdh6/6.3.2/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1
https://archive.cloudera.com/cdh6/6.3.2/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha256
https://archive.cloudera.com/cdh6/6.3.2/parcels/manifest.json

2. 服务器配置

2.1 hosts配置

127.0.0.1   localhost localhost
192.168.5.246   dev-node01
192.168.5.247   dev-node02
192.168.5.248   dev-node03

2.2 关闭iptables、firewalld、selinux

2.3 免密登录（主节点到所有节点，secondnode同样要求）

2.4 对时ntpd

2.5 关闭swap、禁用透明页，写入/etc/profile

echo 0 >  /proc/sys/vm/swappiness
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag
swapoff -a

2.6 安装jdk1.8

2.7 下载mysql-jdbc包

mkdir /usr/share/java/
wget http://central.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar
mv mysql-connector-java-8.0.16.jar mysql-connector-java.jar

2.8 mysql 数据库创建

create database cmserver default charset utf8 collate utf8_general_ci;
grant all on cmserver.* to 'cmserveruser'@'%' identified by 'root';

create database hive default charset utf8 collate utf8_general_ci;
grant all on hive.* to 'hiveuser'@'%' identified by 'root';

create database amon default charset utf8 collate utf8_general_ci;
grant all on amon.* to 'amonuser'@'%' identified by 'root';

create database rman default charset utf8 collate utf8_general_ci;
grant all on rman.* to 'rmanuser'@'%' identified by 'root';

create database oozie default charset utf8 collate utf8_general_ci;
grant all on oozie.* to 'oozieuser'@'%' identified by 'root';

create database hue default charset utf8 collate utf8_general_ci;
grant all on hue.* to 'hueuser'@'%' identified by 'root';

2.9 安装httpd服务

yum install httpd
systemctl start httpd
systemctl enable httpd.service

3. CM安装

3.1 配置本地yum源

mkdir -p /var/www/html/cloudera-repos/
tar -xzvf cm6.3.1-redhat7.tar.gz -C /var/www/html/cloudera-repos/
wget https://archive.cloudera.com/cm6/6.3.1/allkeys.asc

vim /etc/yum.repos.d/cloudera-manager.repo
[cloudera-manager]
name = Cloudera Manager, Version cm6.3.1
baseurl = http://dev-node01/cloudera-repos/cm6.3.1
gpgcheck = 1

yum clean all
yum makecache

rpm --import http://dev-node01/cloudera-repos/cm6.3.1/RPM-GPG-KEY-cloudera

3.2 安装cm

yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server
# 生成sha
sha1sum CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel | awk '{print $1}' > CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha

 

 3.3 配置cm

# 初始化数据库
/opt/cloudera/cm/schema/scm_prepare_database.sh  mysql -h192.168.5.231 -P3306 cmserver cmserveruser root
# 启动cm
systemctl restart cloudera-scm-server

3.4 web安装步骤（admin/admin） 

 

4. FAQ 

4.1  fail to format namenode

问题原因
是由于之前初始化 namenode 在 /dfs/nn 留下了残留数据（失效数据），从而影响再次初始化

解决方法

namenode节点：
rm -rf /dfs/nn

datanode节点：
rm -rf /dfs/dn

4.2 主机运行状况不良

rm -f /var/lib/cloudera-scm-agent/cm_guid
systemctl restart cloudera-scm-agent