CDH 5.16.2 部署
目录
-
-
- 1. 部署的方式
-
- 1.1 bin在线部署
- 1.2 rpm 部署
- 1.3 tar 部署
- 2. 部署前置条件
- 3. 部署初始化集群
- 4. 部署 JDK
- 5. 部署 Mysql
- 6. 部署 Cloudera Manager
-
- 6.1 部署 Cloudera Manager 前置条件
- 6.2 部署 Cloudera Manager
-
Cloudera 官网
- 本次我们部署环境为 CDH 5.16.2
- 本次部署使用的是三台 2 核 16 G 机器
1. 部署的方式
1.1 bin在线部署
- 优点：通过 shell 脚本一键部署，比较方便。
- 缺点：需要互联网在线下载安装包，如果网络抖动则不会成功。
1.2 rpm 部署
- 优点：官网推荐的，部署还算比较简单。
- 缺点：部署 Server、Agent rpm 时候需要网络下载依赖包。
1.3 tar 部署
- 真正的离线部署，本次部署就是采用 tar 部署。
2. 部署前置条件
CDH 5X 部署的硬件条件
- 本次部署采用的是阿里云 Centos 7.2 的操作系统
3. 部署初始化集群
- 所有的机器都需要执行，把内网 IP 加入到 hosts 文件里面
[root@bigdata001 ~]# echo "172.22.95.62 bigdata001">> /etc/hosts
[root@bigdata001 ~]# echo "172.22.95.63 bigdata002">> /etc/hosts
[root@bigdata001 ~]# echo "172.22.95.64 bigdata003">> /etc/hosts
- 关闭防火墙
[root@bigdata001 ~]# systemctl stop firewalld
[root@bigdata001 ~]# systemctl disable firewalld
- 关闭 selinux
[root@bigdata001 ~]# vim /etc/selinux/config
SELINUX=disabled
- 时区同步（如果使用阿里云机器，可以不用做）
[root@bigdata001 ~]# timedatectl set-timezone Asia/Shanghai
- 时钟同步（如果使用阿里云机器，可以不用做）
- 所有机器都需要安装
[root@bigdata001 ~]# yum install -y ntp
- 选取第一台机器作为主机器
- 下面只有第一台机器需要执行
[root@bigdata001 ~]# vim /etc/ntp.conf
#time
server 0.asia.pool.ntp.org
server 1.asia.pool.ntp.org
server 2.asia.pool.ntp.org
server 3.asia.pool.ntp.org
#当外部时间不可⽤时，可使⽤本地硬件时间
server 127.127.1.0 iburst local clock
#允许哪些⽹段的机器来同步时间
restrict 172.22.95.0 mask 255.255.255.0 nomodify notrap
- 第一台机器启动 ntp 服务
[root@bigdata001 ~]# systemctl start ntpd
[root@bigdata001 ~]# systemctl status ntpd
● ntpd.service - Network Time Service
Loaded: loaded (/usr/lib/systemd/system/ntpd.service; enabled; vendor preset: disabled)
Active: active (running) since Wed 2021-02-10 12:15:22 CST; 6min ago
Main PID: 9479 (ntpd)
CGroup: /system.slice/ntpd.service
└─9479 /usr/sbin/ntpd -u ntp:ntp -g
Feb 10 12:15:22 bigdata001 systemd[1]: Starting Network Time Service...
Feb 10 12:15:22 bigdata001 ntpd[9479]: proto: precision = 0.080 usec
Feb 10 12:15:22 bigdata001 ntpd[9479]: 0.0.0.0 c01d 0d kern kernel time sync enabled
Feb 10 12:15:22 bigdata001 systemd[1]: Started Network Time Service.
[root@bigdata001 ~]# ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
LOCAL(0) .LOCL. 10 l 411 64 100 0.000 0.000 0.000
+120.25.115.20 10.137.53.7 2 u 12 64 177 24.430 1081.30 8.310
10.143.33.49 .INIT. 16 u - 1024 0 0.000 0.000 0.000
+100.100.3.1 10.137.55.181 2 u 43 64 377 1.482 1081.81 6.647
+100.100.3.2 10.137.55.181 2 u 43 64 377 1.332 1081.10 7.352
+100.100.3.3 10.137.55.181 2 u 38 64 377 1.470 1080.94 7.428
+203.107.6.88 10.137.38.86 2 u 38 64 377 16.084 1081.27 6.672
10.143.33.50 .INIT. 16 u - 1024 0 0.000 0.000 0.000
10.143.33.51 .INIT. 16 u - 1024 0 0.000 0.000 0.000
10.143.0.44 .INIT. 16 u - 1024 0 0.000 0.000 0.000
10.143.0.45 .INIT. 16 u - 1024 0 0.000 0.000 0.000
10.143.0.46 .INIT. 16 u - 1024 0 0.000 0.000 0.000
+100.100.5.1 10.137.55.181 2 u 29 64 377 1.545 1081.31 7.467
+100.100.5.2 10.137.55.181 2 u 35 64 377 1.819 1082.29 6.611
*100.100.5.3 10.137.55.181 2 u 30 64 377 1.538 1082.21 6.687
- 其他节点禁用服务
[root@bigdata002 ~]# systemctl stop ntpd
[root@bigdata002 ~]# systemctl disable ntpd
Removed symlink /etc/systemd/system/multi-user.target.wants/ntpd.service.
[root@bigdata002 ~]# ntpdate bigdata001
10 Feb 12:24:33 ntpdate[9517]: step time server 172.22.95.62 offset -0.740151 sec
- 同步服务增加调度
[root@bigdata002 ~]# crontab -e
00 00 * * * ntpdate bigdata
4. 部署 JDK
- 上传 JDK 的 tar 包
[root@bigdata001 ~]# mkdir software
[root@bigdata001 ~]# cd software/
[root@bigdata001 software]# ll
total 185292
-rw-r--r-- 1 root root 189736377 Oct 20 21:13 jdk-8u151-linux-x64.tar.gz
[root@bigdata001 software]# cp jdk-8u151-linux-x64.tar.gz /tmp/
[root@bigdata001 software]# cd /tmp/
[root@bigdata001 tmp]# scp jdk-8u151-linux-x64.tar.gz bigdata002:/tmp
[root@bigdata001 tmp]# scp jdk-8u151-linux-x64.tar.gz bigdata003:/tmp
- 三台都执行
[root@bigdata001 ~]# cd /tmp/
[root@bigdata001 tmp]# mkdir /usr/java
[root@bigdata001 tmp]# tar -xzvf jdk-8u151-linux-x64.tar.gz -C /usr/java
[root@bigdata001 tmp]# cd /usr/java/
[root@bigdata001 java]# ll
total 4
drwxr-xr-x 8 10 143 4096 Sep 6 2017 jdk1.8.0_151
[root@bigdata001 java]# chown root:root jdk1.8.0_151/
[root@bigdata001 java]# chown -R root:root jdk1.8.0_151/*
[root@bigdata001 java]# chmod 777 jdk1.8.0_151/
[root@bigdata001 java]# chmod -R 777 jdk1.8.0_151/*
[root@bigdata001 java]# ll
total 4
drwxrwxrwx 8 root root 4096 Sep 6 2017 jdk1.8.0_151
[root@bigdata001 java]# cd ~
[root@bigdata001 ~]# vim .bashrc
#### add env
export JAVA_HOME=/usr/java/jdk1.8.0_151
export PATH=${JAVA_HOME}/bin:${PATH}
[root@bigdata001 ~]# source .bashrc
[root@bigdata001 ~]# which java
/usr/java/jdk1.8.0_151/bin/java
5. 部署 Mysql
Mysql 下载路径
[root@bigdata001 ~]# cd software/
[root@bigdata001 software]# cp mysql-5.7.31-linux-glibc2.12-x86_64.tar.gz /usr/local
[root@bigdata001 software]# cd /usr/local/
[root@bigdata001 local]# tar -xzvf mysql-5.7.31-linux-glibc2.12-x86_64.tar.gz
- 创建mysql用户，并修改Mysql文件权限
[root@bigdata001 local]# useradd mysql
[root@bigdata001 local]# id mysql
uid=1000(mysql) gid=1000(mysql) groups=1000(mysql)
[root@bigdata001 local]# chown mysql:mysql mysql-5.7.31-linux-glibc2.12-x86_64
[root@bigdata001 local]# chown -R mysql:mysql mysql-5.7.31-linux-glibc2.12-x86_64/*
[root@bigdata001 local]# ln -s /usr/local/mysql-5.7.28-linux-glibc2.12-x86_64 mysql
- 修改 Mysql 配置文件
[root@bigdata001 local]# cat /dev/null >/etc/my.cnf
[root@bigdata001 local]# vim /etc/my.cnf
[client]
port=3306
socket=/usr/local/mysql/data/mysql.sock
default-character-set=utf8
[mysqld]
port=3306
socket=/usr/local/mysql/data/mysql.sock
character-set-server=utf8
skip-external-locking
key_buffer_size=256M
sort_buffer_size=2M
read_buffer_size=2M
read_rnd_buffer_size=4M
query_cache_size=32M
max_allowed_packet=16M
myisam_sort_buffer_size=128M
tmp_table_size=32M
table_open_cache=512
thread_cache_size=8
wait_timeout=86400
interactive_timeout=86400
max_connections=600
# Try number of CPU's*2 for thread_concurrency
# thread_concurrency=32
#isolation level and default engine
default-storage-engine=INNODB
transaction-isolation=READ-COMMITTED
server-id=1
basedir=/usr/local/mysql
datadir=/usr/local/mysql/data
pid-file=/usr/local/mysql/data/hostname.pid
#open performance schema
log-warnings
sysdate-is-now
binlog_format=MIXED
log_bin_trust_function_creators=1
log-error=/usr/local/mysql/data/hostname.err
log-bin=/usr/local/mysql/arch/mysql-bin
#other logs
#general_log =1
#general_log_file=/usr/local/mysql/data/general_log.err
#slow_query_log=1
#slow_query_log_file=/usr/local/mysql/data/slow_log.err
#for replication slave
#log-slave-updates
#sync_binlog = 1
#for innodb options
innodb_data_home_dir=/usr/local/mysql/data/
innodb_data_file_path=ibdata1:500M:autoextend
innodb_log_group_home_dir=/usr/local/mysql/arch
innodb_log_files_in_group=2
innodb_log_file_size=200M
innodb_buffer_pool_size=1024M
# innodb_additional_mem_pool_size=50M
innodb_log_buffer_size=16M
innodb_lock_wait_timeout = 100
#innodb_thread_concurrency = 0
innodb_flush_log_at_trx_commit = 1
innodb_locks_unsafe_for_binlog=1
#innodb io features: add for mysql5.5.8
performance_schema
innodb_read_io_threads=4
innodb-write-io-threads=4
innodb-io-capacity=200
#purge threads change default(0) to 1 for purge
innodb_purge_threads=1
innodb_use_native_aio=on
#case-sensitive file names and separate tablespace
innodb_file_per_table=1
lower_case_table_names=1
[mysqldump]
quick
max_allowed_packet=16M
[mysql]
no-auto-rehash
default-character-set=utf8
[mysqlhotcopy]
interactive-timeout
[myisamchk]
key_buffer_size=256M
sort_buffer_size=256M
read_buffer=2M
write_buffer=2M
[root@bigdata001 local]# chown mysql:mysql /etc/my.cnf
[root@bigdata001 local]# chmod 640 /etc/my.cnf
[root@bigdata001 local]# ll /etc/my.cnf
-rw-r----- 1 mysql mysql 2169 Oct 23 14:33 /etc/my.cnf
- 初始化 Mysql
[root@bigdata001 local]# yum -y install libaio
[root@bigdata001 local]# cd mysql
[root@bigdata001 mysql]# ll
total 304
drwxr-xr-x 2 mysql mysql 4096 Jun 2 2020 bin
drwxr-xr-x 2 mysql mysql 4096 Jun 2 2020 docs
drwxr-xr-x 3 mysql mysql 4096 Jun 2 2020 include
drwxr-xr-x 5 mysql mysql 4096 Jun 2 2020 lib
-rw-r--r-- 1 mysql mysql 275393 Jun 2 2020 LICENSE
drwxr-xr-x 4 mysql mysql 4096 Jun 2 2020 man
-rw-r--r-- 1 mysql mysql 587 Jun 2 2020 README
drwxr-xr-x 28 mysql mysql 4096 Jun 2 2020 share
drwxr-xr-x 2 mysql mysql 4096 Jun 2 2020 support-files
[root@bigdata001 mysql]# su mysql
[mysql@bigdata001 mysql]$ mkdir data tmp arch
[mysql@bigdata001 mysql]$ exit
exit
[root@bigdata001 mysql]# /usr/local/mysql/bin/mysqld --user=mysql --basedir=/usr/local/mysql/ --datadir=/usr/local/mysql/data --initialize
[root@bigdata001 mysql]# cd data/
[root@bigdata001 data]# ll
total 512068
-rw-r----- 1 mysql mysql 56 Feb 10 13:26 auto.cnf
-rw------- 1 mysql mysql 1680 Feb 10 13:26 ca-key.pem
-rw-r--r-- 1 mysql mysql 1112 Feb 10 13:26 ca.pem
-rw-r--r-- 1 mysql mysql 1112 Feb 10 13:26 client-cert.pem
-rw------- 1 mysql mysql 1680 Feb 10 13:26 client-key.pem
-rw-r----- 1 mysql mysql 7066 Feb 10 13:26 hostname.err
-rw-r----- 1 mysql mysql 428 Feb 10 13:26 ib_buffer_pool
-rw-r----- 1 mysql mysql 524288000 Feb 10 13:26 ibdata1
drwxr-x--- 2 mysql mysql 4096 Feb 10 13:26 mysql
drwxr-x--- 2 mysql mysql 4096 Feb 10 13:26 performance_schema
-rw------- 1 mysql mysql 1676 Feb 10 13:26 private_key.pem
-rw-r--r-- 1 mysql mysql 452 Feb 10 13:26 public_key.pem
-rw-r--r-- 1 mysql mysql 1112 Feb 10 13:26 server-cert.pem
-rw------- 1 mysql mysql 1676 Feb 10 13:26 server-key.pem
drwxr-x--- 2 mysql mysql 12288 Feb 10 13:26 sys
[root@bigdata001 data]# cat hostname.err |grep password
2021-02-10T05:26:10.084026Z 1 [Note] A temporary password is generated for root@localhost: 46uadmwCoA- 部署开机自启动
[root@bigdata001 data]# cd /usr/local/mysql
[root@bigdata001 mysql]# cp support-files/mysql.server /etc/rc.d/init.d/mysql
[root@bigdata001 mysql]# chmod +x /etc/rc.d/init.d/mysql
[root@bigdata001 mysql]# chkconfig --del mysql
[root@bigdata001 mysql]# chkconfig --add mysql
[root@bigdata001 mysql]# chkconfig --level 345 mysql on
- 登录 Mysql 并修改密码
[root@bigdata001 mysql]# su - mysql
Last login: Wed Feb 10 13:28:49 CST 2021 on pts/0
[mysql@bigdata001 ~]$ cd /usr/local/mysql
[mysql@bigdata001 mysql]$ bin/mysqld_safe &
[mysql@bigdata001 mysql]$ bin/mysql -uroot -p'46uadmwCoA alter user root@localhost identified by '123456';
Query OK, 0 rows affected (0.00 sec)
mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'ruozedata' ;
Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
mysql> exit
Bye
- Mysql 创建 CDH 部署需要的库以及用户
mysql> create database cmf default character set utf8;
Query OK, 1 row affected (0.00 sec)
mysql> grant all privileges on cmf.* to cmf@'%' identified by '123456';
ult character set utf8;
grant all privileges on hive.* to hive@'%' identified by '123456';
flush privileges;Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> create database amon default character set utf8;
Query OK, 1 row affected (0.00 sec)
mysql> grant all privileges on amon.* to amon@'%' identified by '123456';
Query OK, 0 rows affected, 1 warning (0.01 sec)
mysql> create database hive default character set utf8;
Query OK, 1 row affected (0.00 sec)
mysql> grant all privileges on hive.* to hive@'%' identified by '123456';
Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
- 把 Mysql 依赖的 JAR 放到固定的路径下
- 第一个主机先上传 Mysql jar 并执行
[root@bigdata001 ~]# cd software/
[root@bigdata001 software]# cp mysql-connector-java-5.1.47.jar /tmp/
[root@bigdata001 software]# cd /tmp/
[root@bigdata001 tmp]# scp mysql-connector-java-5.1.47.jar bigdata002:/tmp
[root@bigdata001 tmp]# scp mysql-connector-java-5.1.47.jar bigdata003:/tmp
- 所有的机器都需要执行
[root@bigdata001 ~]# cd /tmp/
[root@bigdata001 tmp]# mkdir -p /usr/share/java/
[root@bigdata001 tmp]# mv mysql-connector-java-5.1.47.jar mysql-connector-java.jar
[root@bigdata001 tmp]# mv mysql-connector-java.jar /usr/share/java/
6. 部署 Cloudera Manager
- 需要下载的文件
Cloudera Manager tar包
Parcel 文件
Parcel sha1
manifest
6.1 部署 Cloudera Manager 前置条件
- 先把文件上传到第一台机器上面，并通过第一台机器把文件传输到几台机器上面。
[root@bigdata001 ~]# cd software/
[root@bigdata001 software]# cp cloudera-manager-centos7-cm5.16.2_x86_64.tar.gz /tmp
[root@bigdata001 software]# cd /tmp/
[root@bigdata001 tmp]# scp cloudera-manager-centos7-cm5.16.2_x86_64.tar.gz bigdata002:/tmp
[root@bigdata001 tmp]# scp cloudera-manager-centos7-cm5.16.2_x86_64.tar.gz bigdata003:/tmp
- 后面操作所有的机器，解压 Cloudera Manager
[root@bigdata001 tmp]# mkdir /opt/cloudera-manager
[root@bigdata001 tmp]# tar -xzvf cloudera-manager-centos7-cm5.16.2_x86_64.tar.gz -C /opt/cloudera-manager/
[root@bigdata001 tmp]# cd /opt/cloudera-manager/
[root@bigdata001 cloudera-manager]# ll
total 8
drwxr-xr-x 4 1106 4001 4096 May 18 2019 cloudera
drwxr-xr-x 9 1106 4001 4096 May 18 2019 cm-5.16.2
[root@bigdata001 cloudera-manager]# cd cm-5.16.2/etc/cloudera-scm-agent/
[root@bigdata001 cloudera-scm-agent]# ll
total 12
-rw-r--r-- 1 1106 4001 8905 May 18 2019 config.ini
[root@bigdata001 cloudera-scm-agent]# vim config.ini
server_host=bigdata001
- 由于我选举第一台机器作为 CM Server，所以下面需要操作第一台机器，指定 CM 的 Mysql 路径
[root@bigdata001 cloudera-scm-agent]# cd ../cloudera-scm-server/
[root@bigdata001 cloudera-scm-server]# ll
total 8
-rw------- 1 1106 4001 714 May 18 2019 db.properties
-rw-r--r-- 1 1106 4001 2229 May 18 2019 log4j.properties
[root@bigdata001 cloudera-scm-server]# vim db.properties
com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=bigdata001
com.cloudera.cmf.db.name=cmf
com.cloudera.cmf.db.user=cmf
com.cloudera.cmf.db.password=123456
com.cloudera.cmf.db.setupType=EXTERNAL
- 所有节点创建⽤户
[root@bigdata001 cloudera-scm-server]# useradd --system --home=/opt/cloudera-manager/cm-5.16.1/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
[root@bigdata001 cloudera-scm-server]# chown -R cloudera-scm:cloudera-scm /opt/cloudera-manager
- 选取一台机器作为 Parcel 源，并部署，我选取的是第一台作为 Parcel 源
[root@bigdata001 cloudera-scm-server]# mkdir -p /opt/cloudera/parcel-repo
[root@bigdata001 cloudera-scm-server]# cd ~/software/
[root@bigdata001 software]# mv CDH-5.16.2-1.cdh5.16.2.p0.8-el7.parcel /opt/cloudera/parcel-repo/
[root@bigdata001 software]# mv CDH-5.16.2-1.cdh5.16.2.p0.8-el7.parcel.sha1 /opt/cloudera/parcel-repo
[root@bigdata001 software]# mv manifest.json /opt/cloudera/parcel-repo/
[root@bigdata001 software]# cd /opt/cloudera/parcel-repo
[root@bigdata001 parcel-repo]# mv CDH-5.16.2-1.cdh5.16.2.p0.8-el7.parcel.sha1 CDH-5.16.2-1.cdh5.16.2.p0.8-el7.parcel.sha
[root@bigdata001 parcel-repo]# ll
total 2082872
-rw-r--r-- 1 root root 2132782197 Oct 20 21:16 CDH-5.16.2-1.cdh5.16.2.p0.8-el7.parcel
-rw-r--r-- 1 root root 41 Feb 9 21:37 CDH-5.16.2-1.cdh5.16.2.p0.8-el7.parcel.sha
-rw-r--r-- 1 root root 66804 Oct 20 21:12 manifest.json
6.2 部署 Cloudera Manager
- 创建所有节点的软件安装目录 用户和用户组的权限
- 所有机器都需要执行
[root@bigdata001 cloudera]# mkdir -p /opt/cloudera/parcels
[root@bigdata001 cloudera]# chown -R cloudera-scm:cloudera-scm /opt/cloudera
- 启动 Cloudera Manager Server
[root@bigdata001 cloudera]# /opt/cloudera-manager/cm-5.16.2/etc/init.d/cloudera-scm-server start
- 查看 Log
[root@bigdata001 cloudera-scm-server]# cd /opt/cloudera-manager/cm-5.16.2/log/cloudera-scm-server/
[root@bigdata001 cloudera-scm-server]# tail -200f cloudera-scm-server.log
- 启动所有的 Cloudera Manager Agent
[root@bigdata001 cloudera]# /opt/cloudera-manager/cm-5.16.2/etc/init.d/cloudera-scm-agent start
- 打开阿里云的防火墙
- 打开 WEB 页面
http://115.29.206.70:7180/cmf/login
用户：admin
密码：admin
- 所有机器执行下面命令
[root@bigdata001 cloudera-scm-server]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@bigdata001 cloudera-scm-server]# echo never > /sys/kernel/mm/transparent_hugepage/enabled
- 刷新下检查
