一、前言。
学习大数据组件,最好的方式是直接参照官网。不过官网的教程也让我吃了一坑,在此记录一下。因在个人笔记本资源有限,在此安装为单机版安装
二、搭建。
1.1配置主机名
hostnamectl set-hostname cdh
1.2配置hosts文件
vi /etc/hosts之后进行编辑,加入本机的ip映射关系(非单机时,有多个ip时,均加进来)
192.168.159.133 cdh
1.3关闭selinux
setenforce 0
1.4关闭防火墙
systemctl stop firewalld.service
systemctl disable firewalld.service
1.5配置免密登录
ssh-keygen (后面一路回车就行)
ssh-copy-id cdh(cdh为主机名,非单机时有多个ip时,则执行多次进行免密配置)
1.6配置时间同步(时间服务器使用已有的,在此不介绍时间服务器的搭建1)yum install nt2)vi /etc/ntp.conf加入以下内容server 0.pool.ntp.org
3)sudo systemctl start ntpd
4)sudo systemctl enable ntpd
5)ntpdate -u 0.pool.ntp.org
6)hwclock --systohc
1.7安装JDK
1)下载jdk
2)将jdk放至/usr/java/目录下(一定要放至此目录下,否则Clouder-Manager启动会失败!!!)
3)解压并配置JAVA_HOME与PATH环境变量
4)导入TLS
1.8安装Mysql(采用docker安装)
1)安装docker
yum install docker
system enable docker
vi /etc/docker/daemons.json并加入加速镜像地址 {
"registry-mirrors": ["http://hub-mirror.c.163.com"]
}
system start docker
2)安装mysql
docker run --name mysql -e MYSQL_ROOT_PASSWORD=root -d --restart=always -p 3306:3306 mysql:5.6
3)建库
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'scm@123';
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'rman@123';
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'hue@123';
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON metastore.* TO 'metastore'@'%' IDENTIFIED BY 'metastore@123';
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'sentry@123';
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'nav@123';
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'navms@123';
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie@123';
1.9设置数据库
格式为:/opt/cloudera/cm/schema/scm_prepare_database.sh [options] -h
执行时:/opt/cloudera/cm/schema/scm_prepare_database.sh mysql -h cdh scm root root
2.0安装Cloudera Manager
下载地址: https://archive.cloudera.com/cm6/6.0.1/redhat7/yum/RPMS/x86_64/
下载:
1.cloudera-manager-daemons-6.0.1-610811.el7.x86_64.rpm
2.cloudera-manager-server-6.0.1-610811.el7.x86_64.rpm
3.cloudera-manager-agent-6.0.1-610811.el7.x86_64.rpm
执行rpm安装: rpm -ivh cloudera-*
导入STL:JAVA_HOME=/usr/java/${jdk名称} /opt/cloudera/cm-agent/bin/certmanager setup --configure-services
单机版时执行:
systemctl start cloudera-scm-server
查看日志:tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
显示如下表示服务已经正常起来:INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
vi /etc/cloudera-scm-agent/config.ini,修改server_host=cdh
启动cloudera-scm-agent服务:systemctl start cloudera-scm-agent
查看cloudera-scm-agent状态:systemctl status cloudera-scm-agent
查看cloudera-scm-agent日志:tail -f /var/log/cloudera-scm-agent/cloudera-scm-agent.log
多机版时:
在server端执行cloudera-scm-server与cloudera-scm-agent操作
在agent端只执行cloudera-scm-agent操作
2.1配置CDH
Cloudera Manager Server启动起来以后就可以登录Web界面进行CDH的安装配置工作了。
在浏览器中输入 http://192.168.1.101:7180,默认用户名与密码均为:admin
2.2之后按自己的安装需要去安装大数据组件即可,就不一一介绍了。