CDH5.7.0安装实践

CDH5.7.0安装实践

规划:

ip1 master

ip2 slave

ip3 slave

ip4 slave  (add node && yum repo)--ip4作为后备的添加结点和yum源

一.基础配置(每台机器都要操作)

1.更改hostname

vi /etc/sysconfig/network    然后service network restart使之生效

2.更改ip与hostname的对应关系

vi /etc/hosts                使用uname -a查看是否生效

3.关闭防火墙

service iptables stop          关闭防火墙

service iptables status        查看防火墙是否正在运行

chkconfig iptables off          关闭防火墙的开机自动运行

4.关闭SELinux

修改/etc/selinux/config和/etc/sysconfig/selinux的SELINUX=disabled

机器重启后生效

通过sestatus验证

5.ntp(用于时间同步服务)

5.1安装配置ntp服务

5.2crontab脚本用于时间同步

01 * * * *  /bin/bash /home/adminscripts/ntpd_sync.sh

二.创建本地yum源

注:由于CDH5.7.0使用的是https协议,所以要配置https可访问的httpd

A.准备yum源文件

1.mkdir -p /var/www/html/cm5/redhat/6/x86_64

下载 http://archive.cloudera.com/cm5/repo-as-tarball/5.7.0/cm5.7.0-centos6.tar.gz

解压后 mv cm /var/www/html/cm5/redhat/6/x86_64/

2.mkdir -p /var/www/html/cdh5/parcels/5.7.0/

下载 http://archive.cloudera.com/cdh5/parcels/5.7.0/CDH-5.7.0-1.cdh5.7.0.p0.45-el6.parcel

下载 http://archive.cloudera.com/cdh5/parcels/5.7.0/CDH-5.7.0-1.cdh5.7.0.p0.45-el6.parcel.sha1并改名为CDH-5.7.0-1.cdh5.7.0.p0.45-el6.parcel.sha

mv CDH-5.7.0-1.cdh5.7.0.p0.45-el6.parcel*  /var/www/html/cdh5/parcels/5.7.0/

3.mkdir -p /var/www/html/cdh5/redhat/6/x86_64/

下载 http://archive.cloudera.com/cdh5/repo-as-tarball/5.7.0/cdh5.7.0-centos6.tar.gz

解压后 mv cdh /var/www/html/cdh5/redhat/6/x86_64/

4.mkdir -p /var/www/html/redhat/cdh/

下载 http://archive.cloudera.com/redhat/cdh/RPM-GPG-KEY-cloudera

然后mv RPM-GPG-KEY-cloudera  /var/www/html/redhat/cdh/

B.ip4配置https访问的yum源

1.安装ssl

2.编译安装httpd

2.1编译安装httpd所需要的依赖(参考http://xtony.blog.51cto.com/3964396/836508)

2.2下载httpd-2.4.20.tar.gz并解压(参考http://cwind.blog.51cto.com/62582/1061168)

编译安装: cd httpd-2.4.20

./configure --prefix=/usr/local/apache2 --with-apr=/usr/local/apr --with-apr-util=/usr/local/apr-util/ --with-pcre=/usr/local/pcre --enable-ssl --enable-so

make && make install

2.3生成证书

mkdir -p /usr/local/apache2/conf/ssl.key

cd /usr/local/apache2/conf/ssl.key

//生成服务器私钥：

openssl genrsa -des3 -out server.key 1024

//生成服务器证书请求，并按要求填些相关证书信息：

openssl req -new -key server.key -out server.csr

//签证:

openssl x509 -req -days 700 -in server.csr -signkey server.key -out server.crt

2.4修改httpd.conf文件

①打开LoadModule ssl_module modules/mod_ssl.so和LoadModule rewrite_module modules/mod_rewrite.so

②加入代码:

Listen 443

NameVirtualHost *:443

NameVirtualHost *:80

ServerName ip4

DocumentRoot /var/www/html

ErrorLog /usr/local/apache2/logs/error.log

CustomLog /usr/local/apache2/logs/access.log combined

ServerName ip4

DocumentRoot /var/www/html

ErrorLog /usr/local/apache2/logs/error.log

CustomLog /usr/local/apache2/logs/access.log combined

SSLEngine on

SSLCertificateFile /usr/local/apache2/conf/ssl.key/server.crt

SSLCertificateKeyFile /usr/local/apache2/conf/ssl.key/server.key

SSLEngine off

③修改为

Options Indexes FollowSymLinks

AllowOverride all

Require all granted

RewriteEngine on

RewriteCond  %{HTTPS} !=on

RewriteRule  ^(.*)  https://%{SERVER_NAME} [L,R]

2.5启动

/usr/local/apache2/bin/apachectl start

2.6访问:https://ip4 (访问http也会自动跳转到https)

2.7去掉校验

在/etc/yum.conf中添加sslverify=false

三.安装Cloudera Manager

1.cm安装使用的http服务，所以安装节点需要是可以访问的http协议yum源

2.下载

下载 http://archive-primary.cloudera.com/cm5/installer/5.3.0/cloudera-manager-installer.bin  

chmod +x cloudera-manager-installer.bin

./cloudera-manager-installer.bin

3.界面一直next 和 yes操作

四.安装CDH parcel

1.打开http://ip1:7180的操作页面进行安装

2.注意如果出现:未在已配置的存储库中找到任何 parcel

在"更多选项里"将"远程parcel存储库url"更改为"https://ip4/cdh5/parcels/5.7.0/"即可找到parcel

3.修改swappiness分区

cat /proc/sys/vm/swappiness

sudo sysctl vm.swappiness=0

五.添加节点

CM管理页面操作

六.添加服务

cluster1——添加服务