centos CDH 离线安装步骤

参考

Requirements

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/installation_reqts.html

 

Cloudera Manager 5 Requirements andSupported Versions

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_ig_cm_requirements.html

 

Cloudera Manager Version and DownloadInformation

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_vd.html

Installing the Latest CDH 5 Release

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_ig_cdh5_install.html

下载离线包

OS: Centos6

http://archive.cloudera.com/cm5/cm/5/cloudera-manager-el6-cm5.5.0_x86_64.tar.gz

http://archive.cloudera.com/cdh5/parcels/5.5/CDH-5.5.0-1.cdh5.5.0.p0.8-el6.parcel

http://archive.cloudera.com/cdh5/parcels/5.5/CDH-5.5.0-1.cdh5.5.0.p0.8-el6.parcel.sha1 (*.sha1 改为 *.sha)

http://archive.cloudera.com/cdh5/parcels/5.5/manifest.json

手动安装手册

Installation Path C - Manual InstallationUsing Cloudera Manager Tarballs

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_ig_install_path_c.html

安装操作

Before You Begin

Java Environment Install

Oracle JDK7 官网

http://www.oracle.com/technetwork/java/javase/downloads/java-archive-downloads-javase7-521261.html#jdk-7u80-oth-JPR

 

1) 下载JDK安装包 :

jdk-7u80-linux-x64.rpm

 2) 安装

rpm –ivh jdk-7u80-linux-x64.rpm

echo "JAVA_HOME=/usr/java/jdk1.7.0_80/">> /etc/environment

Install and Configure Databases

ClouderaManager and Managed Service Data Stores

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_ig_installing_configuring_dbs.html

 

mysql install

1)  参考文档

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_ig_mysql.html#cmig_topic_5_5

 

2)  Mysql官网下载：

http://dev.mysql.com/get/Downloads/MySQL-5.6/MySQL-5.6.26-1.el6.x86_64.rpm-bundle.tar

tar –zxvf MySQL-5.6.26-1.el6.x86_64.rpm-bundle.tar

rpm –ivh  MySQL-shared-compat-*.rpm               # RHEL兼容包

rpm –ivh  MySQL-server-*.rpm                              # MySQL服务端程序

rpm –ivh  MySQL-client-*.rpm                       # MySQL客户端程序

rpm –ivh  MySQL-devel-*.rpm                        # MySQL的库和头文件

rpm –ivh  MySQL-shared-*.rpm                       # MySQL的共享库

 # 获取MySQL安装时生成的随机密码

cat /root/.mysql_secret

service mysql start                                                      # 启动MySQL服务

# 进入MySQL，使用之前获取的随机密码

mysql -uroot -p      

# 在MySQL命令行中设置root账户的密码为password

SET PASSWORD FOR 'root'@'localhost' =PASSWORD('password');

 

Creating Databases for ActivityMonitor, Reports Manager, Hive Metastore Server, Sentry Server, ClouderaNavigator Audit Server, and Cloudera Navigator Metadata Server

mysql> create database database DEFAULT CHARACTER SET utf8;

mysql> grant all on database.* TO 'user'@'%' IDENTIFIED BY 'password';

3)  在Mysql中创建数据库和相关用户

 

# Database amon                       Activity Monitor

# Database rman                       Reports Manager

# Database metastore             Hive Metastore Server

# Database sentry                     Sentry server

# Database nav                           Cloudera Navigator Audit Server

# Database navms                     Cloudera Navigator Metadata Server

 

create database amonDEFAULT CHARACTER SET utf8;

grant all on amon.* TO'amon'@'%' IDENTIFIED BY 'amon_password';

 

create database rmanDEFAULT CHARACTER SET utf8;

grant all on rman.* TO'rman'@'%' IDENTIFIED BY 'rman_password';

 

create database metastoreDEFAULT CHARACTER SET utf8;

grant all on metastore.*TO 'hive'@'%' IDENTIFIED BY 'hive_password';

 

create database sentryDEFAULT CHARACTER SET utf8;

grant all on sentry.* TO'sentry'@'%' IDENTIFIED BY 'sentry_password';

 

create database nav DEFAULTCHARACTER SET utf8;

grant all on nav.* TO 'nav'@'%'IDENTIFIED BY 'nav_password';

 

create database navmsDEFAULT CHARACTER SET utf8;

grant all on navms.* TO 'navms'@'%'IDENTIFIED BY 'navms_password';

Installing the MySQL JDBC Driver

Install the JDBCdriver on the Cloudera Manager Server host, as well as hosts to which youassign the Activity Monitor, Reports Manager, Hive Metastore Server,Sentry Server, Cloudera Navigator Audit Server, and Cloudera Navigator MetadataServer roles.

 

  Note: If you already have the JDBCdriver installed on the hosts that need it, you can skip this section. However,MySQL 5.6 requires a driver version 5.1.26 or higher.

 

1）Download the MySQL JDBC driver from http://www.mysql.com/downloads/connector/j/5.1.html.

 

2） Extract the JDBC driver JAR file from the downloaded file. Forexample:

tar zxvf mysql-connector-java-5.1.31.tar.gz

 

3） Copy the JDBC driver, renamed, to the relevant host. For example:

  sudo cpmysql-connector-java-5.1.31/mysql-connector-java-5.1.31-bin.jar/usr/share/java/mysql-connector-java.jar

SSH Configure

Network config

1） 修改网络配置 （所有节点）

sudo vim/etc/sysconfig/network

        NETWORKING=yes

         #修改hostname

  HOSTNAME=ClouderaManager

2） 修改hosts （所有节点）

sudo vim/etc/hosts

127.0.0.1  localhost localhost.localdomain localhost4 localhost4.localdomain4

::1        localhost localhost.localdomain localhost6 localhost6.localdomain6

#添加

10.0.0.227 ClouderaManager

10.0.0.228 Agent01

10.0.0.229 Agent02

3） 重启服务

        sudo /etc/init.d/network restart

ssh config

1） 生成公钥文件

ssh-keygen -t rsa

2） 将所有节点生成的公钥添加到同一个认证文件中

cat ~/.ssh/id_rsa.pub >>~/.ssh/authorized_keys

3）  将认证文件拷贝到所有节点

scp ~/.ssh/authorized_keys root@Agent01:~/.ssh/

         测试 ssh Agent01 是否无密码登陆

FireWall And SELinux Configure

修改所有节点

Firewall

1） 关闭防火墙服务

sudo service iptables stop

2） 配置开机关闭项

sudo chkconfig iptables off

SELinux

1）  修改文件

sudo vim  /etc/selinux/config

#修改SELINUX为disabled

SELINUX=disabled

SELINUXTYPE=targeted

2）  临时生效

setenforce 0 

NTP Configure

Server - ClouderaManager

1） 修改配置文件

sudo vim  /etc/ntp.conf 

//添加或修改restrict 项   

//允许任何IP的客户机都可以进行时间同步

restrict default nomodify notrap

2） 重启服务

service ntpd restart

3） 检查服务

ntpstat

4） 开机启动服务

chkconfig ntpd on

Client  - Agent

1） 修改配置文件

//备份

sudo cp /etc/ntp.conf/etc/ntp.conf_bak

        sudo vim /etc/ntp.conf

        //添加server 并注释掉原有server

#server0.centos.pool.ntp.org iburst

#server1.centos.pool.ntp.org iburst

#server2.centos.pool.ntp.org iburst

#server3.centos.pool.ntp.org iburst

server ClouderaManager

2） 同步Server

ntpdate ClouderaManager

3） 启动服务

service ntpd start

4） 开机启动服务

chkconfig ntpd on

Install httpd

yum install httpd

Installed:

  httpd.x86_640:2.2.15-45.el6.centos                                                                                                                                                         

Dependency Installed:

apr.x86_64 0:1.3.9-5.el6_2

apr-util.x86_64 0:1.3.9-3.el6_0.1

apr-util-ldap.x86_64 0:1.3.9-3.el6_0.1

httpd-tools.x86_640:2.2.15-45.el6.centos

Install the Cloudera ManagerServer and Agents

Server

1）  离线cloudera-manager安装包安装

sudo mkdir  /opt/cloudera-manager

sudo tar xzf cloudera-manager*.tar.gz -C/opt/cloudera-manager

sudo cp /usr/share/java/mysql-connector-java.jar /opt/cloudera-manager/cm-5.0/share/cmf/lib/mysql-connector-java.jar

 

2）  添加用户和修改权限

sudo useradd --system--home=/opt/cloudera-manager/cm-5.0/run/cloudera-scm-server --no-create-home --shell=/bin/false--comment "Cloudera SCM User" cloudera-scm

sudo chown cloudera-scm:cloudera-scm /opt/ cloudera-manager–R

 

3）  在主节点初始化CM数据库

/opt/cloudera-manager/cm-5.0/share/cmf/schema/scm_prepare_database.shmysql -uroot -ppassword scm scm scm

 

4）  离线CDH5安装包目录

sudo mkdir -p /opt/cloudera/parcel-repo

sudo chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo

5） 启动服务

/opt/cloudera-manager/cm-5.0/etc/init.d/cloudera-scm-serverstart

6） 查看日志文件看服务是否正常（另起终端查看启动时间较长）

tail –f  /opt/cloudera-manager/cm-5.0/log/cloudera-scm-server/cloudera-scm-server.log

Agent

1）从Server复制安装文件到Agent

scp –rf /opt/clouderaManager Agent:/opt/

 

2）添加用户和修改权限

sudo useradd --system--home=/opt/cloudera-manager/cm-5.0/run/cloudera-scm-server --no-create-home--shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

sudo chown cloudera-scm:cloudera-scm /opt/ cloudera-manager–R

 

3）修改配置文件

sudo vim/opt/cloudera-manager/cm-5.0/etc/cloudera-scm-agent/config.ini

[General]

# Hostname of the CM server.

server_host=ClouderaManager

# Port that the CM server is listeningon.

server_port=7182

 

4）启动服务

/opt/cloudera-manager/cm-5.0/etc/init.d/cloudera-scm-agentstart

 

5） 查看日志文件看服务是否正常

tail –f /opt/cloudera-manager/cm-5.0/log/cloudera-scm-agent/cloudera-scm-agent.log

 

WebPage Configure

Login

访问: 10.0.0.227:7180

用户: admin 密码: admin