Centos7下Cloudera Manager 5离线部署CDH5

Centos7下Cloudera Manager 5离线部署 CDH5

1.先决条件

• CDH 5 和 CM 5 依赖和支持的版本

• 硬件要求指南

  How-to: Select the Right Hardware for Your New Hadoop Cluster

2.准备工作

• 3台16核32G内存260G硬盘CentOS 7机器

  机器规划：

  HostName	角色	IP
  cdh001	CM节点，NameNode，Active Monitor,DataNode	192.168.31.151
  cdh002	DataNode、 JobTracker	192.168.31.152
  cdh003	DataNode、 JobTracker	192.168.31.153

• CM离线包

  CM下载地址

  • cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz

• CDH离线包

  CDH下载地址

  • CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel
  • CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1 (注意下载后需要重命名为**.sha,否则安装parcel的时候会在线下载)
  • manifest.json

• JDK1.8u162

  • 下载地址

• JDBC

  • 下载地址

• 操作系统依赖

  • psmisc

    CM客户端和服务端启动脚本会使用pstree命令，不安装会报找不到命令

  • libxslt-devel

    不安装的话Hue安装时测试MySQL会报错

    # 所有节点安装
    yum install -y psmisc libxslt-devel wget
    

3.CM安装

3.1.网络配置

• 修改主机名

  # 分别在对应机器上执行
  hostnamectl set-hostname cdh01
  hostnamectl set-hostname cdh02
  hostnamectl set-hostname cdh03
  

  

• 配置hosts文件 (所有节点)

  vi /etc/hosts
  
  192.168.31.151 cdh01
  192.168.31.152 cdh02
  192.168.31.153 cdh03
  
  

  

• 配置ssh免密登录

  # 在cdh01上执行
  ssh-keygen
  # 一路回车默认在~/.ssh目录下生成RSA密钥对id_rsa(私钥)，id_rsa.pub（公钥）
  
  # 添加公钥认证
  ssh-copy-id -i cdh01
  ssh-copy-id -i cdh02
  ssh-copy-id -i cdh03
  # 输入机器用户密码
  
  # 拷贝密钥对到各个机器上去
  scp -r ~/.ssh/ cdh01:~/
  scp -r ~/.ssh/ cdh02:~/
  scp -r ~/.ssh/ cdh03:~/
  

• 关闭防火墙 （所有节点）

  # 检查防火墙状态
  systemctl status firewalld
  # 关闭防火墙
  systemctl stop firewalld
  # 设置防火墙开机不自启 
  systemctl disable firewalld
  
  

• 关闭selinux（所有节点）

  # 临时关闭selinux
  setenforce 0
  
  # 设置开机不自启
  vi /etc/selinux/config
  # 修改"SELINUX=enforcing"为"SELINUX=disabled"
  SELINUX=disabled
  
  # 查看 SELINUX 状态：
  /usr/sbin/sestatus –v
  # SELinux status: enabled（enabled：开启；disabled：关闭）
  

  

3.2.JDK安装

• 官方JDK安装要求

  • The JDK must be 64-bit. Do not use a 32-bit JDK.
  • The installed JDK must be a supported version as documented in CDH and Cloudera Manager Supported JDK Versions.
    • Oracle JDK 7 is supported across all versions of Cloudera Manager 5 and CDH 5.
    • Oracle JDK 8 is supported in C5.3.x and higher.
    • Oracle JDK 9 is not supported in any Cloudera Manager or CDH version.
  • The same version of the Oracle JDK must be installed on each cluster host.
  • The JDK must be installed at /usr/java/jdk-version.

  Important:The RHEL-compatible and Ubuntu operating systems supported by Cloudera Enterprise all use AES-256 encryption by default for tickets. To support AES-256 bit encryption in JDK versions lower than 1.8u161, you must install the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy File on all cluster and Hadoop user machines. Cloudera Manager can automatically install the policy files, or you can install them manually. For JCE Policy File installation instructions, see the README.txtfile included in the jce_policy-x.zip file. JDK 1.8u161 and higher enable unlimited strength encryption by default, and do not require policy files.On SLES platforms, do not install or try to use the IBM Java version bundled with the SLES distribution. CDH does not run correctly with that version.

  根据官方指南因此本次安装选择JDK充分测试过且支持的最高版本JDK1.8u162

  # 上传下载的jdk到cdh01机器上
  
  # 解压缩
  
  # 移动到/opt/java目录下
  mkdir /usr/java/
  tar zxvf jdk-8u162-linux-x64.tar.gz -C /usr/java/
  
  # 配置环境变量
  vi /etc/profile
  # 在最末尾加上
  
  # for jdk
  export JAVA_HOME=/usr/java/jdk1.8.0_162
  export JRE_HOME=${JAVA_HOME}/jre
  export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
  export PATH=${JAVA_HOME}/bin:$PATH
  
  # 生效环境变量
  source /etc/profile
  # 检查是否生效
  java -version
  
  # 将jdk拷贝到其他节点上去
  scp -r /usr/java/ cdh02:/usr/
  scp -r /usr/java/ cdh03:/usr/
  
  # 拷贝环境变量到其他节点上去
  scp /etc/profile cdh02:/etc/
  scp /etc/profile cdh03:/etc/
  
  # 在其他节点检查jdk环境是否生效
  source /etc/profile
  java -version
  

3.3.配置ntp

官方文档

• 安装ntp（所有节点）

  yum install -y ntp
  

• 配置

  • cdh01节点

    # cdh01主节点配置
    vi /etc/ntp.conf
    # 注释默认的server,添加以下server
    server 210.72.145.44 perfer   # 中国国家受时中心
    server 202.112.10.36             # 1.cn.pool.ntp.org
    server 59.124.196.83             # 0.asia.pool.ntp.org
    server  127.127.1.0     # local clock
    fudge   127.127.1.0 stratum 10
    

  • cdh02、cdh03节点

    vi /etc/ntp.conf
    
    ## 子节点把所有server行注释掉，添加下面一行
    server cdh01
    

  • 所有节点

    systemctl start ntpd
    systemctl enable ntpd
    ntpstat
    

    synchronised to NTP server (192.168.31.151) at stratum 12
    time correct to within 909 ms
    polling server every 64 s

3.4.cdh01节点安装配置MySQL

• 安装

  wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm
  rpm -ivh mysql-community-release-el7-5.noarch.rpm
  yum update
  yum install mysql-server
  systemctl start mysqld
  

• 配置

  systemctl stop mysqld
  cp /etc/my.cnf /etc/my.cnf.default
  vi /etc/my.cnf
  
  # 删除原有配置，并修改为如下建议配置
  

  [mysqld]
  datadir=/var/lib/mysql
  socket=/var/lib/mysql/mysql.sock
  transaction-isolation = READ-COMMITTED
  # Disabling symbolic-links is recommended to prevent assorted security risks;
  # to do so, uncomment this line:
  symbolic-links = 0
  
  key_buffer_size = 32M
  max_allowed_packet = 32M
  thread_stack = 256K
  thread_cache_size = 64
  query_cache_limit = 8M
  query_cache_size = 64M
  query_cache_type = 1
  
  max_connections = 550
  #expire_logs_days = 10
  #max_binlog_size = 100M
  
  #log_bin should be on a disk with enough free space.
  #Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your
  #system and chown the specified folder to the mysql user.
  log_bin=/var/lib/mysql/mysql_binary_log
  
  #In later versions of MySQL, if you enable the binary log and do not set
  #a server_id, MySQL will not start. The server_id must be unique within
  #the replicating group.
  server_id=1
  
  binlog_format = mixed
  
  read_buffer_size = 2M
  read_rnd_buffer_size = 16M
  sort_buffer_size = 8M
  join_buffer_size = 8M
  
  # InnoDB settings
  innodb_file_per_table = 1
  innodb_flush_log_at_trx_commit  = 2
  innodb_log_buffer_size = 64M
  innodb_buffer_pool_size = 4G
  innodb_thread_concurrency = 8
  innodb_flush_method = O_DIRECT
  innodb_log_file_size = 512M
  
  [mysqld_safe]
  log-error=/var/log/mysqld.log
  pid-file=/var/run/mysqld/mysqld.pid
  
  sql_mode=STRICT_ALL_TABLES
  
  

  # 设置开机自启
  systemctl enable mysqld
  # 启动
  systemctl start mysqld
   
  

• 设置root密码

  /usr/bin/mysql_secure_installation
  
  [...]
  Enter current password for root (enter for none):
  OK, successfully used password, moving on...
  [...]
  Set root password? [Y/n] Y
  New password:
  Re-enter new password:
  Remove anonymous users? [Y/n] Y
  [...]
  Disallow root login remotely? [Y/n] N
  [...]
  Remove test database and access to it [Y/n] Y
  [...]
  Reload privilege tables now? [Y/n] Y
  All done!
  

• 安装配置MySQL JDBC 驱动

  下载 .tar.gz格式的文件地址：http://www.mysql.com/downloads/connector/j/5.1.html

  wget https://cdn.mysql.com//Downloads/Connector-J/mysql-connector-java-5.1.47.tar.gz
  
  tar zxvf mysql-connector-java-5.1.47.tar.gz
  
  mkdir -p /usr/share/java/
  
  cp mysql-connector-java-5.1.47/mysql-connector-java-5.1.47-bin.jar \
   /usr/share/java/mysql-connector-java.jar
   
  scp -r /usr/share/java/ cdh02:/usr/share/
  scp -r /usr/share/java/ cdh03:/usr/share/
  
  

• 创建对应服务数据库

  mysql -uroot -p
  mysql>
  

  CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
  GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'hue';
  
  CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
  GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY 'hive';
  
  CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
  GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'sentry';
  
  CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
  GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie';
  
  CREATE DATABASE am DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
  GRANT ALL ON am.* TO 'am'@'%' IDENTIFIED BY 'am';
  
  

3.5. 创建用户 cloudera-scm （ 所有 节点）

useradd --system --home=/opt/cm-5.15.1/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

3.6.解压CM包

# cdh01节点
tar -zxvf cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz -C /opt/
mv CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel manifest.json /opt/cloudera/parcel-repo/

chown -R cloudera-scm:cloudera-scm /opt/cm-5.15.1/
mkdir -p /opt/cloudera/parcel-repo
chown -R cloudera-scm:cloudera-scm /opt/cloudera/
mkdir /var/lib/cloudera-scm-server
chown -R cloudera-scm:cloudera-scm /var/lib/cloudera-scm-server

3.7.修改agent配置

# cdh01节点
vi /opt/cm-5.15.1/etc/cloudera-scm-agent/config.ini
server_host=cdh01

3.8.拷贝CM包到所有节点

# cdh01节点
scp -r /opt/cm-5.15.1/ cdh02:/opt/
scp -r /opt/cm-5.15.1/ cdh03:/opt/

# cdh02、cdh03节点
mkdir -p /opt/cloudera/parcel-repo
chown -R cloudera-scm:cloudera-scm /opt/cloudera/

3.9.cdh01节点初始化数据库

# cdh01节点
/opt/cm-5.15.1/share/cmf/schema/scm_prepare_database.sh mysql -hlocalhost -uroot -p123456 --scm-host localhost scm scm scm

3.10.启动服务

# cdh01节点启动cm服务端
/opt/cm-5.15.1/etc/init.d/cloudera-scm-server start
# 查看启动日志
tail -f /opt/cm-5.15.1/log/cloudera-scm-server/cloudera-scm-server.log 
# 查看到如下日志服务启动完成
com.cloudera.server.cmf.WebServerImpl: Started Jetty server.

# cdh01、cdh02、cdh03节点启动客户端
/opt/cm-5.15.1/etc/init.d/cloudera-scm-agent start

3.11.初始化CM

• 访问CM web界面

  http://192.168.31.151:7180/

  用户：admin

  密码：admin

• 配置CM

此处会报两个错误：内存交换空间设置和启用了透明大页面压缩，点击【查看详细信息】查看需要修改设置的主机

• 将机器设置成cloudera官方建议的配置

  sysctl vm.swappiness=10
  
  vi /etc/sysctl.conf
  
  # /etc/sysctl.conf加入或修改vm.swappiness配置
  vm.swappiness=10
  
  # 使配置生效
  sysctl -p
  

  echo never > /sys/kernel/mm/transparent_hugepage/defrag
  echo never > /sys/kernel/mm/transparent_hugepage/enabled
  echo 'echo never > /sys/kernel/mm/transparent_hugepage/defrag' >> /etc/rc.local
  echo 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' >> /etc/rc.local
  cat /etc/rc.local
  
  #!/bin/bash
  # THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES
  #
  # It is highly advisable to create own systemd services or udev rules
  # to run scripts during boot instead of using this file.
  #
  # In contrast to previous versions due to parallel execution during boot
  # this script will NOT be run after all other services.
  #
  # Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure
  # that this script will be executed during boot.
  
  touch /var/lock/subsys/local
  echo never > /sys/kernel/mm/transparent_hugepage/defrag
  echo never > /sys/kernel/mm/transparent_hugepage/enabled
  

  修改完成重新运行检查主机检查，此时警告消除点击【完成】。

4.安装CDH服务

• 选择服务

• 配置角色

• 设置数据库连接

• 集群设置

• 开始安装

• 完成安装