ubuntu 12.04 安装 Cloudera Manager 4.8.2

Cloudera可以简化Hadoop的安装配置过程，自动在集群节点上安装hadoop相关组件，创建用户。Cloudera是Hadoop生态系统中，规模最大、知名度最高的公司

1.准备

系统版本

1 iflab@cdh1:~$ uname -a
2 Linux cdh1 3.11.0-15-generic #25~precise1-Ubuntu SMP Thu Jan 30 17:39:31 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
3 iflab@cdh1:~$

角色分配(需要修改hostname、hosts)

1 IP               主机名
2 222.249.250.211  cdh1    manager master
3 222.249.250.212  cdh2    slave1
4 222.249.250.213  cdh3    slave2
5 222.249.250.214  cdh4    slave3

确保所有机器root密码一致或者sudo免输入密码用户的用户名、密码一致！

2.添加Cloudera源

1 sudo sudo nano /etc/apt/sources.list.d/cloudera.list

写入

1 deb [arch=amd64] http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh precise-cdh4 contrib
2 deb-src http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh precise-cdh4 contrib
3 deb [arch=amd64] http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm precise-cm4 contrib
4 deb-src http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm precise-cm4 contrib

获取apt key

1 curl -s http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh/archive.key | sudo apt-key add -

刷新源列表

1 sudo apt-get update

3.安装java环境

1 sudo apt-get -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold -y install oracle-j2sdk1.6

4.安装mysql以及JDBC驱动

1 sudo apt-get install mysql-server libmysql-java

5.配置mysql

1 sudo nano /etc/mysql/conf.d/mysql_cloudera_manager.cnf

写入

1 [mysqld]
2 transaction-isolation=READ-COMMITTED
3 # Disabling symbolic-links is recommended to prevent assorted security risks;
4 # to do so, uncomment this line:
5 # symbolic-links=0
6
7 key_buffer              = 16M
8 key_buffer_size         = 32M
9 max_allowed_packet      = 16M
10 thread_stack            = 256K
11 thread_cache_size       = 64
12 query_cache_limit       = 8M
13 query_cache_size        = 64M
14 query_cache_type        = 1
15 # Important: see Configuring the Databases and Setting max_connections
16 max_connections         = 550
17
18 # log-bin should be on a disk with enough free space
19 log-bin=/var/log/mysql/mysql_binary_log
20
21 # For MySQL version 5.1.8 or later. Comment out binlog_format for older versions.
22 binlog_format           = mixed
23
24 read_buffer_size = 2M
25 read_rnd_buffer_size = 16M
26 sort_buffer_size = 8M
27 join_buffer_size = 8M
28
29 # InnoDB settings
30 innodb_file_per_table = 1
31 innodb_flush_log_at_trx_commit  = 2
32 innodb_log_buffer_size          = 64M
33 innodb_buffer_pool_size         = 4G
34 innodb_thread_concurrency       = 8
35 innodb_flush_method             = O_DIRECT
36 innodb_log_file_size = 512M
37
38 [mysqld_safe]
39 log-error=/var/log/mysqld.log
40 pid-file=/var/run/mysqld/mysqld.pid

编辑my.cnf
sudo nano /etc/mysql/my.cnf
把下面这一行注释掉

1 #bind-address           = 127.0.0.1

6.配置innodb

1 sudo su
2 mv /var/lib/mysql/ib_logfile* /var/tmp/
3 logout

7.初始化数据库

1 sudo service mysql restart
2 mysql -uroot -p
3
4 create database amon DEFAULT CHARACTER SET utf8;
5 grant all on amon.* TO 'amon'@'%' IDENTIFIED BY 'amon_password';
6 grant all on amon.* TO 'amon'@'cdh1' IDENTIFIED BY 'amon_password';
7 create database smon DEFAULT CHARACTER SET utf8;
8 grant all on smon.* TO 'smon'@'%' IDENTIFIED BY 'smon_password';
9 grant all on smon.* TO 'smon'@'cdh1' IDENTIFIED BY 'smon_password';
10 create database rman DEFAULT CHARACTER SET utf8;
11 grant all on rman.* TO 'rman'@'%' IDENTIFIED BY 'rman_password';
12 grant all on rman.* TO 'rman'@'cdh1' IDENTIFIED BY 'rman_password';
13 create database hmon DEFAULT CHARACTER SET utf8;
14 grant all on hmon.* TO 'hmon'@'%' IDENTIFIED BY 'hmon_password';
15 grant all on hmon.* TO 'hmon'@'cdh1' IDENTIFIED BY 'hmon_password';
16 create database hive DEFAULT CHARACTER SET utf8;
17 grant all on hive.* TO 'hive'@'%' IDENTIFIED BY 'hive_password';
18 grant all on hive.* TO 'hive'@'cdh1' IDENTIFIED BY 'hive_password';

8.安装 cloudera-manager以及agent(因为master也是一个节点)

1 sudo apt-get install cloudera-manager-daemons cloudera-manager-server  cloudera-manager-agent

9.配置cloudera-manager-server数据库

1 sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql  -uroot -p --scm-host localhost scm scm scm_password

10.配置其余节点（cdh2、cdh3、cdh4）

配置其余节点的源

1 sudo scp [email protected]:/etc/apt/sources.list.d/cloudera.list /etc/apt/sources.list.d/cloudera.list

获取apt key

1 curl -s http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh/archive.key | sudo apt-key add -

刷新源列表

1 sudo apt-get update

在其余节点安装jdk

1 sudo apt-get -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold -y install oracle-j2sdk1.6

在其余节点安装cloudera-manager-agent 和cloudera-manager-daemons

1 sudo apt-get install cloudera-manager-agent cloudera-manager-daemons

修改agent的配置文件
sudo nano /etc/cloudera-scm-agent/config.ini
修改server_host＝cdh1

重启每个节点的agent

1 sudo service cloudera-scm-agent  restart

11.重启控制节点cloudera-manager以及agent

1 sudo service cloudera-scm-server restart
2 sudo service cloudera-scm-agent  restart

12.访问web控制台

http://222.249.250.211:7180/ 用户名admin密码admin