搭建Hadoop-HA + ZooKeeper + Yarn + Hive + HBase + Storm环境

前提:搭建Hadoop-HA + ZooKeeper + Yarn + Hive + HBase环境

node01 node02 node03 node04
NameNode01 NameNode02 NameNode03
DataNode01 DataNode02 DataNode03
JournalNode01 JournalNode02 JournalNode03
ZooKeeper01 ZooKeeper02 ZooKeeper03
ZooKeeperFailoverController01 ZooKeeperFailoverController02 ZooKeeperFailoverController03
ResourceManager01 ResourceManager02
NodeManager01 NodeManager02 NodeManager03
MySQL Server MetaStore Server Hive CLI
HMaster01(Backup) HMaster02 HMaster03 HMaster04
HRegionServer01 HRegionServer02 HRegionServer03
Nimbus Supervisor01 Supervisor02
  1. 同步四台主机的时间

yum install ntpdate -y
ntpdate ntp1.aliyun.com

  1. 安装node02、node03、node04上的Storm

tar -zxvf apache-storm-0.10.0.tar.gz -C /opt/storm/

  1. 配置node02、node03、node04上的Storm

在node02上修改/opt/storm/apache-storm-0.10.0/conf/storm.yaml
vim /opt/storm/apache-storm-0.10.0/conf/storm.yaml
添加:

storm.zookeeper.servers:
    - "node02"
    - "node03"
    - "node04"
nimbus.host: "node02"
storm.local.dir: "/opt/storm/tmp"
supervisor.slots.ports:
    - 6700
    - 6701
    - 6702
    - 6703

将node02上的/opt/storm拷贝到node03、node04:
scp -r /opt/storm node03:/opt && scp -r /opt/storm node04:/opt

  1. 配置node02、node03、node04上的环境变量

在node02、node03、node04上修改/etc/profile
vim /etc/profile
添加:

export STORM_HOME=/opt/storm/apache-storm-0.10.0
export PATH=$PATH:$STORM_HOME/bin

在node02、node03、node04上运行:
. /etc/profile

  1. 启动Storm

在node02上运行:
storm nimbus >> /opt/storm/nimbus.out 2>&1 &
storm ui >> /opt/storm/ui.out 2>&1 &
在node03、node04上运行(node01也可以):
storm supervisor >> /opt/storm/supervisor.out 2>&1 &

  1. 查看进程

在node02、node03、node04上运行:
jps

  1. 访问网页
http://192.168.163.192:8080

你可能感兴趣的:(搭建Hadoop-HA + ZooKeeper + Yarn + Hive + HBase + Storm环境)