Storm-0.10.0集群安装
一、环境
两台虚拟机，ubuntu-14.04.3
二、关闭防火墙，配置hosts
root@ubuntu:~# cat /etc/hosts 127.0.0.1 localhost 127.0.1.1 ubuntu 192.168.254.130 storm1 192.168.254.131 storm2
三、安装Java（JDK 6+），并配置环境变量
root@ubuntu:~# cat /etc/profile export JAVA_HOME=/usr/local/jdk1.7.0_80 export JRE_HOME=/$JAVA_HOME/jre export PATH=$JAVA_HOME/bin:$PATH export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
四、安装Python（2.6.6+）
确定系统自带的Python版本，如果是2.6.6+可以不用再安装。
root@ubuntu:~# python -V Python 2.7.6五、搭建ZooKeeper集群
root@ubuntu:/usr/local# wget http://mirrors.cnnic.cn/apache/zookeeper/zookeeper-3.4.6/zookeeper-3.4.7.tar.gz
root@ubuntu:/usr/local# tar -zxvf zookeeper-3.4.7.tar.gz root@ubuntu:/usr/local# cd zookeeper-3.4.7/conf root@ubuntu:/usr/local/zookeeper-3.4.7/conf# cp -p zoo_sample.cfg zoo.cfg root@ubuntu:/usr/local/zookeeper-3.4.7/conf# vim zoo.cfg # The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir=/opt/zookeeper/data # the port at which the clients will connect clientPort=2181 server.1=storm1:7000:7001 server.2=storm2:7000:7001 # the maximum number of client connections. # increase this if you need to handle more clients #maxClientCnxns=60 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir #autopurge.snapRetainCount=3 # Purge task interval in hours # Set to "0" to disable auto purge feature #autopurge.purgeInterval=1
# 手动创建dataDir目录 root@ubuntu:/opt# make zookeeper root@ubuntu:/opt# cd zookeeper root@ubuntu:/opt/zookeeper# make data
# 在dataDir目录下创建myid文件，写id号，用来标识当前主机 root@ubuntu:/opt/zookeeper/data# echo "1" > myidstorm2节点重复上面操作，也可以直接复制，只是myid输入的值是2。
# 启动ZooKeeper root@storm1:/usr/local/zookeeper-3.4.7# bin/zkServer.sh start root@storm2:/usr/local/zookeeper-3.4.7# bin/zkServer.sh start # 查看ZooKeeper状态 root@storm1:/usr/local/zookeeper-3.4.7# bin/zkServer.sh status ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper-3.4.7/bin/../conf/zoo.cfg Mode: follower root@storm1:/usr/local/zookeeper-3.4.7# bin/zkServer.sh status ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper-3.4.7/bin/../conf/zoo.cfg Mode: leader六、安装Storm
# 下载安装Storm root@ubuntu:/usr/local# wget wget http://mirrors.cnnic.cn/apache/storm/apache-storm-0.10.0/apache-storm-0.10.0.tar.gz root@ubuntu:/usr/local# tar -zxvf apache-storm-0.10.0.tar.gz root@ubuntu:/usr/local# mv apache-storm-0.10.0 storm-0.10.0
# 省略...
########### These MUST be filled in for a storm configuration
storm.zookeeper.servers:
- "storm1"
- "storm2"
#
nimbus.host: "storm1"
storm.local.dir: "/opt/storm/data"
supervisor.slots.port:
- 6700
- 6701
- 6702
- 6703
nimbus.childopts: "-Xmx1024m"
ui.childopts: "-Xmx768m"
#
# ##### These may optionally be filled in:
#
## List of custom serializations
# topology.kryo.register:
# - org.mycompany.MyType
# - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
# - org.mycompany.MyDecorator
#
## Locations of the drpc servers
# drpc.servers:
# - "server1"
# - "server2"
## Metrics Consumers
# topology.metrics.consumer.register:
# - class: "backtype.storm.metric.LoggingMetricsConsumer"
# parallelism.hint: 1
# - class: "org.mycompany.MyMetricsConsumer"
# parallelism.hint: 1
# argument:
# - endpoint: "metrics-collector.mycompany.org"
# 创建storm.local.dir目录 root@ubuntu:/opt# make storm root@ubuntu:/opt/storm# mkdir data root@ubuntu:/opt/storm/data# pwd /opt/storm/datastorm2节点重复上面操作，共同部分可以直接复制。
# 启动Storm（确保ZooKeeper已正常启动） root@storm1:/usr/local/storm-0.10.0# bin/storm nimbus >/dev/null 2>&1 & root@storm2:/usr/local/storm-0.10.0# bin/storm supervisor >/dev/null 2>&1 & root@storm1:/usr/local/storm-0.10.0# bin/storm ui >/dev/null 2>&1 &
# 查看启动进程 root@storm1:~# jps 2658 QuorumPeerMain 2696 nimbus 2813 core 3334 Jps root@ubuntu:~# jps 2673 supervisor 3287 Jps 2632 QuorumPeerMain
七、遇到的问题
如果出现IPv6引起的连接问题可以修改Storm启动JVM参数，如下：
nimbus.childopts: "-Xmx1024m -Djava.net.preferIPv4Stack=true"
ui.childopts: "-Xmx768m -Djava.net.preferIPv4Stack=true"