storm安装笔记以及提交拓扑任务

Storm -- Distributed and fault-tolerant realtime computation

这是一个分布式的、容错的实时计算系统

把Storm依赖组件的版本贴出来供各位参考：

java(1.7.0_45)

python(2.6.6)

zookeeper (zookeeper-3.4.5)

zeromq(zeromq-2.1.7)

jzmq(github trunk)

storm(0.8.1)

 

java和Python的安装过程不赘述了，说下其他组件的安装过程

安装zookeeper

wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.5/zookeeper-3.4.5.tar.gz

tar xvf zookeeper-3.4.5.tar.gz
cd zookeeper-3.4.5
cp conf/zoo_sample.cfg conf/zoo.cfg

设置下环境变量

export ZOOKEEPER_HOME=/root/zookeeper/zookeeper-3.4.5

export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.45.x86_64

export PATH=$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$PATH

然后就可以启动zookeeper了

zkServer.sh start

可以运行下zkCli.sh，测试下zookeeper是否可用

 

安装zeromq(安装好后/usr/local/lib下会有相应的库文件)

wget http://download.zeromq.org/zeromq-2.1.7.tar.gz

tar xvf zeromq-2.1.7.tar.gz

cd zeromq-2.1.7

yum install -y gcc gcc-c++.x86_64 uuid-devel.x86_64 libuuid-devel.x86_64

./configure

make

make install

 

安装jzmq

git clone https://github.com/nathanmarz/jzmq.git

./autogen.sh

./configure 

make

make install

 

开始安装storm

wget https://github.com/downloads/nathanmarz/storm/storm-0.8.1.zip

unzip storm-0.8.1.zip

cd storm-0.8.1

修改conf/storm.yaml如下

 storm.zookeeper.servers:

     - "127.0.0.1"

 nimbus.host: "192.168.24.57"

 storm.local.dir: "/root/storm/workdir"

 java.library.path: "/usr/local/lib:/opt/local/lib:/usr/lib"

 topology.debug: "true"

然后将storm/bin加入环境变量

export STORM_HOME=/root/storm/storm-0.8.1

export PATH=$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$STORM_HOME/bin:$PATH

 启动storm

storm nimbus &

storm supervisor &

storm ui &

jps可以看到如下进程，而且日志也没有报错。基本上就ok了，可以提交任务在storm环境中跑了

14157 core

14106 QuorumPeerMain

30978 Jps

14144 supervisor

13995 nimbus

 

提交拓扑的代码

storm jar stormDemo.jar com.storm.example.SimpleTopology testhui3

杀死拓扑

storm kill testhui3

 

默认8080端口提供了一个ui dashboard

enjoy~