Twitter Storm

第 121 章 Twitter Storm

121.1. 单机版

操作系统环境：Ubuntu 13.04 KVM虚拟机

安装 storm 涉及到安装以下包：python、zookeeper、zeromq、jzmq、storm

过程 121.1. Ubuntu + Storm 单机环境安装

1. 安装 zookeeper

   $ sudo apt-get install zookeeper zookeeper-bin zookeeperd
   				

2. 安装 zeromq 与 jzmq

   $ sudo apt-get install openjdk-7-jdk
   $ sudo apt-get install libzmq-dev
   				

   设置 JAVA_HOME 环境变量

   $ export JAVA_HOME="/usr/lib/jvm/java-7-openjdk-amd64"
   				

   编译安装 jzmq

   				
   $ cd /usr/local/src/
   $ git clone git://github.com/nathanmarz/jzmq.git
   $ cd jzmq
   $ ./autogen.sh
   $ ./configure --prefix=/srv/jzmq-2.1.0
   $ javac src/org/zeromq/*.java
   $ touch src/classdist_noinst.stamp
   $ make && make install
   
   $ ls /srv/jzmq-2.1.0/
   lib  share
   				
   				

3. 安装 Storm

   下载地址：https://github.com/nathanmarz/storm/downloads

   $ sudo apt-get install unzip
   
   $ cd /usr/local/src/
   $ $ wget https://github.com/downloads/nathanmarz/storm/storm-0.8.1.zip
   $ unzip storm-0.8.1.zip
   $ $ sudo mv storm-0.8.1 /srv/
   $ sudo ln -s /srv/storm-0.8.1 /srv/storm
   				

    

   				
   cat >> ~/.bashrc <<'EOF'
   
   export JAVA_HOME="/usr/lib/jvm/java-7-openjdk-amd64"
   export ZOOKEEPER_HOME="/usr/lib/zookeeper"
   export STORM_HOME=/srv/storm
   export PATH=$PATH:$PATH:$ZOOKEEPER_HOME/bin:$STORM_HOME/bin
   EOF
   				
   				

   修改 Storm CONF_DIR 目录位置

   $ sed -i 's:CONF_DIR = os.path.expanduser("~/.storm"):CONF_DIR = "/srv/storm-0.8.1/conf":' /srv/storm/bin/storm
   				

4. 启动服务

   启动 zookeeper 服务

   $ sudo service zookeeper start
   				

   启动 storm 服务

   				
   /srv/storm/bin/storm nimbus &
   /srv/storm/bin/storm supervisor &
   /srv/storm/bin/storm ui &
   				
   				

   Storm UI

   http://192.168.6.2:8080/

   查看 Java 进程

   $ jps
   10382 Jps
   10228 supervisor
   10294 core
   10173 nimbus
   				

5. 测试 Storm

   $ sudo apt-get install maven2
   
   $ git clone git://github.com/nathanmarz/storm-starter
   $ cd storm-starter
   $ mvn -f m2-pom.xml package
   				

   先进入Storm UI http://192.168.6.2:8080/, 然后运行下面与命令

   $ storm jar target/storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar storm.starter.WordCountTopology wordcountTop
   				

   在次进入 Storm UI 你会看到 wordcountTop