storm on yarn 最简单安装笔记！！！私货哦！！！

http://blog.csdn.net/jiushuai/article/details/26693311

 

java高级交流群：37341439

有storm 相关问题可以一起讨论哦！！一起学习！

 

 

storm on yarn下载地址： https://github.com/yahoo/storm-yarn

 

下载好的storm-yarn-master.zip 得放到linux进行解压，在windows解压的话，lib目录下的软连接会丢失

 

1:storm on yarn 需要编译

 

unzip  storm-yarn-master.zip

 

然后进入 storm-yarn-master 目录通过如下命令进行编译

 

mvn package -DskipTests

 

注意：我用的maven-3.1.1 和jdk1.7.0_45

 

编译好后解压storm-yarn-master/lib/storm-0.9.0-wip21.zip， 得到  storm-0.9.0-wip21目录。

 

将 得到  storm-0.9.0-wip21 目录移动到 和 storm-yarn-master同级。

最终目录是

 

/usr/local/storm/ storm-yarn-master

/usr/local/storm/ storm-0.9.0-wip21

 

然后配置storm的启停用户的环境变量 

 

vim ~/.basarc

 

添加如下环境变量

 

export JAVA_HOME=/usr/java/jdk1.7.0_45
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar


export STORM_WORK=/opt/storm
export STORM_HOME=$STORM_WORK
export PATH=$PATH:$STORM_WORK/storm-yarn-master/bin:$STORM_WORK/storm-0.9.0-wip21/bin

export HADOOP_INSTALL=/opt/hadoop
export HADOOP_HOME=$HADOOP_INSTALL
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL

 

 

 

 

2:将编译好后的storm-yarn-master/lib/storm.zip 添加进hdfs中，可以通过如下命令

hadoop fs -mkdir -p /lib/storm/0.9.0-wip21

hadoop fs -moveFromLocal storm.zip /lib/storm/0.9.0-wip21

 

 

3：需要在hdfs里面添加storm工作目录,并将storm工作目录的所有者设置为storm job提交用户

hadoop fs -mkdir -p /user/storm

hadoop fs -chown storm /user/storm

 

4：确保storm的启停用户配置环境变量如下：

export JAVA_HOME=/usr/java/jdk1.7.0_45
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export STORM_WORK=/usr/local/storm
export STORM_HOME=$STORM_WORK
export PATH=$PATH:$STORM_WORK/storm-yarn-master/bin:$STORM_WORK/storm-0.9.0-wip21/bin
export HADOOP_INSTALL=/usr/local/hadoop
export HADOOP_HOME=$HADOOP_INSTALL
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL

 

 

5:启动storm

storm-yarn launch /usr/local/storm/storm-0.9.0-wip21/conf/master.yaml

 

6：关闭storm

storm-yarn shutdown -appId application_1398822427232_0143 -output ~/.storm/storm.yaml

 

7:运行wordcount

storm jar /opt/storm/storm-yarn-master/lib/myStorm-0.0.1-SNAPSHOT-jar-with-dependencies.jar com.myStorm.App WordCountTopology -c nimbus.host=192.168.109.241

 

其中nimbus.host是你提交storm到yarn后，yarn会给你分配一个地址，你得去自己找哦。

 

 

总结，有时候发现supervisor启动不来，会发现是内存资源不够。在虚拟机环境的同志要注意这点哦。