搭建 Zookeeper+Storm 集群

1.Set up a Zookeeper cluster

2.Install dependencies on Nimbus and worker machines

3.Download and extract a Storm release to Nimbus and worker machines

4.Fill in mandatory configurations into storm.yaml

5.Launch daemons under supervision using "storm" script and a supervisor of your choice

服务器信息

ubuntu-s-storm-1       137.184.63.172       Zookeeper+Storm

ubuntu-s-storm-2       167.172.138.180     Zookeeper+Storm

ubuntu-s-storm-3       143.244.165.32       Zookeeper+Storm

一、配置 Zookeeper 集群

官方地址:https://zookeeper.apache.org/releases.html

1.下载安装apache-zookeeper

cd /usr/local/src

wget https://dlcdn.apache.org/zookeeper/zookeeper-3.7.1/apache-zookeeper-3.7.1-bin.tar.gz

2.解压

tar -zxvf apache-zookeeper-3.7.1-bin.tar.gz

mv apache-zookeeper-3.7.1-bin ../zookeeper

3. 安装 JDK9

tar -zxvf jdk-8u333-linux-x64.tar.gz && mv jdk1.8.0_333 ../jdk_1.8

cat >> /etc/profile  << EOF

export JAVA_HOME=/usr/local/jdk_1.8/

export JRE_HOME=$JAVA_HOME/jre

export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH

export CLASSPATH=$JAVA_HOME/lib:$JRE_HOME/lib:./

EOF

source /etc/profile

4.创建 data and logs

cd /usr/local/zookeeper && mkdir data logs

5.更改 hosts

cat >> /etc/hosts << EOF

137.184.63.172 ubuntu-s-storm-1

167.172.138.180 ubuntu-s-storm-2

143.244.165.32 ubuntu-s-storm-3

EOF

6.配置zoo.cfg 文件

cd /usr/local/zookeeper/conf && cp zoo_sample.cfg zoo.cfg

sed -i 's/dataDir=.*/dataDir=\/usr\/local\/zookeeper\/data/g' /usr/local/zookeeper/conf/zoo.cfg

cat >> /usr/local/zookeeper/conf/zoo.cfg << EOF

dataLogDir=/usr/local/zookeeper/logs

server.1=137.184.63.172:2881:3881

server.2=167.172.138.180:2881:3881

server.3=143.244.165.32:2881:3881

EOF

说明:服务器。服务器ID=服务器IP地址:服务器间的通信端口:服务器间投票端口

客户端和服务器之间通信的默认端口:2181

服务器间通信的默认端口:2881

服务器间投票的默认端口:3881

7.更改zookeeper的默认端口,以避免与storm发生冲突

echo "admin.serverPort=8800" >> /usr/local/zookeeper/conf/zoo.cfg

8.记录每个服务器的ID,单独执行

ubuntu-s-storm-1

echo 1 >/usr/local/zookeeper/data/myid

ubuntu-s-storm-2

echo 2 >/usr/local/zookeeper/data/myid

ubuntu-s-storm-3

echo 3 >/usr/local/zookeeper/data/myid

8.启动 zookeeper

/usr/local/zookeeper/bin/zkServer.sh start/status/restart/stop

二.在Nimbus和辅助计算机上安装依赖项

1.Java 7+ (Apache Storm 1.x is tested through travis ci against both java 7 and java 8 JDKs),安装java

2.Python 2.6.6 (Python 3.x should work too, but is not tested as part of our CI enviornment) 安装python

apt install  python-is-python3

三.下载Storm版本并将其解压缩到Nimbus和worker机器

1.安装 Storm

wget https://dlcdn.apache.org/storm/apache-storm-2.4.0/apache-storm-2.4.0.tar.gz

tar -zxvf apache-storm-2.4.0.tar.gz

mv apache-storm-2.4.0 ../storm

四.在storm.yaml中填写强制配置

1.配置 storm.yaml

cat >> storm.yaml << EOF

storm.zookeeper.servers:

    - "ubuntu-s-storm-1"

    - "ubuntu-s-storm-2"

    - "ubuntu-s-storm-3"

storm.local.dir: "/usr/local/storm/data"

nimbus.seeds: ["ubuntu-s-storm-1", "ubuntu-s-storm-2", "ubuntu-s-storm-2"]

supervisor.slots.ports:

    - 6700

    - 6701

    - 6702

    - 6703

EOF

2.启动 Storm

nohup /usr/local/storm/bin/storm nimbus &

nohup /usr/local/storm/bin/storm supervisor &

nohup /usr/local/storm/bin/storm ui &

你可能感兴趣的:(ubuntu,1024程序员节)