Zookeeper(1)Introduction and Install on Win7
Zookeeper(1)Introduction and Install on Win7
1. Overview
Zookeeper is a high-performance coordination service for distributed applications.
2. Install the Zookeeper on win7
I download the latest version zookeeper-3.4.3.tar.gz.
Unzip the file on windows, copy the directory to working directory.
Add the directory to my path=D:\tool\zookeeper-3.4.3\bin
copy the configuration file from conf/zoo_sample.cfg to zoo.cfg. Change some content as follow:
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=c://tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
run the command to start the server on windows:
>zkServer.cmd
3. Try the client on win7
>zkCli.cmd -server 127.0.0.1:2181
>help
>ls /
[zookeeper]
Try to create a new znode by running create /zk_test my_data command.
>create /zk_test my_data
Create /zk_test
>ls /
[zookeeper, zk_test]
>get /zk_test
my_data
>set /zk_test junk
>delete /zk_test
>ls /
[zookeeper]
4. Running Replicated ZooKeeper
Standalone mode is convenient for evaluation, some development, and testing. But in production,
we should run Zookeeper in replicated mode.
A replicated group of servers in the same application is called a quorum.
Change the configuration zoo.cfg as follow:
tickTime=2000
initLimit=10
syncLimit=5
dataDir=c://tmp/zookeeper
clientPort=2181
server.1=zoo1:2888:3888
server.2=zoo2:2888:3888
server.3=zoo3:2888:3888
initLimit is timeouts ZooKeeper uses to limit the length of time the ZooKeeper servers in quorum have to connect to a leader.
syncLimit limits how far out of date a server can be from a leader.
If you want to test multiple servers on a single machine, specify the servername as localhost with unique quorum & leader election ports (i.e. 2888:3888, 2889:3889, 2890:3890 in the example above) for each server.X in that server's config file. Of course separate dataDirs and distinct clientPorts are also necessary (in the above replicated example, running on a single localhost, you would still have three config files).
It is not wise to run it on windows from my understanding, so I find a redhat server to do this.
5. Installation on Redhat
>wget http://apache.deathculture.net/zookeeper/zookeeper-3.4.3/zookeeper-3.4.3.tar.gz
>tar zxvf zookeeper-3.4.3.tar.gz
>sudo mv zookeeper-3.4.3/ /opt/tools/
>cp zoo_sample.cfg zoo1.cfg
>vi zoo1.cfg
ckTime=2000
initLimit=10
syncLimit=5
dataDir=/tmp/zookeeper/snapshot/d_1
clientPort=2181
server.1=zoo1:2888:3888
server.2=zoo2:2889:3889
server.3=zoo3:2890:3890
>vi zoo2.cfg
ckTime=2000
initLimit=10
syncLimit=5
dataDir=/tmp/zookeeper/snapshot/d_2
clientPort=2182
server.1=zoo1:2888:3888
server.2=zoo2:2889:3889
server.3=zoo3:2890:3890
>vi zoo3.cfg
ckTime=2000
initLimit=10
syncLimit=5
dataDir=/tmp/zookeeper/snapshot/d_3
clientPort=2183
server.1=zoo1:2888:3888
server.2=zoo2:2889:3889
server.3=zoo3:2890:3890
Create these directory
>mkdir /tmp/zookeeper/snapshot/d_1
>mkdir /tmp/zookeeper/snapshot/d_2
>mkdir /tmp/zookeeper/snapshot/d_3
And create file myid under these directory
>vi /tmp/zookeeper/snapshot/d_1/myid
1
>vi /tmp/zookeeper/snapshot/d_2/myid
2
>vi /tmp/zookeeper/snapshot/d_3/myid
3
Add this to my myprofile path
>sudo vi /etc/profile
PATH=.:$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH:$HOME/bin:$ANT_HOME/bin:/sbin:/opt/tools/zookeeper-3.4.3/bin:/usr/sbin:/usr/local/sbin:/home/luohua/git-1.7.6
>. /etc/profile
start the server with these commands:
>zkServer.sh start zoo1.cfg
>zkServer.sh start zoo2.cfg
>zkServer.sh start zoo3.cfg
Check if the server is started.
>jps
10129 Jps
9783 QuorumPeerMain
9815 QuorumPeerMain
9844 QuorumPeerMain
I try to connect the server with my client. But I got these error message:
Error Message:
Welcome to ZooKeeper!
2012-06-09 14:22:20,543 [myid:] - WARN [main-SendThread(localhost.localdomain:2181):ZooKeeperSaslClient@123] - SecurityException: java.lang.SecurityException: Unable to locate a login configuration occurred when trying to find JAAS configuration.
2012-06-09 14:22:20,544 [myid:] - INFO [main-SendThread(localhost.localdomain:2181):ZooKeeperSaslClient@125] - Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-06-09 14:22:20,590 [myid:] - INFO [main-SendThread(localhost.localdomain:2181):ClientCnxn$SendThread@1053] - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
2012-06-09 14:22:29,264 [myid:] - WARN [main-SendThread(localhost.localdomain:2181):ZooKeeperSaslClient@123] - SecurityException: java.lang.SecurityException: Unable to locate a login configuration occurred when trying to find JAAS configuration.
2012-06-09 14:22:29,264 [myid:] - INFO [main-SendThread(localhost.localdomain:2181):ZooKeeperSaslClient@125] - Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
Solution:
>sudo vi /etc/hosts
ip hadoop_name
Not solve this problem. I will check later.
references:
http://zookeeper.apache.org/
http://blog.csdn.net/baiduforum/article/details/6981456
http://blog.csdn.net/gudaoqianfu/article/details/7327191
http://haohouhou.iteye.com/blog/1424048
1. Overview
Zookeeper is a high-performance coordination service for distributed applications.
2. Install the Zookeeper on win7
I download the latest version zookeeper-3.4.3.tar.gz.
Unzip the file on windows, copy the directory to working directory.
Add the directory to my path=D:\tool\zookeeper-3.4.3\bin
copy the configuration file from conf/zoo_sample.cfg to zoo.cfg. Change some content as follow:
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=c://tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
run the command to start the server on windows:
>zkServer.cmd
3. Try the client on win7
>zkCli.cmd -server 127.0.0.1:2181
>help
>ls /
[zookeeper]
Try to create a new znode by running create /zk_test my_data command.
>create /zk_test my_data
Create /zk_test
>ls /
[zookeeper, zk_test]
>get /zk_test
my_data
>set /zk_test junk
>delete /zk_test
>ls /
[zookeeper]
4. Running Replicated ZooKeeper
Standalone mode is convenient for evaluation, some development, and testing. But in production,
we should run Zookeeper in replicated mode.
A replicated group of servers in the same application is called a quorum.
Change the configuration zoo.cfg as follow:
tickTime=2000
initLimit=10
syncLimit=5
dataDir=c://tmp/zookeeper
clientPort=2181
server.1=zoo1:2888:3888
server.2=zoo2:2888:3888
server.3=zoo3:2888:3888
initLimit is timeouts ZooKeeper uses to limit the length of time the ZooKeeper servers in quorum have to connect to a leader.
syncLimit limits how far out of date a server can be from a leader.
If you want to test multiple servers on a single machine, specify the servername as localhost with unique quorum & leader election ports (i.e. 2888:3888, 2889:3889, 2890:3890 in the example above) for each server.X in that server's config file. Of course separate dataDirs and distinct clientPorts are also necessary (in the above replicated example, running on a single localhost, you would still have three config files).
It is not wise to run it on windows from my understanding, so I find a redhat server to do this.
5. Installation on Redhat
>wget http://apache.deathculture.net/zookeeper/zookeeper-3.4.3/zookeeper-3.4.3.tar.gz
>tar zxvf zookeeper-3.4.3.tar.gz
>sudo mv zookeeper-3.4.3/ /opt/tools/
>cp zoo_sample.cfg zoo1.cfg
>vi zoo1.cfg
ckTime=2000
initLimit=10
syncLimit=5
dataDir=/tmp/zookeeper/snapshot/d_1
clientPort=2181
server.1=zoo1:2888:3888
server.2=zoo2:2889:3889
server.3=zoo3:2890:3890
>vi zoo2.cfg
ckTime=2000
initLimit=10
syncLimit=5
dataDir=/tmp/zookeeper/snapshot/d_2
clientPort=2182
server.1=zoo1:2888:3888
server.2=zoo2:2889:3889
server.3=zoo3:2890:3890
>vi zoo3.cfg
ckTime=2000
initLimit=10
syncLimit=5
dataDir=/tmp/zookeeper/snapshot/d_3
clientPort=2183
server.1=zoo1:2888:3888
server.2=zoo2:2889:3889
server.3=zoo3:2890:3890
Create these directory
>mkdir /tmp/zookeeper/snapshot/d_1
>mkdir /tmp/zookeeper/snapshot/d_2
>mkdir /tmp/zookeeper/snapshot/d_3
And create file myid under these directory
>vi /tmp/zookeeper/snapshot/d_1/myid
1
>vi /tmp/zookeeper/snapshot/d_2/myid
2
>vi /tmp/zookeeper/snapshot/d_3/myid
3
Add this to my myprofile path
>sudo vi /etc/profile
PATH=.:$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH:$HOME/bin:$ANT_HOME/bin:/sbin:/opt/tools/zookeeper-3.4.3/bin:/usr/sbin:/usr/local/sbin:/home/luohua/git-1.7.6
>. /etc/profile
start the server with these commands:
>zkServer.sh start zoo1.cfg
>zkServer.sh start zoo2.cfg
>zkServer.sh start zoo3.cfg
Check if the server is started.
>jps
10129 Jps
9783 QuorumPeerMain
9815 QuorumPeerMain
9844 QuorumPeerMain
I try to connect the server with my client. But I got these error message:
Error Message:
Welcome to ZooKeeper!
2012-06-09 14:22:20,543 [myid:] - WARN [main-SendThread(localhost.localdomain:2181):ZooKeeperSaslClient@123] - SecurityException: java.lang.SecurityException: Unable to locate a login configuration occurred when trying to find JAAS configuration.
2012-06-09 14:22:20,544 [myid:] - INFO [main-SendThread(localhost.localdomain:2181):ZooKeeperSaslClient@125] - Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
2012-06-09 14:22:20,590 [myid:] - INFO [main-SendThread(localhost.localdomain:2181):ClientCnxn$SendThread@1053] - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
2012-06-09 14:22:29,264 [myid:] - WARN [main-SendThread(localhost.localdomain:2181):ZooKeeperSaslClient@123] - SecurityException: java.lang.SecurityException: Unable to locate a login configuration occurred when trying to find JAAS configuration.
2012-06-09 14:22:29,264 [myid:] - INFO [main-SendThread(localhost.localdomain:2181):ZooKeeperSaslClient@125] - Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
Solution:
>sudo vi /etc/hosts
ip hadoop_name
Not solve this problem. I will check later.
references:
http://zookeeper.apache.org/
http://blog.csdn.net/baiduforum/article/details/6981456
http://blog.csdn.net/gudaoqianfu/article/details/7327191
http://haohouhou.iteye.com/blog/1424048