zookeeper

1.局域网内的yum源仓库的配置
将一台机器专门用来做yum源仓库，系统启动后自动挂载
1.编辑vi /etc/fstab
2.在文件末尾中追加
/dev/cdrom /mnt/cdrom iso9660 defaults 0 0
3.安装httpd服务
4.在httpd服务的工作路径下创建软连接指向本地的yum源仓库
httpd的工作目录：/var/www/html/
创建软连接：ln -s /mnt/cdrom/ /var/www/html/centos

2.解决SCP命令不存在问题 yum erase 软件名 删除软件
scp是ssh服务客户端的
1.查看ssh相关软件包
yum list | grep ssh
2.安装客户端SSH服务
yum install openssh-clients.i686
3.scp命令的使用必须两台机器都有shh客户端

3.自动化配置免密登录和安装软件
先安装except服务才能运行
yum install expect
boot.sh

#!/bin/bash

SERVERS="node-3.itcast.cn node-4.itcast.cn"#需要配置的目的ip
PASSWORD=123456  #密码
BASE_SERVER=172.16.203.100

auto_ssh_copy_id() {
    expect -c "set timeout -1;
        spawn ssh-copy-id $1;
        expect {
            *(yes/no)* {send -- yes\r;exp_continue;}
            *assword:* {send -- $2\r;exp_continue;}
            eof        {exit 0;}
        }";
}

ssh_copy_id_to_all() {
    for SERVER in $SERVERS
    do
        auto_ssh_copy_id $SERVER $PASSWORD
    done
}

ssh_copy_id_to_all


for SERVER in $SERVERS
do
    scp install.sh root@$SERVER:/root
    ssh root@$SERVER /root/install.sh
done

install_everyone.sh

#!/bin/bash

BASE_SERVER=mini4
yum install -y wget
wget $BASE_SERVER/soft/jdk-7u45-linux-x64.tar.gz
tar -zxvf jdk-7u45-linux-x64.tar.gz -C /usr/local
cat >> /etc/profile << EOF
export JAVA_HOME=/usr/local/jdk1.7.0_45
export PATH=\$PATH:\$JAVA_HOME/bin
EOF

4.zookeeper
1.简介：
zookeeper是一个分布式协调服务；就是为用户的分布式应用程序提供协调服务
A、zookeeper是为别的分布式程序服务的
B、Zookeeper本身就是一个分布式程序（只要有半数以上节点存活，zk就能正常服务）
C、Zookeeper所提供的服务涵盖：主从协调、服务器节点动态上下线、统一配置管理、分布式共享锁、统一名称服务……
D、虽然说可以提供各种服务，但是zookeeper在底层其实只提供了两个功能：
管理(存储，读取)用户程序提交的元数据；
并为用户程序提供数据节点监听服务；

Zookeeper常用应用场景：
《见图》

Zookeeper集群的角色： Leader 和 follower （Observer）
只要集群中有半数以上节点存活，集群就能提供服务
2.zookeeper的集群机制
半数机制：集群中半数以上机器存活，集群可用。
zookeeper适合装在奇数台机器上！！！

3.安装zookeeper软件
1.上传软件包，可以安装lrzsz服务，直接将将文件拖到窗口中即可
2.解压文件到apps目录
tar -zxvf zookeeper-3.4.5.tar.gz -C apps/
3.为了精简删除源码包；
rm -rf src/ *.xml *.txt
rm -rf docs/ dist-maven/

4.配置配置文件conf/zoo.cfg配置文件
1.将zoo_sample.cfg模板文件复制一份并改名为zoo.cfg
cp zoo_sample.cfg zoo.cfg
2.编辑zoo.cfg文件
1.修改dtaaDir
dataDir=/root/zkdata
2.在文件后追加主机名级其所用的端口
server.1=192.168.186.5:2888:3888 #2888是master和follower通信的端口，3888是投票选举master的端口
server.2=192.168.186.4:2888:3888
server.3=192.168.186.3:2888:3888
3.将机器的myid配置到数据文件zkdata中去
1.创建zkdata目录
mkdir /root/mkdata
2.在zkdata下创建myid文件，并将id写到文件中
echo 1 > myid
4.将apps目录复制到其他主机上
scp -r apps/ [email protected]:/root
5.在其他的机器上配置myid
6.将zookeeper集群的防火墙都关闭
用CRT工具一键式操作
service iptables stop

7.启动zookeeper集群
cd /root/apps/zookeeper-3.4.5/
bin/zkServer.sh start

4.zookeeper命令行的常用操作：
1.连接zookeeper
./zkCli.sh
1.1指定主机的连
bin/zkCli.sh -server 192.168.186.4:2181接
bin/zkCli.sh -server zk01,zk02,zk03接
2.–help查看帮助
3.切换角色
connect 192.168.186.5:2181
4.查看znode
ls
5.创建节点数据
create [-s] [-e] path data acl
-e创建短暂节点，客户端端口节点自动删除
-s创建带有顺序标识的节点
如：create /apps 12345
6.获取数据
get /path
6.1删除：
delete /zk
6.2删除节点：rmr循环删(deleteall)
rmr /zk
6.3修改数据
set /zk "zsl“
7.监听数据
get path [watch] 得到数据并为该节点的数据一次性监听
8.监听子节点
ls path [watch] 一次性
9.退出客户端
quit
5.一键式启动zookeeper集群
关闭zookeeper： ./zkServer.sh stop或jps kill -9 进程编号
export定义的变量，会对自己所在的进程及其子进程生效。ssh远程执行命令不会
自动启动脚本startzk.sh
#!/bin/bash
echo “start zkServer…”
for i in 192.168.186.4 192.168.186.3 192.168.186.5
do
ssh root@$i “source /etc/profile;/root/apps/zookeeper-3.4.5/bin/zkServer.sh start”
done
自动关闭脚本stopzk.sh

#!/bin/bash
echo "stop zkServer..."
for i in 192.168.186.4 192.168.186.3 192.168.186.5
do
ssh root@$i "source /etc/profile;/root/apps/zookeeper-3.4.5/bin/zkServer.sh stop"
done

6.zookeeper的java客户端api

package zookeeper;

import org.apache.zookeeper.*;
import org.apache.zookeeper.data.Stat;
import org.junit.Before;
import org.junit.Test;
import java.io.IOException;
import java.util.List;

/**
 * @program:bigdata
 * @package:zookeeper
 * @filename:simplezkClient.java
 * @create:2019.09.19.14.25
 * @author:Administrator
 * @descrption.简单的zookeeper客户程序
 */
public class simplezkClient {
    private static final String connectString="192.168.186.5:2181,192.168.186.4:2181,192.168.186.3:2181";
    private static final int sessionTimout=2000;
    ZooKeeper zkClient=null;
    @Before
    public void init() throws IOException {
        zkClient=new ZooKeeper(connectString, sessionTimout, new Watcher() {
            public void process(WatchedEvent watchedEvent) {
                //收到事件的处理函数
                System.out.println(watchedEvent.getType()+"..."+watchedEvent.getPath());
                try {
//                    循环注册监听器
                    zkClient.getChildren("/",true);
                } catch (KeeperException e) {
                    e.printStackTrace();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        });
    }

    /*
     * 数据的增删改查
     * */
    //创建数据节点到zk中
    @Test
    public void testCreate() throws Exception {

        //参数1：要创建的节点路径
        //参数2：节点的数据
        //参数3：节点的权限，通常用ZooDefs.Ids.OPEN_ACL_UNSAFE
        //参数4：节点的类型
        String nodeCreated = zkClient.create("/ideal", "dhh".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
    }

    //判断znode是否存在
    @Test
    public void testExist() throws KeeperException, InterruptedException {
        //返回对象是节点除数据的值的封装
        Stat stat = zkClient.exists("/ideal", false);
        System.out.println(stat==null?"not exist":"exist");
    }

    //获取子节点
    @Test
    public void getChildren() throws KeeperException, InterruptedException {
        List children = zkClient.getChildren("/", true);
        for (String child:children){
            System.out.println(child);
        }
        //让程序一直运行
        Thread.sleep(Long.MAX_VALUE);

    }
    //获取znode的数据
    @Test
    public void getData() throws KeeperException, InterruptedException {
        byte[] data = zkClient.getData("/ideal", false, null);
        System.out.println(new String(data));
    }

    //删除znode的数据
    @Test
    public void deleteZode() throws KeeperException, InterruptedException {
        zkClient.delete("/ideal",-1);//-1代表全部删除
    }
    //修改znode的数据
    @Test
    public void setData() throws KeeperException, InterruptedException {
        zkClient.setData("/apps","dhh".getBytes(),-1);
        byte[] data = zkClient.getData("/apps", false, null);
        System.out.println(new String(data));
    }
}

7.zookeeper的应用
1.分布式系统服务器上下线动态感知程序开发：

DistributedServer

package zookeeper.application.topandbottomline;

import org.apache.zookeeper.*;

import java.io.IOException;

/**
 * @program:bigdata
 * @package:zookeeper.application.topandbottomline
 * @filename:DistributedServer.java
 * @create:2019.09.19.16.46
 * @author:Administrator
 * @descrption.
 */
public class DistributedServer {
    private ZooKeeper zk=null;
    private static final String connectString="192.168.186.5:2181,192.168.186.4:2181,192.168.186.3:2181";
    private static final int sessionTimout=2000;
    private static final String parentNode="/services";
    public void getConnect() throws IOException {
        zk=new ZooKeeper(connectString, sessionTimout, new Watcher() {
            public void process(WatchedEvent watchedEvent) {
                System.out.println(watchedEvent.getType()+"..."+watchedEvent.getPath());
                try {
                    zk.getChildren("/",true);
                } catch (KeeperException e) {
                    e.printStackTrace();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        });
    }

    public void registerServer(String hostname) throws KeeperException, InterruptedException {
        String s = zk.create(parentNode + "/service", hostname.getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL_SEQUENTIAL);
        System.out.println(hostname+"is online.."+s);
    }
    //业务功能
    public void handleBussiness(String hostname) throws InterruptedException {
        System.out.println(hostname+"start working..");
        Thread.sleep(Long.MAX_VALUE);
    }
    public static void main(String[] args) throws Exception {
        //获取jk连接
        DistributedServer server = new DistributedServer();
        server.getConnect();
        //利用zk连接注册服务器信息
        server.registerServer(args[0]);
        //启动业务逻辑
        server.handleBussiness(args[0]);
    }
}

DistributedClient

volatile关键字的作用：只能有一个线程复制该数据到其工作空间

package zookeeper.application.topandbottomline;

import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooKeeper;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

/**
 * @program:bigdata
 * @package:zookeeper.application.topandbottomline
 * @filename:DistributedClient.java
 * @create:2019.09.19.18.48
 * @author:Administrator
 * @descrption.
 */
public class DistributedClient {
    private ZooKeeper zk=null;
    private static final String connectString="192.168.186.5:2181,192.168.186.4:2181,192.168.186.3:2181";
    private static final int sessionTimout=2000;
    private static final String parentNode="/services";
    //加volatile
    private volatile List serviceList;
    public void getConnect() throws IOException {
        zk=new ZooKeeper(connectString, sessionTimout, new Watcher() {
            public void process(WatchedEvent watchedEvent) {
                System.out.println(watchedEvent.getType()+"..."+watchedEvent.getPath());
                try {
                    getServiceList();
                } catch (KeeperException e) {
                    e.printStackTrace();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        });
    }


    public void getServiceList() throws KeeperException, InterruptedException {
        List children = zk.getChildren(parentNode, true);
        List services =new ArrayList();
        for (String child:services){
            byte[] data = zk.getData(parentNode + "/" + child, false, null);
            services.add(new String(data));
        }
        serviceList=services;
        System.out.println(serviceList);
    }

    //业务功能
    public void handleBussiness() throws InterruptedException {
        System.out.println("client start working..");
        Thread.sleep(Long.MAX_VALUE);
    }

    public static void main(String[] args) throws Exception {
        //获取zk连接
        DistributedClient client = new DistributedClient();
        client.getConnect();

        //获取services的子节点信息并监听，从中获取服务器列表
        client.getServiceList();
        client.handleBussiness();

    }
}

zookeeper的监听和其他线程都是守护线程（随着主线程的消失而消失）

2.分布式共享锁的程序逻辑流程