基于mysql5.6之GTID新特性的主从模式下的amoeba代理

系统：centos6.4

master：192.168.248.200

slave：192.168.248.201

amoeba：192.168.248.202

master：

为了方便选用的mysql为通用二进制：mysql-5.6.30-linux-glibc2.5-x86_64.tar.gz

tar xf mysql-5.6.30-linux-glibc2.5-x86_64.tar.gz -C /usr/local/

ln -s mysql-5.6.30-linux-glibc2.5-x86_64 /usr/local/mysql

useradd -r mysql 

mkdir /mydata/data -pv 

chown -R root:mysql /usr/local/mysql

chown -R mysql:mysql /mydata/data

cd /uar/local/mysql

scripts/mysql_install_db --user=mysql --datadir=/mydata/data

vi /etc/profile.d/mysqld.sh

export PATH=$PATH:/usr/local/mysql/bin

. /etc/profile.d/mysqld.sh

cp support-files/mysql.server /etc/init.d/mysqld

vi my.cnf

###################启用新特性GTID#############################

datadir = /mydata/data:数据目录

binlog-format = row：二进制格式

log-slave-updates = true：从服务器日志更新

gtid-mode = on：启用GTID特性

enforce-gtid-consistency = true：是否强制GTID一致性

master-info-repository = TABLE：master.info用表的格式保存

relay-log-info-repository = TABLE：relay-log.info用表的格式保存

sync-master-info = 1：确保无信息丢失

slave-parallel-workers = 2：设置从服务器sql线程个数

binlog-checksum = CRC32：启用二进制日志校验功能

master-verify-checksum = 1

slave-sql-verify-checksum = 1

binlog-rows-query-log_events = 1：记录二进制日志的额外信息（并非强制）

server-id = 200

report-port = 3306

port = 3306

datadir = /mydata/data

socket = /tmp/mysql.sock

report-host = master

log-bin = mysql-bin

innodb_file_per_table = 1

确保无误后，可以开始启动mysql服务器了

service mysqld start

如有错误，记得一定得看数据目录下的错误日志，如master.err

slave:

tar xf mysql-5.6.30-linux-glibc2.5-x86_64.tar.gz -C /usr/local/

ln -s mysql-5.6.30-linux-glibc2.5-x86_64 /usr/local/mysql

useradd -r mysql 

mkdir /mydata/data -pv 

chown -R root:mysql /usr/local/mysql

chown -R mysql:mysql /mydata/data

cd /uar/local/mysql

scripts/mysql_install_db --user=mysql --datadir=/mydata/data

vi /etc/profile.d/mysqld.sh

export PATH=$PATH:/usr/local/mysql/bin

. /etc/profile.d/mysqld.sh

cp support-files/mysql.server /etc/init.d/mysqld

vi my.cnf

###################启用新特性GTID#############################

datadir = /mydata/data:数据目录

relay-log = relay-mysql

binlog-format = row：二进制格式

log-slave-updates = true：从服务器日志更新

gtid-mode = on：启用GTID特性

enforce-gtid-consistency = true：是否强制GTID一致性

master-info-repository = TABLE：master.info用表的格式保存

relay-log-info-repository = TABLE：relay-log.info用表的格式保存

sync-master-info = 1：确保无信息丢失

slave-parallel-workers = 2：设置从服务器sql线程个数

binlog-checksum = CRC32：启用二进制日志校验功能

master-verify-checksum = 1

slave-sql-verify-checksum = 1

binlog-rows-query-log_events = 1：记录二进制日志的额外信息（并非强制）

server-id = 201

report-port = 3306

port = 3306

datadir = /mydata/data

socket = /tmp/mysql.sock

report-host = slave

log-bin = mysql-bin

innodb_file_per_table = 1

于是开始创建复制的账号密码：

master：

grant replication slave on *.* to 'rep'@'192.168.248.%' identified by 'redhat';

flush privileges;

注意：主从服务器的防火墙和selinux关闭掉：

service iptables stop

setenforce 0

sed -i 's#SELINUX=enforce#SELINUX=permissive#g' /etc/selinux/config

开始主从复制步骤：

slave:

mysql>change master to master_host='192.168.248.200',master_user='rep',master_password='redhat',master_auto_position=1;

mysql>start slave;

mysql>show slave status\G

mysql> show slave status\G

*************************** 1. row ***************************

               Slave_IO_State: Waiting for master to send event

                  Master_Host: 192.168.248.200

                  Master_User: rep

                  Master_Port: 3306

                Connect_Retry: 60

              Master_Log_File: mysql-bin.000001

          Read_Master_Log_Pos: 1091

               Relay_Log_File: relay-bin.000003

                Relay_Log_Pos: 986

        Relay_Master_Log_File: mysql-bin.000001

             Slave_IO_Running: Yes

            Slave_SQL_Running: Yes

              Replicate_Do_DB: 

          Replicate_Ignore_DB: 

           Replicate_Do_Table: 

       Replicate_Ignore_Table: 

      Replicate_Wild_Do_Table: 

  Replicate_Wild_Ignore_Table: 

                   Last_Errno: 0

                   Last_Error: 

                 Skip_Counter: 0

          Exec_Master_Log_Pos: 1091

              Relay_Log_Space: 1796

              Until_Condition: None

               Until_Log_File: 

                Until_Log_Pos: 0

           Master_SSL_Allowed: No

           Master_SSL_CA_File: 

           Master_SSL_CA_Path: 

              Master_SSL_Cert: 

            Master_SSL_Cipher: 

               Master_SSL_Key: 

        Seconds_Behind_Master: 0

Master_SSL_Verify_Server_Cert: No

                Last_IO_Errno: 0

                Last_IO_Error: 

               Last_SQL_Errno: 0

               Last_SQL_Error: 

  Replicate_Ignore_Server_Ids: 

             Master_Server_Id: 200

                  Master_UUID: c5b08767-1124-11e6-82a6-000c293d8d1b

             Master_Info_File: mysql.slave_master_info

                    SQL_Delay: 0

          SQL_Remaining_Delay: NULL

      Slave_SQL_Running_State: Slave has read all relay log; waiting for the slave I/O thread to update it

           Master_Retry_Count: 86400

                  Master_Bind: 

      Last_IO_Error_Timestamp: 

     Last_SQL_Error_Timestamp: 

               Master_SSL_Crl: 

           Master_SSL_Crlpath: 

           Retrieved_Gtid_Set: c5b08767-1124-11e6-82a6-000c293d8d1b:1-5

            Executed_Gtid_Set: 8812b0d5-1126-11e6-82b2-000c29b7e7fb:1-3,

c5b08767-1124-11e6-82a6-000c293d8d1b:1-5

                Auto_Position: 1

1 row in set (0.00 sec)

两个yes，说明了主从复制基于GTID已经成功了

于是开始搭建amoeba代理服务器

192.168.248.202：

由于amoeba是基于java开发的于是必须要支持jdk:

jdk-6u45-linux-x64.bin

cd /root

./jdk-6u45-linux-x64.bin

vi /etc/profile.d/java.sh

export JAVA_HOME=/root/jdk1.6.0_45/

export PATH=$PATH:$JAVA_HOME

. /etc/profile.d/java.sh

java -versiom

[root@RS1 ~]# java -version

java version "1.6.0_45"

Java(TM) SE Runtime Environment (build 1.6.0_45-b06)

Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)

安装下载好的amoeba：

amoeba-mysql-3.0.4-BETA-distribution.zip

unzip amoeba-mysql-3.0.4-BETA-distribution.zip /usr/local

mv amoeba-mysql-3.0.4-BETA-distribution/* /usr/local/amoeba

cd /usr/local/amoeba

cd conf

vi amoeba.xml

<?xml version="1.0" encoding="gbk"?>

<!DOCTYPE amoeba:configuration SYSTEM "amoeba.dtd">

<amoeba:configuration xmlns:amoeba="http://amoeba.meidusa.com/">

        <proxy>

                <service name="Amoeba for Mysql" class="com.meidusa.amoeba.mysql.server.MySQLService">

                        <property name="port">3306</property>:代理服务器的端口

                        <!-- bind ipAddress -->

                        <property name="ipAddress">192.168.248.202</property>：amoeba服务器ip

                        <property name="connectionFactory">

                                <bean class="com.meidusa.amoeba.mysql.net.MysqlClientConnectionFactory">

                                        <property name="sendBufferSize">128</property>

                                        <property name="receiveBufferSize">64</property>

                                </bean>

                        </property>

                        <property name="authenticateProvider">

                                <bean class="com.meidusa.amoeba.mysql.server.MysqlClientAuthenticator">

                                        <property name="user">root</property>：设定amoeba服务器的登录用户

                                        <property name="password">redhat</property>：设置amoeba服务器的登录密码

                                        <property name="filter">

                                                <bean class="com.meidusa.toolkit.net.authenticate.server.IPAccessController">

                                                </bean>

                                        </property>

                                </bean>

                        </property>

                </service>

                <runtime class="com.meidusa.amoeba.mysql.context.MysqlRuntimeContext">

                        <!-- proxy server client process thread size -->

                        <property name="executeThreadSize">128</property>

                        <!-- per connection cache prepared statement size  -->

                        <property name="statementCacheSize">500</property>

                        <!-- query timeout( default: 60 second , TimeUnit:second) -->

                        <property name="queryTimeout">60</property>

                </runtime>

        </proxy>

        <!--

                Each ConnectionManager will start as thread

                manager responsible for the Connection IO read , Death Detection

        -->

        <connectionManagerList>

                <connectionManager name="defaultManager" class="com.meidusa.toolkit.net.MultiConnectionManagerWrapper">

                        <property name="subManagerClassName">com.meidusa.toolkit.net.AuthingableConnectionManager</property>

                </connectionManager>

        </connectionManagerList>

                <!-- default using file loader -->

        <dbServerLoader class="com.meidusa.amoeba.context.DBServerConfigFileLoader">

                <property name="configFile">${amoeba.home}/conf/dbServers.xml</property>

        </dbServerLoader>

        <queryRouter class="com.meidusa.amoeba.mysql.parser.MysqlQueryRouter">

                <property name="ruleLoader">

                        <bean class="com.meidusa.amoeba.route.TableRuleFileLoader">

                                <property name="ruleFile">${amoeba.home}/conf/rule.xml</property>

                                <property name="functionFile">${amoeba.home}/conf/ruleFunctionMap.xml</property>

                        </bean>

                </property>

                <property name="sqlFunctionFile">${amoeba.home}/conf/functionMap.xml</property>

                <property name="LRUMapSize">1500</property>

                <property name="defaultPool">server1</property>：默认池，一般为master也就是server1

                <!--

                <property name="writePool">server1</property>：写池：master

                <property name="readPool">multiPool</property>：读池：slave也就是server2

                -->

                <property name="needParse">true</property>

        </queryRouter>

</amoeba:configuration>

#############################################################################

vi dbServers.xml

<?xml version="1.0" encoding="gbk"?>

<!DOCTYPE amoeba:dbServers SYSTEM "dbserver.dtd">

<amoeba:dbServers xmlns:amoeba="http://amoeba.meidusa.com/">

                <!--

                        Each dbServer needs to be configured into a Pool,

                         such as 'multiPool' dbServer

                -->

        <dbServer name="abstractServer" abstractive="true">

                <factoryConfig class="com.meidusa.amoeba.mysql.net.MysqlServerConnectionFactory">

                        <property name="connectionManager">${defaultManager}</property>

                        <property name="sendBufferSize">64</property>

                        <property name="receiveBufferSize">128</property>

                        <!-- mysql port -->

                        <property name="port">3306</property>：监听的端口

                        <!-- mysql schema -->

                        <property name="schema">test</property>：默认的数据库

                        <!-- mysql user -->

                        <property name="user">amoeba</property>：amoeba所起用的后端用户名

                        <property name="password">redhat</property>amoeba所起用的后端用户名密码

                </factoryConfig>

                <poolConfig class="com.meidusa.toolkit.common.poolable.PoolableObjectPool">

                        <property name="maxActive">500</property>

                        <property name="maxIdle">500</property>

                        <property name="minIdle">1</property>

                        <property name="minEvictableIdleTimeMillis">600000</property>

                        <property name="timeBetweenEvictionRunsMillis">600000</property>

                        <property name="testOnBorrow">true</property>

                        <property name="testOnReturn">true</property>

                        <property name="testWhileIdle">true</property>

                </poolConfig>

        </dbServer>

        <dbServer name="server1"  parent="abstractServer">：定义主库节点

                <factoryConfig>

                        <!-- mysql ip -->

                        <property name="ipAddress">192.168.248.200</property>

                </factoryConfig>

        </dbServer>

        <dbServer name="server2"  parent="abstractServer">：定义从库节点

                <factoryConfig>

                        <!-- mysql ip -->

                        <property name="ipAddress">192.168.248.201</property>

                </factoryConfig>

        </dbServer>

        <dbServer name="multiPool" virtual="true">：定义multi池

                <poolConfig class="com.meidusa.amoeba.server.MultipleServerPool">

                        <!-- Load balancing strategy: 1=ROUNDROBIN , 2=WEIGHTBASED , 3=HA-->

                        <property name="loadbalance">1</property>：负载均衡算法：轮询

                        <!-- Separated by commas,such as: server1,server2,server1 -->

                        <property name="poolNames">server2</property>：从服务器，有多个时，用逗号隔开

                </poolConfig>

        </dbServer>

</amoeba:dbServers>

定义完之后，启动amoeba

/usr/local/amoeba/bin/launcher &:放置在后端执行

[root@RS1 conf]# netstat -tnlp

Active Internet connections (only servers)

Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name   

tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      1082/sshd           

tcp        0      0 127.0.0.1:25                0.0.0.0:*                   LISTEN      1158/master         

tcp        0      0 :::80                       :::*                        LISTEN      2288/httpd          

tcp        0      0 :::22                       :::*                        LISTEN      1082/sshd           

tcp        0      0 ::1:25                      :::*                        LISTEN      1158/master         

tcp        0      0 ::ffff:192.168.248.202:3306 :::*                        LISTEN      4219/java    

可以看见java程序进程已经启动了，即amoeba已经启动了

用一个可以登录的mysql客户端登录amoeba服务器:

[root@master ~]# mysql -uroot -p -h192.168.248.202

Enter password: 

Welcome to the MySQL monitor.  Commands end with ; or \g.

Your MySQL connection id is 567038057

Server version: 5.1.45-mysql-amoeba-proxy-3.0.4-BETA MySQL Community Server (GPL):可以看出已经成功登陆

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> 

断开从服务器的复制：

stop slave;

主从服务器各为amoeba创建一个默认数据库为test的用户，与配置文件一致：

master：

grant all privileges on *.* to 'amoeba'@'192.168.248.%' identified by 'redhat';

slave:

grant all privileges on *.* to 'amoeba'@'192.168.248.%' identified by 'redhat';

创建完之后再开启复制功能：

start slave；

回到amoeba服务器：

[root@master ~]# mysql -uroot -p -h192.168.248.202

Enter password: 

Welcome to the MySQL monitor.  Commands end with ; or \g.

Your MySQL connection id is 567038057

Server version: 5.1.45-mysql-amoeba-proxy-3.0.4-BETA MySQL Community Server (GPL):可以看出已经成功登陆

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

执行读写操作的sql语句，主从服务器各自用tcpdump进行抓包：

master：

[root@master mysql]# tcpdump  -i eth0 -s0 -nn -A tcp  dst port 3306 and dst host 192.168.248.200

slave：

[root@slave mysql]# tcpdump  -i eth0 -s0 -nn -A tcp  dst port 3306 and dst host 192.168.248.201

记录：

不知道为啥主从抓包的时候，读写操作的sql语句都在主服务器上可以抓到，而从服务器一直都抓不到

改变了amoeba的配置文件依然还是完全的实现都写操作，这里记录一下结果