Amoeba For MySQL_测试Amoeba的数据切分功能_CentOS6.5下进行
测试Amoeba的数据分片功能
1. 测试目的
l 验证Amoeba具有数据分片功能。
l 测试Amoeba进行数据切片后进行聚合的情况，是否支持常见的条件查询操作、SQL聚合操作等。
2. 测试环境
2.1 操作系统
| 操作系统类型 |
配置 |
| Windows 7 旗舰版 |
处理器: AMD A8-3500M APU with Radeon(tm) HD Graphic 1.50HZ 内存: 4.00GB ( 3.48G可用 ) 系统类型: 64位操作系统 |
| VMWare 10 CentOS6.5 |
处理器数量: 1 内存: 512MB 系统类型: 64位操作系统 |
2.2 服务器配置
| 服务器名 |
配置 |
| server1(同时充当Amoeba代理服务器)
|
IP: 192.168.52.128 OS: VMWare 10 CentOS6.5 MySQL Version: 5.5.38 x64 Java Version: 1.6 x64 (Amoeba依赖) Amoeba Version: amoeba-mysql-3.0.4-BETA |
| server2 |
IP: 192.168.52.129 OS: VMWare 10 CentOS6.5 MySQL Version: 5.5.38 x64 |
| testClient(模拟客户端应用进行测试) |
IP: 192.168.16.125 OS: Windows 7旗舰版 MySQL Version: 5.5.37 x86 |
3. 测试前的配置
3.1 确保testClient可以连接到server1和server2
在server1和server2上分别对来自testClient[192.168.16.125]的远程登录进行授权。
在server1和server2上的MySQL Server上用root登录执行以下授权命令
grant allprivileges on *.* to ‘root’ @ ‘[ip地址]’ identified by ‘[登录密码]’ with grant option;
3.2 配置Amoeba代理服务器上的dbServers.xml、rule.xml和amoeba.xml文件
( 需要特别注意红色部分的设定 )
dbServers.xml
| <?xml version="1.0" encoding="gbk"?>
<!DOCTYPE amoeba:dbServers SYSTEM "dbserver.dtd"> <amoeba:dbServers xmlns:amoeba="http://amoeba.meidusa.com/">
<!-- Each dbServer needs to be configured into a Pool, If you need to configure multiple dbServer with load balancing that can be simplified by the following configuration: add attribute with name virtual = "true" in dbServer, but the configuration does not allow the element with name factoryConfig such as 'multiPool' dbServer -->
<dbServer name="abstractServer" abstractive="true"> <factoryConfig class="com.meidusa.amoeba.mysql.net.MysqlServerConnectionFactory"> <property name="connectionManager">${defaultManager}</property> <property name="sendBufferSize">64</property> <property name="receiveBufferSize">128</property>
<!-- mysql port --> <property name="port">3306</property>
<!-- mysql schema --> <property name="schema">test</property>
<!-- mysql user --> <property name="user">root</property>
<property name="password">root</property> </factoryConfig>
<poolConfig class="com.meidusa.toolkit.common.poolable.PoolableObjectPool"> <property name="maxActive">500</property> <property name="maxIdle">500</property> <property name="minIdle">1</property> <property name="minEvictableIdleTimeMillis">600000</property> <property name="timeBetweenEvictionRunsMillis">600000</property> <property name="testOnBorrow">true</property> <property name="testOnReturn">true</property> <property name="testWhileIdle">true</property> </poolConfig> </dbServer>
<dbServer name="server1" parent="abstractServer"> <factoryConfig> <!-- mysql ip --> <property name="ipAddress">192.168.52.128</property> </factoryConfig> </dbServer>
<dbServer name="server2" parent="abstractServer"> <factoryConfig> <!-- mysql ip --> <property name="ipAddress">192.168.52.129</property> </factoryConfig> </dbServer>
<dbServer name="multiPool" virtual="true"> <poolConfig class="com.meidusa.amoeba.server.MultipleServerPool"> <!-- Load balancing strategy: 1=ROUNDROBIN , 2=WEIGHTBASED , 3=HA--> <property name="loadbalance">1</property>
<!-- Separated by commas,such as: server1,server2,server1 --> <property name="poolNames">server1,server2</property> </poolConfig> </dbServer>
</amoeba:dbServers> |
rule.xml
| <?xml version="1.0" encoding="gbk"?> <!DOCTYPE amoeba:rule SYSTEM "rule.dtd">
<amoeba:rule xmlns:amoeba="http://amoeba.meidusa.com/"> <tableRule name="staff" schema="test" defaultPools="server1,server2">
<rule name="rule1" ruleResult="POOLNAME"> <parameters>ID</parameters> <expression><![CDATA[ var division = ID % 2; case division when 0 then 'server1'; when 1 then 'server2'; end case; ]]></expression> </rule> </tableRule> </amoeba:rule> |
amoeba.xml
| <?xml version="1.0" encoding="gbk"?>
<!DOCTYPE amoeba:configuration SYSTEM "amoeba.dtd"> <amoeba:configuration xmlns:amoeba="http://amoeba.meidusa.com/">
<proxy>
<!-- service class must implements com.meidusa.amoeba.service.Service --> <service name="Amoeba for Mysql" class="com.meidusa.amoeba.mysql.server.MySQLService"> <!-- port --> <property name="port">8066</property>
<!-- bind ipAddress --> <!-- <property name="ipAddress">127.0.0.1</property> -->
<property name="connectionFactory"> <bean class="com.meidusa.amoeba.mysql.net.MysqlClientConnectionFactory"> <property name="sendBufferSize">128</property> <property name="receiveBufferSize">64</property> </bean> </property>
<property name="authenticateProvider"> <bean class="com.meidusa.amoeba.mysql.server.MysqlClientAuthenticator">
<property name="user">root</property>
<property name="password">root</property>
<property name="filter"> <bean class="com.meidusa.toolkit.net.authenticate.server.IPAccessController"> <property name="ipFile">${amoeba.home}/conf/access_list.conf</property> </bean> </property> </bean> </property>
</service>
<runtime class="com.meidusa.amoeba.mysql.context.MysqlRuntimeContext">
<!-- proxy server client process thread size --> <property name="executeThreadSize">128</property>
<!-- per connection cache prepared statement size --> <property name="statementCacheSize">500</property>
<!-- query timeout( default: 60 second , TimeUnit:second) --> <property name="queryTimeout">60</property> </runtime>
</proxy>
<!-- Each ConnectionManager will start as thread manager responsible for the Connection IO read , Death Detection --> <connectionManagerList> <connectionManager name="defaultManager" class="com.meidusa.toolkit.net.MultiConnectionManagerWrapper"> <property name="subManagerClassName">com.meidusa.toolkit.net.AuthingableConnectionManager</property> </connectionManager> </connectionManagerList>
<!-- default using file loader --> <dbServerLoader class="com.meidusa.amoeba.context.DBServerConfigFileLoader"> <property name="configFile">${amoeba.home}/conf/dbServers.xml</property> </dbServerLoader>
<queryRouter class="com.meidusa.amoeba.mysql.parser.MysqlQueryRouter"> <property name="ruleLoader"> <bean class="com.meidusa.amoeba.route.TableRuleFileLoader"> <property name="ruleFile">${amoeba.home}/conf/rule.xml</property> <property name="functionFile">${amoeba.home}/conf/ruleFunctionMap.xml</property> </bean> </property> <property name="sqlFunctionFile">${amoeba.home}/conf/functionMap.xml</property> <property name="LRUMapSize">1500</property> <property name="defaultPool">multiPool</property>
<!-- <property name="writePool">server1</property> <property name="readPool">server2</property> -->
<property name="needParse">true</property> </queryRouter> </amoeba:configuration> |
4. 测试
4.1 启动数据库服务
分别启动Server1和Server2上的MySQL数据库服务(默认情况下是开机启动的)：
service mysqlstart
4.2 启动Amoeba监听程序
| # cd /usr/local/amoeba/bin # launcher
log4j:WARN log4j config load completed from file:/usr/local/amoeba/conf/log4j.xml 2014-07-08 00:39:06,909 INFO context.MysqlRuntimeContext - Amoeba for Mysql current versoin=5.1.45-mysql-amoeba-proxy-3.0.4-BETA log4j:WARN ip access config load completed from file:/usr/local/amoeba/conf/access_list.conf 2014-07-08 00:39:07,735 INFO net.ServerableConnectionManager - Server listening on 0.0.0.0/0.0.0.0:8066. 2014-07-08 00:40:35 [INFO] Project Name=Amoeba-MySQL, PID=3167 , starting... log4j:WARN log4j config load completed from file:/usr/local/amoeba/conf/log4j.xml 2014-07-08 00:40:36,597 INFO context.MysqlRuntimeContext - Amoeba for Mysql current versoin=5.1.45-mysql-amoeba-proxy-3.0.4-BETA log4j:WARN ip access config load completed from file:/usr/local/amoeba/conf/access_list.conf 2014-07-08 00:40:37,501 INFO net.ServerableConnectionManager - Server listening on 0.0.0.0/0.0.0.0:8066. 2014-07-08 00:41:26 [INFO] ignore signal:HUP 2014-07-08 00:41:56 [INFO] Project Name=Amoeba-MySQL, PID=3211 , starting... log4j:WARN log4j config load completed from file:/usr/local/amoeba/conf/log4j.xml 2014-07-08 00:41:57,745 INFO context.MysqlRuntimeContext - Amoeba for Mysql current versoin=5.1.45-mysql-amoeba-proxy-3.0.4-BETA log4j:WARN ip access config load completed from file:/usr/local/amoeba/conf/access_list.conf 2014-07-08 00:41:58,669 INFO net.ServerableConnectionManager - Server listening on 0.0.0.0/0.0.0.0:8066. |
4.3 在testClient进行测试
通过8066端口连接amoeba代理服务器
通过观察Server version确保是通过Amoeba代理服务器连接上的
测试一（数据水平切分）
这里只进行数据水平切分测试
进行DML操作
| use test; insert into staff(id, name) values(1, ‘AAA’); insert into staff(id, name) values(2, ‘BBB’); insert into staff(id, name) values(3, ‘CCC’); insert into staff(id, name) values(4, ‘DDD’); insert into staff(id, name) values(5, ‘EEE’); |
查询插入情况
(1) testClient连接8066端口情况下查询
(2)Server1连接3306端口情况下查询
| [root@localhost bin]# mysql -uroot -hlocalhost -P3306 -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 10 Server version: 5.5.38-log MySQL Community Server (GPL)
Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> use test; Database changed mysql> select * from staff; +------+------+ | id | name | +------+------+ | 2 | BBB | | 4 | DDD | +------+------+ 2 rows in set (0.01 sec) |
(3)Server2 连接3306端口情况下查询
| [root@localhost Desktop]# mysql -uroot -hlocalhost -P3306 -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 19 Server version: 5.5.38-log MySQL Community Server (GPL)
Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> use test; Database changed mysql> select * from staff; +------+------+ | id | name | +------+------+ | 1 | AAA | | 3 | CCC | | 5 | EEE | +------+------+ 3 rows in set (0.00 sec) |
试验结论：
通过以上试验，证明Amoeba确实具有数据分片功能（至少具有数据水平切分功能）。
测试二（对查询操作的支持）
确保testClient通过8066端口连接Amoeba代理服务器并执行以下命令
测试where语句
测试like语句
测试count语句
测试order排序
试验结论：
通过观察测试结果可以发现，amoeba实现数据水平切分以后，支持常规的where条件查询、模糊匹配查询，但是对于聚合函数不支持，包括count、sum等，那么可以得出对于group by和having同样是不支持的。