垂直拆分
水平拆分
垂直+水平拆分
拦截
Mycat 的原理中最重要的一个动词是“拦截”,它拦截了用户发送过来的 SQL 语句,首先对 SQL 语句做了
一些特定的分析:如分片分析、路由分析、读写分离分析、缓存分析等,然后将此 SQL 发往后端的真实数据库,
并将返回的结果做适当的处理,最终再返回给用户
这种方式把数据库的分布式从代码中解耦出来,程序员察觉不出来后台使用mycat还是mysql。
1.6版本架构图:
(1)下载安装包:去官网( http://www.mycat.io/ )
Mycat-server-1.6-RELEASE-20161028204710-linux.tar.gz
(2)解压缩文件拷贝到linux下 /usr/local/
tar -zxvf Mycat-server-1.6-RELEASE-20161028204710-linux.tar.gz -C /usr/local/
(3)安装完成!
schema.xml
作用:定义逻辑库,表、分片节点等内容
rule.xml
作用:定义分片规则
server.xml
作用:定义用户以及系统相关变量,如端口等.
<mycat:schema xmlns:mycat="http://io.mycat/">
<schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100" dataNode="dn1">schema>
<dataNode name="dn1" dataHost="localhost1" database="db1" />
<dataHost name="localhost1" maxCon="1000" minCon="10" balance="0"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()heartbeat>
<writeHost host="hostM1" url="localhost:3306" user="root" password="123456">
writeHost>
dataHost>
mycat:schema>
(1):逻辑库!schema中的name是虚拟出来的逻辑库的数据库名
(2):dataNode中的name对应schema中的dataNode的值,可以有多个dataNode,根据需求指定!
database是真实数据库中的库名
(3):dataHost中的name对应dataNode中的dataHost得值!
balance指的是负载均衡的类型(后面有详细说明)
<user name="mycat">
<property name="password">654321property>
<property name="schemas">TESTDBproperty>
user>
mysql -uroot -p123123 -h 192.168.154.1 -P 3306
mysql -uroot -p123123 -h 192.168.154.154 -P 3306
如本机远程访问报错,请建对应用户
grant all privileges on . to root@'缺少的host' identified by '123123';
控制台启动 :去mycat/bin 目录下 mycat console
后台启动 :去mycat/bin 目录下 mycat start
域名解析失败
在 127.0.0.1 后面增加你的机器名
(2)修改后重新启动网络服务
mysql -uroot -p654321 -P9066 -h192.168.67.131
命令:
show database
show @@help
mysql -uroot -p654321 -P8066 -h192.168.67.131
<mycat:schema xmlns:mycat="http://io.mycat/">
<schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100" dataNode="dn1">
schema>
<dataNode name="dn1" dataHost="host1" database="user" />
<dataHost name="host1" maxCon="1000" minCon="10" balance="2"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()heartbeat>
<writeHost host="hostm1" url="主机(写数据的服务器)IP地址:3306" user="用户名"
password="密码">
<readHost host="hosts1" url="从机(读机器的服务器)IP地址:3306" user="用户名"
password="密码">
readHost>
writeHost>
dataHost>
mycat:schema>
配置server.xml
<user name="mycat">
<property name="password">654321property>
<property name="schemas">TESTDBproperty>
user>
(1)创建库/表(从机和主机都要有)
create database user;
create table t_replica
( id int auto_increment primary key,
name varchar(200)
);
(2)分别在主机和从机两个库下插入
insert into t_replica(name) values (@@hostname)
(3)使用mycat账号登录数据库
mysql -umycat -p654321 -h 192.168.186.100(mycat所在的服务器IP)-P 3306
use TESTDB;
select * from t_replica;
#客户表 rows:20万
CREATE TABLE customer(
id INT AUTO_INCREMENT,
NAME VARCHAR(200),
PRIMARY KEY(id)
);
#订单表 rows:600万
CREATE TABLE orders(
id INT AUTO_INCREMENT,
order_type INT,
customer_id INT,
amount DECIMAL(10,2),
PRIMARY KEY(id)
);
#订单详细表 rows:600万
CREATE TABLE orders_detail(
id INT AUTO_INCREMENT,
detail VARCHAR(2000),
order_id INT,
PRIMARY KEY(id)
);
#订单状态字典表 rows:20
CREATE TABLE dict_order_type(
id INT AUTO_INCREMENT,
order_type VARCHAR(200),
PRIMARY KEY(id)
);
#测试联合查询
select o.*,od.detail
from orders o
inner join orders_detail od on o.id =od.order_id;
<mycat:schema xmlns:mycat="http://io.mycat/">
<schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100" dataNode="dn1">
<table name="customer" dataNode="dn2" >table>
schema>
<dataNode name="dn1" dataHost="host1" database="shopping" />
<dataNode name="dn2" dataHost="host2" database="shopping" />
<dataHost name="host1" maxCon="1000" minCon="10" balance="2"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()heartbeat>
<writeHost host="hostm1" url="第一台数据库服务器IP:3306" user="root"
password="123123">
writeHost>
dataHost>
<dataHost name="host2" maxCon="1000" minCon="10" balance="0"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()heartbeat>
<writeHost host="hostm2" url="第二台数据库服务器IP:3306" user="root"
password="123123">
writeHost>
dataHost>
mycat:schema>
<mycat:schema xmlns:mycat="http://io.mycat/">
<schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100" dataNode="dn1">
<table name="customer" dataNode="dn2" >table>
<table name="orders" dataNode="dn1,dn2" rule="mod_rule" >table>
schema>
<dataNode name="dn1" dataHost="host1" database="shopping" />
<dataNode name="dn2" dataHost="host2" database="shopping" />
<dataHost name="host1" maxCon="1000" minCon="10" balance="2"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()heartbeat>
<writeHost host="hostm1" url="第一台数据库服务器IP:3306" user="root"
password="123123">
writeHost>
dataHost>
<dataHost name="host2" maxCon="1000" minCon="10" balance="0"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()heartbeat>
<writeHost host="hostm2" url="第二台数据库服务器IP:3306" user="root"
password="123123">
writeHost>
dataHost>
<tableRule name="mod_rule">
<rule>
<columns>customer_idcolumns>
<algorithm>mod-longalgorithm>
rule>
tableRule>
<function name="mod-long" class="io.mycat.route.function.PartitionByMod">
<property name="count">2property>
function>
为了相关联的表的行尽量分在一个库下
schema.xml
<mycat:schema xmlns:mycat="http://io.mycat/">
<schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100" dataNode="dn1">
<table name="customer" dataNode="dn2" >table>
<table name="orders" dataNode="dn1,dn2" rule="mod_rule" >
<childTable name="orders_detail" primaryKey="id" joinKey="order_id" parentKey="id" />
table>
<table name="dict_status" dataNode="dn1,dn2" type="global" >table>
schema>
<dataNode name="dn1" dataHost="host1" database="shopping" />
<dataNode name="dn2" dataHost="host2" database="shopping" />
<dataHost name="host1" maxCon="1000" minCon="10" balance="2"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()heartbeat>
<writeHost host="hostm1" url="第一台数据库服务器IP:3306" user="root"
password="123123">
writeHost>
dataHost>
<dataHost name="host2" maxCon="1000" minCon="10" balance="0"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()heartbeat>
<writeHost host="hostm2" url="第二台数据库服务器IP" user="root"
password="123123">
writeHost>
dataHost>
mycat:schema>
建相关表
#客户表 rows:20万
Create table customer(
id int auto_increment,
name varchar(200),
primary key(id)
);
#订单表 rows:600万
create table orders(
id int auto_increment,
order_type int,
customer_id int,
amount decimal(10,2),
PRIMARY KEY(id)
);
#订单详情表 rows:600万
create table orders_detail(
id INT AUTO_INCREMENT,
detail varchar(2000),
order_id int,
PRIMARY KEY(id)
);
#订单状态字典表 rows:20
create table dict_order_type(
id INT AUTO_INCREMENT,
order_type varchar(200),
PRIMARY KEY(id)
);
TRUNCATE TABLE
insert into customer values(100,'zhang3');
INSERT INTO customer VALUES(101,'li4');
insert into orders(id,order_type,customer_id,amount) values(1,101,100,100100);
INSERT INTO orders(id,order_type,customer_id,amount) VALUES(2,101,100,100300);
INSERT INTO orders(id,order_type,customer_id,amount) VALUES(3,101,101,120000);
INSERT INTO orders(id,order_type,customer_id,amount) VALUES(4,101,101,103000);
INSERT INTO orders(id,order_type,customer_id,amount) VALUES(5,102,101,100400);
INSERT INTO orders(id,order_type,customer_id,amount) VALUES(6,102,100,100020);
insert into dict_order_type values(101,'type1');
insert into dict_order_type values(101,'type1');
select * from orders o inner join customer c on o.customer_id=c.id;
select * from orders o inner join dict_order_type ot on ot.id=o.order_type;
select * from orders
SELECT * FROM orders o INNER JOIN orders_detail od ON od.order_id=o.id;
insert into orders_detail(id,detail,order_id) values(1,'detail1',1);
INSERT INTO orders_detail(id,detail,order_id) VALUES(2,'detail1',2);
INSERT INTO orders_detail(id,detail,order_id) VALUES(3,'detail1',3);
INSERT INTO orders_detail(id,detail,order_id) VALUES(4,'detail1',4);
INSERT INTO orders_detail(id,detail,order_id) VALUES(5,'detail1',5);
INSERT INTO orders_detail(id,detail,order_id) VALUES(6,'detail1',6);
use TESTDB;
delete from dict_order_type;
insert into dict_order_type(id,order_type) values(101,'type1');
INSERT INTO dict_order_type(id,order_type) VALUES(102,'type2');
alter table orders change id id bigint;
insert into `orders`(id,amount,customer_id,order_type) values( 1277286175459119104,1000,103,101);
select * from orders;
设定为全局的表,会直接复制给每个数据库一份,所有写操作也会同步给多个库。
所以全局表一般不能是大数据表或者更新频繁的表
一般是字典表或者系统表为宜。
schema.xml
<mycat:schema xmlns:mycat="http://io.mycat/">
<schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100" dataNode="dn1">
<table name="customer" dataNode="dn2" >table>
<table name="orders" dataNode="dn1,dn2" rule="mod_rule" >table>
<table name="dict_order_type" dataNode="dn1,dn2" type="global" >table>
schema>
<dataNode name="dn1" dataHost="host1" database="atguigu_mc" />
<dataNode name="dn2" dataHost="host2" database="atguigu_sm" />
<dataHost name="host1" maxCon="1000" minCon="10" balance="2"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()heartbeat>
<writeHost host="hostm1" url="192.168.67.1:3306" user="root"
password="123123">
writeHost>
dataHost>
<dataHost name="host2" maxCon="1000" minCon="10" balance="0"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()heartbeat>
<writeHost host="hostm2" url="192.168.67.1:3306" user="root"
password="123123">
writeHost>
dataHost>
win10
CREATE TABLE MYCAT_SEQUENCE (NAME VARCHAR(50) NOT NULL,current_value INT NOT
NULL,increment INT NOT NULL DEFAULT 100, PRIMARY KEY(NAME)) ENGINE=INNODB;
DELIMITER $$
CREATE FUNCTION mycat_seq_currval(seq_name VARCHAR(50)) RETURNS VARCHAR(64)
DETERMINISTIC
BEGIN
DECLARE retval VARCHAR(64);
SET retval="-999999999,null";
SELECT CONCAT(CAST(current_value AS CHAR),",",CAST(increment AS CHAR)) INTO retval FROM
MYCAT_SEQUENCE WHERE NAME = seq_name;
RETURN retval;
END $$
DELIMITER ;
DELIMITER $$
CREATE FUNCTION mycat_seq_setval(seq_name VARCHAR(50),VALUE INTEGER) RETURNS VARCHAR(64)
DETERMINISTIC
BEGIN
UPDATE MYCAT_SEQUENCE
SET current_value = VALUE
WHERE NAME = seq_name;
RETURN mycat_seq_currval(seq_name);
END $$
DELIMITER ;
DELIMITER $$
CREATE FUNCTION mycat_seq_nextval(seq_name VARCHAR(50)) RETURNS VARCHAR(64)
DETERMINISTIC
BEGIN
UPDATE MYCAT_SEQUENCE
SET current_value = current_value + increment WHERE NAME = seq_name;
RETURN mycat_seq_currval(seq_name);
END $$
DELIMITER ;
SELECT * FROM MYCAT_SEQUENCE
TRUNCATE TABLE MYCAT_SEQUENCE
##增加要用的序列
INSERT INTO MYCAT_SEQUENCE(NAME,current_value,increment) VALUES ('ORDERS', 400000,
100);
sequence_db_conf.properties
vim sequence_db_conf.properties
意思是 ORDERS这个序列在dn1这个节点上,具体dn1节点是哪台机子,请参考schema.xml
server.xml
vim server.xml
insert into orders(id,amount,customer_id,order_type) values( 1277286175467507712,1000,101,102);
长度:18位
缺点:比较长
根据业务逻辑组合
可以利用 redis的单线程原子性 incr来生成序列
首先要准备一些服务器,并且创建数据库(其中没有任何数据)!根据已有的数据库,分析哪些表、哪些库需要分表、分库,配置好Mycat的配置文件,就可以使用了!
(1)将服务器上对应的账号开放远程访问权限
(2)关闭防火墙
(3)服务器在同一个网关下
(4)修改完Mycat的配置文件,需要重新启动(mycat console)
(5)要保证数据库都能通过远程访问
(6)固定好服务器的IP地址