linux安装mycat2 搭建主从复制 读写分离 分库分表
linux安装mycat2 搭建读写分离 分库分表
- 一、安装mycat2
-
- 1. 下载mycat2压缩包
- 2. 解压
- 3. 修改权限
- 4. 配置物理地址
- 5. 启动mycat
- 6. 连接mycat
- 二、主从复制
-
- 1.环境说明
- 2.配置主从复制
- 三、读写分离
-
- 1.创建数据源
- 2.创建集群
- 3.创建逻辑库，为其配置集群
- 4.测试读写分离
- 四、集群搭建
-
- 1.新建容器
- 2.配置集群的主从复制
-
- 2.1.修改M1S2
- 2.2.修改M2
- 2.3.修改M2S1
- 五、mycat操作双主双从
-
- 1.添加数据源
- 2.修改集群配置
- 3.测试
- 4.结论
- 六、分库分表
-
- 1.配置
- 2.分库分表
- 3.关联表分表
- 4.分片算法
一、安装mycat2
1. 下载mycat2压缩包
[root@VM-0-11-centos ~]~ mkdir -p /mydata/mycat
[root@VM-0-11-centos ~]~ cd /mydata/mycat
[root@VM-0-11-centos mycat]~ wget http://dl.mycat.org.cn/2.0/install-template/mycat2-install-template-1.21.zip
[root@VM-0-11-centos mycat]~ wget http://dl.mycat.org.cn/2.0/1.21-release/mycat2-1.21-release-jar-with-dependencies.jar
2. 解压
[root@VM-0-11-centos mycat]~ yum -y install unzip
[root@VM-0-11-centos mycat]~ unzip mycat2-install-template-1.21.zip
[root@VM-0-11-centos mycat]~ cp /mydata/mycat/mycat2-1.21-release-jar-with-dependencies.jar /mydata/mycat/mycat/lib/ #解压后将mycat2-1.21-release-jar-with-dependencies.jar移动到lib目录下
3. 修改权限
将bin目录的文件加执行权限
[root@VM-0-11-centos mycat]~ cd /mydata/mycat/mycat/bin
[root@VM-0-11-centos bin]~ chmod +x *
4. 配置物理地址
为mycat代理连接启动时需要有一个默认的数据源，所以我们在启动的时候先为其准备一个数据源，用已启动的mysql就可以。然后配置物理库地址，配置文件位置：/mydata/mycat/mycat/conf/datasources/prototypeDs.datasource.json
{
"dbType":"mysql",
"idleTimeout":60000,
"initSqls":[],
"initSqlsGetConnection":true,
"instanceType":"READ_WRITE",
"maxCon":1000,
"maxConnectTimeout":3000,
"maxRetryCount":5,
"minCon":1,
"name":"prototypeDs",
"password":密码,
"type":"JDBC",
"url":"jdbc:mysql://localhost:端口号?useUnicode=true&serverTimezone=Asia/Shanghai&characterEncoding=UTF-8",
"user":用户名,
"weight":0
}
5. 启动mycat
[root@VM-0-11-centos mycat]~ cd /mydata/mycat/mycat/bin
[root@VM-0-11-centos bin]~ ./mycat start
6. 连接mycat
通过navicate连接mycat，注意端口号为8066，用户名、密码在/mydata/mycat/mycat/conf/users/{用户名}.user.json下查看
二、主从复制
1.环境说明
使用docker启动两个mysql实例
[root@VM-0-11-centos ~]~ docker run --name M1 -p 3307:3306 -e MYSQL_ROOT_PASSWORD=123456 -d mysql:5.7 --lower_case_table_names=1
[root@VM-0-11-centos ~]~ docker run --name M1S1 -p 3308:3306 -e MYSQL_ROOT_PASSWORD=123456 -d mysql:5.7 --lower_case_table_names=1
名称 Ip Port
M1 172.17.0.3 3307
M1S1 172.17.0.2 3308
2.配置主从复制
创建临时文件夹mysqlms，将两个mysql容器的配置文件复制出来
[root@VM-0-11-centos ~]~ mkdir -p /mydata/mysqlms
[root@VM-0-11-centos ~]~ cd /mydata/mysqlms
[root@VM-0-11-centos mysqlms]~ docker cp M1:/etc/mysql/conf.d/docker.cnf m1.cnf
[root@VM-0-11-centos mysqlms]~ docker cp M1S1:/etc/mysql/conf.d/docker.cnf m1s1.cnf
m1.cnf文件底部添加
server-id=1
log-bin=master.bin
m1s1.cnf文件底部添加
server-id=2
将修改的内容同步到容器里面
[root@VM-0-11-centos mysqlms]~ docker cp m1.cnf M1:/etc/mysql/conf.d/docker.cnf
[root@VM-0-11-centos mysqlms]~ docker cp m1s1.cnf M1S1:/etc/mysql/conf.d/docker.cnf
[root@VM-0-11-centos mysqlms]~ docker restart M1 M1S1 #重启容器
之后使用navicate就可以测试能否连接这两个mysql了
首先进入主机进行相关配置
[root@VM-0-11-centos mysqlms]~ docker exec -it M1 bash #进入M1容器内部
[root@VM-0-11-centos mysqlms]~ mysql -uroot -p123456
mysql> create user 'rep'@'%' identified by '123456'; #创建用户名为rep的用户
mysql> grant replication slave on *.* to 'rep'@'%'; #给用户授予权限
mysql> flush privileges; #刷新权限
进入从机进行相关配置
[root@VM-0-11-centos mysqlms]~ docker exec -it M1S1 bash #进入M1S1容器内部
[root@VM-0-11-centos mysqlms]~ mysql -uroot -p123456
mysql> change master to master_host="主机的ip地址",master_port=3307,master_user="rep",master_password="123456",master_log_file="master.000001",master_log_pos=745;
master_log_file和master_log_pos需要登录主机执行以下命令查看
mysql> show master status;
继续在从机执行
mysql> start slave
mysql> show slave status \G;
成功的标识
三、读写分离
读写分离概念：主库负责数据的写入和读取，同时将数据同步进从库中，从库只负责读数据，所以配置读写分离前必须先配置主从复制
使用navicate登入到mycat，也就是8066这个端口
1.创建数据源
首先创建读写的数据源
/*+ mycat:createDataSource{
"dbType":"mysql",
"idleTimeout":60000,
"initSqls":[],
"initSqlsGetConnection":true,
"instanceType":"READ_WRITE",
"maxCon":1000,
"maxConnectTimeout":3000,
"maxRetryCount":5,
"minCon":1,
"name":"m1",
"password":"123456",
"type":"JDBC",
"url":"jdbc:mysql://127.0.0.1:3307?useUnicode=true&serverTimezone=UTC&characterEncoding=UTF-8",
"user":"root",
"weight":0
} */;
创建读的数据源
/*+ mycat:createDataSource{
"dbType":"mysql",
"idleTimeout":60000,
"initSqls":[],
"initSqlsGetConnection":true,
"instanceType":"READ",
"maxCon":1000,
"maxConnectTimeout":3000,
"maxRetryCount":5,
"minCon":1,
"name":"m1s1",
"password":"123456",
"type":"JDBC",
"url":"jdbc:mysql://127.0.0.1:3308?useUnicode=true&serverTimezone=UTC&characterEncoding=UTF-8",
"user":"root",
"weight":0
} */;
查询数据源
/*+ mycat:showDataSources{} */
2.创建集群
配置主从，masters(主)和replicas(从)分别对应着m1和m1s1
/*! mycat:createCluster{
"clusterType":"MASTER_SLAVE",
"heartbeat":{
"heartbeatTimeout":1000,
"maxRetry":3,
"minSwitchTimeInterval":300,
"slaveThreshold":0
},
"masters":[
"m1"
],
"maxCon":2000,
"name":"prototype",
"readBalanceType":"BALANCE_ALL",
"replicas":[
"m1s1"
],
"switchType":"SWITCH"
} */;
查询集群
/*+ mycat:showClusters{} */
关联着m1和m1s1数据源，m1负责读写，m1s1负责读
3.创建逻辑库，为其配置集群
CREATE DATABASE db1 DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
执行完后发现除了mycat，主从库也新增了一个db1
修改数据源
[root@VM-0-11-centos mysqlms]~ vim /mydata/mycat/mycat/conf/schemas/db1.schema.json
在里面添加，让它指向刚刚创建的name为prototype的集群
"targetName":"prototype"
重启mycat
[root@VM-0-11-centos bin]~ cd /mydata/mycat/mycat
[root@VM-0-11-centos bin]~ ./mycat restart
4.测试读写分离
在mycat的db1库中新建一张SYS_USER表
CREATE TABLE SYS_USER(
ID BIGINT PRIMARY KEY,
USERNAME VARCHAR(200) NOT NULL,
ADDRESS VARCHAR(500)
)
执行完成后查看db1.schema.json文件，发现它记录了sql语句
分别往主库和从库中添加一条数据，要不一样的
之后在mycat的db1里面查询数据，会发现每次查询的结果不一样，至此主从搭建完成，读写分离测试成功。
四、集群搭建
从mycat上删除db1，主从库也会相继删除
1.新建容器
新启动三个mysql容器，进行集群搭建
docker run --name M1S2 -p 3309:3306 -e MYSQL_ROOT_PASSWORD=123456 -d mysql:5.7 --lower_case_table_names=1
docker run --name M2 -p 3310:3306 -e MYSQL_ROOT_PASSWORD=123456 -d mysql:5.7 --lower_case_table_names=1
docker run --name M2S1 -p 3311:3306 -e MYSQL_ROOT_PASSWORD=123456 -d mysql:5.7 --lower_case_table_names=1
修改M1S2、M2、M2S1的配置
同步到容器中并重启
#docker cp m1.cnf M1:/etc/mysql/conf.d/docker.cnf
#docker cp m1s1.cnf M1S1:/etc/mysql/conf.d/docker.cnf
docker cp m1s2.cnf M1S2:/etc/mysql/conf.d/docker.cnf
docker cp m2.cnf M2:/etc/mysql/conf.d/docker.cnf
docker cp m2s1.cnf M2S1:/etc/mysql/conf.d/docker.cnf
docker restart M1S2 M2 M2S1 #重启新建的三个容器
2.配置集群的主从复制
M1和M1S1之前修改过了，不需要修改，先查看M1的master_status
[root@VM-0-11-centos mysqlms]# docker exec -it M1 bash
root@2941bb5bde79:/# mysql -uroot -p123456
mysql> show master status;
已发生变化
2.1.修改M1S2
docker exec -it M1S2 bash
mysql -uroot -p123456
mysql> change master to master_host="43.154.193.211",master_port=3307,master_user="rep",master_password="123456",master_log_file="m1的",master_log_pos=m1的;
mysql> start slave ;#启动主从
mysql> show slave status \G;#查看状态
2.2.修改M2
M2看作是M1的从机，M1也可以看作是M2的从机，两者互相复制
docker exec -it M2 bash
mysql -uroot -p123456
mysql> create user 'rep1'@'%' identified by '123456';
mysql> grant replication slave on *.* to 'rep1'@'%';
mysql> flush privileges;
mysql> change master to master_host="43.154.193.211",master_port=3307,master_user="rep",master_password="123456",master_log_file="m1的",master_log_pos=m1的;
mysql> start slave ;#启动主从
mysql> show slave status \G;#查看状态
2.3.修改M2S1
docker exec -it M2S1 bash
mysql -uroot -p123456
mysql> change master to master_host="43.154.193.211",master_port=3310,master_user="rep1",master_password="123456",master_log_file="m2的",master_log_pos=m2的;
mysql> start slave ;#启动主从
mysql> show slave status \G;#查看状态
之后就可以验证集群是否成功
五、mycat操作双主双从
1.添加数据源
添加M1S2读的数据源
/*+ mycat:createDataSource{
"dbType":"mysql",
"idleTimeout":60000,
"initSqls":[],
"initSqlsGetConnection":true,
"instanceType":"READ",
"maxCon":1000,
"maxConnectTimeout":3000,
"maxRetryCount":5,
"minCon":1,
"name":"m1s2",
"password":"123456",
"type":"JDBC",
"url":"jdbc:mysql://127.0.0.1:3309?useUnicode=true&serverTimezone=UTC&characterEncoding=UTF-8",
"user":"root",
"weight":0
} */;
添加M2的数据源
/*+ mycat:createDataSource{
"dbType":"mysql",
"idleTimeout":60000,
"initSqls":[],
"initSqlsGetConnection":true,
"instanceType":"READ_WRITE",
"maxCon":1000,
"maxConnectTimeout":3000,
"maxRetryCount":5,
"minCon":1,
"name":"m2",
"password":"123456",
"type":"JDBC",
"url":"jdbc:mysql://127.0.0.1:3310?useUnicode=true&serverTimezone=UTC&characterEncoding=UTF-8",
"user":"root",
"weight":0
} */;
添加M2S1的数据源
/*+ mycat:createDataSource{
"dbType":"mysql",
"idleTimeout":60000,
"initSqls":[],
"initSqlsGetConnection":true,
"instanceType":"READ",
"maxCon":1000,
"maxConnectTimeout":3000,
"maxRetryCount":5,
"minCon":1,
"name":"m2s1",
"password":"123456",
"type":"JDBC",
"url":"jdbc:mysql://127.0.0.1:3311?useUnicode=true&serverTimezone=UTC&characterEncoding=UTF-8",
"user":"root",
"weight":0
} */;
2.修改集群配置
{
"clusterType":"MASTER_SLAVE",
"heartbeat":{
"heartbeatTimeout":1000,
"maxRetryCount":0,
"minSwitchTimeInterval":300,
"showLog":true,
"slaveThreshold":0.0
},
"masters":[
"m1","m2"
],
"maxCon":2000,
"name":"prototype",
"readBalanceType":"BALANCE_ALL",
"replicas":[
"m1s1","m1s2","m2s1"
],
"switchType":"SWITCH"
}
3.测试
在mycat里新建数据库db1，创建sys_user表
CREATE DATABASE db1 DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
create table sys_user(
id bigint primary key,
username varchar(200) not null,
address varchar(500)
)
修改上面5个MySQL数据库中db1库中sys_user表里面的数据，让它不一样，再在MyCAT里面执行查询 ，会发现结果一直在变
4.结论
一主一从：一台主机负责所有写请求，一台从机负责所有读请求。
双主双从：两台主机，两台从机，一台主机负责写，其它三台负责读，当负责写的主机宕掉后，备机就会接替职责成为主机负责写，宕掉的主机重启后会成为备机负责读。
六、分库分表
1.配置
这里只模拟单机模式下的分库分表，使用默认的数据源prototypeDs.datasource.json
{
"dbType":"mysql",
"idleTimeout":60000,
"initSqls":[],
"initSqlsGetConnection":true,
"instanceType":"READ_WRITE",
"maxCon":1000,
"maxConnectTimeout":3000,
"maxRetryCount":5,
"minCon":1,
"name":"prototypeDs",
"password":"jadfsdfgasdw",
"type":"JDBC",
"url":"jdbc:mysql://localhost:3306?useUnicode=true&serverTimezone=Asia/Shanghai&characterEncoding=UTF-8",
"user":"root",
"weight":0
}
创建集群，只将prototypeDs数据源作为主机，不设置从机
{
"clusterType":"MASTER_SLAVE",
"heartbeat":{
"heartbeatTimeout":1000,
"maxRetryCount":3,
"minSwitchTimeInterval":300,
"showLog":false,
"slaveThreshold":0.0
},
"masters":[
"prototypeDs"
],
"maxCon":2000,
"name":"c0",
"readBalanceType":"BALANCE_ALL",
"replicas":[],
"switchType":"SWITCH"
}
在mycat中添加数据库db1，并创建广播表travelrecord(关键字BROADCAST)
#添加数据库db1
CREATE DATABASE db1;
#在建表语句中加上关键字 BROADCAST（广播，即为全局表）
CREATE TABLE db1.`travelrecord` (
`id` bigint NOT NULL AUTO_INCREMENT,
`user_id` varchar(100) DEFAULT NULL,
`traveldate` date DEFAULT NULL,
`fee` decimal(10,0) DEFAULT NULL,
`days` int DEFAULT NULL,
`blob` longblob,
PRIMARY KEY (`id`),
KEY `id` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 BROADCAST;
进入/mydata/mycat/mycat/conf/schemas/farm.schema.json
{
"customTables":{},
"globalTables":{
"travelrecord":{
"broadcast":[
{
"targetName":"c0"
}
],
"createTableSQL":"CREATE TABLE farm.`travelrecord` (\n\t`id` bigint NOT NULL AUTO_INCREMENT,\n\t`user_id` varchar(100) DEFAULT NULL,\n\t`traveldate` date DEFAULT NULL,\n\t`fee` decimal(10, 0) DEFAULT NULL,\n\t`days` int DEFAULT NULL,\n\t`blob` longblob,\n\tPRIMARY KEY (`id`),\n\tKEY `id` (`id`)\n) BROADCAST ENGINE = InnoDB CHARSET = utf8"
}
},
"normalProcedures":{},
"normalTables":{},
"schemaName":"farm",
"shardingTables":{
},
"views":{}
}
2.分库分表
在 Mycat 终端直接运行建表语句进行数据分片，dbpartition是分库规则，tbpartition是分表规则，tbpartitions 1标识分一张表， dbpartitions 2表示分两个库
CREATE TABLE farm.orders(
id BIGINT NOT NULL AUTO_INCREMENT,
order_type INT,
customer_id INT,
amount DECIMAL(10,2),
PRIMARY KEY(id),
KEY `id` (`id`)
)ENGINE=INNODB DEFAULT CHARSET=utf8
dbpartition BY mod_hash(customer_id) tbpartition BY mod_hash(customer_id)
tbpartitions 1 dbpartitions 2;
往orders表中增加数据
INSERT INTO orders(id,order_type,customer_id,amount)
VALUES(1,101,100,100100);
INSERT INTO orders(id,order_type,customer_id,amount)
VALUES(2,101,100,100300);
INSERT INTO orders(id,order_type,customer_id,amount)
VALUES(3,101,101,120000);
INSERT INTO orders(id,order_type,customer_id,amount)
VALUES(4,101,101,103000);
INSERT INTO orders(id,order_type,customer_id,amount)
VALUES(5,102,101,100400);
INSERT INTO orders(id,order_type,customer_id,amount)
VALUES(6,102,100,100020);
SELECT * FROM orders;
执行完后可以看到在mycat客户端里新增了一个farm库，里面有一张orders表
在本地数据库中新增了farm_0和farm_1，里面各有一张orders表
orders_0中数据
orders_1中数据
3.关联表分表
将orders的关联表orders_detail也分表，在 Mycat 终端直接运行建表语句进行数据分片
CREATE TABLE orders_detail(
`id` BIGINT NOT NULL AUTO_INCREMENT,
detail VARCHAR(2000),
order_id INT,
PRIMARY KEY(id)
)ENGINE=INNODB DEFAULT CHARSET=utf8
dbpartition BY mod_hash(order_id) tbpartition BY mod_hash(order_id)
tbpartitions 1 dbpartitions 2;
INSERT INTO orders_detail(id,detail,order_id) VALUES(1,'detail1',1);
INSERT INTO orders_detail(id,detail,order_id) VALUES(2,'detail1',2);
INSERT INTO orders_detail(id,detail,order_id) VALUES(3,'detail1',3);
INSERT INTO orders_detail(id,detail,order_id) VALUES(4,'detail1',4);
INSERT INTO orders_detail(id,detail,order_id) VALUES(5,'detail1',5);
INSERT INTO orders_detail(id,detail,order_id) VALUES(6,'detail1',6);
执行后，观察发现虽然分片算法一样，但分片字段不同最终存储的位置可能就会不同
对比数据节点1
对比数据节点2
在mycat中关联查询也能查询出所有数据
4.分片算法
