OGG同步MySQL数据

Mysql 5.6->Mysql 5.6 (linux -> win)

以《installing and Configuring Oracle GoldenGate for MySQL.pdf》为准。

字符集转换只能支持库级的字符集
源端必须是innodb存储引擎
目标库是mysql的话，replicat通过native api连接

用户：要有对information_schema数据库读的权限。
源：select any table或者select on database.table的权限；读my.cnf权限；读日志的权限。
目标：DML权限，select any table等权限

具体不支持的操作和数据类型，参考手册中的1.3.5-1.3.8  
如：DDL不支持

MYSQL的安装：
1、rpm -qa|grep mysql
mysql-libs-5.1.71-1.el6.x86_64

rpm -e mysql-libs-5.1.71-1.el6.x86_64 --nodeps

rpm -Uhv MySQL-server-advanced-5.6.23-1.el6.x86_64.rpm
service mysql start

more /root/.mysql_secret  root初始密码

mysql -u root -h localhost -p
set password for root@'localhost'=password("oracle");   修改root密码

rpm -ihv MySQL-client-advanced-5.6.23-1.el6.x86_64.rpm

2、创建用户以及测试表

（1）源端创建ggs用户：

mysql -u root -h localhost -p

create user ggs;

set password for ggs=password('oracle');

创建测试表及授权

create database testdb character set utf8;
grant select on testdb.* to ggs;

use testdb;
create table tb01(id int,name varchar(10)) engine=innodb;

（2）目标端创建ggs用户：
mysql -u root -h localhost -p

create user ggs;

set password for ggs=password('oracle');

创建测试表及授权：
create database testdb;
GRANT INSERT,UPDATE,DELETE,SELECT ON testdb.* TO ggs;
grant drop on testdb.* to ggs;   --如果需要复制truncate的话

use testdb;
create table tb01(id int,name varchar(10)) engine=innodb;

3、源端启用mysql log
cp /usr/share/mysql/my-default.cnf /etc/my.cnf
vi /etc/my.cnf 

#注意文件夹对MySQL用户有权限
log-bin=/var/lib/mysql/binary-log
log-bin-index=/var/lib/mysql/binary-log.index
binlog_format=row

service mysqld restart
show variables like '%log_bin%';

4、安装ogg

su - mysql  (goldengate安装在mysql用户下，不能用oracle用户了，否则会没有权限)

unzip Oracle\ GoldenGate\ V12.1.2.1.0\ for\ MySQL\ on\ Linux\ x86-64\ V47367-01.zip -d ogg4mysql
cd ogg4mysql
tar xvf ggs_Linux_x64_MySQL_64bit.tar

./ggsci
create subdirs

dblogin sourcedb testdb, userid ggs, password password01

如果报错：
ln -s /var/lib/mysql/mysql.sock /tmp/mysql.sock
或者export MYSQL_UNIX_PORT=/var/lib/mysql/mysql.sock

5、配置ogg进程：
（1）配置mgr（源和目标）
edit param mgr
port 8809

start mgr

（2）配置源端进程
edit param ext1

extract ext1
setenv (MYSQL_HOME="/var/lib/mysql")
DBOPTIONS HOST 127.0.0.1, CONNECTIONPORT 3306
sourcedb testdb@localhost:3306, userid ggs, password oracle
tranlogoptions altlogdest /var/lib/mysql/binary-log.index
exttrail ./dirdat/et
dynamicresolution
gettruncates
table testdb.tb01;

--mysql中是dbname.tablename

add extract ext1,tranlog,begin now
add exttrail ./dirdat/et, extract ext1

edit param pmp1

extract pmp1
rmthost 134.178.1.140, mgrport 8809
rmttrail ./dirdat/rt
passthru
gettruncates
table testdb.tb01;

add extract pmp1, exttrailsource ./dirdat/et
add rmttrail ./dirdat/rt, extract pmp1

（3）异构一定要生成定义文件
ggsci>edit param defgen
DEFSFILE ./dirdef/test.def               
sourcedb testdb@localhost:3306 USERID ggs, PASSWORD password01
TABLE testdb.tb01;

(DEFSFILE ./dirdef/orcldef CHARSET UTF-8 可以设置字符集)

exit ggsci

rm ./dirdef/test.def
./defgen paramfile dirprm/defgen.prm

生成test.def文件，ftp传输到目标的dirdef文件目录下。

目标rep1进程修改：
--ASSUMETARGETDEFS
SOURCEDEFS ./dirdef/test.def

（4）配置目标端进程
目标主机：
edit param rep1

replicat rep1
DBOPTIONS HOST 127.0.0.1, CONNECTIONPORT 3306
targetdb testdb ,userid ggs, password oracle
REPERROR DEFAULT, ABEND
discardfile ./dirrpt/rep1.dsc, append, megabytes 50
--assumetargetdefs
SOURCEDEFS ./dirdef/test.def
gettruncates
dynamicresolution
map testdb.tb01, target testdb.tb01;

add replicat rep1, exttrail ./dirdat/rt, nodbcheckpoint

3、字符集问题

假设需要插入中文，实际前面建库是不行的，插入不了，必须是utf8或者gbk

源端：
mysql> show variables like '%char%';
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | utf8                       |
| character_set_connection | utf8                       |
| character_set_database   | latin1                     |
| character_set_filesystem | binary                     |
| character_set_results    | utf8                       |
| character_set_server     | latin1                     |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
数据库字符集是latin1

ext1：

CLIENT CHARACTER SET         = "utf8mb4"
SERVER CHARACTER SET         = "latin1"
DATABASE CHARACTER SET       = "latin1"
LOCALE INFORMATION           = "en"
DATABASE COLLATION           = "casesensitive"

目标：
mysql> show variables like '%char%';
+--------------------------+----------------------------------------------------
-----------+
| Variable_name            | Value
           |
+--------------------------+----------------------------------------------------
-----------+
| character_set_client     | gbk
           |
| character_set_connection | gbk
           |
| character_set_database   | utf8
           |
| character_set_filesystem | binary
           |
| character_set_results    | gbk
           |
| character_set_server     | utf8
           |
| character_set_system     | utf8
           |
| character_sets_dir       | C:\Program Files (x86)\MySQL\MySQL Server 5.6\share
\charsets\ |
+--------------------------+----------------------------------------------------
-----------+
数据库字符集是utf8

rep1：
Database Language and Character Set:
CLIENT CHARACTER SET         = "utf8mb4"
SERVER CHARACTER SET         = "utf8mb4"
DATABASE CHARACTER SET       = "utf8mb4"
LOCALE INFORMATION           = "en"
DATABASE COLLATION           = "caseinsensitive"

所以必须建库：
create database testdb2 character set utf8;

同时在mysql中可以设置客户端：
mysql>set names utf8;

在goldengate参数中设置sessioncharset

如：replicat设置：
targetdb testdb, userid ggs, password oracle, sessioncharset utf8

建议建库是全部用gbk或者utf8；