DataX工具部署与使用(MySQL to Oracle)
目录：
-
- 一、准备环境
-
- （1）安装JDK
- （2）检查Python版本
- （3）DataX 解压及测试
- 二、同步测试
-
- 1、配置清单
- 2、操作示例
-
- （1）同步测试环境
- （2）准备测试表（MySQL）
- （3）准备测试表（Oracle）
- （4）创建并编辑json文件
- （5）执行json配置文件
- 三、故障处理
-
- 1、告警日志
- 2、故障分析
- 3、问题处理
-
- （1）方式一：禁用 SSL
- （2）方式二：升级驱动
一、准备环境
1、创建用户组和目录
groupadd -g 1500 datax
useradd -g datax -u 1500 datax
mkdir /datax
chown datax:datax /datax
2、系统环境
- Linux
- JDK(1.8以上，推荐1.8)
- Python(2或3都可以)
- Apache Maven 3.x (Compile DataX)（本文使用datax工具包直接解压，未采用编译方式）
（1）安装JDK
安装JDK（1.8.0_361）
# rpm -ivh jdk-8u361-linux-x64.rpm
warning: jdk-8u361-linux-x64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Preparing... ################################# [100%]
Updating / installing...
1:jdk1.8-2000:1.8.0_361-fcs ################################# [100%]
Unpacking JAR files...
tools.jar...
plugin.jar...
javaws.jar...
deploy.jar...
rt.jar...
jsse.jar...
charsets.jar...
localedata.jar...
检查JDK版本
# java -version
java version "1.8.0_361"
Java(TM) SE Runtime Environment (build 1.8.0_361-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.361-b09, mixed mode)
（2）检查Python版本
# python -V
Python 2.7.5
我这边使用的是linux7自带的python版本
（3）DataX 解压及测试
Download ：DataX下载地址，上传到/datax目录
解压缩：
# tar -zxvf datax.tar.gz
自检脚本：
python {YOUR_DATAX_HOME}/bin/datax.py {YOUR_DATAX_HOME}/job/job.json
示例：
# python /datax/datax/bin/datax.py /datax/datax/job/job.json
DataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.
2023-07-31 10:56:16.407 [main] INFO MessageSource - JVM TimeZone: GMT+08:00, Locale: zh_CN
2023-07-31 10:56:16.412 [main] INFO MessageSource - use Locale: zh_CN timeZone: sun.util.calendar.ZoneInfo[id="GMT+08:00",offset=28800000,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
2023-07-31 10:56:16.499 [main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2023-07-31 10:56:16.513 [main] INFO Engine - the machine info =>
osInfo: Oracle Corporation 1.8 25.361-b09
jvmInfo: Linux amd64 5.4.17-2011.6.2.el7uek.x86_64
cpu num: 4
totalPhysicalMemory: -0.00G
freePhysicalMemory: -0.00G
maxFileDescriptorCount: -1
currentOpenFileDescriptorCount: -1
GC Names [PS MarkSweep, PS Scavenge]
MEMORY_NAME | allocation_size | init_size
PS Eden Space | 256.00MB | 256.00MB
Code Cache | 240.00MB | 2.44MB
Compressed Class Space | 1,024.00MB | 0.00MB
PS Survivor Space | 42.50MB | 42.50MB
PS Old Gen | 683.00MB | 683.00MB
Metaspace | -0.00MB | 0.00MB
2023-07-31 10:56:16.549 [main] INFO Engine -
{
"content":[
{
"reader":{
"name":"streamreader",
"parameter":{
"column":[
{
"type":"string",
"value":"DataX"
},
{
"type":"long",
"value":19890604
},
{
"type":"date",
"value":"1989-06-04 00:00:00"
},
{
"type":"bool",
"value":true
},
{
"type":"bytes",
"value":"test"
}
],
"sliceRecordCount":100000
}
},
"writer":{
"name":"streamwriter",
"parameter":{
"encoding":"UTF-8",
"print":false
}
}
}
],
"setting":{
"errorLimit":{
"percentage":0.02,
"record":0
},
"speed":{
"channel":1
}
}
}
2023-07-31 10:56:16.589 [main] WARN Engine - prioriy set to 0, because NumberFormatException, the value is: null
2023-07-31 10:56:16.594 [main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2023-07-31 10:56:16.594 [main] INFO JobContainer - DataX jobContainer starts job.
2023-07-31 10:56:16.599 [main] INFO JobContainer - Set jobId = 0
2023-07-31 10:56:16.813 [job-0] INFO JobContainer - jobContainer starts to do prepare ...
2023-07-31 10:56:16.814 [job-0] INFO JobContainer - DataX Reader.Job [streamreader] do prepare work .
2023-07-31 10:56:16.816 [job-0] INFO JobContainer - DataX Writer.Job [streamwriter] do prepare work .
2023-07-31 10:56:16.817 [job-0] INFO JobContainer - jobContainer starts to do split ...
2023-07-31 10:56:16.818 [job-0] INFO JobContainer - Job set Channel-Number to 1 channels.
2023-07-31 10:56:16.821 [job-0] INFO JobContainer - DataX Reader.Job [streamreader] splits to [1] tasks.
2023-07-31 10:56:16.823 [job-0] INFO JobContainer - DataX Writer.Job [streamwriter] splits to [1] tasks.
2023-07-31 10:56:16.879 [job-0] INFO JobContainer - jobContainer starts to do schedule ...
2023-07-31 10:56:16.891 [job-0] INFO JobContainer - Scheduler starts [1] taskGroups.
2023-07-31 10:56:16.896 [job-0] INFO JobContainer - Running by standalone Mode.
2023-07-31 10:56:16.914 [taskGroup-0] INFO TaskGroupContainer - taskGroupId=[0] start [1] channels for [1] tasks.
2023-07-31 10:56:16.922 [taskGroup-0] INFO Channel - Channel set byte_speed_limit to -1, No bps activated.
2023-07-31 10:56:16.923 [taskGroup-0] INFO Channel - Channel set record_speed_limit to -1, No tps activated.
2023-07-31 10:56:16.943 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started
2023-07-31 10:56:17.247 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[305]ms
2023-07-31 10:56:17.249 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] completed it's tasks.
2023-07-31 10:56:26.930 [job-0] INFO StandAloneJobContainerCommunicator - Total 100000 records, 2600000 bytes | Speed 253.91KB/s, 10000 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.035s | All Task WaitReaderTime 0.062s | Percentage 100.00%
2023-07-31 10:56:26.931 [job-0] INFO AbstractScheduler - Scheduler accomplished all tasks.
2023-07-31 10:56:26.932 [job-0] INFO JobContainer - DataX Writer.Job [streamwriter] do post work.
2023-07-31 10:56:26.933 [job-0] INFO JobContainer - DataX Reader.Job [streamreader] do post work.
2023-07-31 10:56:26.933 [job-0] INFO JobContainer - DataX jobId [0] completed successfully.
2023-07-31 10:56:26.935 [job-0] INFO HookInvoker - No hook invoked, because base dir not exists or is a file: /datax/datax/hook
2023-07-31 10:56:26.939 [job-0] INFO JobContainer -
[total cpu info] =>
averageCpu | maxDeltaCpu | minDeltaCpu
-1.00% | -1.00% | -1.00%
[total gc info] =>
NAME | totalGCCount | maxDeltaGCCount | minDeltaGCCount | totalGCTime | maxDeltaGCTime | minDeltaGCTime
PS MarkSweep | 0 | 0 | 0 | 0.000s | 0.000s | 0.000s
PS Scavenge | 0 | 0 | 0 | 0.000s | 0.000s | 0.000s
2023-07-31 10:56:26.939 [job-0] INFO JobContainer - PerfTrace not enable!
2023-07-31 10:56:26.940 [job-0] INFO StandAloneJobContainerCommunicator - Total 100000 records, 2600000 bytes | Speed 253.91KB/s, 10000 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.035s | All Task WaitReaderTime 0.062s | Percentage 100.00%
2023-07-31 10:56:26.945 [job-0] INFO JobContainer -
任务启动时刻 : 2023-07-31 10:56:16
任务结束时刻 : 2023-07-31 10:56:26
任务总计耗时 : 10s
任务平均流量 : 253.91KB/s
记录写入速度 : 10000rec/s
读出记录总数 : 100000
读写失败总数 : 0
自检脚本无法通过报错处理：https://developer.aliyun.com/ask/376622?spm=a2c6h.13148508.0.0.1d734f0eKYBCG7
二、同步测试
1、配置清单
在两个不同数据库之间使用datax数据迁移可以通过命令查看配置模板： python datax.py -r {Sourcedb_READER} -w {Targetdb_WRITER}
示例：查看MySQL到Oracle数据传输json文件模板
# python /datax/datax/bin/datax.py -r mysqlreader -w oraclewriter
DataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.
Please refer to the mysqlreader document:
https://github.com/alibaba/DataX/blob/master/mysqlreader/doc/mysqlreader.md
Please refer to the oraclewriter document:
https://github.com/alibaba/DataX/blob/master/oraclewriter/doc/oraclewriter.md
Please save the following configuration as a json file and use
python {DATAX_HOME}/bin/datax.py {JSON_FILE_NAME}.json
to run the job.
{
"job": {
"content": [
{
"reader": {
"name": "mysqlreader",
"parameter": {
"column": [],
"connection": [
{
"jdbcUrl": [],
"table": []
}
],
"password": "",
"username": "",
"where": ""
}
},
"writer": {
"name": "oraclewriter",
"parameter": {
"column": [],
"connection": [
{
"jdbcUrl": "",
"table": []
}
],
"password": "",
"preSql": [],
"username": ""
}
}
}
],
"setting": {
"speed": {
"channel": ""
}
}
}
}
详情参阅：
mysqlreader 文档： https://github.com/alibaba/DataX/blob/master/mysqlreader/doc/mysqlreader.md
oraclewriter 文档： https://github.com/alibaba/DataX/blob/master/oraclewriter/doc/oraclewriter.md
2、操作示例
（1）同步测试环境
| OS版本 | Ol7.9 | Ol7.9 |
|---|---|---|
| DB版本 | MySQL5.7 | Oracle19c (经典模式) |
| 同步方向 | Reader | Writer |
| IP | 192.168.48.30 | 192.168.48.21 |
| 端口 | 3306 | 1521 |
| 数据库名 | datax | orcl |
| 用户/密码 | root/root123 | shuaige/shuaige123 |
| 表名 | datax_test1 | datax_test1 |
两个虚拟机均使用
OracleLinux7.9操作系统最小化安装，关闭防火墙和selinux
（2）准备测试表（MySQL）
-- 1、登录mysql
------------------------------------------------------------------------------------
$ mysql -uroot -proot123 -S /mysql/mysql.sock
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 22
Server version: 5.7.38 MySQL Community Server (GPL)
Copyright (c) 2000, 2022, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
-- 2、创建数据库
------------------------------------------------------------------------------------
mysql> create database datax;
Query OK, 1 row affected (0.00 sec)
-- 3、查看数据库
------------------------------------------------------------------------------------
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| datax |
| mysql |
| performance_schema |
| sys |
+--------------------+
5 rows in set (0.00 sec)
-- 4、切换数据库
------------------------------------------------------------------------------------
mysql> \u datax
Database changed
-- 5、创建测试表
------------------------------------------------------------------------------------
mysql> create table datax_test1 (id INT(30) primary key comment '编号',name VARCHAR(1000) not null comment '姓名');
Query OK, 0 rows affected (0.11 sec)
-- 6、检查新建表
------------------------------------------------------------------------------------
mysql> show tables;
+-----------------+
| Tables_in_datax |
+-----------------+
| datax_test1 |
+-----------------+
1 row in set (0.00 sec)
-- 7、插入测试数据
------------------------------------------------------------------------------------
mysql> insert into datax_test1(id,name) value ('1','小明');
Query OK, 1 row affected (0.36 sec)
-- 8、提交事务
------------------------------------------------------------------------------------
mysql> commit;
Query OK, 0 rows affected (0.00 sec)
-- 9、查询测试数据
------------------------------------------------------------------------------------
mysql> select * from datax.datax_test1;
+----+--------+
| id | name |
+----+--------+
| 1 | 小明 |
+----+--------+
1 row in set (0.00 sec)
（3）准备测试表（Oracle）
-- 1、登录oracle
------------------------------------------------------------------------------------
$ sqlplus / as sysdba
SQL*Plus: Release 19.0.0.0.0 - Production on Mon Jul 31 11:39:17 2023
Version 19.3.0.0.0
Copyright (c) 1982, 2019, Oracle. All rights reserved.
Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.3.0.0.0
-- 1、创建测试表
------------------------------------------------------------------------------------
SYS@orcl> create table shuaige.datax_test1 (id number(30) primary key,name VARCHAR2(1000) not null);
Table created.
注意更改用户名和表空间,不指定使用用户默认表空间
（4）创建并编辑json文件
在/datax/datax/job目录下用mysqlreader to oraclewriter模板配置json文件
vi /datax/datax/job/mysql2oracle_datax_test1.json
编辑配置模板按照实际环境修改：
{
"job": {
"content": [
{
"reader": {
"name": "mysqlreader",
"parameter": {
"column": ["*"],
"splitPk": "ID",
"connection": [
{
"jdbcUrl": ["jdbc:mysql://192.168.48.30:3306/datax"],
"table": ["datax_test1"]
}
],
"password": "root123",
"username": "root",
"where": ""
}
},
"writer": {
"name": "oraclewriter",
"parameter": {
"column": ["*"],
"connection": [
{
"jdbcUrl": "jdbc:oracle:thin:@192.168.48.21:1521:orcl",
"table": ["datax_test1"]
}
],
"password": "shuaige123",
"preSql": ["truncate table datax_test1"],
"username": "shuaige"
}
}
}
],
"setting": {
"speed": {
"channel": "4"
}
}
}
}
（5）执行json配置文件
# python /datax/datax/bin/datax.py /datax/datax/job/mysql2oracle_datax_test1.json
DataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.
2023-07-31 11:46:55.446 [main] INFO MessageSource - JVM TimeZone: GMT+08:00, Locale: zh_CN
2023-07-31 11:46:55.451 [main] INFO MessageSource - use Locale: zh_CN timeZone: sun.util.calendar.ZoneInfo[id="GMT+08:00",offset=28800000,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
2023-07-31 11:46:55.538 [main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2023-07-31 11:46:55.553 [main] INFO Engine - the machine info =>
osInfo: Oracle Corporation 1.8 25.361-b09
jvmInfo: Linux amd64 5.4.17-2011.6.2.el7uek.x86_64
cpu num: 4
totalPhysicalMemory: -0.00G
freePhysicalMemory: -0.00G
maxFileDescriptorCount: -1
currentOpenFileDescriptorCount: -1
GC Names [PS MarkSweep, PS Scavenge]
MEMORY_NAME | allocation_size | init_size
PS Eden Space | 256.00MB | 256.00MB
Code Cache | 240.00MB | 2.44MB
Compressed Class Space | 1,024.00MB | 0.00MB
PS Survivor Space | 42.50MB | 42.50MB
PS Old Gen | 683.00MB | 683.00MB
Metaspace | -0.00MB | 0.00MB
2023-07-31 11:46:55.605 [main] INFO Engine -
{
"content":[
{
"reader":{
"name":"mysqlreader",
"parameter":{
"column":[
"*"
],
"connection":[
{
"jdbcUrl":[
"jdbc:mysql://192.168.48.30:3306/datax"
],
"table":[
"datax_test1"
]
}
],
"password":"*******",
"splitPk":"ID",
"username":"root",
"where":""
}
},
"writer":{
"name":"oraclewriter",
"parameter":{
"column":[
"*"
],
"connection":[
{
"jdbcUrl":"jdbc:oracle:thin:@192.168.48.21:1521:orcl",
"table":[
"datax_test1"
]
}
],
"password":"**********",
"preSql":[
"truncate table datax_test1"
],
"username":"shuaige"
}
}
}
],
"setting":{
"speed":{
"channel":"4"
}
}
}
2023-07-31 11:46:55.657 [main] WARN Engine - prioriy set to 0, because NumberFormatException, the value is: null
2023-07-31 11:46:55.661 [main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2023-07-31 11:46:55.662 [main] INFO JobContainer - DataX jobContainer starts job.
2023-07-31 11:46:55.668 [main] INFO JobContainer - Set jobId = 0
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
2023-07-31 11:46:56.996 [job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://192.168.48.30:3306/datax?&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
2023-07-31 11:46:56.999 [job-0] WARN OriginalConfPretreatmentUtil - 您的配置文件中的列配置存在一定的风险. 因为您未配置读取数据库表的列，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2023-07-31 11:46:58.519 [job-0] INFO OriginalConfPretreatmentUtil - table:[datax_test1] all columns:[
ID,NAME
].
2023-07-31 11:46:58.520 [job-0] WARN OriginalConfPretreatmentUtil - 您的配置文件中的列配置信息存在风险. 因为您配置的写入数据库表的列为*，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2023-07-31 11:46:58.524 [job-0] INFO OriginalConfPretreatmentUtil - Write data [
INSERT INTO %s (ID,NAME) VALUES(?,?)
], which jdbcUrl like:[jdbc:oracle:thin:@192.168.48.21:1521:orcl]
2023-07-31 11:46:58.525 [job-0] INFO JobContainer - jobContainer starts to do prepare ...
2023-07-31 11:46:58.526 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] do prepare work .
2023-07-31 11:46:58.528 [job-0] INFO JobContainer - DataX Writer.Job [oraclewriter] do prepare work .
2023-07-31 11:46:58.690 [job-0] INFO CommonRdbmsWriter$Job - Begin to execute preSqls:[truncate table datax_test1]. context info:jdbc:oracle:thin:@192.168.48.21:1521:orcl.
2023-07-31 11:46:58.766 [job-0] INFO JobContainer - jobContainer starts to do split ...
2023-07-31 11:46:58.767 [job-0] INFO JobContainer - Job set Channel-Number to 4 channels.
2023-07-31 11:46:58.880 [job-0] INFO SingleTableSplitUtil - split pk [sql=SELECT MIN(ID),MAX(ID) FROM datax_test1] is running...
2023-07-31 11:46:58.960 [job-0] INFO SingleTableSplitUtil - After split(), allQuerySql=[
select * from datax_test1 where (1 <= ID AND ID <= 1)
select * from datax_test1 where ID IS NULL
].
2023-07-31 11:46:58.961 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] splits to [2] tasks.
2023-07-31 11:46:58.963 [job-0] INFO JobContainer - DataX Writer.Job [oraclewriter] splits to [2] tasks.
2023-07-31 11:46:59.013 [job-0] INFO JobContainer - jobContainer starts to do schedule ...
2023-07-31 11:46:59.022 [job-0] INFO JobContainer - Scheduler starts [1] taskGroups.
2023-07-31 11:46:59.028 [job-0] INFO JobContainer - Running by standalone Mode.
2023-07-31 11:46:59.043 [taskGroup-0] INFO TaskGroupContainer - taskGroupId=[0] start [2] channels for [2] tasks.
2023-07-31 11:46:59.050 [taskGroup-0] INFO Channel - Channel set byte_speed_limit to -1, No bps activated.
2023-07-31 11:46:59.050 [taskGroup-0] INFO Channel - Channel set record_speed_limit to -1, No tps activated.
2023-07-31 11:46:59.067 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started
2023-07-31 11:46:59.073 [0-0-0-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from datax_test1 where (1 <= ID AND ID <= 1)
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 11:46:59.075 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[1] attemptCount[1] is started
2023-07-31 11:46:59.076 [0-0-1-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from datax_test1 where ID IS NULL
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 11:46:59.196 [0-0-0-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from datax_test1 where (1 <= ID AND ID <= 1)
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 11:46:59.273 [0-0-1-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from datax_test1 where ID IS NULL
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 11:46:59.880 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[1] is successed, used[806]ms
2023-07-31 11:46:59.981 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[916]ms
2023-07-31 11:46:59.982 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] completed it's tasks.
2023-07-31 11:47:09.063 [job-0] INFO StandAloneJobContainerCommunicator - Total 1 records, 3 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 100.00%
2023-07-31 11:47:09.063 [job-0] INFO AbstractScheduler - Scheduler accomplished all tasks.
2023-07-31 11:47:09.064 [job-0] INFO JobContainer - DataX Writer.Job [oraclewriter] do post work.
2023-07-31 11:47:09.066 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] do post work.
2023-07-31 11:47:09.067 [job-0] INFO JobContainer - DataX jobId [0] completed successfully.
2023-07-31 11:47:09.069 [job-0] INFO HookInvoker - No hook invoked, because base dir not exists or is a file: /datax/datax/hook
2023-07-31 11:47:09.073 [job-0] INFO JobContainer -
[total cpu info] =>
averageCpu | maxDeltaCpu | minDeltaCpu
-1.00% | -1.00% | -1.00%
[total gc info] =>
NAME | totalGCCount | maxDeltaGCCount | minDeltaGCCount | totalGCTime | maxDeltaGCTime | minDeltaGCTime
PS MarkSweep | 1 | 1 | 1 | 0.236s | 0.236s | 0.236s
PS Scavenge | 1 | 1 | 1 | 0.012s | 0.012s | 0.012s
2023-07-31 11:47:09.075 [job-0] INFO JobContainer - PerfTrace not enable!
2023-07-31 11:47:09.077 [job-0] INFO StandAloneJobContainerCommunicator - Total 1 records, 3 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 100.00%
2023-07-31 11:47:09.079 [job-0] INFO JobContainer -
任务启动时刻 : 2023-07-31 11:46:55
任务结束时刻 : 2023-07-31 11:47:09
任务总计耗时 : 13s
任务平均流量 : 0B/s
记录写入速度 : 0rec/s
读出记录总数 : 1
读写失败总数 : 0
执行成功，检查Oracle数据库查看数据是否同步成功
SYS@orcl> select * from shuaige.datax_test1;
ID NAME
------ --------------
1 小明
数据传输成功，如果在执行json文件报错，一定要先看好json文件格式，用户密码大小写，[ ]，" “等，“channel"参数至少为“1”，“preSql”:[””] SQL条件可以不加，但是有主键冲突的行会报错传输失败。
三、故障处理
1、告警日志
在编写配置文件并行运行时产生以下报错
# python /datax/datax/bin/datax.py /datax/datax/job/mysql2oracle_datax_test1.json
DataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.
2023-07-31 16:55:49.827 [main] INFO MessageSource - JVM TimeZone: GMT+08:00, Locale: zh_CN
2023-07-31 16:55:49.831 [main] INFO MessageSource - use Locale: zh_CN timeZone: sun.util.calendar.ZoneInfo[id="GMT+08:00",offset=28800000,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
2023-07-31 16:55:49.918 [main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2023-07-31 16:55:49.931 [main] INFO Engine - the machine info =>
osInfo: Oracle Corporation 1.8 25.361-b09
jvmInfo: Linux amd64 5.4.17-2011.6.2.el7uek.x86_64
cpu num: 4
totalPhysicalMemory: -0.00G
freePhysicalMemory: -0.00G
maxFileDescriptorCount: -1
currentOpenFileDescriptorCount: -1
GC Names [PS MarkSweep, PS Scavenge]
MEMORY_NAME | allocation_size | init_size
PS Eden Space | 256.00MB | 256.00MB
Code Cache | 240.00MB | 2.44MB
Compressed Class Space | 1,024.00MB | 0.00MB
PS Survivor Space | 42.50MB | 42.50MB
PS Old Gen | 683.00MB | 683.00MB
Metaspace | -0.00MB | 0.00MB
2023-07-31 16:55:49.978 [main] INFO Engine -
{
"content":[
{
"reader":{
"name":"mysqlreader",
"parameter":{
"column":[
"*"
],
"connection":[
{
"jdbcUrl":[
"jdbc:mysql://192.168.48.30:3306/datax"
],
"table":[
"datax_test1"
]
}
],
"password":"*******",
"splitPk":"ID",
"username":"root",
"where":""
}
},
"writer":{
"name":"oraclewriter",
"parameter":{
"column":[
"*"
],
"connection":[
{
"jdbcUrl":"jdbc:oracle:thin:@192.168.48.21:1521:orcl",
"table":[
"datax_test1"
]
}
],
"password":"**********",
"preSql":[
"truncate table datax_test1"
],
"username":"shuaige"
}
}
}
],
"setting":{
"speed":{
"channel":"4"
}
}
}
2023-07-31 16:55:50.021 [main] WARN Engine - prioriy set to 0, because NumberFormatException, the value is: null
2023-07-31 16:55:50.026 [main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2023-07-31 16:55:50.026 [main] INFO JobContainer - DataX jobContainer starts job.
2023-07-31 16:55:50.030 [main] INFO JobContainer - Set jobId = 0
Mon Jul 31 16:55:50 GMT+08:00 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2023-07-31 16:55:51.121 [job-0] WARN DBUtil - test connection of [jdbc:mysql://192.168.48.30:3306/datax] failed, for Code:[MYSQLErrCode-02], Description:[数据库服务的IP地址或者Port错误，请检查填写的IP地址和Port或者联系DBA确认IP地址和Port是否正确。如果是同步中心用户请联系DBA确认idb上录入的IP和PORT信息和数据库的当前实际信息是一致的]. - 具体错误信息为：com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 536 milliseconds ago. The last packet sent successfully to the server was 511 milliseconds ago..
2023-07-31 16:55:51.130 [job-0] ERROR RetryUtil - Exception when calling callable, 异常Msg:DataX无法连接对应的数据库，可能原因是：1) 配置的ip/port/database/jdbc错误，无法连接。2) 配置的username/password错误，鉴权失败。请和DBA确认该数据库的连接信息是否正确。
java.lang.Exception: DataX无法连接对应的数据库，可能原因是：1) 配置的ip/port/database/jdbc错误，无法连接。2) 配置的username/password错误，鉴权失败。请和DBA确认该数据库的连接信息是否正确。
at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:71) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:51) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.common.util.RetryUtil$Retry.call(RetryUtil.java:164) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.common.util.RetryUtil$Retry.doRetry(RetryUtil.java:111) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.common.util.RetryUtil.executeWithRetry(RetryUtil.java:30) [datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.util.DBUtil.chooseJdbcUrl(DBUtil.java:51) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.dealJdbcAndTable(OriginalConfPretreatmentUtil.java:92) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:59) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:33) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.reader.CommonRdbmsReader$Job.init(CommonRdbmsReader.java:55) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.mysqlreader.MysqlReader$Job.init(MysqlReader.java:37) [mysqlreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:673) [datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:303) [datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113) [datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.Engine.start(Engine.java:93) [datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.Engine.entry(Engine.java:175) [datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.Engine.main(Engine.java:208) [datax-core-0.0.1-SNAPSHOT.jar:na]
2023-07-31 16:55:52.132 [job-0] ERROR RetryUtil - Exception when calling callable, 即将尝试执行第1次重试.本次重试计划等待[1000]ms,实际等待[1000]ms, 异常Msg:[DataX无法连接对应的数据库，可能原因是：1) 配置的ip/port/database/jdbc错误，无法连接。2) 配置的username/password错误，鉴权失败。请和DBA确认该数据库的连接信息是否正确。]
Mon Jul 31 16:55:52 GMT+08:00 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2023-07-31 16:55:52.148 [job-0] WARN DBUtil - test connection of [jdbc:mysql://192.168.48.30:3306/datax] failed, for Code:[MYSQLErrCode-02], Description:[数据库服务的IP地址或者Port错误，请检查填写的IP地址和Port或者联系DBA确认IP地址和Port是否正确。如果是同步中心用户请联系DBA确认idb上录入的IP和PORT信息和数据库的当前实际信息是一致的]. - 具体错误信息为：com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 3 milliseconds ago. The last packet sent successfully to the server was 2 milliseconds ago..
2023-07-31 16:55:54.150 [job-0] ERROR RetryUtil - Exception when calling callable, 即将尝试执行第2次重试.本次重试计划等待[2000]ms,实际等待[2000]ms, 异常Msg:[DataX无法连接对应的数据库，可能原因是：1) 配置的ip/port/database/jdbc错误，无法连接。2) 配置的username/password错误，鉴权失败。请和DBA确认该数据库的连接信息是否正确。]
Mon Jul 31 16:55:54 GMT+08:00 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2023-07-31 16:55:54.168 [job-0] WARN DBUtil - test connection of [jdbc:mysql://192.168.48.30:3306/datax] failed, for Code:[MYSQLErrCode-02], Description:[数据库服务的IP地址或者Port错误，请检查填写的IP地址和Port或者联系DBA确认IP地址和Port是否正确。如果是同步中心用户请联系DBA确认idb上录入的IP和PORT信息和数据库的当前实际信息是一致的]. - 具体错误信息为：com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 3 milliseconds ago. The last packet sent successfully to the server was 2 milliseconds ago..
2023-07-31 16:55:58.170 [job-0] ERROR RetryUtil - Exception when calling callable, 即将尝试执行第3次重试.本次重试计划等待[4000]ms,实际等待[4000]ms, 异常Msg:[DataX无法连接对应的数据库，可能原因是：1) 配置的ip/port/database/jdbc错误，无法连接。2) 配置的username/password错误，鉴权失败。请和DBA确认该数据库的连接信息是否正确。]
Mon Jul 31 16:55:58 GMT+08:00 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2023-07-31 16:55:58.177 [job-0] WARN DBUtil - test connection of [jdbc:mysql://192.168.48.30:3306/datax] failed, for Code:[MYSQLErrCode-02], Description:[数据库服务的IP地址或者Port错误，请检查填写的IP地址和Port或者联系DBA确认IP地址和Port是否正确。如果是同步中心用户请联系DBA确认idb上录入的IP和PORT信息和数据库的当前实际信息是一致的]. - 具体错误信息为：com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 3 milliseconds ago. The last packet sent successfully to the server was 2 milliseconds ago..
2023-07-31 16:56:06.180 [job-0] ERROR RetryUtil - Exception when calling callable, 即将尝试执行第4次重试.本次重试计划等待[8000]ms,实际等待[8001]ms, 异常Msg:[DataX无法连接对应的数据库，可能原因是：1) 配置的ip/port/database/jdbc错误，无法连接。2) 配置的username/password错误，鉴权失败。请和DBA确认该数据库的连接信息是否正确。]
Mon Jul 31 16:56:06 GMT+08:00 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2023-07-31 16:56:06.195 [job-0] WARN DBUtil - test connection of [jdbc:mysql://192.168.48.30:3306/datax] failed, for Code:[MYSQLErrCode-02], Description:[数据库服务的IP地址或者Port错误，请检查填写的IP地址和Port或者联系DBA确认IP地址和Port是否正确。如果是同步中心用户请联系DBA确认idb上录入的IP和PORT信息和数据库的当前实际信息是一致的]. - 具体错误信息为：com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 3 milliseconds ago. The last packet sent successfully to the server was 2 milliseconds ago..
2023-07-31 16:56:22.198 [job-0] ERROR RetryUtil - Exception when calling callable, 即将尝试执行第5次重试.本次重试计划等待[16000]ms,实际等待[16001]ms, 异常Msg:[DataX无法连接对应的数据库，可能原因是：1) 配置的ip/port/database/jdbc错误，无法连接。2) 配置的username/password错误，鉴权失败。请和DBA确认该数据库的连接信息是否正确。]
Mon Jul 31 16:56:22 GMT+08:00 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2023-07-31 16:56:22.206 [job-0] WARN DBUtil - test connection of [jdbc:mysql://192.168.48.30:3306/datax] failed, for Code:[MYSQLErrCode-02], Description:[数据库服务的IP地址或者Port错误，请检查填写的IP地址和Port或者联系DBA确认IP地址和Port是否正确。如果是同步中心用户请联系DBA确认idb上录入的IP和PORT信息和数据库的当前实际信息是一致的]. - 具体错误信息为：com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 2 milliseconds ago. The last packet sent successfully to the server was 1 milliseconds ago..
2023-07-31 16:56:54.207 [job-0] ERROR RetryUtil - Exception when calling callable, 即将尝试执行第6次重试.本次重试计划等待[32000]ms,实际等待[32000]ms, 异常Msg:[DataX无法连接对应的数据库，可能原因是：1) 配置的ip/port/database/jdbc错误，无法连接。2) 配置的username/password错误，鉴权失败。请和DBA确认该数据库的连接信息是否正确。]
Mon Jul 31 16:56:54 GMT+08:00 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2023-07-31 16:56:54.220 [job-0] WARN DBUtil - test connection of [jdbc:mysql://192.168.48.30:3306/datax] failed, for Code:[MYSQLErrCode-02], Description:[数据库服务的IP地址或者Port错误，请检查填写的IP地址和Port或者联系DBA确认IP地址和Port是否正确。如果是同步中心用户请联系DBA确认idb上录入的IP和PORT信息和数据库的当前实际信息是一致的]. - 具体错误信息为：com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 2 milliseconds ago. The last packet sent successfully to the server was 1 milliseconds ago..
2023-07-31 16:56:54.227 [job-0] ERROR JobContainer - Exception when job run
com.alibaba.datax.common.exception.DataXException: Code:[DBUtilErrorCode-10], Description:[连接数据库失败. 请检查您的 账号、密码、数据库名称、IP、Port或者向 DBA 寻求帮助(注意网络环境).]. - 数据库连接失败. 因为根据您配置的连接信息,无法从:jdbc:mysql://192.168.48.30:3306/datax 中找到可连接的jdbcUrl. 请检查您的配置并作出修改. - java.lang.Exception: DataX无法连接对应的数据库，可能原因是：1) 配置的ip/port/database/jdbc错误，无法连接。2) 配置的username/password错误，鉴权失败。请和DBA确认该数据库的连接信息是否正确。
at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:71)
at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:51)
at com.alibaba.datax.common.util.RetryUtil$Retry.call(RetryUtil.java:164)
at com.alibaba.datax.common.util.RetryUtil$Retry.doRetry(RetryUtil.java:111)
at com.alibaba.datax.common.util.RetryUtil.executeWithRetry(RetryUtil.java:30)
at com.alibaba.datax.plugin.rdbms.util.DBUtil.chooseJdbcUrl(DBUtil.java:51)
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.dealJdbcAndTable(OriginalConfPretreatmentUtil.java:92)
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:59)
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:33)
at com.alibaba.datax.plugin.rdbms.reader.CommonRdbmsReader$Job.init(CommonRdbmsReader.java:55)
at com.alibaba.datax.plugin.reader.mysqlreader.MysqlReader$Job.init(MysqlReader.java:37)
at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:673)
at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:303)
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113)
at com.alibaba.datax.core.Engine.start(Engine.java:93)
at com.alibaba.datax.core.Engine.entry(Engine.java:175)
at com.alibaba.datax.core.Engine.main(Engine.java:208)
at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:41) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.util.DBUtil.chooseJdbcUrl(DBUtil.java:77) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.dealJdbcAndTable(OriginalConfPretreatmentUtil.java:92) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:59) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:33) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.reader.CommonRdbmsReader$Job.init(CommonRdbmsReader.java:55) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.mysqlreader.MysqlReader$Job.init(MysqlReader.java:37) ~[mysqlreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:673) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:303) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.Engine.start(Engine.java:93) [datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.Engine.entry(Engine.java:175) [datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.Engine.main(Engine.java:208) [datax-core-0.0.1-SNAPSHOT.jar:na]
Caused by: java.lang.Exception: DataX无法连接对应的数据库，可能原因是：1) 配置的ip/port/database/jdbc错误，无法连接。2) 配置的username/password错误，鉴权失败。请和DBA确认该数据库的连接信息是否正确。
at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:71) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:51) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.common.util.RetryUtil$Retry.call(RetryUtil.java:164) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.common.util.RetryUtil$Retry.doRetry(RetryUtil.java:111) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.common.util.RetryUtil.executeWithRetry(RetryUtil.java:30) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.util.DBUtil.chooseJdbcUrl(DBUtil.java:51) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
... 11 common frames omitted
2023-07-31 16:56:54.241 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
2023-07-31 16:56:54.243 [job-0] ERROR Engine -
经DataX智能分析,该任务最可能的错误原因是:
com.alibaba.datax.common.exception.DataXException: Code:[DBUtilErrorCode-10], Description:[连接数据库失败. 请检查您的 账号、密码、数据库名称、IP、Port或者向 DBA 寻求帮助(注意网络环境).]. - 数据库连接失败. 因为根据您配置的连接信息,无法从:jdbc:mysql://192.168.48.30:3306/datax 中找到可连接的jdbcUrl. 请检查您的配置并作出修改. - java.lang.Exception: DataX无法连接对应的数据库，可能原因是：1) 配置的ip/port/database/jdbc错误，无法连接。2) 配置的username/password错误，鉴权失败。请和DBA确认该数据库的连接信息是否正确。
at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:71)
at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:51)
at com.alibaba.datax.common.util.RetryUtil$Retry.call(RetryUtil.java:164)
at com.alibaba.datax.common.util.RetryUtil$Retry.doRetry(RetryUtil.java:111)
at com.alibaba.datax.common.util.RetryUtil.executeWithRetry(RetryUtil.java:30)
at com.alibaba.datax.plugin.rdbms.util.DBUtil.chooseJdbcUrl(DBUtil.java:51)
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.dealJdbcAndTable(OriginalConfPretreatmentUtil.java:92)
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:59)
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:33)
at com.alibaba.datax.plugin.rdbms.reader.CommonRdbmsReader$Job.init(CommonRdbmsReader.java:55)
at com.alibaba.datax.plugin.reader.mysqlreader.MysqlReader$Job.init(MysqlReader.java:37)
at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:673)
at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:303)
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113)
at com.alibaba.datax.core.Engine.start(Engine.java:93)
at com.alibaba.datax.core.Engine.entry(Engine.java:175)
at com.alibaba.datax.core.Engine.main(Engine.java:208)
at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:41)
at com.alibaba.datax.plugin.rdbms.util.DBUtil.chooseJdbcUrl(DBUtil.java:77)
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.dealJdbcAndTable(OriginalConfPretreatmentUtil.java:92)
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:59)
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:33)
at com.alibaba.datax.plugin.rdbms.reader.CommonRdbmsReader$Job.init(CommonRdbmsReader.java:55)
at com.alibaba.datax.plugin.reader.mysqlreader.MysqlReader$Job.init(MysqlReader.java:37)
at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:673)
at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:303)
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113)
at com.alibaba.datax.core.Engine.start(Engine.java:93)
at com.alibaba.datax.core.Engine.entry(Engine.java:175)
at com.alibaba.datax.core.Engine.main(Engine.java:208)
Caused by: java.lang.Exception: DataX无法连接对应的数据库，可能原因是：1) 配置的ip/port/database/jdbc错误，无法连接。2) 配置的username/password错误，鉴权失败。请和DBA确认该数据库的连接信息是否正确。
at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:71)
at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:51)
at com.alibaba.datax.common.util.RetryUtil$Retry.call(RetryUtil.java:164)
at com.alibaba.datax.common.util.RetryUtil$Retry.doRetry(RetryUtil.java:111)
at com.alibaba.datax.common.util.RetryUtil.executeWithRetry(RetryUtil.java:30)
at com.alibaba.datax.plugin.rdbms.util.DBUtil.chooseJdbcUrl(DBUtil.java:51)
... 11 more
2、故障分析
我们可以确认jdbcurl和数据库信息并未填写错误，那么问题可能由于别的原因产生，可以看到上述日志中还打印了一个告警信息：
Mon Jul 31 15:57:52 GMT+08:00 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
告警信息翻译过来大概是：“不建议在没有服务器身份验证的情况下建立 SSL 连接。根据 MySQL 5.5.45+、5.6.26+ 和 5.7.6+ 要求，如果未设置显式选项，则必须默认建立 SSL 连接”，而描述中也说了：“为了符合不使用 SSL 的现有应用程序，验证服务器证书属性设置为“false”。您需要通过设置 useSSL=false 来显式禁用 SSL”
3、问题处理
（1）方式一：禁用 SSL
更改配置文件中连接MySQL数据库的"jdbcUrl": ["jdbc:mysql://192.168.48.30:3306/datax?useSSL=false"]
再次执行json文件输出如下：
# python /datax/datax/bin/datax.py /datax/datax/job/mysql2oracle_datax_test1.json
DataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.
2023-07-31 16:06:51.640 [main] INFO MessageSource - JVM TimeZone: GMT+08:00, Locale: zh_CN
2023-07-31 16:06:51.646 [main] INFO MessageSource - use Locale: zh_CN timeZone: sun.util.calendar.ZoneInfo[id="GMT+08:00",offset=28800000,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
2023-07-31 16:06:51.723 [main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2023-07-31 16:06:51.738 [main] INFO Engine - the machine info =>
osInfo: Oracle Corporation 1.8 25.361-b09
jvmInfo: Linux amd64 5.4.17-2011.6.2.el7uek.x86_64
cpu num: 4
totalPhysicalMemory: -0.00G
freePhysicalMemory: -0.00G
maxFileDescriptorCount: -1
currentOpenFileDescriptorCount: -1
GC Names [PS MarkSweep, PS Scavenge]
MEMORY_NAME | allocation_size | init_size
PS Eden Space | 256.00MB | 256.00MB
Code Cache | 240.00MB | 2.44MB
Compressed Class Space | 1,024.00MB | 0.00MB
PS Survivor Space | 42.50MB | 42.50MB
PS Old Gen | 683.00MB | 683.00MB
Metaspace | -0.00MB | 0.00MB
2023-07-31 16:06:51.787 [main] INFO Engine -
{
"content":[
{
"reader":{
"name":"mysqlreader",
"parameter":{
"column":[
"*"
],
"connection":[
{
"jdbcUrl":[
"jdbc:mysql://192.168.48.30:3306/datax?useSSL=false"
],
"table":[
"datax_test1"
]
}
],
"password":"*******",
"splitPk":"ID",
"username":"root",
"where":""
}
},
"writer":{
"name":"oraclewriter",
"parameter":{
"column":[
"*"
],
"connection":[
{
"jdbcUrl":"jdbc:oracle:thin:@192.168.48.21:1521:orcl",
"table":[
"datax_test1"
]
}
],
"password":"**********",
"preSql":[
"truncate table datax_test1"
],
"username":"shuaige"
}
}
}
],
"setting":{
"speed":{
"channel":"4"
}
}
}
2023-07-31 16:06:51.832 [main] WARN Engine - prioriy set to 0, because NumberFormatException, the value is: null
2023-07-31 16:06:51.836 [main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2023-07-31 16:06:51.836 [main] INFO JobContainer - DataX jobContainer starts job.
2023-07-31 16:06:51.840 [main] INFO JobContainer - Set jobId = 0
2023-07-31 16:06:52.542 [job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://192.168.48.30:3306/datax?useSSL=false&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
2023-07-31 16:06:52.545 [job-0] WARN OriginalConfPretreatmentUtil - 您的配置文件中的列配置存在一定的风险. 因为您未配置读取数据库表的列，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2023-07-31 16:06:53.197 [job-0] INFO OriginalConfPretreatmentUtil - table:[datax_test1] all columns:[
ID,NAME
].
2023-07-31 16:06:53.197 [job-0] WARN OriginalConfPretreatmentUtil - 您的配置文件中的列配置信息存在风险. 因为您配置的写入数据库表的列为*，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2023-07-31 16:06:53.200 [job-0] INFO OriginalConfPretreatmentUtil - Write data [
INSERT INTO %s (ID,NAME) VALUES(?,?)
], which jdbcUrl like:[jdbc:oracle:thin:@192.168.48.21:1521:orcl]
2023-07-31 16:06:53.201 [job-0] INFO JobContainer - jobContainer starts to do prepare ...
2023-07-31 16:06:53.203 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] do prepare work .
2023-07-31 16:06:53.204 [job-0] INFO JobContainer - DataX Writer.Job [oraclewriter] do prepare work .
2023-07-31 16:06:53.289 [job-0] INFO CommonRdbmsWriter$Job - Begin to execute preSqls:[truncate table datax_test1]. context info:jdbc:oracle:thin:@192.168.48.21:1521:orcl.
2023-07-31 16:06:53.341 [job-0] INFO JobContainer - jobContainer starts to do split ...
2023-07-31 16:06:53.342 [job-0] INFO JobContainer - Job set Channel-Number to 4 channels.
2023-07-31 16:06:53.376 [job-0] INFO SingleTableSplitUtil - split pk [sql=SELECT MIN(ID),MAX(ID) FROM datax_test1] is running...
2023-07-31 16:06:53.416 [job-0] INFO SingleTableSplitUtil - After split(), allQuerySql=[
select * from datax_test1 where (1 <= ID AND ID <= 1)
select * from datax_test1 where ID IS NULL
].
2023-07-31 16:06:53.417 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] splits to [2] tasks.
2023-07-31 16:06:53.419 [job-0] INFO JobContainer - DataX Writer.Job [oraclewriter] splits to [2] tasks.
2023-07-31 16:06:53.473 [job-0] INFO JobContainer - jobContainer starts to do schedule ...
2023-07-31 16:06:53.481 [job-0] INFO JobContainer - Scheduler starts [1] taskGroups.
2023-07-31 16:06:53.486 [job-0] INFO JobContainer - Running by standalone Mode.
2023-07-31 16:06:53.500 [taskGroup-0] INFO TaskGroupContainer - taskGroupId=[0] start [2] channels for [2] tasks.
2023-07-31 16:06:53.508 [taskGroup-0] INFO Channel - Channel set byte_speed_limit to -1, No bps activated.
2023-07-31 16:06:53.509 [taskGroup-0] INFO Channel - Channel set record_speed_limit to -1, No tps activated.
2023-07-31 16:06:53.531 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started
2023-07-31 16:06:53.536 [0-0-0-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from datax_test1 where (1 <= ID AND ID <= 1)
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?useSSL=false&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 16:06:53.541 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[1] attemptCount[1] is started
2023-07-31 16:06:53.542 [0-0-1-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from datax_test1 where ID IS NULL
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?useSSL=false&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 16:06:53.560 [0-0-0-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from datax_test1 where (1 <= ID AND ID <= 1)
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?useSSL=false&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 16:06:53.572 [0-0-1-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from datax_test1 where ID IS NULL
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?useSSL=false&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 16:06:53.909 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[1] is successed, used[371]ms
2023-07-31 16:06:54.111 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[583]ms
2023-07-31 16:06:54.113 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] completed it's tasks.
2023-07-31 16:07:03.526 [job-0] INFO StandAloneJobContainerCommunicator - Total 1 records, 3 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 100.00%
2023-07-31 16:07:03.526 [job-0] INFO AbstractScheduler - Scheduler accomplished all tasks.
2023-07-31 16:07:03.527 [job-0] INFO JobContainer - DataX Writer.Job [oraclewriter] do post work.
2023-07-31 16:07:03.528 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] do post work.
2023-07-31 16:07:03.529 [job-0] INFO JobContainer - DataX jobId [0] completed successfully.
2023-07-31 16:07:03.531 [job-0] INFO HookInvoker - No hook invoked, because base dir not exists or is a file: /datax/datax/hook
2023-07-31 16:07:03.535 [job-0] INFO JobContainer -
[total cpu info] =>
averageCpu | maxDeltaCpu | minDeltaCpu
-1.00% | -1.00% | -1.00%
[total gc info] =>
NAME | totalGCCount | maxDeltaGCCount | minDeltaGCCount | totalGCTime | maxDeltaGCTime | minDeltaGCTime
PS MarkSweep | 1 | 1 | 1 | 0.049s | 0.049s | 0.049s
PS Scavenge | 1 | 1 | 1 | 0.017s | 0.017s | 0.017s
2023-07-31 16:07:03.535 [job-0] INFO JobContainer - PerfTrace not enable!
2023-07-31 16:07:03.537 [job-0] INFO StandAloneJobContainerCommunicator - Total 1 records, 3 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 100.00%
2023-07-31 16:07:03.540 [job-0] INFO JobContainer -
任务启动时刻 : 2023-07-31 16:06:51
任务结束时刻 : 2023-07-31 16:07:03
任务总计耗时 : 11s
任务平均流量 : 0B/s
记录写入速度 : 0rec/s
读出记录总数 : 1
读写失败总数 : 0
（2）方式二：升级驱动
其实我们也可以通过升级datax3.0中的“MySQL Connector Java（MySQL官方提供的Java语言连接MySQL数据库的驱动程序）”来实现对mysql数据库的连接
1.下载新版驱动：MySQL Connector Java
2.将安装包上传至服务器/datax/datax/plugin/reader/mysqlreader/libs下，更改用户属主
chown datax.datax /datax/datax/plugin/reader/mysqlreader/libs/mysql-connector-java-8.0.28.jar
3.备份datax3.0中自带的mysql-connector-java-5.1.47.jar
mv /datax/datax/plugin/reader/mysqlreader/libs/mysql-connector-java-5.1.47.jar /datax/datax/plugin/reader/mysqlreader/libs/mysql-connector-java-5.1.47.jar.bak
ll mysql*
-rw-r--r-- 1 datax datax 1007502 Oct 26 2022 mysql-connector-java-5.1.47.jar.bak
-rw-r--r-- 1 datax datax 2476480 Jul 31 09:30 mysql-connector-java-8.0.28.jar
4.再次执行json文件输出如下：
# python /datax/datax/bin/datax.py /datax/datax/job/mysql2oracle_datax_test1.json
DataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.
2023-07-31 16:47:38.717 [main] INFO MessageSource - JVM TimeZone: GMT+08:00, Locale: zh_CN
2023-07-31 16:47:38.722 [main] INFO MessageSource - use Locale: zh_CN timeZone: sun.util.calendar.ZoneInfo[id="GMT+08:00",offset=28800000,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
2023-07-31 16:47:38.817 [main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2023-07-31 16:47:38.833 [main] INFO Engine - the machine info =>
osInfo: Oracle Corporation 1.8 25.361-b09
jvmInfo: Linux amd64 5.4.17-2011.6.2.el7uek.x86_64
cpu num: 4
totalPhysicalMemory: -0.00G
freePhysicalMemory: -0.00G
maxFileDescriptorCount: -1
currentOpenFileDescriptorCount: -1
GC Names [PS MarkSweep, PS Scavenge]
MEMORY_NAME | allocation_size | init_size
PS Eden Space | 256.00MB | 256.00MB
Code Cache | 240.00MB | 2.44MB
Compressed Class Space | 1,024.00MB | 0.00MB
PS Survivor Space | 42.50MB | 42.50MB
PS Old Gen | 683.00MB | 683.00MB
Metaspace | -0.00MB | 0.00MB
2023-07-31 16:47:38.885 [main] INFO Engine -
{
"content":[
{
"reader":{
"name":"mysqlreader",
"parameter":{
"column":[
"*"
],
"connection":[
{
"jdbcUrl":[
"jdbc:mysql://192.168.48.30:3306/datax"
],
"table":[
"datax_test1"
]
}
],
"password":"*******",
"splitPk":"ID",
"username":"root",
"where":""
}
},
"writer":{
"name":"oraclewriter",
"parameter":{
"column":[
"*"
],
"connection":[
{
"jdbcUrl":"jdbc:oracle:thin:@192.168.48.21:1521:orcl",
"table":[
"datax_test1"
]
}
],
"password":"**********",
"preSql":[
"truncate table datax_test1"
],
"username":"shuaige"
}
}
}
],
"setting":{
"speed":{
"channel":"4"
}
}
}
2023-07-31 16:47:38.929 [main] WARN Engine - prioriy set to 0, because NumberFormatException, the value is: null
2023-07-31 16:47:38.934 [main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2023-07-31 16:47:38.935 [main] INFO JobContainer - DataX jobContainer starts job.
2023-07-31 16:47:38.938 [main] INFO JobContainer - Set jobId = 0
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
2023-07-31 16:47:40.332 [job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://192.168.48.30:3306/datax?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
2023-07-31 16:47:40.335 [job-0] WARN OriginalConfPretreatmentUtil - 您的配置文件中的列配置存在一定的风险. 因为您未配置读取数据库表的列，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2023-07-31 16:47:40.964 [job-0] INFO OriginalConfPretreatmentUtil - table:[datax_test1] all columns:[
ID,NAME
].
2023-07-31 16:47:40.965 [job-0] WARN OriginalConfPretreatmentUtil - 您的配置文件中的列配置信息存在风险. 因为您配置的写入数据库表的列为*，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2023-07-31 16:47:40.970 [job-0] INFO OriginalConfPretreatmentUtil - Write data [
INSERT INTO %s (ID,NAME) VALUES(?,?)
], which jdbcUrl like:[jdbc:oracle:thin:@192.168.48.21:1521:orcl]
2023-07-31 16:47:40.972 [job-0] INFO JobContainer - jobContainer starts to do prepare ...
2023-07-31 16:47:40.978 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] do prepare work .
2023-07-31 16:47:40.979 [job-0] INFO JobContainer - DataX Writer.Job [oraclewriter] do prepare work .
2023-07-31 16:47:41.063 [job-0] INFO CommonRdbmsWriter$Job - Begin to execute preSqls:[truncate table datax_test1]. context info:jdbc:oracle:thin:@192.168.48.21:1521:orcl.
2023-07-31 16:47:41.225 [job-0] INFO JobContainer - jobContainer starts to do split ...
2023-07-31 16:47:41.226 [job-0] INFO JobContainer - Job set Channel-Number to 4 channels.
2023-07-31 16:47:41.312 [job-0] INFO SingleTableSplitUtil - split pk [sql=SELECT MIN(ID),MAX(ID) FROM datax_test1] is running...
2023-07-31 16:47:41.394 [job-0] INFO SingleTableSplitUtil - After split(), allQuerySql=[
select * from datax_test1 where (1 <= ID AND ID <= 1)
select * from datax_test1 where ID IS NULL
].
2023-07-31 16:47:41.395 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] splits to [2] tasks.
2023-07-31 16:47:41.397 [job-0] INFO JobContainer - DataX Writer.Job [oraclewriter] splits to [2] tasks.
2023-07-31 16:47:41.433 [job-0] INFO JobContainer - jobContainer starts to do schedule ...
2023-07-31 16:47:41.440 [job-0] INFO JobContainer - Scheduler starts [1] taskGroups.
2023-07-31 16:47:41.445 [job-0] INFO JobContainer - Running by standalone Mode.
2023-07-31 16:47:41.455 [taskGroup-0] INFO TaskGroupContainer - taskGroupId=[0] start [2] channels for [2] tasks.
2023-07-31 16:47:41.463 [taskGroup-0] INFO Channel - Channel set byte_speed_limit to -1, No bps activated.
2023-07-31 16:47:41.464 [taskGroup-0] INFO Channel - Channel set record_speed_limit to -1, No tps activated.
2023-07-31 16:47:41.480 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started
2023-07-31 16:47:41.487 [0-0-0-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from datax_test1 where (1 <= ID AND ID <= 1)
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 16:47:41.488 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[1] attemptCount[1] is started
2023-07-31 16:47:41.489 [0-0-1-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select * from datax_test1 where ID IS NULL
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 16:47:41.542 [0-0-0-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from datax_test1 where (1 <= ID AND ID <= 1)
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 16:47:41.582 [0-0-1-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select * from datax_test1 where ID IS NULL
] jdbcUrl:[jdbc:mysql://192.168.48.30:3306/datax?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true].
2023-07-31 16:47:41.791 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[1] is successed, used[303]ms
2023-07-31 16:47:41.993 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[515]ms
2023-07-31 16:47:41.995 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] completed it's tasks.
2023-07-31 16:47:51.480 [job-0] INFO StandAloneJobContainerCommunicator - Total 1 records, 3 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 100.00%
2023-07-31 16:47:51.480 [job-0] INFO AbstractScheduler - Scheduler accomplished all tasks.
2023-07-31 16:47:51.481 [job-0] INFO JobContainer - DataX Writer.Job [oraclewriter] do post work.
2023-07-31 16:47:51.482 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] do post work.
2023-07-31 16:47:51.483 [job-0] INFO JobContainer - DataX jobId [0] completed successfully.
2023-07-31 16:47:51.486 [job-0] INFO HookInvoker - No hook invoked, because base dir not exists or is a file: /datax/datax/hook
2023-07-31 16:47:51.493 [job-0] INFO JobContainer -
[total cpu info] =>
averageCpu | maxDeltaCpu | minDeltaCpu
-1.00% | -1.00% | -1.00%
[total gc info] =>
NAME | totalGCCount | maxDeltaGCCount | minDeltaGCCount | totalGCTime | maxDeltaGCTime | minDeltaGCTime
PS MarkSweep | 1 | 1 | 1 | 0.053s | 0.053s | 0.053s
PS Scavenge | 1 | 1 | 1 | 0.020s | 0.020s | 0.020s
2023-07-31 16:47:51.494 [job-0] INFO JobContainer - PerfTrace not enable!
2023-07-31 16:47:51.497 [job-0] INFO StandAloneJobContainerCommunicator - Total 1 records, 3 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 100.00%
2023-07-31 16:47:51.502 [job-0] INFO JobContainer -
任务启动时刻 : 2023-07-31 16:47:38
任务结束时刻 : 2023-07-31 16:47:51
任务总计耗时 : 12s
任务平均流量 : 0B/s
记录写入速度 : 0rec/s
读出记录总数 : 1
读写失败总数 : 0
至此结束，有什么问题欢迎留言。