Canal部署过程中的错误

错误整理的不是很全面，有一些可能在后期整理文档的过程中丢失了，如果有人遇到了未整理的错误，也可以私信我

错误一：NullPointerException

日志：

ERROR c.a.o.canal.adapter.launcher.loader.CanalAdapterService - ## something goes wrong when starting up the canal client adapters:
java.lang.NullPointerException: null

原因是：https://www.cnblogs.com/lukairui/p/12461339.html

简单介绍：

​ 在canal-adapter/conf文件夹下的application.yml文件中

源码中getMqServers()这个方法在调用的时候，返回了空值

错误二：MySQLNonTransientConnectionException

日志：

MySQLNonTransientConnectionException: Could not create connection to database server.

原因：

更改一下url就好了

url: jdbc:mysql://localhost:3306/yinliu?characterEncoding=utf8&useSSL=false&serverTimezone=UTC&rewriteBatchedStatements=true
 
driver: com.mysql.cj.jdbc.Driver

错误三：ESConnection

日志：

ERROR c.a.otter.canal.client.adapter.es7x.support.ESConnection - [test_user/fCjeNaRFQhynESPY9hfN-Q][[test_user][2]] ElasticsearchException[Elasticsearch exception [type=document_missing_exception, reason=[_doc][10]: document missing]]

原因：

修改的mysql数据库数据，在es中不存在。

错误四：UnknownHostException

日志：

java.lang.RuntimeException: java.net.UnknownHostException: http: Name or service not known
        at com.alibaba.otter.canal.client.adapter.es7x.ES7xAdapter.init(ES7xAdapter.java:54) ~[client-adapter.es7x-1.1.5-SNAPSHOT-jar-with-dependencies.jar:na]
        at com.alibaba.otter.canal.adapter.launcher.loader.CanalAdapterLoader.loadAdapter(CanalAdapterLoader.java:225) [client-adapter.launcher-1.1.5.jar:na]
        ......
        at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:395) ~[spring-boot-2.0.1.RELEASE.jar:2.0.1.RELEASE]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:327) ~[spring-boot-2.0.1.RELEASE.jar:2.0.1.RELEASE]
        at com.alibaba.otter.canal.adapter.launcher.CanalAdapterApplication.main(CanalAdapterApplication.java:19) ~[client-adapter.launcher-1.1.5.jar:na]
Caused by: java.net.UnknownHostException: http: Name or service not known
        at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) ~[na:1.8.0_292]
        at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929) ~[na:1.8.0_292]
        at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324) ~[na:1.8.0_292]
        at java.net.InetAddress.getAllByName0(InetAddress.java:1277) ~[na:1.8.0_292]
        at java.net.InetAddress.getAllByName(InetAddress.java:1193) ~[na:1.8.0_292]
        at java.net.InetAddress.getAllByName(InetAddress.java:1127) ~[na:1.8.0_292]
        at java.net.InetAddress.getByName(InetAddress.java:1077) ~[na:1.8.0_292]
        at com.alibaba.otter.canal.client.adapter.es7x.support.ESConnection.(ESConnection.java:84) ~[client-adapter.es7x-1.1.5-SNAPSHOT-jar-with-dependencies.jar:na]
        at com.alibaba.otter.canal.client.adapter.es7x.ES7xAdapter.init(ES7xAdapter.java:45) ~[client-adapter.es7x-1.1.5-SNAPSHOT-jar-with-dependencies.jar:na]
        ... 42 common frames omitted

原因：

      - name: es7
        key: es-test
        hosts: http://127.0.0.1:9200 # 127.0.0.1:9200 for rest mode
        properties:
          mode: rest # transport # or rest
#          # security.auth: test:123456 #  only used for rest mode
          cluster.name: es

修改：

      - name: es7
        key: es-test
        hosts: 127.0.0.1:9200 # 127.0.0.1:9200 for rest mode
        properties:
          mode: rest # transport # or rest
#          # security.auth: test:123456 #  only used for rest mode
          cluster.name: es

错误五：CommunicationsException

日志：连接mysql时候报错08001

ERROR com.alibaba.druid.pool.DruidDataSource - create connection SQLException, url: jdbc:mysql://127.0.0.1:3306/test?useUnicode=true, errorCode 0, state 08S01
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

原因：

MySQL的连接字符串需要配置ServerTimezone属性，可以用UTC、Hongkong、Asiz/Shanghai等。

如果使用UTC的话会有8小时时差，所以建议使用Hongkong或Asiz/Shanghai。

修改：conf/application.yml

  srcDataSources:
    defaultDS:
      url: jdbc:mysql://127.0.0.1:3306/test?useUnicode=true&useSSL=false&serverTimezone=Asia/Shanghai&characterEncoding=utf-8&autoReconnect=true
      username: canal
      password: Canal@123

错误六：CanalClientException

日志：

 ERROR c.a.otter.canal.adapter.launcher.loader.AdapterProcessor - process error!
com.alibaba.otter.canal.protocol.exception.CanalClientException: java.net.ConnectException: Connection refused
        at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.doConnect(SimpleCanalConnector.java:198) ~[na:na]
        at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.connect(SimpleCanalConnector.java:115) ~[na:na]
        at com.alibaba.otter.canal.connector.tcp.consumer.CanalTCPConsumer.connect(CanalTCPConsumer.java:63) ~[na:na]
        at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.process(AdapterProcessor.java:184) ~[client-adapter.launcher-1.1.5.jar:na]
        at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_292]
Caused by: java.net.ConnectException: Connection refused
        at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_292]
        at sun.nio.ch.Net.connect(Net.java:482) ~[na:1.8.0_292]
        at sun.nio.ch.Net.connect(Net.java:474) ~[na:1.8.0_292]
        at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:647) ~[na:1.8.0_292]
        at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.doConnect(SimpleCanalConnector.java:150) ~[na:na]
        ... 4 common frames omitted

原因：配置端口不对或对应端口服务异常

修改：

错误七：ClassCastException

日志：

 ERROR c.a.o.canal.adapter.launcher.loader.CanalAdapterLoader - Load canal adapter: es7 failed
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassCastException: com.alibaba.druid.pool.DruidDataSource cannot be cast to com.alibaba.druid.pool.DruidDataSource
        at com.alibaba.otter.canal.client.adapter.es7x.ES7xAdapter.init(ES7xAdapter.java:54) ~[client-adapter.es7x-1.1.5-jar-with-dependencies.jar:na]
        at com.alibaba.otter.canal.adapter.launcher.loader.CanalAdapterLoader.loadAdapter(CanalAdapterLoader.java:225) [client-adapter.launcher-1.1.5.jar:na]
        at com.alibaba.otter.canal.adapter.launcher.loader.CanalAdapterLoader.init(CanalAdapterLoader.java:56) [client-adapter.launcher-1.1.5.jar:na]
        ......
        at com.alibaba.otter.canal.client.adapter.es7x.ES7xAdapter.init(ES7xAdapter.java:52) ~[client-adapter.es7x-1.1.5-jar-with-dependencies.jar:na]
        ... 42 common frames omitted
Caused by: java.lang.ClassCastException: com.alibaba.druid.pool.DruidDataSource cannot be cast to com.alibaba.druid.pool.DruidDataSource
        at com.alibaba.otter.canal.client.adapter.es.core.ESAdapter.addSyncConfigToCache(ESAdapter.java:146) ~[client-adapter.es7x-1.1.5-jar-with-dependencies.jar:na]
        at com.alibaba.otter.canal.client.adapter.es.core.ESAdapter.init(ESAdapter.java:75) ~[client-adapter.es7x-1.1.5-jar-with-dependencies.jar:na]
        ... 43 common frames omitted

原因：canaladapter-1.1.5版本的druid包冲突

修改：

• 下载v1.1.5-alpha-2版本的
• 解压后，将canaladapter-1.1.5的plugin里面的client-adapter.es7x-1.1.5-jar-with-dependencies.jar替换为v1.1.5-alpha-2版本中plugin里面的client-adapter.es7x-1.1.5-SNAPSHOT-jar-with-dependencies.jar

错误八：IllegalStateException

日志：

java.lang.IllegalStateException: Extension instance(name: es7, class: interface com.alibaba.otter.canal.client.adapter.OuterAdapter) could not be instantiated: class could not be found

原因：

​ 文件下配置有问题

修改：

错误九：65536

ERROR: [5] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
[2]: memory locking requested for elasticsearch process but memory is not locked
[3]: max number of threads [1024] for user [wangxiang] is too low, increase to at least [4096]
[4]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[5]: system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
1.修改vi /etc/security/limits.conf文件，增加配置

      soft    nofile          65536
      hard    nofile          65536

切换到root用户，编辑vi /etc/security/limits.conf配置文件，添加如下内容:
soft memlock unlimited
hard memlock unlimited
备注：* 代表Linux所有用户名称

最大线程个数太低。修改配置文件 vi /etc/security/limits.d/90-nproc.conf
soft nproc 4096
1
hard nproc 4096
1
修改/etc/sysctl.conf文件，增加配置
vm.max_map_count=262144
或者
vm.max_map_count=655360
执行命令sysctl -p生效

canal程序包API

网址： http://alibaba.github.io/canal/apidocs/1.0.13/index.html?com/alibaba/otter/canal/client/package-use.html