flink-connector-hbase 任务提交yarn出现类冲突bugfix

第一种报错信息：

java.lang.RuntimeException: Cannot create connection to HBase.
	at org.apache.flink.connector.hbase.sink.HBaseSinkFunction.open(HBaseSinkFunction.java:99)
	at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:34)
	at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
	at org.apache.flink.table.runtime.operators.sink.SinkOperator.open(SinkOperator.java:58)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:437)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:574)
	at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.call(StreamTaskActionExecutor.java:100)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:554)
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:756)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: java.lang.reflect.UndeclaredThrowableException
	at org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:65)
	at org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:36)
	at org.apache.flink.connector.hbase.sink.HBaseSinkFunction.open(HBaseSinkFunction.java:72)
	... 12 more
Caused by: java.lang.reflect.UndeclaredThrowableException
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1769)
	at org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:347)
	at org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:61)
	... 14 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.client.ConnectionFactory.lambda$createConnection$0(ConnectionFactory.java:62)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
	... 16 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.hbase.client.ClusterStatusListener$MulticastListener not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2299)
	at org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.client.ConnectionImplementation.<init>(ConnectionImplementation.java:339)
	... 24 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.hbase.client.ClusterStatusListener$MulticastListener not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2267)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2291)
	... 25 more
Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.hbase.client.ClusterStatusListener$MulticastListener not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2171)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2265)
	... 26 more

第二种报错信息：

Caused by: java.lang.RuntimeException: java.lang.RuntimeException: class org.apache.hadoop.hbase.client.ClusterStatusListener$MulticastListener not org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.client.ClusterStatusListener$Listener
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2299)
	at org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.client.ConnectionImplementation.<init>(ConnectionImplementation.java:339)
	... 37 more
Caused by: java.lang.RuntimeException: class org.apache.hadoop.hbase.client.ClusterStatusListener$MulticastListener not org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.client.ClusterStatusListener$Listener
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2293)

配置：

<dependencies>
        <dependency>
            <groupId>org.apache.flinkgroupId>
            <artifactId>flink-sql-connector-hbase-2.2_${scala.binary.version}artifactId>
            <version>${flink.version}version>
        dependency>
        <dependency>
            <groupId>org.apache.hadoopgroupId>
            <artifactId>hadoop-commonartifactId>
            <version>${hadoop-client.version}version>
            <scope>providedscope>
            <exclusions>
                <exclusion>
                    <artifactId>commons-cliartifactId>
                    <groupId>commons-cligroupId>
                exclusion>
            exclusions>
        dependency>
    dependencies>

产生问题原因：
flink-sql-connector-hbase-2.2_2.11中有一个配置文件hbase-default.xml，在打包的时候会打进来，在这个配置文件中有一个配置项为：

<property>
		<name>hbase.status.listener.classname>
		<value>org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.client.ClusterStatusListener$MulticastListenervalue>
		<description>
			Implementation of the status listener with a multicast message.
		description>
	property>

这个配置是导致问题产生的原因，如果引入flink-sql-connector-hbase-2.2_2.11，这个依赖中产生的hbase配置文件和hbase-common依赖自带的配置文件是不一样的，当任务提交到yarn上时，如果依赖的是flink-sql-connector-hbase-2.2_2.11，那么hbase.status.listener.class配置项的value应该为org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.client.ClusterStatusListener$MulticastListener，但是由于种种原因，最后代码读取配置文件时，读到该项配置为org.apache.hadoop.hbase.client.ClusterStatusListener$MulticastListener ，导致上述报错信息产生。

解决方法：

1. 将flink-sql-connector-hbase-2.2_2.11改为flink-connector-hbase-2.2_2.11，并且引入hbase-client，解决问题（推荐）
2. 更改ConnectionImplementation源码：将其中的getClass方法写死，直接手动指定listenerClass （不推荐）

 this.metaCache = new MetaCache(this.metrics);
        boolean shouldListen = conf.getBoolean("hbase.status.published", false);
        this.hostnamesCanChange = conf.getBoolean("hbase.resolve.hostnames.on.failure", true);
        // 不去配置文件中获取，直接手动指定类
        Class<? extends Listener> listenerClass = ClusterStatusListener.DEFAULT_STATUS_LISTENER_CLASS;
        //Class listenerClass = conf.getClass("hbase.status.listener.class", ClusterStatusListener.DEFAULT_STATUS_LISTENER_CLASS, Listener.class);
        LOG.warn("listenerClass.name = {}",listenerClass.getName());
        this.alternateBufferedMutatorClassName = this.conf.get("hbase.client.bufferedmutator.classname");