如何源码编译seaTunnel

如何源码编译seaTunnel

参考Set Up Develop Environment

编译前准备

下列软件需要提前安装好

• Git
• Java ( JDK8/JDK11) 并设置JAVA_HOME 环境变量
• Scala (只支持 scala 2.11.12 )
• JetBrains IDEA .

下载源码并编译

git clone [email protected]:apache/seatunnel.git
cd seatunnel
# 切换到指定分支
git checkout 2.3.3

# 编译安装整个项目
mvn install -Dmaven.test.skip

# 编译整个seaTunnel
mvn clean package -pl seatunnel-dist -am -Dmaven.test.skip=true

#编译指定的模块
mvn clean package -pl seatunnel-connectors-v2/connector-redis -am -DskipTests -T 1C

编译碰到的问题

找不到seatunnel-flink-starter-common

针对2.3.3版本，编译时如果报：seatunnel-flink-starter-common找不到，则需要做如下操作：

• 找到seatunnel\seatunnel-core\seatunnel-flink-starter\seatunnel-flink-15-starter\pom.xml文件
• 将seatunnel-flink-starter-common的版本从${revision}修改为${project.version}

<dependency>
            <groupId>org.apache.seatunnelgroupId>
            <artifactId>seatunnel-flink-starter-commonartifactId>
            <version>${project.version}version>
        dependency>

Jindo Oss编译失败

删除掉不需要的connector，

• 找到seatunnel\seatunnel-connectors-v2\pom.xml文件，将如下connector删除

   <modules>
          <module>connector-commonmodule>
          <module>connector-cdcmodule>
          <module>connector-clickhousemodule>
          <module>connector-consolemodule>
          <module>connector-fakemodule>
          <module>connector-httpmodule>
          <module>connector-jdbcmodule>
          <module>connector-kafkamodule>
          <module>connector-socketmodule>
          <module>connector-hivemodule>
          <module>connector-filemodule> 
          <module>connector-assertmodule>
          <module>connector-emailmodule>
          <module>connector-elasticsearchmodule>  
          <module>connector-redismodule>
          <module>connector-starrocksmodule>
  		<module>connector-paimonmodule>
  		
      modules>
  

• 找到seatunnel\seatunnel-connectors-v2\pom.xml文件，将如下connector删除

  		 <modules>
          <module>connector-file-basemodule>
          <module>connector-file-hadoopmodule>
          <module>connector-file-localmodule>
          <module>connector-file-ossmodule>
          <module>connector-file-ftpmodule>
          <module>connector-file-base-hadoopmodule>
          <module>connector-file-sftpmodule>
          <module>connector-file-s3module>
  		
      modules>
  

seatunnel-dist编译失败

将:seatunnel-dist\pom.xml中缺少的依赖注释掉（如将下面的依赖注释掉）

				 <dependency>
                    <groupId>org.apache.seatunnelgroupId>
                    <artifactId>connector-http-feishuartifactId>
                    <version>${project.version}version>
                    <scope>providedscope>
                dependency>
                <dependency>
                    <groupId>org.apache.seatunnelgroupId>
                    <artifactId>connector-http-wechatartifactId>
                    <version>${project.version}version>
                    <scope>providedscope>
                dependency>
				<dependency>
                    <groupId>org.apache.seatunnelgroupId>
                    <artifactId>connector-http-myhoursartifactId>
                    <version>${project.version}version>
                    <scope>providedscope>
                dependency>
                <dependency>
                    <groupId>org.apache.seatunnelgroupId>
                    <artifactId>connector-http-lemlistartifactId>
                    <version>${project.version}version>
                    <scope>providedscope>
                dependency>
				<dependency>
                    <groupId>org.apache.seatunnelgroupId>
                    <artifactId>connector-file-jindo-ossartifactId>
                    <version>${project.version}version>
                    <scope>providedscope>
                dependency>
                <dependency>
                    <groupId>org.apache.seatunnelgroupId>
                    <artifactId>connector-file-cosartifactId>
                    <version>${project.version}version>
                    <scope>providedscope>
                dependency>