编译flink1.9.0 报flink-fs-hadoop-shaded找不到

编译flink1.9.0 报flink-fs-hadoop-shaded找不到

1.Flink 源码下载

git clone [email protected]:apache/flink.git

然后你可以切换到项目的不同分支，执行下面命令将代码切换到 release-1.9 分支：

git checkout release-1.9

2.Flink 源码编译

mvn clean install -Dmaven.test.skip=true -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true

• -Dmaven.test.skip：跳过测试代码
• -Dmaven.javadoc.skip：跳过 javadoc 检查
• -Dcheckstyle.skip：跳过代码风格检查

maven 编译的时候跳过这些检查，这样可以减少很多时间，还可能会减少错误的发生。

注意：你的 maven 的 settings.xml 文件的 mirror 添加下面这个(这样才能下载到某些下载不了的依赖)。

  nexus-aliyun
  *,!jeecg,!jeecg-snapshots,!mapr-releases
  Nexus aliyun
  http://maven.aliyun.com/nexus/content/groups/public



  mapr-public
  mapr-releases
  mapr-releases
  https://maven.aliyun.com/repository/mapr-public

编译报错

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:12 min
[INFO] Finished at: 2020-02-22T09:00:29+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project flink-s3-fs-hadoop: Could not resolve dependencies for project org.apache.flink:flink-s3-fs-hadoop:jar:1.9-SNAPSHOT: Could not find artifact org.apache.flink:flink-fs-hadoop-shaded:jar:tests:1.9-SNAPSHOT in nexus-aliyun (http://maven.aliyun.com/nexus/content/groups/public) -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :flink-s3-fs-hadoop

解决方法

在网站 https://www.mvnjar.com/
搜索对应的jar包

下载后添加到本地

192:flink lijun$ mvn install:install-file -DgroupId=org.apache.flink -DartifactId=flink-fs-hadoop-shaded -Dversion=1.9-SNAPSHOT -Dpackaging=jar  -Dfile=/Users/lijun/Downloads/flink-s3-fs-hadoop-1.9.0.jar
[INFO] Scanning for projects...
[WARNING] 
[WARNING] Some problems were encountered while building the effective model for org.apache.flink:flink-scala_2.11:jar:1.9-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.flink:flink-scala_${scala.binary.version}:[unknown-version], /Users/lijun/IT/workspace/workspace_idea/bigData/flink/flink-scala/pom.xml, line 31, column 14
[WARNING] 'build.plugins.plugin.version' for org.codehaus.mojo:build-helper-maven-plugin is missing. @ org.apache.flink:flink-scala_${scala.binary.version}:[unknown-version], /Users/lijun/IT/workspace/workspace_idea/bigData/flink/flink-scala/pom.xml, line 215, column 12
[WARNING] 
[WARNING] Some problems were encountered while building the effective model for org.apache.flink:flink-runtime_2.11:jar:1.9-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.flink:flink-runtime_${scala.binary.version}:[unknown-version], /Users/lijun/IT/workspace/workspace_idea/bigData/flink/flink-runtime/pom.xml, line 32, column 14
[WARNING] 'build.plugins.plugin.version' for org.codehaus.mojo:build-helper-maven-plugin is missing. @ org.apache.flink:flink-runtime_${scala.binary.version}:[unknown-version], /Users/lijun/IT/workspace/workspace_idea/bigData/flink/flink-runtime/pom.xml, line 424, column 12
[WARNING] Some problems were encountered while building the effective model for org.apache.flink:flink-table-planner_2.11:jar:1.9-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.flink:flink-table-planner_${scala.binary.version}:[unknown-version], /Users/lijun/IT/workspace/workspace_idea/bigData/flink/flink-table/flink-table-planner/pom.xml, line 30, column 14
[WARNING] 

[WARNING] Some problems were encountered while building the effective model for or
[INFO] flink-plugins-test ................................. SKIPPED
[INFO] dummy-fs ........................................... SKIPPED
[INFO] another-dummy-fs ................................... SKIPPED
[INFO] flink-tpch-test .................................... SKIPPED
[INFO] flink-contrib ...................................... SKIPPED
[INFO] flink-connector-wikiedits .......................... SKIPPED
[INFO] flink-yarn-tests ................................... SKIPPED
[INFO] flink-fs-tests ..................................... SKIPPED
[INFO] flink-docs ......................................... SKIPPED
[INFO] flink-ml-parent .................................... SKIPPED
[INFO] flink-ml-api ....................................... SKIPPED
[INFO] flink-ml-lib 1.9-SNAPSHOT .......................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.672 s
[INFO] Finished at: 2020-02-22T09:11:41+08:00
[INFO] ------------------------------------------------------------------------
192:flink lijun$ 

问题解决