beam问题

一、

警告: Could not read Source statistics: {}

java.lang.IllegalArgumentException: Error matching the pattern or glob /home/admin/flink-1.3.0/README.txt: status NOT_FOUND
at org.apache.beam.sdk.repackaged.com.google.common.base.Preconditions.checkArgument(Preconditions.java:383)
at org.apache.beam.sdk.io.FileBasedSource.getEstimatedSizeBytes(FileBasedSource.java:209)
at org.apache.beam.runners.flink.translation.wrappers.SourceInputFormat.getStatistics(SourceInputFormat.java:89)
at org.apache.flink.optimizer.dag.DataSourceNode.computeOperatorSpecificDefaultEstimates(DataSourceNode.java:166)
at org.apache.flink.optimizer.dag.OptimizerNode.computeOutputEstimates(OptimizerNode.java:589)
at org.apache.flink.optimizer.traversals.IdAndEstimatesVisitor.postVisit(IdAndEstimatesVisitor.java:61)
at org.apache.flink.optimizer.traversals.IdAndEstimatesVisitor.postVisit(IdAndEstimatesVisitor.java:32)
at org.apache.flink.optimizer.dag.DataSourceNode.accept(DataSourceNode.java:250)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:520)
at org.apache.flink.optimizer.dag.SingleInputNode.accept(SingleInputNode.java:515)
at org.apache.flink.optimizer.dag.DataSinkNode.accept(DataSinkNode.java:248)
at org.apache.flink.optimizer.Optimizer.compile(Optimizer.java:477)
at org.apache.flink.optimizer.Optimizer.compile(Optimizer.java:398)
at org.apache.flink.client.program.ClusterClient.getOptimizedPlan(ClusterClient.java:333)
at org.apache.flink.client.program.ClusterClient.getOptimizedPlan(ClusterClient.java:739)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:428)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:404)
at org.apache.flink.client.RemoteExecutor.executePlanWithJars(RemoteExecutor.java:211)
at org.apache.flink.client.RemoteExecutor.executePlan(RemoteExecutor.java:188)
at org.apache.flink.api.java.RemoteEnvironment.execute(RemoteEnvironment.java:172)
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:118)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:283)

at org.apache.beam.examples.WordCount.main(WordCount.java:185)

在idea上运行beam的时候，出现这种警告，但是不影响最终结果的输出。

不过，运行的会卡3分钟多，去下载jar文件。

十月 24, 2017 6:29:26 下午 org.apache.flink.runtime.client.JobSubmissionClientActor$1 call
信息: Upload jar files to job manager akka.tcp://flink@flinkmaster-1:6123/user/jobmanager.
十月 24, 2017 6:29:26 下午 org.apache.flink.runtime.blob.BlobClient uploadJarFiles
信息: Blob client connecting to akka.tcp://flink@flinkmaster-1:6123/user/jobmanager
十月 24, 2017 6:32:49 下午 org.apache.flink.runtime.client.JobSubmissionClientActor$1 call
信息: Submit job to the job manager akka.tcp://flink@flinkmaster-1:6123/user/jobmanager.

二、

mvn exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount -Pflink-runner -Dexec.args="--runner=FlinkRunner \

--inputFile=/home/admin/flink/flink-1.3.2/README.txt \

--output=/tmp/counts \

--flinkMaster=hbase-1:6123 \

--filesToStage=target/word-count-beam-bundled-0.1.jar

执行这个命令式，参数=之间不要留有空格，否则会报错

WARNING] 

java.lang.reflect.InvocationTargetException

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)

at java.lang.Thread.run(Thread.java:748)

Caused by: java.lang.IllegalArgumentException: Argument 'hbase-1:36398' does not begin with '--'

at org.apache.beam.sdk.repackaged.com.google.common.base.Preconditions.checkArgument(Preconditions.java:191)

at org.apache.beam.sdk.options.PipelineOptionsFactory.parseCommandLine(PipelineOptionsFactory.java:1513)

at org.apache.beam.sdk.options.PipelineOptionsFactory.access$200(PipelineOptionsFactory.java:104)

at org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:288)

at org.apache.beam.examples.WordCount.main(WordCount.java:174)

... 6 more

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 2.817s

[INFO] Finished at: Tue Oct 24 21:50:21 CST 2017

[INFO] Final Memory: 19M/342M

[INFO] ------------------------------------------------------------------------

[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project word-count-beam: An exception occured while executing the Java class. null: InvocationTargetException: Argument 'hbase-1:36398' does not begin with '--' -> [Help 1]

[ERROR] 

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR] 

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException