flink系列1 程序win上启动

2018-01-11 14:30:24,595 INFO  org.apache.flink.runtime.jobmanager.JobManager                - --------------------------------------------------------------------------------
2018-01-11 14:30:24,596 INFO  org.apache.flink.runtime.jobmanager.JobManager                -  Starting JobManager (Version: 1.4.0, Rev:3a9d9f2, Date:06.12.2017 @ 11:08:40 UTC)
2018-01-11 14:30:24,596 INFO  org.apache.flink.runtime.jobmanager.JobManager                -  OS current user: astron
2018-01-11 14:30:24,797 ERROR org.apache.hadoop.util.Shell                                  - Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
    at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:379)
    at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:394)
    at org.apache.hadoop.util.Shell.(Shell.java:387)
    at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
    at org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
    at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
    at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:791)
    at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:761)
    at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:634)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.flink.runtime.util.EnvironmentInformation.getHadoopUser(EnvironmentInformation.java:96)
    at org.apache.flink.runtime.util.EnvironmentInformation.logEnvironmentInfo(EnvironmentInformation.java:285)
    at org.apache.flink.runtime.jobmanager.JobManager$.main(JobManager.scala:1904) at org.apache.flink.runtime.jobmanager.JobManager.main(JobManager.scala) 2018-01-11 14:30:24,808 WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2018-01-11 14:30:24,863 INFO org.apache.flink.runtime.jobmanager.JobManager - Current Hadoop/Kerberos user: astron 2018-01-11 14:30:24,863 INFO org.apache.flink.runtime.jobmanager.JobManager - JVM: Java HotSpot(TM) 64-Bit Server VM - Oracle Corporation - 1.8/25.151-b12 2018-01-11 14:30:24,863 INFO org.apache.flink.runtime.jobmanager.JobManager - Maximum heap size: 736 MiBytes 2018-01-11 14:30:24,863 INFO org.apache.flink.runtime.jobmanager.JobManager - JAVA_HOME: C:\Program Files\Java\jdk1.8.0_151 2018-01-11 14:30:24,865 INFO org.apache.flink.runtime.jobmanager.JobManager - Hadoop version: 2.7.3 2018-01-11 14:30:24,865 INFO org.apache.flink.runtime.jobmanager.JobManager - JVM Options: 2018-01-11 14:30:24,865 INFO org.apache.flink.runtime.jobmanager.JobManager - -Xms768m 2018-01-11 14:30:24,865 INFO org.apache.flink.runtime.jobmanager.JobManager - -Xmx768m 2018-01-11 14:30:24,866 INFO org.apache.flink.runtime.jobmanager.JobManager - -Dlog.file=D:\flink-1.4.0-bin-hadoop27-scala_2.11\flink-1.4.0\bin\..\log\flink-astron-jobmanager-CDYJY-astronPC.log 2018-01-11 14:30:24,866 INFO org.apache.flink.runtime.jobmanager.JobManager - -Dlogback.configurationFile=file:D:\flink-1.4.0-bin-hadoop27-scala_2.11\flink-1.4.0\bin\..\conf/logback.xml 2018-01-11 14:30:24,866 INFO org.apache.flink.runtime.jobmanager.JobManager - -Dlog4j.configuration=file:D:\flink-1.4.0-bin-hadoop27-scala_2.11\flink-1.4.0\bin\..\conf/log4j.properties 2018-01-11 14:30:24,866 INFO org.apache.flink.runtime.jobmanager.JobManager - Program Arguments: 2018-01-11 14:30:24,866 INFO org.apache.flink.runtime.jobmanager.JobManager - --configDir 2018-01-11 14:30:24,866 INFO org.apache.flink.runtime.jobmanager.JobManager - D:\flink-1.4.0-bin-hadoop27-scala_2.11\flink-1.4.0\bin\..\conf 2018-01-11 14:30:24,866 INFO org.apache.flink.runtime.jobmanager.JobManager - --executionMode 2018-01-11 14:30:24,866 INFO org.apache.flink.runtime.jobmanager.JobManager - local 2018-01-11 14:30:24,867 INFO org.apache.flink.runtime.jobmanager.JobManager - Classpath: D:\flink-1.4.0-bin-hadoop27-scala_2.11\flink-1.4.0\bin\..\lib\flink-dist_2.11-1.4.0.jar;D:\flink-1.4.0-bin-hadoop27-scala_2.11\flink-1.4.0\bin\..\lib\flink-python_2.11-1.4.0.jar;D:\flink-1.4.0-bin-hadoop27-scala_2.11\flink-1.4.0\bin\..\lib\flink-shaded-hadoop2-uber-1.4.0.jar;D:\flink-1.4.0-bin-hadoop27-scala_2.11\flink-1.4.0\bin\..\lib\log4j-1.2.17.jar;D:\flink-1.4.0-bin-hadoop27-scala_2.11\flink-1.4.0\bin\..\lib\slf4j-log4j12-1.7.7.jar; 2018-01-11 14:30:24,867 INFO org.apache.flink.runtime.jobmanager.JobManager - -------------------------------------------------------------------------------- 2018-01-11 14:30:24,868 INFO org.apache.flink.runtime.jobmanager.JobManager - Registered UNIX signal handlers for [TERM, INT] 2018-01-11 14:30:24,962 INFO org.apache.flink.runtime.jobmanager.JobManager - Loading configuration from D:\flink-1.4.0-bin-hadoop27-scala_2.11\flink-1.4.0\bin\..\conf 2018-01-11 14:30:24,965 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.rpc.address, localhost 2018-01-11 14:30:24,965 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.rpc.port, 6123 2018-01-11 14:30:24,965 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.heap.mb, 1024 2018-01-11 14:30:24,965 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.heap.mb, 1024 2018-01-11 14:30:24,965 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.numberOfTaskSlots, 1 2018-01-11 14:30:24,966 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.memory.preallocate, false 2018-01-11 14:30:24,966 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: parallelism.default, 1 2018-01-11 14:30:24,966 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: web.port, 8081 2018-01-11 14:30:24,988 INFO org.apache.flink.runtime.jobmanager.JobManager - Starting JobManager without high-availability 2018-01-11 14:30:24,990 INFO org.apache.flink.runtime.jobmanager.JobManager - Starting JobManager on localhost:6123 with execution mode LOCAL 2018-01-11 14:30:25,020 INFO org.apache.flink.runtime.security.modules.HadoopModule - Hadoop user set to astron (auth:SIMPLE) 2018-01-11 14:30:25,154 INFO org.apache.flink.runtime.jobmanager.JobManager - Trying to start actor system at localhost:6123 2018-01-11 14:30:25,421 INFO akka.event.slf4j.Slf4jLogger - Slf4jLogger started 2018-01-11 14:30:25,749 INFO akka.remote.Remoting - Starting remoting 2018-01-11 14:30:25,875 INFO akka.remote.Remoting - Remoting started; listening on addresses :[akka.tcp://flink@localhost:6123] 2018-01-11 14:30:25,880 INFO org.apache.flink.runtime.jobmanager.JobManager - Actor system started at akka.tcp://flink@localhost:6123 2018-01-11 14:30:25,893 INFO org.apache.flink.runtime.metrics.MetricRegistryImpl - No metrics reporter configured, no metrics will be exposed/reported. 2018-01-11 14:30:25,897 INFO org.apache.flink.runtime.jobmanager.JobManager - Starting JobManager web frontend 2018-01-11 14:30:25,906 INFO org.apache.flink.runtime.webmonitor.WebMonitorUtils - Determined location of JobManager log file: D:\flink-1.4.0-bin-hadoop27-scala_2.11\flink-1.4.0\bin\..\log\flink-astron-jobmanager-CDYJY-astronPC.log 2018-01-11 14:30:25,906 INFO org.apache.flink.runtime.webmonitor.WebMonitorUtils - Determined location of JobManager stdout file: D:\flink-1.4.0-bin-hadoop27-scala_2.11\flink-1.4.0\bin\..\log\flink-astron-jobmanager-CDYJY-astronPC.out 2018-01-11 14:30:25,907 INFO org.apache.flink.runtime.webmonitor.WebRuntimeMonitor - Using directory C:\Users\astron\AppData\Local\Temp\flink-web-ad3b3ad8-6065-492a-a629-3285bdb7048b for the web interface files 2018-01-11 14:30:25,908 INFO org.apache.flink.runtime.webmonitor.WebRuntimeMonitor - Using directory C:\Users\astron\AppData\Local\Temp\flink-web-f04857c0-59e1-4be8-b594-af61a0926bf5 for web frontend JAR file uploads 2018-01-11 14:30:26,121 INFO org.apache.flink.runtime.webmonitor.WebRuntimeMonitor - Web frontend listening at localhost:8081 2018-01-11 14:30:26,122 INFO org.apache.flink.runtime.jobmanager.JobManager - Starting JobManager actor 2018-01-11 14:30:26,128 INFO org.apache.flink.runtime.blob.BlobServer - Created BLOB server storage directory C:\Users\astron\AppData\Local\Temp\blobStore-bb1ca34d-2f58-44e1-97dc-2ca033c0d4a1 2018-01-11 14:30:26,130 INFO org.apache.flink.runtime.blob.BlobServer - Started BLOB server at 0.0.0.0:62101 - max concurrent requests: 50 - max backlog: 1000 2018-01-11 14:30:26,171 INFO org.apache.flink.runtime.jobmanager.JobManager - Starting embedded TaskManager for JobManager's LOCAL execution mode 2018-01-11 14:30:26,172 INFO org.apache.flink.runtime.jobmanager.MemoryArchivist - Started memory archivist akka://flink/user/archive 2018-01-11 14:30:26,174 INFO org.apache.flink.runtime.jobmanager.JobManager - Starting JobManager at akka.tcp://flink@localhost:6123/user/jobmanager. 2018-01-11 14:30:26,180 INFO org.apache.flink.runtime.jobmanager.JobManager - JobManager akka.tcp://flink@localhost:6123/user/jobmanager was granted leadership with leader session ID Some(00000000-0000-0000-0000-000000000000). 2018-01-11 14:30:26,181 INFO org.apache.flink.runtime.io.network.netty.NettyConfig - NettyConfig [server address: localhost/127.0.0.1, server port: 0, ssl enabled: false, memory segment size (bytes): 32768, transport type: NIO, number of server threads: 1 (manual), number of client threads: 1 (manual), server connect backlog: 0 (use Netty's default), client connect timeout (sec): 120, send/receive buffer size (bytes): 0 (use Netty's default)] 2018-01-11 14:30:26,184 INFO org.apache.flink.runtime.taskexecutor.TaskManagerConfiguration - Messages have a max timeout of 10000 ms 2018-01-11 14:30:26,187 INFO org.apache.flink.runtime.taskexecutor.TaskManagerServices - Temporary file directory 'C:\Users\astron\AppData\Local\Temp': total 118 GB, usable 52 GB (44.07% usable) 2018-01-11 14:30:26,322 INFO org.apache.flink.runtime.io.network.buffer.NetworkBufferPool - Allocated 72 MB for network buffer pool (number of memory segments: 2306, bytes per segment: 32768). 2018-01-11 14:30:26,336 WARN org.apache.flink.runtime.query.QueryableStateUtils - Could not load Queryable State Client Proxy. Probable reason: flink-queryable-state-runtime is not in the classpath. Please put the corresponding jar from the opt to the lib folder. 2018-01-11 14:30:26,336 WARN org.apache.flink.runtime.query.QueryableStateUtils - Could not load Queryable State Server. Probable reason: flink-queryable-state-runtime is not in the classpath. Please put the corresponding jar from the opt to the lib folder. 2018-01-11 14:30:26,337 INFO org.apache.flink.runtime.io.network.NetworkEnvironment - Starting the network environment and its components. 2018-01-11 14:30:26,345 INFO org.apache.flink.runtime.io.network.netty.NettyClient - Successful initialization (took 4 ms). 2018-01-11 14:30:26,348 INFO org.apache.flink.runtime.io.network.netty.NettyServer - Successful initialization (took 3 ms). Listening on SocketAddress /127.0.0.1:62106. 2018-01-11 14:30:26,489 INFO org.apache.flink.runtime.taskexecutor.TaskManagerServices - Limiting managed memory to 0.7 of the currently free heap space (460 MB), memory will be allocated lazily. 2018-01-11 14:30:26,494 INFO org.apache.flink.runtime.io.disk.iomanager.IOManager - I/O manager uses directory C:\Users\astron\AppData\Local\Temp\flink-io-4371bfee-efa7-4c35-b7bf-4952bbbb49b1 for spill files. 2018-01-11 14:30:26,499 INFO org.apache.flink.runtime.filecache.FileCache - User file cache uses directory C:\Users\astron\AppData\Local\Temp\flink-dist-cache-90a4b18a-a40f-4633-a27a-6c84dacd1ceb 2018-01-11 14:30:26,510 INFO org.apache.flink.runtime.filecache.FileCache - User file cache uses directory C:\Users\astron\AppData\Local\Temp\flink-dist-cache-93b68834-c782-4766-aea7-6882e40cc0e5 2018-01-11 14:30:26,516 INFO org.apache.flink.runtime.taskmanager.TaskManager - Starting TaskManager actor at akka://flink/user/taskmanager#-1317514981. 2018-01-11 14:30:26,516 INFO org.apache.flink.runtime.taskmanager.TaskManager - TaskManager data connection information: 48b153b53a217da7885d2ca8f8a102bb @ localscen.risk.jd.com (dataPort=62106) 2018-01-11 14:30:26,517 INFO org.apache.flink.runtime.taskmanager.TaskManager - TaskManager has 1 task slot(s). 2018-01-11 14:30:26,517 INFO org.apache.flink.runtime.taskmanager.TaskManager - Memory usage stats: [HEAP: 96/736/736 MB, NON HEAP: 39/39/-1 MB (used/committed/max)] 2018-01-11 14:30:26,518 INFO org.apache.flink.runtime.clusterframework.standalone.StandaloneResourceManager - Trying to associate with JobManager leader akka.tcp://flink@localhost:6123/user/jobmanager 2018-01-11 14:30:26,520 INFO org.apache.flink.runtime.taskmanager.TaskManager - Trying to register at JobManager akka.tcp://flink@localhost:6123/user/jobmanager (attempt 1, timeout: 500 milliseconds) 2018-01-11 14:30:26,522 INFO org.apache.flink.runtime.clusterframework.standalone.StandaloneResourceManager - Resource Manager associating with leading JobManager Actor[akka://flink/user/jobmanager#-1749419493] - leader session 00000000-0000-0000-0000-000000000000 2018-01-11 14:30:26,522 INFO org.apache.flink.runtime.clusterframework.standalone.StandaloneResourceManager - TaskManager 48b153b53a217da7885d2ca8f8a102bb has started. 2018-01-11 14:30:26,524 INFO org.apache.flink.runtime.instance.InstanceManager - Registered TaskManager at localscen (akka://flink/user/taskmanager) as 5e42b32cf44ae20c6400962296261621. Current number of registered hosts is 1. Current number of alive task slots is 1. 2018-01-11 14:30:26,525 INFO org.apache.flink.runtime.taskmanager.TaskManager - Successful registration at JobManager (akka://flink/user/jobmanager), starting network stack and library cache. 2018-01-11 14:30:26,527 INFO org.apache.flink.runtime.taskmanager.TaskManager - Determined BLOB server address to be localhost/127.0.0.1:62101. Starting BLOB cache. 2018-01-11 14:30:26,529 INFO org.apache.flink.runtime.blob.PermanentBlobCache - Created BLOB cache storage directory C:\Users\astron\AppData\Local\Temp\blobStore-2a3d34f5-5506-4fcd-9c22-c2186ea19952 2018-01-11 14:30:26,531 INFO org.apache.flink.runtime.blob.TransientBlobCache - Created BLOB cache storage directory C:\Users\astron\AppData\Local\Temp\blobStore-7b11a823-bb0b-4961-a2a6-01f4cf602195