第71课：Spark SQL窗口函数解密与实战学习笔记
第71课：Spark SQL窗口函数解密与实战学习笔记
本期内容：
1 SparkSQL窗口函数解析
2 SparkSQL窗口函数实战
窗口函数是Spark内置函数中最有价值的函数，因为很多关于分组的统计往往都使用了窗口函数。
Window Aggregates (Windows)
Window Aggregates (aka Windows) operate on a group of rows (a row set) called awindow to apply aggregation on. They calculate a value for every input row for its window.
| Note |
Window-based framework is available as an experimental feature since Spark1.4.0. |
Spark SQL supports three kinds of window aggregate function: ranking functions, analyticfunctions, and aggregate functions.
A window specification defines the partitioning, ordering, and frame boundaries.
Window Aggregate Functions
A window aggregate function calculates a return value over a set of rows called windowthat are somehow related to the current row.
| Note |
Window functions are also called over functions due to how they are applied using Column’s over function. |
Although similar to aggregate functions, a window function does not group rows into a single output row and retains their separate identities. A window function can access rows that are linked to the current row.
| Tip |
See Examples section in this document. |
Spark SQL supports three kinds of window functions:
·
ranking functions
·
·
analytic functions
·
·
aggregate functions
·
| Table 1. Window functions in Spark SQL (see Introducing Window Functions in Spark SQL) |
||
|
|
SQL |
DataFrame API |
| Ranking functions |
RANK |
rank |
| DENSE_RANK |
dense_rank |
|
| PERCENT_RANK |
percent_rank |
|
| NTILE |
ntile |
|
| ROW_NUMBER |
row_number |
|
| Analytic functions |
CUME_DIST |
cume_dist |
| LAG |
lag |
|
| LEAD |
lead |
|
For aggregate functions, you can use the existing aggregate functions as window functions, e.g. sum, avg, min, max and count.
You can mark a function window by OVER clause after a function in SQL, e.g. avg(revenue) OVER (…) or over method on a function in the Dataset API, e.g. rank().over(…).
When executed, a window function computes a value for each row in a window.
| Note |
Window functions belong to Window functions group in Spark’s Scala API. |
窗口函数中最重要的是row_number。row_bumber是对分组进行排序，所谓分组排序就是说在分组的基础上再进行排序。
先看一下GroupedTopN的代码：
package SparkSQLByScala
import org.apache.spark.{SparkContext, SparkConf}
/**
* 复杂Top N案例实战
*
* @author DT大数据梦工厂
* 新浪微博：http://weibo.com/ilovepains/
*/
object TopNGroup {
def main(args: Array[String]) {
/**
* 第1步：创建Spark的配置对象SparkConf，设置Spark程序的运行时的配置信息，
* 例如说通过setMaster来设置程序要链接的Spark集群的Master的URL,如果设置
* 为local，则代表Spark程序在本地运行，特别适合于机器配置条件非常差（例如
* 只有1G的内存）的初学者 *
*/
val conf = new SparkConf() //创建SparkConf对象
conf.setAppName("Top N Basically!") //设置应用程序的名称，在程序运行的监控界面可以看到名称
// conf.setMaster("spark://Master:7077") //此时，程序在Spark集群
conf.setMaster("local")
/**
* 第2步：创建SparkContext对象
* SparkContext是Spark程序所有功能的唯一入口，无论是采用Scala、Java、Python、R等都必须有一个SparkContext
* SparkContext核心作用：初始化Spark应用程序运行所需要的核心组件，包括DAGScheduler、TaskScheduler、SchedulerBackend
* 同时还会负责Spark程序往Master注册程序等
* SparkContext是整个Spark应用程序中最为至关重要的一个对象
*/
val sc = new SparkContext(conf) //创建SparkContext对象，通过传入SparkConf实例来定制Spark运行的具体参数和配置信息
sc.setLogLevel("OFF")
/**
* 第3步：根据具体的数据来源（HDFS、HBase、Local FS、DB、S3等）通过SparkContext来创建RDD
* RDD的创建基本有三种方式：根据外部的数据来源（例如HDFS）、根据Scala集合、由其它的RDD操作
* 数据会被RDD划分成为一系列的Partitions，分配到每个Partition的数据属于一个Task的处理范畴
*/
val lines = sc.textFile("D://DT-IMF//testdata//topNGroup.txt") //读取本地文件并设置为一个Partition
val groupRDD = lines.map(line =>(line.split(" ")(0),line.split(" ")(1).toInt)).groupByKey()
val top5 = groupRDD.map(pair => (pair._1,pair._2.toList.sortWith(_>_).take(5))).sortByKey()
top5.collect().foreach(pair => {
println(pair._1 + ":")
pair._2.foreach(println)
println("*********************")
})
}
}
运行结果：
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
16/04/12 00:39:20 INFO SparkContext: Running Spark version 1.6.0
16/04/12 00:39:32 INFO SecurityManager: Changing view acls to: think
16/04/12 00:39:32 INFO SecurityManager: Changing modify acls to: think
16/04/12 00:39:32 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(think); users with modify permissions: Set(think)
16/04/12 00:39:34 INFO Utils: Successfully started service 'sparkDriver' on port 63433.
16/04/12 00:39:35 INFO Slf4jLogger: Slf4jLogger started
16/04/12 00:39:35 INFO Remoting: Starting remoting
16/04/12 00:39:36 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://[email protected]:63448]
16/04/12 00:39:36 INFO Utils: Successfully started service 'sparkDriverActorSystem' on port 63448.
16/04/12 00:39:36 INFO SparkEnv: Registering MapOutputTracker
16/04/12 00:39:36 INFO SparkEnv: Registering BlockManagerMaster
16/04/12 00:39:36 INFO DiskBlockManager: Created local directory at C:\Users\think\AppData\Local\Temp\blockmgr-c37d158d-2af8-4630-afb4-63d7093546e5
16/04/12 00:39:36 INFO MemoryStore: MemoryStore started with capacity 1773.8 MB
16/04/12 00:39:37 INFO SparkEnv: Registering OutputCommitCoordinator
16/04/12 00:39:37 INFO Utils: Successfully started service 'SparkUI' on port 4040.
16/04/12 00:39:37 INFO SparkUI: Started SparkUI at http://192.168.56.1:4040
16/04/12 00:39:38 INFO Executor: Starting executor ID driver on host localhost
16/04/12 00:39:38 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 63460.
16/04/12 00:39:38 INFO NettyBlockTransferService: Server created on 63460
16/04/12 00:39:38 INFO BlockManagerMaster: Trying to register BlockManager
16/04/12 00:39:38 INFO BlockManagerMasterEndpoint: Registering block manager localhost:63460 with 1773.8 MB RAM, BlockManagerId(driver, localhost, 63460)
16/04/12 00:39:38 INFO BlockManagerMaster: Registered BlockManager
Hadoop:
99
98
97
96
69
*********************
Spark:
195
100
99
98
91
*********************
下面使用SparkSQL的方式重新编写并执行：
package SparkSQLByScala
import org.apache.spark.SparkConf
import org.apache.spark.SparkContext
import org.apache.spark.sql.hive.HiveContext
object SparkSQLWindowFunctionOps {
def main(args: Array[String]): Unit = {
val conf = new SparkConf() //创建SparkConf对象
conf.setAppName("SparkSQLWindowFuntionOps") //设置应用第69课：Spark SQL通过Hive数据源实战程序的名称，在程序运行的监控界面可以看到名称
conf.setMaster("spark://slq1:7077") //此时，程序在Spark集群
val sc = new SparkContext(conf) //创建SparkContext对象，通过传入SparkConf实例来定制Spark运行的具体参数和配置信息
/**
* 第一：在目前企业级大数据Spark开发的时候绝大多数情况下是采用Hive作为数据仓库的；
* Spark提供了Hive的支持功能，Spark通过HiveContext可以直接操作Hive中的数据；
* 基于HiveContext我们可以使用sql/hql两种方式才编写SQL语句对Hive进行操作，包括
* 创建表、删除表、往表里导入数据以及用SQL语法构造各种SQL语句对表中的数据进行CRUD操作
* 第二：我们也可以直接通过saveAsTable的方式把DataFrame中的数据保存到Hive数据仓库中；
* 第三：可以直接通过HiveContext.table方法来直接加载Hive中的表而生成DataFrame
*/
val hiveContext = new HiveContext(sc)
hiveContext.sql("use hive") //使用名称为hive的数据库，我们接下来所有的表的操作都位于这个库中
/**
* 如果要创建的表存在的话就删除，然后创建我们要导入数据的表
*/
hiveContext.sql("DROP TABLE IF EXISTS scores")
hiveContext.sql("CREATE TABLE IF NOT EXISTS scores(name STRING,score INT) "
+"ROW FORMAT DELIMITED FIELDS TERMINATED BY ' ' LINES TERMINATED BY '\\n'")
//把要处理的出数据导入到Hive的表中
hiveContext.sql("LOAD DATA LOCAL INPATH '/home/richard/slq/spark/160330/topNGroup.txt' INTO TABLE scores")
/**
* 使用子查询的方式完成目标数据的提取，在目标数据内幕使用窗口函数row_number来进行分组排序：
* PARTITION BY :指定窗口函数分组的Key；
* ORDER BY：分组后进行排序；
*/
val result = hiveContext.sql("SELECT name,score "
+ "FROM ("
+ "SELECT "
+ "name,"
+ "score,"
+ "row_number() OVER (PARTITION BY name ORDER BY score DESC) rank"
+" FROM scores "
+ ") sub_scores "
+ "WHERE rank <=4")
result.show(); //在Driver的控制台上打印出结果内容
//把数据保存在Hive数据仓库中
hiveContext.sql("DROP TABLE IF EXISTS sortedResultScores")
result.saveAsTable("sortedResultScores")
}
}
执行结果：
[richard@slq1 160330]$ ./SparkAppsScala.sh
16/04/13 00:09:13 INFO spark.SparkContext: Running Spark version 1.6.0
16/04/13 00:09:17 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/04/13 00:09:20 INFO spark.SecurityManager: Changing view acls to: richard
16/04/13 00:09:20 INFO spark.SecurityManager: Changing modify acls to: richard
16/04/13 00:09:20 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(richard); users with modify permissions: Set(richard)
16/04/13 00:09:27 INFO util.Utils: Successfully started service 'sparkDriver' on port 49808.
16/04/13 00:09:31 INFO slf4j.Slf4jLogger: Slf4jLogger started
16/04/13 00:09:32 INFO Remoting: Starting remoting
16/04/13 00:09:35 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://[email protected]:36186]
16/04/13 00:09:35 INFO util.Utils: Successfully started service 'sparkDriverActorSystem' on port 36186.
16/04/13 00:09:35 INFO spark.SparkEnv: Registering MapOutputTracker
16/04/13 00:09:36 INFO spark.SparkEnv: Registering BlockManagerMaster
16/04/13 00:09:36 INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-65c08292-a1ea-4c14-98b3-4fb355b2dd25
16/04/13 00:09:37 INFO storage.MemoryStore: MemoryStore started with capacity 517.4 MB
16/04/13 00:09:38 INFO spark.SparkEnv: Registering OutputCommitCoordinator
16/04/13 00:09:40 INFO server.Server: jetty-8.y.z-SNAPSHOT
16/04/13 00:09:41 INFO server.AbstractConnector: Started [email protected]:4040
16/04/13 00:09:41 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.
16/04/13 00:09:41 INFO ui.SparkUI: Started SparkUI at http://192.168.1.121:4040
16/04/13 00:09:41 INFO spark.HttpFileServer: HTTP File server directory is /tmp/spark-103237c0-b116-4a08-b441-8ea4b3de3e99/httpd-dc709bf3-3487-4c46-9a7e-d2cb3ded5212
16/04/13 00:09:41 INFO spark.HttpServer: Starting HTTP Server
16/04/13 00:09:41 INFO server.Server: jetty-8.y.z-SNAPSHOT
16/04/13 00:09:41 INFO server.AbstractConnector: Started [email protected]:38484
16/04/13 00:09:41 INFO util.Utils: Successfully started service 'HTTP file server' on port 38484.
16/04/13 00:09:42 INFO spark.SparkContext: Added JAR file:/home/richard/slq/spark/160330/SparkSQLWindowFunctionOps.jar at http://192.168.1.121:38484/jars/SparkSQLWindowFunctionOps.jar with timestamp 1460477382113
16/04/13 00:09:43 INFO client.AppClient$ClientEndpoint: Connecting to master spark://slq1:7077...
16/04/13 00:09:45 INFO cluster.SparkDeploySchedulerBackend: Connected to Spark cluster with app ID app-20160413000945-0000
16/04/13 00:09:45 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 52549.
16/04/13 00:09:45 INFO netty.NettyBlockTransferService: Server created on 52549
16/04/13 00:09:45 INFO storage.BlockManagerMaster: Trying to register BlockManager
16/04/13 00:09:45 INFO storage.BlockManagerMasterEndpoint: Registering block manager 192.168.1.121:52549 with 517.4 MB RAM, BlockManagerId(driver, 192.168.1.121, 52549)
16/04/13 00:09:45 INFO storage.BlockManagerMaster: Registered BlockManager
16/04/13 00:09:46 INFO client.AppClient$ClientEndpoint: Executor added: app-20160413000945-0000/0 on worker-20160412234000-192.168.1.121-35019 (192.168.1.121:35019) with 1 cores
16/04/13 00:09:46 INFO cluster.SparkDeploySchedulerBackend: Granted executor ID app-20160413000945-0000/0 on hostPort 192.168.1.121:35019 with 1 cores, 1024.0 MB RAM
16/04/13 00:09:46 INFO client.AppClient$ClientEndpoint: Executor added: app-20160413000945-0000/1 on worker-20160412233939-192.168.1.123-37351 (192.168.1.123:37351) with 1 cores
16/04/13 00:09:46 INFO cluster.SparkDeploySchedulerBackend: Granted executor ID app-20160413000945-0000/1 on hostPort 192.168.1.123:37351 with 1 cores, 1024.0 MB RAM
16/04/13 00:09:46 INFO client.AppClient$ClientEndpoint: Executor added: app-20160413000945-0000/2 on worker-20160412233939-192.168.1.122-42079 (192.168.1.122:42079) with 1 cores
16/04/13 00:09:46 INFO cluster.SparkDeploySchedulerBackend: Granted executor ID app-20160413000945-0000/2 on hostPort 192.168.1.122:42079 with 1 cores, 1024.0 MB RAM
16/04/13 00:09:48 INFO client.AppClient$ClientEndpoint: Executor updated: app-20160413000945-0000/2 is now RUNNING
16/04/13 00:09:48 INFO client.AppClient$ClientEndpoint: Executor updated: app-20160413000945-0000/1 is now RUNNING
16/04/13 00:09:49 INFO client.AppClient$ClientEndpoint: Executor updated: app-20160413000945-0000/0 is now RUNNING
16/04/13 00:09:53 INFO cluster.SparkDeploySchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.0
16/04/13 00:10:34 INFO cluster.SparkDeploySchedulerBackend: Registered executor NettyRpcEndpointRef(null) (slq2:39600) with ID 2
16/04/13 00:10:35 INFO cluster.SparkDeploySchedulerBackend: Registered executor NettyRpcEndpointRef(null) (slq3:53352) with ID 1
16/04/13 00:10:35 INFO storage.BlockManagerMasterEndpoint: Registering block manager slq2:34090 with 517.4 MB RAM, BlockManagerId(2, slq2, 34090)
16/04/13 00:10:35 INFO storage.BlockManagerMasterEndpoint: Registering block manager slq3:43697 with 517.4 MB RAM, BlockManagerId(1, slq3, 43697)
16/04/13 00:10:41 INFO hive.HiveContext: Initializing execution hive, version 1.2.1
16/04/13 00:10:44 INFO client.ClientWrapper: Inspected Hadoop version: 2.6.0
16/04/13 00:10:44 INFO client.ClientWrapper: Loaded org.apache.hadoop.hive.shims.Hadoop23Shims for Hadoop version 2.6.0
16/04/13 00:10:51 INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
16/04/13 00:10:52 INFO metastore.ObjectStore: ObjectStore, initialize called
16/04/13 00:10:55 INFO DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
16/04/13 00:10:55 INFO DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored
16/04/13 00:10:59 WARN DataNucleus.Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies)
16/04/13 00:11:09 WARN DataNucleus.Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies)
16/04/13 00:11:23 INFO cluster.SparkDeploySchedulerBackend: Registered executor NettyRpcEndpointRef(null) (slq1:41641) with ID 0
16/04/13 00:11:25 INFO storage.BlockManagerMasterEndpoint: Registering block manager slq1:44779 with 517.4 MB RAM, BlockManagerId(0, slq1, 44779)
16/04/13 00:11:43 INFO metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
16/04/13 00:11:58 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
16/04/13 00:11:58 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
16/04/13 00:12:18 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
16/04/13 00:12:18 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
16/04/13 00:12:24 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
16/04/13 00:12:24 INFO metastore.ObjectStore: Initialized ObjectStore
16/04/13 00:12:29 WARN metastore.ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.0
16/04/13 00:12:32 WARN metastore.ObjectStore: Failed to get database default, returning NoSuchObjectException
16/04/13 00:12:36 INFO metastore.HiveMetaStore: Added admin role in metastore
16/04/13 00:12:36 INFO metastore.HiveMetaStore: Added public role in metastore
16/04/13 00:12:38 INFO metastore.HiveMetaStore: No user is added in admin role, since config is empty
16/04/13 00:12:42 INFO metastore.HiveMetaStore: 0: get_all_databases
16/04/13 00:12:43 INFO HiveMetaStore.audit: ugi=richard ip=unknown-ip-addr cmd=get_all_databases
16/04/13 00:12:43 INFO metastore.HiveMetaStore: 0: get_functions: db=default pat=*
16/04/13 00:12:43 INFO HiveMetaStore.audit: ugi=richard ip=unknown-ip-addr cmd=get_functions: db=default pat=*
16/04/13 00:12:43 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MResourceUri" is tagged as "embedded-only" so does not have its own datastore table.
16/04/13 00:12:52 INFO session.SessionState: Created local directory: /tmp/1ecbd29a-b42c-49c9-bbbb-af5a4226101f_resources
16/04/13 00:12:52 INFO session.SessionState: Created HDFS directory: /tmp/hive/richard/1ecbd29a-b42c-49c9-bbbb-af5a4226101f
16/04/13 00:12:53 INFO session.SessionState: Created local directory: /tmp/richard/1ecbd29a-b42c-49c9-bbbb-af5a4226101f
16/04/13 00:12:53 INFO session.SessionState: Created HDFS directory: /tmp/hive/richard/1ecbd29a-b42c-49c9-bbbb-af5a4226101f/_tmp_space.db
16/04/13 00:12:54 INFO hive.HiveContext: default warehouse location is /user/hive/warehouse
16/04/13 00:12:54 INFO hive.HiveContext: Initializing HiveMetastoreConnection version 1.2.1 using Spark classes.
16/04/13 00:12:55 INFO client.ClientWrapper: Inspected Hadoop version: 2.6.0
16/04/13 00:12:55 INFO client.ClientWrapper: Loaded org.apache.hadoop.hive.shims.Hadoop23Shims for Hadoop version 2.6.0
16/04/13 00:13:02 INFO hive.metastore: Trying to connect to metastore with URI thrift://slq1:9083
16/04/13 00:13:03 INFO hive.metastore: Connected to metastore.
16/04/13 00:13:08 INFO session.SessionState: Created local directory: /tmp/110134cb-6db5-4fa2-805f-d2381f07f293_resources
16/04/13 00:13:08 INFO session.SessionState: Created HDFS directory: /tmp/hive/richard/110134cb-6db5-4fa2-805f-d2381f07f293
16/04/13 00:13:08 INFO session.SessionState: Created local directory: /tmp/richard/110134cb-6db5-4fa2-805f-d2381f07f293
16/04/13 00:13:08 INFO session.SessionState: Created HDFS directory: /tmp/hive/richard/110134cb-6db5-4fa2-805f-d2381f07f293/_tmp_space.db
16/04/13 00:13:13 INFO parse.ParseDriver: Parsing command: use hive
16/04/13 00:13:24 INFO parse.ParseDriver: Parse Completed
16/04/13 00:13:34 INFO log.PerfLogger:
16/04/13 00:13:34 INFO log.PerfLogger:
16/04/13 00:13:34 INFO log.PerfLogger:
16/04/13 00:13:35 INFO log.PerfLogger:
16/04/13 00:13:35 INFO parse.ParseDriver: Parsing command: use hive
16/04/13 00:13:44 INFO parse.ParseDriver: Parse Completed
16/04/13 00:13:44 INFO log.PerfLogger:
16/04/13 00:13:44 INFO log.PerfLogger:
16/04/13 00:13:46 INFO ql.Driver: Semantic Analysis Completed
16/04/13 00:13:46 INFO log.PerfLogger:
16/04/13 00:13:46 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null)
16/04/13 00:13:46 INFO log.PerfLogger:
16/04/13 00:13:46 INFO metadata.Hive: Dumping metastore api call timing information for : compilation phase
16/04/13 00:13:46 INFO metadata.Hive: Total time spent in this metastore function was greater than 1000ms : getAllDatabases_()=1180
16/04/13 00:13:46 INFO metadata.Hive: Total time spent in this metastore function was greater than 1000ms : getFunctions_(String, String, )=2031
16/04/13 00:13:46 INFO ql.Driver: Concurrency mode is disabled, not creating a lock manager
16/04/13 00:13:46 INFO log.PerfLogger:
16/04/13 00:13:46 INFO ql.Driver: Starting command(queryId=richard_20160413001335_6afa4359-a624-4d0e-93e7-f946cc51e16f): use hive
16/04/13 00:13:47 INFO log.PerfLogger:
16/04/13 00:13:47 INFO log.PerfLogger:
16/04/13 00:13:47 INFO log.PerfLogger:
16/04/13 00:13:47 INFO ql.Driver: Starting task [Stage-0:DDL] in serial mode
16/04/13 00:13:47 INFO log.PerfLogger:
16/04/13 00:13:47 INFO log.PerfLogger:
16/04/13 00:13:47 INFO ql.Driver: OK
16/04/13 00:13:47 INFO log.PerfLogger:
16/04/13 00:13:47 INFO log.PerfLogger:
16/04/13 00:13:47 INFO log.PerfLogger:
16/04/13 00:13:49 INFO parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS scores
16/04/13 00:13:49 INFO parse.ParseDriver: Parse Completed
16/04/13 00:13:50 INFO log.PerfLogger:
16/04/13 00:13:50 INFO log.PerfLogger:
16/04/13 00:13:50 INFO log.PerfLogger:
16/04/13 00:13:50 INFO log.PerfLogger:
16/04/13 00:13:50 INFO parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS scores
16/04/13 00:13:50 INFO parse.ParseDriver: Parse Completed
16/04/13 00:13:50 INFO log.PerfLogger:
16/04/13 00:13:50 INFO log.PerfLogger:
16/04/13 00:13:51 INFO ql.Driver: Semantic Analysis Completed
16/04/13 00:13:51 INFO log.PerfLogger:
16/04/13 00:13:51 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null)
16/04/13 00:13:51 INFO log.PerfLogger:
16/04/13 00:13:51 INFO ql.Driver: Concurrency mode is disabled, not creating a lock manager
16/04/13 00:13:51 INFO log.PerfLogger:
16/04/13 00:13:51 INFO ql.Driver: Starting command(queryId=richard_20160413001350_a6ab0056-c235-481a-bbe1-7203d6b51396): DROP TABLE IF EXISTS scores
16/04/13 00:13:51 INFO log.PerfLogger:
16/04/13 00:13:51 INFO log.PerfLogger:
16/04/13 00:13:51 INFO log.PerfLogger:
16/04/13 00:13:52 INFO ql.Driver: Starting task [Stage-0:DDL] in serial mode
16/04/13 00:13:52 ERROR metadata.Hive: Table scores not found: hive.scores table not found
16/04/13 00:13:52 INFO log.PerfLogger:
16/04/13 00:13:52 INFO log.PerfLogger:
16/04/13 00:13:52 INFO ql.Driver: OK
16/04/13 00:13:52 INFO log.PerfLogger:
16/04/13 00:13:52 INFO log.PerfLogger:
16/04/13 00:13:52 INFO log.PerfLogger:
16/04/13 00:13:54 INFO parse.ParseDriver: Parsing command: CREATE TABLE IF NOT EXISTS scores(name STRING,score INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ' ' LINES TERMINATED BY '\n'
16/04/13 00:13:54 INFO parse.ParseDriver: Parse Completed
16/04/13 00:13:54 INFO log.PerfLogger:
16/04/13 00:13:54 INFO log.PerfLogger:
16/04/13 00:13:54 INFO log.PerfLogger:
16/04/13 00:13:54 INFO log.PerfLogger:
16/04/13 00:13:54 INFO parse.ParseDriver: Parsing command: CREATE TABLE IF NOT EXISTS scores(name STRING,score INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ' ' LINES TERMINATED BY '\n'
16/04/13 00:13:54 INFO parse.ParseDriver: Parse Completed
16/04/13 00:13:54 INFO log.PerfLogger:
16/04/13 00:13:54 INFO log.PerfLogger:
16/04/13 00:13:55 INFO parse.CalcitePlanner: Starting Semantic Analysis
16/04/13 00:13:57 INFO parse.CalcitePlanner: Creating table hive.scores position=27
16/04/13 00:13:58 INFO ql.Driver: Semantic Analysis Completed
16/04/13 00:13:58 INFO log.PerfLogger:
16/04/13 00:13:58 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null)
16/04/13 00:13:58 INFO log.PerfLogger:
16/04/13 00:13:58 INFO ql.Driver: Concurrency mode is disabled, not creating a lock manager
16/04/13 00:13:58 INFO log.PerfLogger:
16/04/13 00:13:58 INFO ql.Driver: Starting command(queryId=richard_20160413001354_bda65e1a-360c-4e85-9a0c-f06297a5176b): CREATE TABLE IF NOT EXISTS scores(name STRING,score INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ' ' LINES TERMINATED BY '\n'
16/04/13 00:13:59 INFO log.PerfLogger:
16/04/13 00:13:59 INFO log.PerfLogger:
16/04/13 00:13:59 INFO log.PerfLogger:
16/04/13 00:13:59 INFO ql.Driver: Starting task [Stage-0:DDL] in serial mode
16/04/13 00:14:16 INFO log.PerfLogger:
16/04/13 00:14:16 INFO metadata.Hive: Dumping metastore api call timing information for : execution phase
16/04/13 00:14:16 INFO metadata.Hive: Total time spent in this metastore function was greater than 1000ms : createTable_(Table, )=14357
16/04/13 00:14:16 INFO log.PerfLogger:
16/04/13 00:14:16 INFO ql.Driver: OK
16/04/13 00:14:16 INFO log.PerfLogger:
16/04/13 00:14:16 INFO log.PerfLogger:
16/04/13 00:14:16 INFO log.PerfLogger:
16/04/13 00:14:16 INFO parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH '/home/richard/slq/spark/160330/topNGroup.txt' INTO TABLE scores
16/04/13 00:14:16 INFO parse.ParseDriver: Parse Completed
16/04/13 00:14:16 INFO log.PerfLogger:
16/04/13 00:14:16 INFO log.PerfLogger:
16/04/13 00:14:16 INFO log.PerfLogger:
16/04/13 00:14:16 INFO log.PerfLogger:
16/04/13 00:14:16 INFO parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH '/home/richard/slq/spark/160330/topNGroup.txt' INTO TABLE scores
16/04/13 00:14:16 INFO parse.ParseDriver: Parse Completed
16/04/13 00:14:16 INFO log.PerfLogger:
16/04/13 00:14:16 INFO log.PerfLogger:
16/04/13 00:14:19 INFO ql.Driver: Semantic Analysis Completed
16/04/13 00:14:19 INFO log.PerfLogger:
16/04/13 00:14:19 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null)
16/04/13 00:14:19 INFO log.PerfLogger:
16/04/13 00:14:19 INFO metadata.Hive: Dumping metastore api call timing information for : compilation phase
16/04/13 00:14:19 INFO metadata.Hive: Total time spent in this metastore function was greater than 1000ms : getTable_(String, String, )=1996
16/04/13 00:14:19 INFO ql.Driver: Concurrency mode is disabled, not creating a lock manager
16/04/13 00:14:19 INFO log.PerfLogger:
16/04/13 00:14:19 INFO ql.Driver: Starting command(queryId=richard_20160413001416_347e381c-8656-4e78-8caf-9aa580899d45): LOAD DATA LOCAL INPATH '/home/richard/slq/spark/160330/topNGroup.txt' INTO TABLE scores
16/04/13 00:14:19 INFO log.PerfLogger:
16/04/13 00:14:19 INFO log.PerfLogger:
16/04/13 00:14:19 INFO log.PerfLogger:
16/04/13 00:14:19 INFO ql.Driver: Starting task [Stage-0:MOVE] in serial mode
16/04/13 00:14:19 INFO exec.Task: Loading data to table hive.scores from file:/home/richard/slq/spark/160330/topNGroup.txt
16/04/13 00:14:27 INFO metadata.Hive: Renaming src: file:/home/richard/slq/spark/160330/topNGroup.txt, dest: hdfs://slq1:9000/user/hive/warehouse/hive.db/scores/topNGroup.txt, Status:true
16/04/13 00:14:29 INFO log.PerfLogger:
16/04/13 00:14:29 INFO ql.Driver: Starting task [Stage-1:STATS] in serial mode
16/04/13 00:14:29 INFO exec.StatsTask: Executing stats task
16/04/13 00:14:30 INFO exec.Task: Table hive.scores stats: [numFiles=1, totalSize=192]
16/04/13 00:14:30 INFO log.PerfLogger:
16/04/13 00:14:30 INFO metadata.Hive: Dumping metastore api call timing information for : execution phase
16/04/13 00:14:30 INFO metadata.Hive: Total time spent in this metastore function was greater than 1000ms : alter_table_(String, String, Table, boolean, )=1684
16/04/13 00:14:30 INFO log.PerfLogger:
16/04/13 00:14:30 INFO ql.Driver: OK
16/04/13 00:14:30 INFO log.PerfLogger:
16/04/13 00:14:30 INFO log.PerfLogger:
16/04/13 00:14:30 INFO log.PerfLogger:
16/04/13 00:14:31 INFO parse.ParseDriver: Parsing command: SELECT name,score FROM (SELECT name,score,row_number() OVER (PARTITION BY name ORDER BY score DESC) rank FROM scores ) sub_scores WHERE rank <=4
16/04/13 00:14:31 INFO parse.ParseDriver: Parse Completed
16/04/13 00:14:39 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
16/04/13 00:14:40 INFO storage.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 520.3 KB, free 520.3 KB)
16/04/13 00:14:40 INFO storage.MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 41.9 KB, free 562.3 KB)
16/04/13 00:14:40 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on 192.168.1.121:52549 (size: 41.9 KB, free: 517.4 MB)
16/04/13 00:14:40 INFO spark.SparkContext: Created broadcast 0 from show at SparkSQLWindowFunctionOps.scala:52
16/04/13 00:14:43 INFO mapred.FileInputFormat: Total input paths to process : 1
16/04/13 00:14:44 INFO spark.SparkContext: Starting job: show at SparkSQLWindowFunctionOps.scala:52
16/04/13 00:14:44 INFO scheduler.DAGScheduler: Registering RDD 8 (show at SparkSQLWindowFunctionOps.scala:52)
16/04/13 00:14:44 INFO scheduler.DAGScheduler: Got job 0 (show at SparkSQLWindowFunctionOps.scala:52) with 1 output partitions
16/04/13 00:14:44 INFO scheduler.DAGScheduler: Final stage: ResultStage 1 (show at SparkSQLWindowFunctionOps.scala:52)
16/04/13 00:14:44 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 0)
16/04/13 00:14:45 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 0)
16/04/13 00:14:45 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 0 (MapPartitionsRDD[8] at show at SparkSQLWindowFunctionOps.scala:52), which has no missing parents
16/04/13 00:14:45 INFO storage.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 10.2 KB, free 572.5 KB)
16/04/13 00:14:45 INFO storage.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 5.4 KB, free 577.9 KB)
16/04/13 00:14:45 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on 192.168.1.121:52549 (size: 5.4 KB, free: 517.4 MB)
16/04/13 00:14:45 INFO spark.SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:1006
16/04/13 00:14:45 INFO scheduler.DAGScheduler: Submitting 2 missing tasks from ShuffleMapStage 0 (MapPartitionsRDD[8] at show at SparkSQLWindowFunctionOps.scala:52)
16/04/13 00:14:45 INFO scheduler.TaskSchedulerImpl: Adding task set 0.0 with 2 tasks
16/04/13 00:14:45 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, slq3, partition 0,NODE_LOCAL, 2227 bytes)
16/04/13 00:14:45 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, slq2, partition 1,NODE_LOCAL, 2227 bytes)
16/04/13 00:14:48 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on slq3:43697 (size: 5.4 KB, free: 517.4 MB)
16/04/13 00:14:49 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on slq2:34090 (size: 5.4 KB, free: 517.4 MB)
16/04/13 00:14:57 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on slq3:43697 (size: 41.9 KB, free: 517.4 MB)
16/04/13 00:14:58 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 in memory on slq2:34090 (size: 41.9 KB, free: 517.4 MB)
16/04/13 00:15:11 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 25520 ms on slq3 (1/2)
16/04/13 00:15:12 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 26675 ms on slq2 (2/2)
16/04/13 00:15:12 INFO scheduler.DAGScheduler: ShuffleMapStage 0 (show at SparkSQLWindowFunctionOps.scala:52) finished in 26.866 s
16/04/13 00:15:12 INFO scheduler.DAGScheduler: looking for newly runnable stages
16/04/13 00:15:12 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool
16/04/13 00:15:12 INFO scheduler.DAGScheduler: running: Set()
16/04/13 00:15:12 INFO scheduler.DAGScheduler: waiting: Set(ResultStage 1)
16/04/13 00:15:12 INFO scheduler.DAGScheduler: failed: Set()
16/04/13 00:15:12 INFO scheduler.DAGScheduler: Submitting ResultStage 1 (MapPartitionsRDD[15] at show at SparkSQLWindowFunctionOps.scala:52), which has no missing parents
16/04/13 00:15:13 INFO storage.MemoryStore: Block broadcast_2 stored as values in memory (estimated size 15.5 KB, free 593.4 KB)
16/04/13 00:15:13 INFO storage.MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 8.0 KB, free 601.4 KB)
16/04/13 00:15:13 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on 192.168.1.121:52549 (size: 8.0 KB, free: 517.4 MB)
16/04/13 00:15:13 INFO spark.SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:1006
16/04/13 00:15:13 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from ResultStage 1 (MapPartitionsRDD[15] at show at SparkSQLWindowFunctionOps.scala:52)
16/04/13 00:15:13 INFO scheduler.TaskSchedulerImpl: Adding task set 1.0 with 1 tasks
16/04/13 00:15:13 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 1.0 (TID 2, slq3, partition 0,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:13 INFO storage.BlockManagerInfo: Added broadcast_2_piece0 in memory on slq3:43697 (size: 8.0 KB, free: 517.4 MB)
16/04/13 00:15:15 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to slq3:53352
16/04/13 00:15:15 INFO spark.MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 165 bytes
16/04/13 00:15:17 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 1.0 (TID 2) in 4229 ms on slq3 (1/1)
16/04/13 00:15:17 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool
16/04/13 00:15:17 INFO scheduler.DAGScheduler: ResultStage 1 (show at SparkSQLWindowFunctionOps.scala:52) finished in 4.240 s
16/04/13 00:15:17 INFO scheduler.DAGScheduler: Job 0 finished: show at SparkSQLWindowFunctionOps.scala:52, took 33.072636 s
16/04/13 00:15:17 INFO spark.SparkContext: Starting job: show at SparkSQLWindowFunctionOps.scala:52
16/04/13 00:15:17 INFO scheduler.DAGScheduler: Got job 1 (show at SparkSQLWindowFunctionOps.scala:52) with 199 output partitions
16/04/13 00:15:17 INFO scheduler.DAGScheduler: Final stage: ResultStage 3 (show at SparkSQLWindowFunctionOps.scala:52)
16/04/13 00:15:17 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 2)
16/04/13 00:15:17 INFO scheduler.DAGScheduler: Missing parents: List()
16/04/13 00:15:17 INFO scheduler.DAGScheduler: Submitting ResultStage 3 (MapPartitionsRDD[15] at show at SparkSQLWindowFunctionOps.scala:52), which has no missing parents
16/04/13 00:15:18 INFO storage.MemoryStore: Block broadcast_3 stored as values in memory (estimated size 15.5 KB, free 616.9 KB)
16/04/13 00:15:18 INFO storage.MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 8.0 KB, free 624.9 KB)
16/04/13 00:15:18 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on 192.168.1.121:52549 (size: 8.0 KB, free: 517.3 MB)
16/04/13 00:15:18 INFO spark.SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:1006
16/04/13 00:15:18 INFO scheduler.DAGScheduler: Submitting 199 missing tasks from ResultStage 3 (MapPartitionsRDD[15] at show at SparkSQLWindowFunctionOps.scala:52)
16/04/13 00:15:18 INFO scheduler.TaskSchedulerImpl: Adding task set 3.0 with 199 tasks
16/04/13 00:15:18 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 3.0 (TID 3, slq2, partition 1,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:18 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 3.0 (TID 4, slq3, partition 2,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:18 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on slq3:43697 (size: 8.0 KB, free: 517.3 MB)
16/04/13 00:15:18 INFO storage.BlockManagerInfo: Added broadcast_3_piece0 in memory on slq2:34090 (size: 8.0 KB, free: 517.4 MB)
16/04/13 00:15:18 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 3.0 (TID 5, slq3, partition 3,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:18 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 3.0 (TID 4) in 360 ms on slq3 (1/199)
16/04/13 00:15:19 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 3.0 (TID 6, slq3, partition 4,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:19 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 3.0 (TID 5) in 256 ms on slq3 (2/199)
16/04/13 00:15:19 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 3.0 (TID 7, slq3, partition 5,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:19 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 3.0 (TID 6) in 238 ms on slq3 (3/199)
16/04/13 00:15:19 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 3.0 (TID 8, slq3, partition 6,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:19 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 3.0 (TID 7) in 167 ms on slq3 (4/199)
16/04/13 00:15:19 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 3.0 (TID 9, slq3, partition 7,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:19 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 3.0 (TID 8) in 172 ms on slq3 (5/199)
16/04/13 00:15:19 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 3.0 (TID 10, slq3, partition 8,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:19 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 3.0 (TID 9) in 203 ms on slq3 (6/199)
16/04/13 00:15:20 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 3.0 (TID 11, slq3, partition 9,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:20 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 3.0 (TID 10) in 170 ms on slq3 (7/199)
16/04/13 00:15:20 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 3.0 (TID 12, slq3, partition 10,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:20 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 3.0 (TID 11) in 208 ms on slq3 (8/199)
16/04/13 00:15:20 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to slq2:39600
16/04/13 00:15:20 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 3.0 (TID 13, slq3, partition 11,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:20 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 3.0 (TID 12) in 181 ms on slq3 (9/199)
16/04/13 00:15:21 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 3.0 (TID 14, slq3, partition 12,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:21 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 3.0 (TID 13) in 1337 ms on slq3 (10/199)
16/04/13 00:15:22 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 3.0 (TID 15, slq3, partition 13,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:22 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 3.0 (TID 14) in 273 ms on slq3 (11/199)
16/04/13 00:15:23 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 3.0 (TID 16, slq2, partition 14,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:23 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 3.0 (TID 3) in 4843 ms on slq2 (12/199)
16/04/13 00:15:23 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 3.0 (TID 17, slq3, partition 15,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:23 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 3.0 (TID 15) in 1718 ms on slq3 (13/199)
16/04/13 00:15:23 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 3.0 (TID 18, slq2, partition 16,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:23 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 3.0 (TID 16) in 394 ms on slq2 (14/199)
16/04/13 00:15:23 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 3.0 (TID 19, slq3, partition 17,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:23 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 3.0 (TID 17) in 255 ms on slq3 (15/199)
16/04/13 00:15:24 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 3.0 (TID 20, slq2, partition 18,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:24 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 3.0 (TID 18) in 425 ms on slq2 (16/199)
16/04/13 00:15:24 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 3.0 (TID 21, slq3, partition 19,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:24 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 3.0 (TID 19) in 323 ms on slq3 (17/199)
16/04/13 00:15:24 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 3.0 (TID 22, slq2, partition 20,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:24 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 3.0 (TID 20) in 328 ms on slq2 (18/199)
16/04/13 00:15:24 INFO scheduler.TaskSetManager: Starting task 20.0 in stage 3.0 (TID 23, slq3, partition 21,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:24 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 3.0 (TID 21) in 350 ms on slq3 (19/199)
16/04/13 00:15:24 INFO scheduler.TaskSetManager: Starting task 21.0 in stage 3.0 (TID 24, slq2, partition 22,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:24 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 3.0 (TID 22) in 279 ms on slq2 (20/199)
16/04/13 00:15:24 INFO scheduler.TaskSetManager: Starting task 22.0 in stage 3.0 (TID 25, slq3, partition 23,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:24 INFO scheduler.TaskSetManager: Finished task 20.0 in stage 3.0 (TID 23) in 296 ms on slq3 (21/199)
16/04/13 00:15:25 INFO scheduler.TaskSetManager: Starting task 23.0 in stage 3.0 (TID 26, slq3, partition 24,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:25 INFO scheduler.TaskSetManager: Finished task 22.0 in stage 3.0 (TID 25) in 245 ms on slq3 (22/199)
16/04/13 00:15:25 INFO scheduler.TaskSetManager: Starting task 24.0 in stage 3.0 (TID 27, slq2, partition 25,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:25 INFO scheduler.TaskSetManager: Finished task 21.0 in stage 3.0 (TID 24) in 367 ms on slq2 (23/199)
16/04/13 00:15:25 INFO scheduler.TaskSetManager: Starting task 25.0 in stage 3.0 (TID 28, slq3, partition 26,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:25 INFO scheduler.TaskSetManager: Finished task 23.0 in stage 3.0 (TID 26) in 249 ms on slq3 (24/199)
16/04/13 00:15:25 INFO scheduler.TaskSetManager: Starting task 26.0 in stage 3.0 (TID 29, slq2, partition 27,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:25 INFO scheduler.TaskSetManager: Finished task 24.0 in stage 3.0 (TID 27) in 302 ms on slq2 (25/199)
16/04/13 00:15:25 INFO scheduler.TaskSetManager: Starting task 27.0 in stage 3.0 (TID 30, slq3, partition 28,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:25 INFO scheduler.TaskSetManager: Finished task 25.0 in stage 3.0 (TID 28) in 232 ms on slq3 (26/199)
16/04/13 00:15:25 INFO scheduler.TaskSetManager: Starting task 28.0 in stage 3.0 (TID 31, slq3, partition 29,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:25 INFO scheduler.TaskSetManager: Finished task 27.0 in stage 3.0 (TID 30) in 213 ms on slq3 (27/199)
16/04/13 00:15:25 INFO scheduler.TaskSetManager: Starting task 29.0 in stage 3.0 (TID 32, slq2, partition 30,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:25 INFO scheduler.TaskSetManager: Finished task 26.0 in stage 3.0 (TID 29) in 448 ms on slq2 (28/199)
16/04/13 00:15:26 INFO scheduler.TaskSetManager: Starting task 30.0 in stage 3.0 (TID 33, slq3, partition 31,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:26 INFO scheduler.TaskSetManager: Finished task 28.0 in stage 3.0 (TID 31) in 236 ms on slq3 (29/199)
16/04/13 00:15:26 INFO scheduler.TaskSetManager: Starting task 31.0 in stage 3.0 (TID 34, slq2, partition 32,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:26 INFO scheduler.TaskSetManager: Finished task 29.0 in stage 3.0 (TID 32) in 302 ms on slq2 (30/199)
16/04/13 00:15:26 INFO scheduler.TaskSetManager: Starting task 32.0 in stage 3.0 (TID 35, slq3, partition 33,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:26 INFO scheduler.TaskSetManager: Finished task 30.0 in stage 3.0 (TID 33) in 249 ms on slq3 (31/199)
16/04/13 00:15:26 INFO scheduler.TaskSetManager: Starting task 33.0 in stage 3.0 (TID 36, slq2, partition 34,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:26 INFO scheduler.TaskSetManager: Starting task 34.0 in stage 3.0 (TID 37, slq3, partition 35,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:26 INFO scheduler.TaskSetManager: Finished task 31.0 in stage 3.0 (TID 34) in 316 ms on slq2 (32/199)
16/04/13 00:15:26 INFO scheduler.TaskSetManager: Finished task 32.0 in stage 3.0 (TID 35) in 238 ms on slq3 (33/199)
16/04/13 00:15:26 INFO scheduler.TaskSetManager: Starting task 35.0 in stage 3.0 (TID 38, slq2, partition 36,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:26 INFO scheduler.TaskSetManager: Finished task 33.0 in stage 3.0 (TID 36) in 253 ms on slq2 (34/199)
16/04/13 00:15:26 INFO scheduler.TaskSetManager: Starting task 36.0 in stage 3.0 (TID 39, slq3, partition 37,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:26 INFO scheduler.TaskSetManager: Finished task 34.0 in stage 3.0 (TID 37) in 284 ms on slq3 (35/199)
16/04/13 00:15:27 INFO scheduler.TaskSetManager: Starting task 37.0 in stage 3.0 (TID 40, slq2, partition 38,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:27 INFO scheduler.TaskSetManager: Starting task 38.0 in stage 3.0 (TID 41, slq3, partition 39,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:27 INFO scheduler.TaskSetManager: Finished task 35.0 in stage 3.0 (TID 38) in 301 ms on slq2 (36/199)
16/04/13 00:15:27 INFO scheduler.TaskSetManager: Finished task 36.0 in stage 3.0 (TID 39) in 268 ms on slq3 (37/199)
16/04/13 00:15:27 INFO scheduler.TaskSetManager: Starting task 39.0 in stage 3.0 (TID 42, slq3, partition 40,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:27 INFO scheduler.TaskSetManager: Finished task 38.0 in stage 3.0 (TID 41) in 274 ms on slq3 (38/199)
16/04/13 00:15:27 INFO scheduler.TaskSetManager: Starting task 40.0 in stage 3.0 (TID 43, slq2, partition 41,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:27 INFO scheduler.TaskSetManager: Finished task 37.0 in stage 3.0 (TID 40) in 324 ms on slq2 (39/199)
16/04/13 00:15:27 INFO scheduler.TaskSetManager: Starting task 41.0 in stage 3.0 (TID 44, slq3, partition 42,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:27 INFO scheduler.TaskSetManager: Finished task 39.0 in stage 3.0 (TID 42) in 304 ms on slq3 (40/199)
16/04/13 00:15:27 INFO scheduler.TaskSetManager: Starting task 42.0 in stage 3.0 (TID 45, slq2, partition 43,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:27 INFO scheduler.TaskSetManager: Finished task 40.0 in stage 3.0 (TID 43) in 286 ms on slq2 (41/199)
16/04/13 00:15:27 INFO scheduler.TaskSetManager: Starting task 43.0 in stage 3.0 (TID 46, slq3, partition 44,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:27 INFO scheduler.TaskSetManager: Finished task 41.0 in stage 3.0 (TID 44) in 158 ms on slq3 (42/199)
16/04/13 00:15:27 INFO scheduler.TaskSetManager: Starting task 44.0 in stage 3.0 (TID 47, slq2, partition 45,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:27 INFO scheduler.TaskSetManager: Finished task 42.0 in stage 3.0 (TID 45) in 292 ms on slq2 (43/199)
16/04/13 00:15:27 INFO scheduler.TaskSetManager: Starting task 45.0 in stage 3.0 (TID 48, slq3, partition 46,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:27 INFO scheduler.TaskSetManager: Finished task 43.0 in stage 3.0 (TID 46) in 196 ms on slq3 (44/199)
16/04/13 00:15:28 INFO scheduler.TaskSetManager: Starting task 46.0 in stage 3.0 (TID 49, slq3, partition 47,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:28 INFO scheduler.TaskSetManager: Finished task 45.0 in stage 3.0 (TID 48) in 189 ms on slq3 (45/199)
16/04/13 00:15:28 INFO scheduler.TaskSetManager: Starting task 47.0 in stage 3.0 (TID 50, slq2, partition 48,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:28 INFO scheduler.TaskSetManager: Finished task 44.0 in stage 3.0 (TID 47) in 248 ms on slq2 (46/199)
16/04/13 00:15:28 INFO scheduler.TaskSetManager: Starting task 48.0 in stage 3.0 (TID 51, slq3, partition 49,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:28 INFO scheduler.TaskSetManager: Finished task 46.0 in stage 3.0 (TID 49) in 230 ms on slq3 (47/199)
16/04/13 00:15:28 INFO scheduler.TaskSetManager: Starting task 49.0 in stage 3.0 (TID 52, slq2, partition 50,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:28 INFO scheduler.TaskSetManager: Finished task 47.0 in stage 3.0 (TID 50) in 295 ms on slq2 (48/199)
16/04/13 00:15:28 INFO scheduler.TaskSetManager: Starting task 50.0 in stage 3.0 (TID 53, slq3, partition 51,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:28 INFO scheduler.TaskSetManager: Finished task 48.0 in stage 3.0 (TID 51) in 199 ms on slq3 (49/199)
16/04/13 00:15:28 INFO scheduler.TaskSetManager: Starting task 51.0 in stage 3.0 (TID 54, slq2, partition 52,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:28 INFO scheduler.TaskSetManager: Finished task 49.0 in stage 3.0 (TID 52) in 180 ms on slq2 (50/199)
16/04/13 00:15:28 INFO scheduler.TaskSetManager: Starting task 52.0 in stage 3.0 (TID 55, slq3, partition 53,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:28 INFO scheduler.TaskSetManager: Finished task 50.0 in stage 3.0 (TID 53) in 248 ms on slq3 (51/199)
16/04/13 00:15:28 INFO scheduler.TaskSetManager: Starting task 53.0 in stage 3.0 (TID 56, slq2, partition 54,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:28 INFO scheduler.TaskSetManager: Finished task 51.0 in stage 3.0 (TID 54) in 238 ms on slq2 (52/199)
16/04/13 00:15:29 INFO scheduler.TaskSetManager: Starting task 54.0 in stage 3.0 (TID 57, slq2, partition 55,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:29 INFO scheduler.TaskSetManager: Finished task 53.0 in stage 3.0 (TID 56) in 213 ms on slq2 (53/199)
16/04/13 00:15:29 INFO scheduler.TaskSetManager: Starting task 55.0 in stage 3.0 (TID 58, slq3, partition 56,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:29 INFO scheduler.TaskSetManager: Finished task 52.0 in stage 3.0 (TID 55) in 397 ms on slq3 (54/199)
16/04/13 00:15:29 INFO scheduler.TaskSetManager: Starting task 56.0 in stage 3.0 (TID 59, slq2, partition 57,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:29 INFO scheduler.TaskSetManager: Finished task 54.0 in stage 3.0 (TID 57) in 323 ms on slq2 (55/199)
16/04/13 00:15:29 INFO scheduler.TaskSetManager: Starting task 57.0 in stage 3.0 (TID 60, slq3, partition 58,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:29 INFO scheduler.TaskSetManager: Finished task 55.0 in stage 3.0 (TID 58) in 287 ms on slq3 (56/199)
16/04/13 00:15:29 INFO scheduler.TaskSetManager: Starting task 58.0 in stage 3.0 (TID 61, slq2, partition 59,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:29 INFO scheduler.TaskSetManager: Finished task 56.0 in stage 3.0 (TID 59) in 240 ms on slq2 (57/199)
16/04/13 00:15:29 INFO scheduler.TaskSetManager: Starting task 59.0 in stage 3.0 (TID 62, slq3, partition 60,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:29 INFO scheduler.TaskSetManager: Finished task 57.0 in stage 3.0 (TID 60) in 252 ms on slq3 (58/199)
16/04/13 00:15:29 INFO scheduler.TaskSetManager: Starting task 60.0 in stage 3.0 (TID 63, slq2, partition 61,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:29 INFO scheduler.TaskSetManager: Finished task 58.0 in stage 3.0 (TID 61) in 203 ms on slq2 (59/199)
16/04/13 00:15:29 INFO scheduler.TaskSetManager: Starting task 61.0 in stage 3.0 (TID 64, slq3, partition 62,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:29 INFO scheduler.TaskSetManager: Finished task 59.0 in stage 3.0 (TID 62) in 259 ms on slq3 (60/199)
16/04/13 00:15:30 INFO scheduler.TaskSetManager: Starting task 62.0 in stage 3.0 (TID 65, slq2, partition 63,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:30 INFO scheduler.TaskSetManager: Finished task 60.0 in stage 3.0 (TID 63) in 222 ms on slq2 (61/199)
16/04/13 00:15:30 INFO scheduler.TaskSetManager: Starting task 63.0 in stage 3.0 (TID 66, slq3, partition 64,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:30 INFO scheduler.TaskSetManager: Finished task 61.0 in stage 3.0 (TID 64) in 193 ms on slq3 (62/199)
16/04/13 00:15:30 INFO scheduler.TaskSetManager: Starting task 64.0 in stage 3.0 (TID 67, slq2, partition 65,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:30 INFO scheduler.TaskSetManager: Starting task 65.0 in stage 3.0 (TID 68, slq3, partition 66,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:30 INFO scheduler.TaskSetManager: Finished task 62.0 in stage 3.0 (TID 65) in 269 ms on slq2 (63/199)
16/04/13 00:15:30 INFO scheduler.TaskSetManager: Finished task 63.0 in stage 3.0 (TID 66) in 175 ms on slq3 (64/199)
16/04/13 00:15:30 INFO scheduler.TaskSetManager: Starting task 66.0 in stage 3.0 (TID 69, slq3, partition 67,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:30 INFO scheduler.TaskSetManager: Starting task 67.0 in stage 3.0 (TID 70, slq2, partition 68,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:30 INFO scheduler.TaskSetManager: Finished task 65.0 in stage 3.0 (TID 68) in 168 ms on slq3 (65/199)
16/04/13 00:15:30 INFO scheduler.TaskSetManager: Finished task 64.0 in stage 3.0 (TID 67) in 172 ms on slq2 (66/199)
16/04/13 00:15:30 INFO scheduler.TaskSetManager: Starting task 68.0 in stage 3.0 (TID 71, slq3, partition 69,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:30 INFO scheduler.TaskSetManager: Finished task 66.0 in stage 3.0 (TID 69) in 269 ms on slq3 (67/199)
16/04/13 00:15:30 INFO scheduler.TaskSetManager: Starting task 69.0 in stage 3.0 (TID 72, slq2, partition 70,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:30 INFO scheduler.TaskSetManager: Finished task 67.0 in stage 3.0 (TID 70) in 417 ms on slq2 (68/199)
16/04/13 00:15:30 INFO scheduler.TaskSetManager: Starting task 70.0 in stage 3.0 (TID 73, slq3, partition 71,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:30 INFO scheduler.TaskSetManager: Finished task 68.0 in stage 3.0 (TID 71) in 178 ms on slq3 (69/199)
16/04/13 00:15:31 INFO scheduler.TaskSetManager: Starting task 71.0 in stage 3.0 (TID 74, slq2, partition 72,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:31 INFO scheduler.TaskSetManager: Starting task 72.0 in stage 3.0 (TID 75, slq3, partition 73,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:31 INFO scheduler.TaskSetManager: Finished task 69.0 in stage 3.0 (TID 72) in 188 ms on slq2 (70/199)
16/04/13 00:15:31 INFO scheduler.TaskSetManager: Finished task 70.0 in stage 3.0 (TID 73) in 164 ms on slq3 (71/199)
16/04/13 00:15:31 INFO scheduler.TaskSetManager: Starting task 73.0 in stage 3.0 (TID 76, slq2, partition 74,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:31 INFO scheduler.TaskSetManager: Finished task 71.0 in stage 3.0 (TID 74) in 221 ms on slq2 (72/199)
16/04/13 00:15:31 INFO scheduler.TaskSetManager: Starting task 74.0 in stage 3.0 (TID 77, slq2, partition 75,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:31 INFO scheduler.TaskSetManager: Finished task 73.0 in stage 3.0 (TID 76) in 155 ms on slq2 (73/199)
16/04/13 00:15:31 INFO scheduler.TaskSetManager: Starting task 75.0 in stage 3.0 (TID 78, slq3, partition 76,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:31 INFO scheduler.TaskSetManager: Finished task 72.0 in stage 3.0 (TID 75) in 515 ms on slq3 (74/199)
16/04/13 00:15:31 INFO scheduler.TaskSetManager: Starting task 76.0 in stage 3.0 (TID 79, slq2, partition 77,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:31 INFO scheduler.TaskSetManager: Finished task 74.0 in stage 3.0 (TID 77) in 189 ms on slq2 (75/199)
16/04/13 00:15:31 INFO scheduler.TaskSetManager: Starting task 77.0 in stage 3.0 (TID 80, slq3, partition 78,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:31 INFO scheduler.TaskSetManager: Finished task 75.0 in stage 3.0 (TID 78) in 164 ms on slq3 (76/199)
16/04/13 00:15:31 INFO scheduler.TaskSetManager: Starting task 78.0 in stage 3.0 (TID 81, slq2, partition 79,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:31 INFO scheduler.TaskSetManager: Finished task 76.0 in stage 3.0 (TID 79) in 195 ms on slq2 (77/199)
16/04/13 00:15:31 INFO scheduler.TaskSetManager: Starting task 79.0 in stage 3.0 (TID 82, slq3, partition 80,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:31 INFO scheduler.TaskSetManager: Finished task 77.0 in stage 3.0 (TID 80) in 164 ms on slq3 (78/199)
16/04/13 00:15:32 INFO scheduler.TaskSetManager: Starting task 80.0 in stage 3.0 (TID 83, slq2, partition 81,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:32 INFO scheduler.TaskSetManager: Finished task 78.0 in stage 3.0 (TID 81) in 223 ms on slq2 (79/199)
16/04/13 00:15:32 INFO scheduler.TaskSetManager: Starting task 81.0 in stage 3.0 (TID 84, slq3, partition 82,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:32 INFO scheduler.TaskSetManager: Finished task 79.0 in stage 3.0 (TID 82) in 298 ms on slq3 (80/199)
16/04/13 00:15:32 INFO scheduler.TaskSetManager: Starting task 82.0 in stage 3.0 (TID 85, slq2, partition 83,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:32 INFO scheduler.TaskSetManager: Finished task 80.0 in stage 3.0 (TID 83) in 257 ms on slq2 (81/199)
16/04/13 00:15:32 INFO scheduler.TaskSetManager: Starting task 83.0 in stage 3.0 (TID 86, slq3, partition 84,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:32 INFO scheduler.TaskSetManager: Finished task 81.0 in stage 3.0 (TID 84) in 226 ms on slq3 (82/199)
16/04/13 00:15:32 INFO scheduler.TaskSetManager: Starting task 84.0 in stage 3.0 (TID 87, slq2, partition 85,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:32 INFO scheduler.TaskSetManager: Finished task 82.0 in stage 3.0 (TID 85) in 208 ms on slq2 (83/199)
16/04/13 00:15:32 INFO scheduler.TaskSetManager: Starting task 85.0 in stage 3.0 (TID 88, slq3, partition 86,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:32 INFO scheduler.TaskSetManager: Finished task 83.0 in stage 3.0 (TID 86) in 177 ms on slq3 (84/199)
16/04/13 00:15:32 INFO scheduler.TaskSetManager: Starting task 86.0 in stage 3.0 (TID 89, slq2, partition 87,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:32 INFO scheduler.TaskSetManager: Finished task 84.0 in stage 3.0 (TID 87) in 219 ms on slq2 (85/199)
16/04/13 00:15:32 INFO scheduler.TaskSetManager: Starting task 87.0 in stage 3.0 (TID 90, slq3, partition 88,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:32 INFO scheduler.TaskSetManager: Finished task 85.0 in stage 3.0 (TID 88) in 271 ms on slq3 (86/199)
16/04/13 00:15:33 INFO scheduler.TaskSetManager: Starting task 88.0 in stage 3.0 (TID 91, slq3, partition 89,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:33 INFO scheduler.TaskSetManager: Finished task 87.0 in stage 3.0 (TID 90) in 223 ms on slq3 (87/199)
16/04/13 00:15:33 INFO scheduler.TaskSetManager: Starting task 89.0 in stage 3.0 (TID 92, slq3, partition 90,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:33 INFO scheduler.TaskSetManager: Finished task 88.0 in stage 3.0 (TID 91) in 260 ms on slq3 (88/199)
16/04/13 00:15:33 INFO scheduler.TaskSetManager: Starting task 90.0 in stage 3.0 (TID 93, slq3, partition 91,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:33 INFO scheduler.TaskSetManager: Finished task 89.0 in stage 3.0 (TID 92) in 258 ms on slq3 (89/199)
16/04/13 00:15:33 INFO scheduler.TaskSetManager: Starting task 91.0 in stage 3.0 (TID 94, slq3, partition 92,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:33 INFO scheduler.TaskSetManager: Finished task 90.0 in stage 3.0 (TID 93) in 318 ms on slq3 (90/199)
16/04/13 00:15:34 INFO scheduler.TaskSetManager: Starting task 92.0 in stage 3.0 (TID 95, slq3, partition 93,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:34 INFO scheduler.TaskSetManager: Finished task 91.0 in stage 3.0 (TID 94) in 259 ms on slq3 (91/199)
16/04/13 00:15:34 INFO scheduler.TaskSetManager: Starting task 93.0 in stage 3.0 (TID 96, slq3, partition 94,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:34 INFO scheduler.TaskSetManager: Finished task 92.0 in stage 3.0 (TID 95) in 278 ms on slq3 (92/199)
16/04/13 00:15:34 INFO scheduler.TaskSetManager: Starting task 94.0 in stage 3.0 (TID 97, slq2, partition 95,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:34 INFO scheduler.TaskSetManager: Finished task 86.0 in stage 3.0 (TID 89) in 1834 ms on slq2 (93/199)
16/04/13 00:15:34 INFO scheduler.TaskSetManager: Starting task 95.0 in stage 3.0 (TID 98, slq3, partition 96,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:34 INFO scheduler.TaskSetManager: Finished task 93.0 in stage 3.0 (TID 96) in 160 ms on slq3 (94/199)
16/04/13 00:15:34 INFO scheduler.TaskSetManager: Starting task 96.0 in stage 3.0 (TID 99, slq2, partition 97,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:34 INFO scheduler.TaskSetManager: Finished task 94.0 in stage 3.0 (TID 97) in 192 ms on slq2 (95/199)
16/04/13 00:15:34 INFO scheduler.TaskSetManager: Starting task 97.0 in stage 3.0 (TID 100, slq3, partition 98,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:34 INFO scheduler.TaskSetManager: Finished task 95.0 in stage 3.0 (TID 98) in 218 ms on slq3 (96/199)
16/04/13 00:15:34 INFO scheduler.TaskSetManager: Starting task 98.0 in stage 3.0 (TID 101, slq2, partition 99,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:34 INFO scheduler.TaskSetManager: Finished task 96.0 in stage 3.0 (TID 99) in 227 ms on slq2 (97/199)
16/04/13 00:15:34 INFO scheduler.TaskSetManager: Starting task 99.0 in stage 3.0 (TID 102, slq3, partition 100,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:34 INFO scheduler.TaskSetManager: Finished task 97.0 in stage 3.0 (TID 100) in 165 ms on slq3 (98/199)
16/04/13 00:15:35 INFO scheduler.TaskSetManager: Starting task 100.0 in stage 3.0 (TID 103, slq2, partition 101,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:35 INFO scheduler.TaskSetManager: Finished task 98.0 in stage 3.0 (TID 101) in 187 ms on slq2 (99/199)
16/04/13 00:15:35 INFO scheduler.TaskSetManager: Starting task 101.0 in stage 3.0 (TID 104, slq3, partition 102,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:35 INFO scheduler.TaskSetManager: Finished task 99.0 in stage 3.0 (TID 102) in 197 ms on slq3 (100/199)
16/04/13 00:15:35 INFO scheduler.TaskSetManager: Starting task 102.0 in stage 3.0 (TID 105, slq3, partition 103,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:35 INFO scheduler.TaskSetManager: Finished task 101.0 in stage 3.0 (TID 104) in 183 ms on slq3 (101/199)
16/04/13 00:15:35 INFO scheduler.TaskSetManager: Starting task 103.0 in stage 3.0 (TID 106, slq2, partition 104,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:35 INFO scheduler.TaskSetManager: Finished task 100.0 in stage 3.0 (TID 103) in 233 ms on slq2 (102/199)
16/04/13 00:15:35 INFO scheduler.TaskSetManager: Starting task 104.0 in stage 3.0 (TID 107, slq3, partition 105,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:35 INFO scheduler.TaskSetManager: Finished task 102.0 in stage 3.0 (TID 105) in 219 ms on slq3 (103/199)
16/04/13 00:15:35 INFO scheduler.TaskSetManager: Starting task 105.0 in stage 3.0 (TID 108, slq2, partition 106,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:35 INFO scheduler.TaskSetManager: Finished task 103.0 in stage 3.0 (TID 106) in 330 ms on slq2 (104/199)
16/04/13 00:15:35 INFO scheduler.TaskSetManager: Starting task 106.0 in stage 3.0 (TID 109, slq3, partition 107,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:35 INFO scheduler.TaskSetManager: Finished task 104.0 in stage 3.0 (TID 107) in 310 ms on slq3 (105/199)
16/04/13 00:15:35 INFO scheduler.TaskSetManager: Starting task 107.0 in stage 3.0 (TID 110, slq2, partition 108,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:35 INFO scheduler.TaskSetManager: Finished task 105.0 in stage 3.0 (TID 108) in 223 ms on slq2 (106/199)
16/04/13 00:15:35 INFO scheduler.TaskSetManager: Starting task 108.0 in stage 3.0 (TID 111, slq3, partition 109,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:35 INFO scheduler.TaskSetManager: Finished task 106.0 in stage 3.0 (TID 109) in 170 ms on slq3 (107/199)
16/04/13 00:15:36 INFO scheduler.TaskSetManager: Starting task 109.0 in stage 3.0 (TID 112, slq2, partition 110,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:36 INFO scheduler.TaskSetManager: Finished task 107.0 in stage 3.0 (TID 110) in 210 ms on slq2 (108/199)
16/04/13 00:15:37 INFO scheduler.TaskSetManager: Starting task 110.0 in stage 3.0 (TID 113, slq3, partition 111,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:37 INFO scheduler.TaskSetManager: Finished task 108.0 in stage 3.0 (TID 111) in 1166 ms on slq3 (109/199)
16/04/13 00:15:37 INFO scheduler.TaskSetManager: Starting task 111.0 in stage 3.0 (TID 114, slq2, partition 112,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:37 INFO scheduler.TaskSetManager: Finished task 109.0 in stage 3.0 (TID 112) in 1105 ms on slq2 (110/199)
16/04/13 00:15:37 INFO scheduler.TaskSetManager: Starting task 112.0 in stage 3.0 (TID 115, slq3, partition 113,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:37 INFO scheduler.TaskSetManager: Finished task 110.0 in stage 3.0 (TID 113) in 307 ms on slq3 (111/199)
16/04/13 00:15:37 INFO scheduler.TaskSetManager: Starting task 113.0 in stage 3.0 (TID 116, slq2, partition 114,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:37 INFO scheduler.TaskSetManager: Finished task 111.0 in stage 3.0 (TID 114) in 484 ms on slq2 (112/199)
16/04/13 00:15:37 INFO scheduler.TaskSetManager: Starting task 114.0 in stage 3.0 (TID 117, slq3, partition 115,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:37 INFO scheduler.TaskSetManager: Finished task 112.0 in stage 3.0 (TID 115) in 317 ms on slq3 (113/199)
16/04/13 00:15:37 INFO scheduler.TaskSetManager: Starting task 115.0 in stage 3.0 (TID 118, slq2, partition 116,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:37 INFO scheduler.TaskSetManager: Finished task 113.0 in stage 3.0 (TID 116) in 154 ms on slq2 (114/199)
16/04/13 00:15:37 INFO scheduler.TaskSetManager: Starting task 116.0 in stage 3.0 (TID 119, slq2, partition 117,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:37 INFO scheduler.TaskSetManager: Finished task 115.0 in stage 3.0 (TID 118) in 169 ms on slq2 (115/199)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Starting task 117.0 in stage 3.0 (TID 120, slq3, partition 118,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Finished task 114.0 in stage 3.0 (TID 117) in 262 ms on slq3 (116/199)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Starting task 118.0 in stage 3.0 (TID 121, slq2, partition 119,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Finished task 116.0 in stage 3.0 (TID 119) in 163 ms on slq2 (117/199)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Starting task 119.0 in stage 3.0 (TID 122, slq3, partition 120,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Finished task 117.0 in stage 3.0 (TID 120) in 168 ms on slq3 (118/199)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Starting task 120.0 in stage 3.0 (TID 123, slq2, partition 121,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Finished task 118.0 in stage 3.0 (TID 121) in 139 ms on slq2 (119/199)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Starting task 121.0 in stage 3.0 (TID 124, slq3, partition 122,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Finished task 119.0 in stage 3.0 (TID 122) in 196 ms on slq3 (120/199)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Starting task 122.0 in stage 3.0 (TID 125, slq2, partition 123,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Finished task 120.0 in stage 3.0 (TID 123) in 202 ms on slq2 (121/199)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Starting task 123.0 in stage 3.0 (TID 126, slq3, partition 124,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Finished task 121.0 in stage 3.0 (TID 124) in 186 ms on slq3 (122/199)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Starting task 124.0 in stage 3.0 (TID 127, slq2, partition 125,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Finished task 122.0 in stage 3.0 (TID 125) in 185 ms on slq2 (123/199)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Starting task 125.0 in stage 3.0 (TID 128, slq3, partition 126,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Finished task 123.0 in stage 3.0 (TID 126) in 234 ms on slq3 (124/199)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Starting task 126.0 in stage 3.0 (TID 129, slq2, partition 127,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Finished task 124.0 in stage 3.0 (TID 127) in 230 ms on slq2 (125/199)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Starting task 127.0 in stage 3.0 (TID 130, slq3, partition 128,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:38 INFO scheduler.TaskSetManager: Finished task 125.0 in stage 3.0 (TID 128) in 173 ms on slq3 (126/199)
16/04/13 00:15:39 INFO scheduler.TaskSetManager: Starting task 128.0 in stage 3.0 (TID 131, slq2, partition 129,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:39 INFO scheduler.TaskSetManager: Finished task 126.0 in stage 3.0 (TID 129) in 204 ms on slq2 (127/199)
16/04/13 00:15:39 INFO scheduler.TaskSetManager: Starting task 129.0 in stage 3.0 (TID 132, slq3, partition 130,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:39 INFO scheduler.TaskSetManager: Finished task 127.0 in stage 3.0 (TID 130) in 205 ms on slq3 (128/199)
16/04/13 00:15:39 INFO scheduler.TaskSetManager: Starting task 130.0 in stage 3.0 (TID 133, slq2, partition 131,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:39 INFO scheduler.TaskSetManager: Finished task 128.0 in stage 3.0 (TID 131) in 177 ms on slq2 (129/199)
16/04/13 00:15:39 INFO scheduler.TaskSetManager: Starting task 131.0 in stage 3.0 (TID 134, slq3, partition 132,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:39 INFO scheduler.TaskSetManager: Finished task 129.0 in stage 3.0 (TID 132) in 231 ms on slq3 (130/199)
16/04/13 00:15:39 INFO scheduler.TaskSetManager: Starting task 132.0 in stage 3.0 (TID 135, slq2, partition 133,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:39 INFO scheduler.TaskSetManager: Finished task 130.0 in stage 3.0 (TID 133) in 206 ms on slq2 (131/199)
16/04/13 00:15:39 INFO scheduler.TaskSetManager: Starting task 133.0 in stage 3.0 (TID 136, slq3, partition 134,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:39 INFO scheduler.TaskSetManager: Finished task 131.0 in stage 3.0 (TID 134) in 243 ms on slq3 (132/199)
16/04/13 00:15:39 INFO scheduler.TaskSetManager: Starting task 134.0 in stage 3.0 (TID 137, slq2, partition 135,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:39 INFO scheduler.TaskSetManager: Finished task 132.0 in stage 3.0 (TID 135) in 203 ms on slq2 (133/199)
16/04/13 00:15:39 INFO scheduler.TaskSetManager: Starting task 135.0 in stage 3.0 (TID 138, slq2, partition 136,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:39 INFO scheduler.TaskSetManager: Finished task 134.0 in stage 3.0 (TID 137) in 195 ms on slq2 (134/199)
16/04/13 00:15:39 INFO scheduler.TaskSetManager: Starting task 136.0 in stage 3.0 (TID 139, slq3, partition 137,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:39 INFO scheduler.TaskSetManager: Finished task 133.0 in stage 3.0 (TID 136) in 256 ms on slq3 (135/199)
16/04/13 00:15:40 INFO scheduler.TaskSetManager: Starting task 137.0 in stage 3.0 (TID 140, slq2, partition 138,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:40 INFO scheduler.TaskSetManager: Finished task 135.0 in stage 3.0 (TID 138) in 209 ms on slq2 (136/199)
16/04/13 00:15:40 INFO scheduler.TaskSetManager: Starting task 138.0 in stage 3.0 (TID 141, slq3, partition 139,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:40 INFO scheduler.TaskSetManager: Finished task 136.0 in stage 3.0 (TID 139) in 204 ms on slq3 (137/199)
16/04/13 00:15:40 INFO scheduler.TaskSetManager: Starting task 139.0 in stage 3.0 (TID 142, slq2, partition 140,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:40 INFO scheduler.TaskSetManager: Finished task 137.0 in stage 3.0 (TID 140) in 188 ms on slq2 (138/199)
16/04/13 00:15:40 INFO scheduler.TaskSetManager: Starting task 140.0 in stage 3.0 (TID 143, slq3, partition 141,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:40 INFO scheduler.TaskSetManager: Finished task 138.0 in stage 3.0 (TID 141) in 190 ms on slq3 (139/199)
16/04/13 00:15:40 INFO scheduler.TaskSetManager: Starting task 141.0 in stage 3.0 (TID 144, slq3, partition 142,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:40 INFO scheduler.TaskSetManager: Finished task 140.0 in stage 3.0 (TID 143) in 185 ms on slq3 (140/199)
16/04/13 00:15:40 INFO scheduler.TaskSetManager: Starting task 142.0 in stage 3.0 (TID 145, slq2, partition 143,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:40 INFO scheduler.TaskSetManager: Finished task 139.0 in stage 3.0 (TID 142) in 249 ms on slq2 (141/199)
16/04/13 00:15:40 INFO scheduler.TaskSetManager: Starting task 143.0 in stage 3.0 (TID 146, slq3, partition 144,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:40 INFO scheduler.TaskSetManager: Starting task 144.0 in stage 3.0 (TID 147, slq2, partition 145,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:40 INFO scheduler.TaskSetManager: Finished task 141.0 in stage 3.0 (TID 144) in 358 ms on slq3 (142/199)
16/04/13 00:15:40 INFO scheduler.TaskSetManager: Finished task 142.0 in stage 3.0 (TID 145) in 338 ms on slq2 (143/199)
16/04/13 00:15:40 INFO scheduler.TaskSetManager: Starting task 145.0 in stage 3.0 (TID 148, slq2, partition 146,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:40 INFO scheduler.TaskSetManager: Finished task 144.0 in stage 3.0 (TID 147) in 189 ms on slq2 (144/199)
16/04/13 00:15:41 INFO scheduler.TaskSetManager: Starting task 146.0 in stage 3.0 (TID 149, slq3, partition 147,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:41 INFO scheduler.TaskSetManager: Finished task 143.0 in stage 3.0 (TID 146) in 225 ms on slq3 (145/199)
16/04/13 00:15:41 INFO scheduler.TaskSetManager: Starting task 147.0 in stage 3.0 (TID 150, slq2, partition 148,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:41 INFO scheduler.TaskSetManager: Finished task 145.0 in stage 3.0 (TID 148) in 223 ms on slq2 (146/199)
16/04/13 00:15:41 INFO scheduler.TaskSetManager: Starting task 148.0 in stage 3.0 (TID 151, slq3, partition 149,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:41 INFO scheduler.TaskSetManager: Finished task 146.0 in stage 3.0 (TID 149) in 227 ms on slq3 (147/199)
16/04/13 00:15:42 INFO scheduler.TaskSetManager: Starting task 149.0 in stage 3.0 (TID 152, slq2, partition 150,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:42 INFO scheduler.TaskSetManager: Finished task 147.0 in stage 3.0 (TID 150) in 1611 ms on slq2 (148/199)
16/04/13 00:15:42 INFO scheduler.TaskSetManager: Starting task 150.0 in stage 3.0 (TID 153, slq3, partition 151,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:42 INFO scheduler.TaskSetManager: Finished task 148.0 in stage 3.0 (TID 151) in 1657 ms on slq3 (149/199)
16/04/13 00:15:42 INFO scheduler.TaskSetManager: Starting task 151.0 in stage 3.0 (TID 154, slq2, partition 152,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:42 INFO scheduler.TaskSetManager: Finished task 149.0 in stage 3.0 (TID 152) in 167 ms on slq2 (150/199)
16/04/13 00:15:43 INFO scheduler.TaskSetManager: Starting task 152.0 in stage 3.0 (TID 155, slq3, partition 153,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:43 INFO scheduler.TaskSetManager: Finished task 150.0 in stage 3.0 (TID 153) in 225 ms on slq3 (151/199)
16/04/13 00:15:43 INFO scheduler.TaskSetManager: Starting task 153.0 in stage 3.0 (TID 156, slq2, partition 154,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:43 INFO scheduler.TaskSetManager: Finished task 151.0 in stage 3.0 (TID 154) in 236 ms on slq2 (152/199)
16/04/13 00:15:43 INFO scheduler.TaskSetManager: Starting task 154.0 in stage 3.0 (TID 157, slq2, partition 155,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:43 INFO scheduler.TaskSetManager: Finished task 153.0 in stage 3.0 (TID 156) in 137 ms on slq2 (153/199)
16/04/13 00:15:43 INFO scheduler.TaskSetManager: Starting task 155.0 in stage 3.0 (TID 158, slq3, partition 156,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:43 INFO scheduler.TaskSetManager: Finished task 152.0 in stage 3.0 (TID 155) in 288 ms on slq3 (154/199)
16/04/13 00:15:43 INFO scheduler.TaskSetManager: Starting task 156.0 in stage 3.0 (TID 159, slq2, partition 157,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:43 INFO scheduler.TaskSetManager: Finished task 154.0 in stage 3.0 (TID 157) in 187 ms on slq2 (155/199)
16/04/13 00:15:43 INFO scheduler.TaskSetManager: Starting task 157.0 in stage 3.0 (TID 160, slq2, partition 158,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:43 INFO scheduler.TaskSetManager: Finished task 156.0 in stage 3.0 (TID 159) in 173 ms on slq2 (156/199)
16/04/13 00:15:43 INFO scheduler.TaskSetManager: Starting task 158.0 in stage 3.0 (TID 161, slq3, partition 159,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:43 INFO scheduler.TaskSetManager: Finished task 155.0 in stage 3.0 (TID 158) in 333 ms on slq3 (157/199)
16/04/13 00:15:43 INFO scheduler.TaskSetManager: Starting task 159.0 in stage 3.0 (TID 162, slq2, partition 160,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:43 INFO scheduler.TaskSetManager: Finished task 157.0 in stage 3.0 (TID 160) in 255 ms on slq2 (158/199)
16/04/13 00:15:43 INFO scheduler.TaskSetManager: Starting task 160.0 in stage 3.0 (TID 163, slq3, partition 161,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:43 INFO scheduler.TaskSetManager: Finished task 158.0 in stage 3.0 (TID 161) in 248 ms on slq3 (159/199)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Starting task 161.0 in stage 3.0 (TID 164, slq2, partition 162,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Finished task 159.0 in stage 3.0 (TID 162) in 124 ms on slq2 (160/199)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Starting task 162.0 in stage 3.0 (TID 165, slq3, partition 163,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Finished task 160.0 in stage 3.0 (TID 163) in 182 ms on slq3 (161/199)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Starting task 163.0 in stage 3.0 (TID 166, slq2, partition 164,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Finished task 161.0 in stage 3.0 (TID 164) in 191 ms on slq2 (162/199)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Starting task 164.0 in stage 3.0 (TID 167, slq3, partition 165,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Finished task 162.0 in stage 3.0 (TID 165) in 190 ms on slq3 (163/199)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Starting task 165.0 in stage 3.0 (TID 168, slq2, partition 166,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Finished task 163.0 in stage 3.0 (TID 166) in 199 ms on slq2 (164/199)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Starting task 166.0 in stage 3.0 (TID 169, slq3, partition 167,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Finished task 164.0 in stage 3.0 (TID 167) in 177 ms on slq3 (165/199)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Starting task 167.0 in stage 3.0 (TID 170, slq2, partition 168,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Finished task 165.0 in stage 3.0 (TID 168) in 204 ms on slq2 (166/199)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Starting task 168.0 in stage 3.0 (TID 171, slq3, partition 169,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Finished task 166.0 in stage 3.0 (TID 169) in 147 ms on slq3 (167/199)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Starting task 169.0 in stage 3.0 (TID 172, slq2, partition 170,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Finished task 167.0 in stage 3.0 (TID 170) in 153 ms on slq2 (168/199)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Starting task 170.0 in stage 3.0 (TID 173, slq3, partition 171,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Finished task 168.0 in stage 3.0 (TID 171) in 172 ms on slq3 (169/199)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Starting task 171.0 in stage 3.0 (TID 174, slq3, partition 172,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Finished task 170.0 in stage 3.0 (TID 173) in 154 ms on slq3 (170/199)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Starting task 172.0 in stage 3.0 (TID 175, slq2, partition 173,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:44 INFO scheduler.TaskSetManager: Finished task 169.0 in stage 3.0 (TID 172) in 217 ms on slq2 (171/199)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Starting task 173.0 in stage 3.0 (TID 176, slq3, partition 174,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Finished task 171.0 in stage 3.0 (TID 174) in 148 ms on slq3 (172/199)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Starting task 174.0 in stage 3.0 (TID 177, slq2, partition 175,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Finished task 172.0 in stage 3.0 (TID 175) in 171 ms on slq2 (173/199)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Starting task 175.0 in stage 3.0 (TID 178, slq3, partition 176,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Finished task 173.0 in stage 3.0 (TID 176) in 114 ms on slq3 (174/199)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Starting task 176.0 in stage 3.0 (TID 179, slq2, partition 177,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Finished task 174.0 in stage 3.0 (TID 177) in 180 ms on slq2 (175/199)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Starting task 177.0 in stage 3.0 (TID 180, slq3, partition 178,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Finished task 175.0 in stage 3.0 (TID 178) in 173 ms on slq3 (176/199)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Starting task 178.0 in stage 3.0 (TID 181, slq2, partition 179,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Finished task 176.0 in stage 3.0 (TID 179) in 157 ms on slq2 (177/199)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Starting task 179.0 in stage 3.0 (TID 182, slq3, partition 180,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Finished task 177.0 in stage 3.0 (TID 180) in 176 ms on slq3 (178/199)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Starting task 180.0 in stage 3.0 (TID 183, slq2, partition 181,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Finished task 178.0 in stage 3.0 (TID 181) in 178 ms on slq2 (179/199)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Starting task 181.0 in stage 3.0 (TID 184, slq3, partition 182,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Finished task 179.0 in stage 3.0 (TID 182) in 200 ms on slq3 (180/199)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Starting task 182.0 in stage 3.0 (TID 185, slq2, partition 183,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Finished task 180.0 in stage 3.0 (TID 183) in 200 ms on slq2 (181/199)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Starting task 183.0 in stage 3.0 (TID 186, slq3, partition 184,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:45 INFO scheduler.TaskSetManager: Finished task 181.0 in stage 3.0 (TID 184) in 156 ms on slq3 (182/199)
16/04/13 00:15:46 INFO scheduler.TaskSetManager: Starting task 184.0 in stage 3.0 (TID 187, slq2, partition 185,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:46 INFO scheduler.TaskSetManager: Finished task 182.0 in stage 3.0 (TID 185) in 168 ms on slq2 (183/199)
16/04/13 00:15:46 INFO scheduler.TaskSetManager: Starting task 185.0 in stage 3.0 (TID 188, slq2, partition 186,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:46 INFO scheduler.TaskSetManager: Finished task 184.0 in stage 3.0 (TID 187) in 308 ms on slq2 (184/199)
16/04/13 00:15:46 INFO scheduler.TaskSetManager: Starting task 186.0 in stage 3.0 (TID 189, slq3, partition 187,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:46 INFO scheduler.TaskSetManager: Finished task 183.0 in stage 3.0 (TID 186) in 689 ms on slq3 (185/199)
16/04/13 00:15:46 INFO scheduler.TaskSetManager: Starting task 187.0 in stage 3.0 (TID 190, slq2, partition 188,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:46 INFO scheduler.TaskSetManager: Finished task 185.0 in stage 3.0 (TID 188) in 302 ms on slq2 (186/199)
16/04/13 00:15:47 INFO scheduler.TaskSetManager: Starting task 188.0 in stage 3.0 (TID 191, slq2, partition 189,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:47 INFO scheduler.TaskSetManager: Finished task 187.0 in stage 3.0 (TID 190) in 1335 ms on slq2 (187/199)
16/04/13 00:15:47 INFO scheduler.TaskSetManager: Starting task 189.0 in stage 3.0 (TID 192, slq3, partition 190,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:47 INFO scheduler.TaskSetManager: Finished task 186.0 in stage 3.0 (TID 189) in 1421 ms on slq3 (188/199)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Starting task 190.0 in stage 3.0 (TID 193, slq2, partition 191,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Finished task 188.0 in stage 3.0 (TID 191) in 191 ms on slq2 (189/199)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Starting task 191.0 in stage 3.0 (TID 194, slq3, partition 192,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Finished task 189.0 in stage 3.0 (TID 192) in 189 ms on slq3 (190/199)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Starting task 192.0 in stage 3.0 (TID 195, slq2, partition 193,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Finished task 190.0 in stage 3.0 (TID 193) in 120 ms on slq2 (191/199)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Starting task 193.0 in stage 3.0 (TID 196, slq3, partition 194,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Finished task 191.0 in stage 3.0 (TID 194) in 131 ms on slq3 (192/199)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Starting task 194.0 in stage 3.0 (TID 197, slq2, partition 195,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Finished task 192.0 in stage 3.0 (TID 195) in 178 ms on slq2 (193/199)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Starting task 195.0 in stage 3.0 (TID 198, slq3, partition 196,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Finished task 193.0 in stage 3.0 (TID 196) in 156 ms on slq3 (194/199)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Starting task 196.0 in stage 3.0 (TID 199, slq2, partition 197,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Finished task 194.0 in stage 3.0 (TID 197) in 142 ms on slq2 (195/199)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Starting task 197.0 in stage 3.0 (TID 200, slq3, partition 198,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Finished task 195.0 in stage 3.0 (TID 198) in 228 ms on slq3 (196/199)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Starting task 198.0 in stage 3.0 (TID 201, slq2, partition 199,NODE_LOCAL, 2070 bytes)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Finished task 196.0 in stage 3.0 (TID 199) in 173 ms on slq2 (197/199)
16/04/13 00:15:48 INFO scheduler.TaskSetManager: Finished task 197.0 in stage 3.0 (TID 200) in 140 ms on slq3 (198/199)
16/04/13 00:15:49 INFO scheduler.TaskSetManager: Finished task 198.0 in stage 3.0 (TID 201) in 350 ms on slq2 (199/199)
16/04/13 00:15:49 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 3.0, whose tasks have all completed, from pool
16/04/13 00:15:49 INFO scheduler.DAGScheduler: ResultStage 3 (show at SparkSQLWindowFunctionOps.scala:52) finished in 30.506 s
16/04/13 00:15:49 INFO scheduler.DAGScheduler: Job 1 finished: show at SparkSQLWindowFunctionOps.scala:52, took 31.292156 s
+------+-----+
| name|score|
+------+-----+
| Spark| 195|
| Spark| 100|
| Spark| 99|
| Spark| 98|
|Hadoop| 99|
|Hadoop| 98|
|Hadoop| 97|
|Hadoop| 96|
+------+-----+
16/04/13 00:15:49 INFO parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS sortedResultScores
16/04/13 00:15:49 INFO parse.ParseDriver: Parse Completed
16/04/13 00:15:49 INFO log.PerfLogger:
16/04/13 00:15:49 INFO log.PerfLogger:
16/04/13 00:15:49 INFO log.PerfLogger:
16/04/13 00:15:49 INFO log.PerfLogger:
16/04/13 00:15:49 INFO parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS sortedResultScores
16/04/13 00:15:49 INFO parse.ParseDriver: Parse Completed
16/04/13 00:15:49 INFO log.PerfLogger:
16/04/13 00:15:49 INFO log.PerfLogger:
16/04/13 00:15:49 INFO ql.Driver: Semantic Analysis Completed
16/04/13 00:15:49 INFO log.PerfLogger:
16/04/13 00:15:49 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null)
16/04/13 00:15:49 INFO log.PerfLogger:
16/04/13 00:15:49 INFO ql.Driver: Concurrency mode is disabled, not creating a lock manager
16/04/13 00:15:49 INFO log.PerfLogger:
16/04/13 00:15:49 INFO ql.Driver: Starting command(queryId=richard_20160413001549_f93344a5-ba69-4a81-9035-9672eb2d02af): DROP TABLE IF EXISTS sortedResultScores
16/04/13 00:15:49 INFO log.PerfLogger:
16/04/13 00:15:49 INFO log.PerfLogger:
16/04/13 00:15:49 INFO log.PerfLogger:
16/04/13 00:15:49 INFO ql.Driver: Starting task [Stage-0:DDL] in serial mode
16/04/13 00:15:49 ERROR metadata.Hive: Table sortedResultScores not found: hive.sortedResultScores table not found
16/04/13 00:15:49 INFO log.PerfLogger:
16/04/13 00:15:49 INFO log.PerfLogger:
16/04/13 00:15:49 INFO ql.Driver: OK
16/04/13 00:15:49 INFO log.PerfLogger:
16/04/13 00:15:49 INFO log.PerfLogger:
16/04/13 00:15:49 INFO log.PerfLogger:
16/04/13 00:15:52 INFO storage.MemoryStore: Block broadcast_4 stored as values in memory (estimated size 520.3 KB, free 1145.2 KB)
16/04/13 00:15:53 INFO storage.MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 41.9 KB, free 1187.2 KB)
16/04/13 00:15:53 INFO storage.BlockManagerInfo: Added broadcast_4_piece0 in memory on 192.168.1.121:52549 (size: 41.9 KB, free: 517.3 MB)
16/04/13 00:15:53 INFO spark.SparkContext: Created broadcast 4 from saveAsTable at SparkSQLWindowFunctionOps.scala:56
16/04/13 00:15:53 INFO mapred.FileInputFormat: Total input paths to process : 1
16/04/13 00:15:54 INFO Configuration.deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id
16/04/13 00:15:54 INFO Configuration.deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
16/04/13 00:15:54 INFO Configuration.deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
16/04/13 00:15:54 INFO Configuration.deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap
16/04/13 00:15:54 INFO Configuration.deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
16/04/13 00:15:54 INFO parquet.ParquetRelation: Using default output committer for Parquet: org.apache.parquet.hadoop.ParquetOutputCommitter
16/04/13 00:15:54 INFO datasources.DefaultWriterContainer: Using user defined output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
16/04/13 00:15:55 INFO spark.SparkContext: Starting job: saveAsTable at SparkSQLWindowFunctionOps.scala:56
16/04/13 00:15:55 INFO scheduler.DAGScheduler: Registering RDD 21 (saveAsTable at SparkSQLWindowFunctionOps.scala:56)
16/04/13 00:15:55 INFO scheduler.DAGScheduler: Got job 2 (saveAsTable at SparkSQLWindowFunctionOps.scala:56) with 200 output partitions
16/04/13 00:15:55 INFO scheduler.DAGScheduler: Final stage: ResultStage 5 (saveAsTable at SparkSQLWindowFunctionOps.scala:56)
16/04/13 00:15:55 INFO scheduler.DAGScheduler: Parents of final stage: List(ShuffleMapStage 4)
16/04/13 00:15:55 INFO scheduler.DAGScheduler: Missing parents: List(ShuffleMapStage 4)
16/04/13 00:15:55 INFO scheduler.DAGScheduler: Submitting ShuffleMapStage 4 (MapPartitionsRDD[21] at saveAsTable at SparkSQLWindowFunctionOps.scala:56), which has no missing parents
16/04/13 00:15:55 INFO storage.MemoryStore: Block broadcast_5 stored as values in memory (estimated size 10.2 KB, free 1197.4 KB)
16/04/13 00:15:55 INFO storage.MemoryStore: Block broadcast_5_piece0 stored as bytes in memory (estimated size 5.4 KB, free 1202.8 KB)
16/04/13 00:15:55 INFO storage.BlockManagerInfo: Added broadcast_5_piece0 in memory on 192.168.1.121:52549 (size: 5.4 KB, free: 517.3 MB)
16/04/13 00:15:55 INFO spark.SparkContext: Created broadcast 5 from broadcast at DAGScheduler.scala:1006
16/04/13 00:15:55 INFO scheduler.DAGScheduler: Submitting 2 missing tasks from ShuffleMapStage 4 (MapPartitionsRDD[21] at saveAsTable at SparkSQLWindowFunctionOps.scala:56)
16/04/13 00:15:55 INFO scheduler.TaskSchedulerImpl: Adding task set 4.0 with 2 tasks
16/04/13 00:15:55 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 4.0 (TID 202, slq1, partition 0,NODE_LOCAL, 2227 bytes)
16/04/13 00:15:55 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 4.0 (TID 203, slq3, partition 1,NODE_LOCAL, 2227 bytes)
16/04/13 00:15:55 INFO storage.BlockManagerInfo: Added broadcast_5_piece0 in memory on slq3:43697 (size: 5.4 KB, free: 517.3 MB)
16/04/13 00:15:55 INFO storage.BlockManagerInfo: Added broadcast_4_piece0 in memory on slq3:43697 (size: 41.9 KB, free: 517.3 MB)
16/04/13 00:16:00 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 4.0 (TID 203) in 4897 ms on slq3 (1/2)
16/04/13 00:16:03 INFO storage.BlockManagerInfo: Added broadcast_5_piece0 in memory on slq1:44779 (size: 5.4 KB, free: 517.4 MB)
16/04/13 00:16:17 INFO storage.BlockManagerInfo: Added broadcast_4_piece0 in memory on slq1:44779 (size: 41.9 KB, free: 517.4 MB)
16/04/13 00:16:45 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 4.0 (TID 202) in 49967 ms on slq1 (2/2)
16/04/13 00:16:45 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 4.0, whose tasks have all completed, from pool
16/04/13 00:16:45 INFO scheduler.DAGScheduler: ShuffleMapStage 4 (saveAsTable at SparkSQLWindowFunctionOps.scala:56) finished in 49.971 s
16/04/13 00:16:45 INFO scheduler.DAGScheduler: looking for newly runnable stages
16/04/13 00:16:45 INFO scheduler.DAGScheduler: running: Set()
16/04/13 00:16:45 INFO scheduler.DAGScheduler: waiting: Set(ResultStage 5)
16/04/13 00:16:45 INFO scheduler.DAGScheduler: failed: Set()
16/04/13 00:16:45 INFO scheduler.DAGScheduler: Submitting ResultStage 5 (MapPartitionsRDD[26] at saveAsTable at SparkSQLWindowFunctionOps.scala:56), which has no missing parents
16/04/13 00:16:46 INFO storage.MemoryStore: Block broadcast_6 stored as values in memory (estimated size 77.5 KB, free 1280.3 KB)
16/04/13 00:16:46 INFO storage.MemoryStore: Block broadcast_6_piece0 stored as bytes in memory (estimated size 29.9 KB, free 1310.2 KB)
16/04/13 00:16:46 INFO storage.BlockManagerInfo: Added broadcast_6_piece0 in memory on 192.168.1.121:52549 (size: 29.9 KB, free: 517.3 MB)
16/04/13 00:16:46 INFO spark.SparkContext: Created broadcast 6 from broadcast at DAGScheduler.scala:1006
16/04/13 00:16:46 INFO scheduler.DAGScheduler: Submitting 200 missing tasks from ResultStage 5 (MapPartitionsRDD[26] at saveAsTable at SparkSQLWindowFunctionOps.scala:56)
16/04/13 00:16:46 INFO scheduler.TaskSchedulerImpl: Adding task set 5.0 with 200 tasks
16/04/13 00:16:46 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 5.0 (TID 204, slq1, partition 0,NODE_LOCAL, 2070 bytes)
16/04/13 00:16:46 INFO scheduler.TaskSetManager: Starting task 1.0 in stage 5.0 (TID 205, slq3, partition 1,NODE_LOCAL, 2070 bytes)
16/04/13 00:16:46 INFO storage.BlockManagerInfo: Added broadcast_6_piece0 in memory on slq3:43697 (size: 29.9 KB, free: 517.3 MB)
16/04/13 00:16:47 INFO storage.BlockManagerInfo: Added broadcast_6_piece0 in memory on slq1:44779 (size: 29.9 KB, free: 517.3 MB)
16/04/13 00:16:47 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 1 to slq3:53352
16/04/13 00:16:47 INFO spark.MapOutputTrackerMaster: Size of output statuses for shuffle 1 is 165 bytes
16/04/13 00:16:50 INFO scheduler.TaskSetManager: Starting task 2.0 in stage 5.0 (TID 206, slq2, partition 2,ANY, 2070 bytes)
16/04/13 00:16:50 INFO storage.BlockManagerInfo: Added broadcast_6_piece0 in memory on slq2:34090 (size: 29.9 KB, free: 517.3 MB)
16/04/13 00:16:51 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 1 to slq1:41641
16/04/13 00:16:52 INFO spark.MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 1 to slq2:39600
16/04/13 00:17:02 INFO scheduler.TaskSetManager: Starting task 3.0 in stage 5.0 (TID 207, slq3, partition 3,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:03 INFO scheduler.TaskSetManager: Finished task 1.0 in stage 5.0 (TID 205) in 16281 ms on slq3 (1/200)
16/04/13 00:17:04 INFO scheduler.TaskSetManager: Starting task 4.0 in stage 5.0 (TID 208, slq3, partition 4,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:04 INFO scheduler.TaskSetManager: Finished task 3.0 in stage 5.0 (TID 207) in 1355 ms on slq3 (2/200)
16/04/13 00:17:05 INFO scheduler.TaskSetManager: Starting task 5.0 in stage 5.0 (TID 209, slq3, partition 5,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:05 INFO scheduler.TaskSetManager: Finished task 4.0 in stage 5.0 (TID 208) in 1347 ms on slq3 (3/200)
16/04/13 00:17:06 INFO scheduler.TaskSetManager: Starting task 6.0 in stage 5.0 (TID 210, slq3, partition 6,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:06 INFO scheduler.TaskSetManager: Finished task 5.0 in stage 5.0 (TID 209) in 1224 ms on slq3 (4/200)
16/04/13 00:17:06 INFO scheduler.TaskSetManager: Finished task 2.0 in stage 5.0 (TID 206) in 16861 ms on slq2 (5/200)
16/04/13 00:17:07 INFO scheduler.TaskSetManager: Starting task 7.0 in stage 5.0 (TID 211, slq3, partition 7,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:07 INFO scheduler.TaskSetManager: Finished task 6.0 in stage 5.0 (TID 210) in 943 ms on slq3 (6/200)
16/04/13 00:17:08 INFO scheduler.TaskSetManager: Starting task 8.0 in stage 5.0 (TID 212, slq3, partition 8,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:08 INFO scheduler.TaskSetManager: Finished task 7.0 in stage 5.0 (TID 211) in 922 ms on slq3 (7/200)
16/04/13 00:17:09 INFO scheduler.TaskSetManager: Starting task 9.0 in stage 5.0 (TID 213, slq3, partition 9,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:09 INFO scheduler.TaskSetManager: Finished task 8.0 in stage 5.0 (TID 212) in 926 ms on slq3 (8/200)
16/04/13 00:17:10 INFO scheduler.TaskSetManager: Starting task 10.0 in stage 5.0 (TID 214, slq3, partition 10,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:10 INFO scheduler.TaskSetManager: Finished task 9.0 in stage 5.0 (TID 213) in 794 ms on slq3 (9/200)
16/04/13 00:17:11 INFO scheduler.TaskSetManager: Starting task 11.0 in stage 5.0 (TID 215, slq3, partition 11,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:11 INFO scheduler.TaskSetManager: Finished task 10.0 in stage 5.0 (TID 214) in 970 ms on slq3 (10/200)
16/04/13 00:17:12 INFO scheduler.TaskSetManager: Starting task 12.0 in stage 5.0 (TID 216, slq3, partition 12,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:12 INFO scheduler.TaskSetManager: Finished task 11.0 in stage 5.0 (TID 215) in 743 ms on slq3 (11/200)
16/04/13 00:17:12 INFO scheduler.TaskSetManager: Starting task 13.0 in stage 5.0 (TID 217, slq3, partition 13,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:12 INFO scheduler.TaskSetManager: Finished task 12.0 in stage 5.0 (TID 216) in 793 ms on slq3 (12/200)
16/04/13 00:17:16 INFO scheduler.TaskSetManager: Starting task 14.0 in stage 5.0 (TID 218, slq2, partition 14,ANY, 2070 bytes)
16/04/13 00:17:16 INFO scheduler.TaskSetManager: Starting task 15.0 in stage 5.0 (TID 219, slq3, partition 15,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:16 INFO scheduler.TaskSetManager: Finished task 13.0 in stage 5.0 (TID 217) in 3133 ms on slq3 (13/200)
16/04/13 00:17:16 INFO scheduler.TaskSetManager: Finished task 14.0 in stage 5.0 (TID 218) in 958 ms on slq2 (14/200)
16/04/13 00:17:17 INFO scheduler.TaskSetManager: Starting task 16.0 in stage 5.0 (TID 220, slq3, partition 16,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:17 INFO scheduler.TaskSetManager: Finished task 15.0 in stage 5.0 (TID 219) in 1905 ms on slq3 (15/200)
16/04/13 00:17:19 INFO scheduler.TaskSetManager: Starting task 17.0 in stage 5.0 (TID 221, slq3, partition 17,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:19 INFO scheduler.TaskSetManager: Finished task 16.0 in stage 5.0 (TID 220) in 1286 ms on slq3 (16/200)
16/04/13 00:17:20 INFO scheduler.TaskSetManager: Starting task 18.0 in stage 5.0 (TID 222, slq3, partition 18,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:20 INFO scheduler.TaskSetManager: Finished task 17.0 in stage 5.0 (TID 221) in 979 ms on slq3 (17/200)
16/04/13 00:17:21 INFO scheduler.TaskSetManager: Starting task 19.0 in stage 5.0 (TID 223, slq3, partition 19,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:21 INFO scheduler.TaskSetManager: Finished task 18.0 in stage 5.0 (TID 222) in 1045 ms on slq3 (18/200)
16/04/13 00:17:22 INFO scheduler.TaskSetManager: Starting task 20.0 in stage 5.0 (TID 224, slq3, partition 20,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:22 INFO scheduler.TaskSetManager: Finished task 19.0 in stage 5.0 (TID 223) in 1283 ms on slq3 (19/200)
16/04/13 00:17:22 INFO scheduler.TaskSetManager: Starting task 21.0 in stage 5.0 (TID 225, slq1, partition 21,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:22 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 5.0 (TID 204) in 36170 ms on slq1 (20/200)
16/04/13 00:17:23 INFO scheduler.TaskSetManager: Starting task 22.0 in stage 5.0 (TID 226, slq3, partition 22,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:23 INFO scheduler.TaskSetManager: Finished task 20.0 in stage 5.0 (TID 224) in 808 ms on slq3 (21/200)
16/04/13 00:17:24 INFO scheduler.TaskSetManager: Starting task 23.0 in stage 5.0 (TID 227, slq3, partition 23,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:25 INFO scheduler.TaskSetManager: Finished task 22.0 in stage 5.0 (TID 226) in 2056 ms on slq3 (22/200)
16/04/13 00:17:25 INFO scheduler.TaskSetManager: Starting task 24.0 in stage 5.0 (TID 228, slq1, partition 24,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:25 INFO scheduler.TaskSetManager: Finished task 21.0 in stage 5.0 (TID 225) in 2501 ms on slq1 (23/200)
16/04/13 00:17:26 INFO scheduler.TaskSetManager: Starting task 25.0 in stage 5.0 (TID 229, slq3, partition 25,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:26 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on slq3:43697 in memory (size: 41.9 KB, free: 517.3 MB)
16/04/13 00:17:26 INFO scheduler.TaskSetManager: Finished task 23.0 in stage 5.0 (TID 227) in 2448 ms on slq3 (24/200)
16/04/13 00:17:26 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on slq2:34090 in memory (size: 41.9 KB, free: 517.4 MB)
16/04/13 00:17:26 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on 192.168.1.121:52549 in memory (size: 41.9 KB, free: 517.3 MB)
16/04/13 00:17:27 INFO spark.ContextCleaner: Cleaned accumulator 2
16/04/13 00:17:27 INFO spark.ContextCleaner: Cleaned accumulator 3
16/04/13 00:17:27 INFO spark.ContextCleaner: Cleaned accumulator 4
16/04/13 00:17:27 INFO spark.ContextCleaner: Cleaned accumulator 5
16/04/13 00:17:27 INFO spark.ContextCleaner: Cleaned accumulator 6
16/04/13 00:17:27 INFO spark.ContextCleaner: Cleaned shuffle 0
16/04/13 00:17:27 INFO spark.ContextCleaner: Cleaned accumulator 7
16/04/13 00:17:27 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on 192.168.1.121:52549 in memory (size: 5.4 KB, free: 517.3 MB)
16/04/13 00:17:27 INFO scheduler.TaskSetManager: Starting task 26.0 in stage 5.0 (TID 230, slq3, partition 26,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:27 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on slq3:43697 in memory (size: 5.4 KB, free: 517.3 MB)
16/04/13 00:17:27 INFO scheduler.TaskSetManager: Finished task 25.0 in stage 5.0 (TID 229) in 1319 ms on slq3 (25/200)
16/04/13 00:17:27 INFO storage.BlockManagerInfo: Removed broadcast_1_piece0 on slq2:34090 in memory (size: 5.4 KB, free: 517.4 MB)
16/04/13 00:17:27 INFO scheduler.TaskSetManager: Starting task 27.0 in stage 5.0 (TID 231, slq1, partition 27,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:27 INFO scheduler.TaskSetManager: Finished task 24.0 in stage 5.0 (TID 228) in 2529 ms on slq1 (26/200)
16/04/13 00:17:28 INFO spark.ContextCleaner: Cleaned accumulator 8
16/04/13 00:17:28 INFO storage.BlockManagerInfo: Removed broadcast_2_piece0 on 192.168.1.121:52549 in memory (size: 8.0 KB, free: 517.3 MB)
16/04/13 00:17:28 INFO storage.BlockManagerInfo: Removed broadcast_2_piece0 on slq3:43697 in memory (size: 8.0 KB, free: 517.3 MB)
16/04/13 00:17:28 INFO spark.ContextCleaner: Cleaned accumulator 9
16/04/13 00:17:28 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on 192.168.1.121:52549 in memory (size: 8.0 KB, free: 517.3 MB)
16/04/13 00:17:28 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on slq3:43697 in memory (size: 8.0 KB, free: 517.3 MB)
16/04/13 00:17:28 INFO storage.BlockManagerInfo: Removed broadcast_3_piece0 on slq2:34090 in memory (size: 8.0 KB, free: 517.4 MB)
16/04/13 00:17:28 INFO scheduler.TaskSetManager: Starting task 28.0 in stage 5.0 (TID 232, slq3, partition 28,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:28 INFO scheduler.TaskSetManager: Finished task 26.0 in stage 5.0 (TID 230) in 938 ms on slq3 (27/200)
16/04/13 00:17:28 INFO storage.BlockManagerInfo: Removed broadcast_5_piece0 on 192.168.1.121:52549 in memory (size: 5.4 KB, free: 517.3 MB)
16/04/13 00:17:29 INFO storage.BlockManagerInfo: Removed broadcast_5_piece0 on slq3:43697 in memory (size: 5.4 KB, free: 517.3 MB)
16/04/13 00:17:29 INFO storage.BlockManagerInfo: Removed broadcast_5_piece0 on slq1:44779 in memory (size: 5.4 KB, free: 517.3 MB)
16/04/13 00:17:29 INFO scheduler.TaskSetManager: Starting task 29.0 in stage 5.0 (TID 233, slq3, partition 29,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:29 INFO scheduler.TaskSetManager: Finished task 28.0 in stage 5.0 (TID 232) in 691 ms on slq3 (28/200)
16/04/13 00:17:30 INFO scheduler.TaskSetManager: Starting task 30.0 in stage 5.0 (TID 234, slq3, partition 30,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:30 INFO scheduler.TaskSetManager: Finished task 29.0 in stage 5.0 (TID 233) in 642 ms on slq3 (29/200)
16/04/13 00:17:30 INFO scheduler.TaskSetManager: Starting task 31.0 in stage 5.0 (TID 235, slq1, partition 31,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:30 INFO scheduler.TaskSetManager: Finished task 27.0 in stage 5.0 (TID 231) in 2449 ms on slq1 (30/200)
16/04/13 00:17:30 INFO scheduler.TaskSetManager: Starting task 32.0 in stage 5.0 (TID 236, slq3, partition 32,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:30 INFO scheduler.TaskSetManager: Finished task 30.0 in stage 5.0 (TID 234) in 613 ms on slq3 (31/200)
16/04/13 00:17:31 INFO scheduler.TaskSetManager: Starting task 33.0 in stage 5.0 (TID 237, slq3, partition 33,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:31 INFO scheduler.TaskSetManager: Finished task 32.0 in stage 5.0 (TID 236) in 572 ms on slq3 (32/200)
16/04/13 00:17:31 INFO scheduler.TaskSetManager: Starting task 34.0 in stage 5.0 (TID 238, slq1, partition 34,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:31 INFO scheduler.TaskSetManager: Finished task 31.0 in stage 5.0 (TID 235) in 1068 ms on slq1 (33/200)
16/04/13 00:17:31 INFO scheduler.TaskSetManager: Starting task 35.0 in stage 5.0 (TID 239, slq3, partition 35,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:31 INFO scheduler.TaskSetManager: Finished task 33.0 in stage 5.0 (TID 237) in 660 ms on slq3 (34/200)
16/04/13 00:17:32 INFO scheduler.TaskSetManager: Starting task 36.0 in stage 5.0 (TID 240, slq3, partition 36,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:32 INFO scheduler.TaskSetManager: Finished task 35.0 in stage 5.0 (TID 239) in 484 ms on slq3 (35/200)
16/04/13 00:17:32 INFO scheduler.TaskSetManager: Starting task 37.0 in stage 5.0 (TID 241, slq1, partition 37,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:32 INFO scheduler.TaskSetManager: Finished task 34.0 in stage 5.0 (TID 238) in 1098 ms on slq1 (36/200)
16/04/13 00:17:33 INFO scheduler.TaskSetManager: Starting task 38.0 in stage 5.0 (TID 242, slq3, partition 38,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:33 INFO scheduler.TaskSetManager: Finished task 36.0 in stage 5.0 (TID 240) in 735 ms on slq3 (37/200)
16/04/13 00:17:33 INFO scheduler.TaskSetManager: Starting task 39.0 in stage 5.0 (TID 243, slq3, partition 39,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:33 INFO scheduler.TaskSetManager: Finished task 38.0 in stage 5.0 (TID 242) in 533 ms on slq3 (38/200)
16/04/13 00:17:33 INFO scheduler.TaskSetManager: Starting task 40.0 in stage 5.0 (TID 244, slq1, partition 40,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:33 INFO scheduler.TaskSetManager: Finished task 37.0 in stage 5.0 (TID 241) in 1455 ms on slq1 (39/200)
16/04/13 00:17:34 INFO scheduler.TaskSetManager: Starting task 41.0 in stage 5.0 (TID 245, slq3, partition 41,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:34 INFO scheduler.TaskSetManager: Finished task 39.0 in stage 5.0 (TID 243) in 618 ms on slq3 (40/200)
16/04/13 00:17:34 INFO scheduler.TaskSetManager: Starting task 42.0 in stage 5.0 (TID 246, slq3, partition 42,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:34 INFO scheduler.TaskSetManager: Finished task 41.0 in stage 5.0 (TID 245) in 664 ms on slq3 (41/200)
16/04/13 00:17:35 INFO scheduler.TaskSetManager: Starting task 43.0 in stage 5.0 (TID 247, slq3, partition 43,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:35 INFO scheduler.TaskSetManager: Finished task 42.0 in stage 5.0 (TID 246) in 592 ms on slq3 (42/200)
16/04/13 00:17:35 INFO scheduler.TaskSetManager: Starting task 44.0 in stage 5.0 (TID 248, slq1, partition 44,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:35 INFO scheduler.TaskSetManager: Finished task 40.0 in stage 5.0 (TID 244) in 1927 ms on slq1 (43/200)
16/04/13 00:17:36 INFO scheduler.TaskSetManager: Starting task 45.0 in stage 5.0 (TID 249, slq3, partition 45,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:36 INFO scheduler.TaskSetManager: Finished task 43.0 in stage 5.0 (TID 247) in 773 ms on slq3 (44/200)
16/04/13 00:17:36 INFO scheduler.TaskSetManager: Starting task 46.0 in stage 5.0 (TID 250, slq3, partition 46,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:36 INFO scheduler.TaskSetManager: Finished task 45.0 in stage 5.0 (TID 249) in 541 ms on slq3 (45/200)
16/04/13 00:17:37 INFO scheduler.TaskSetManager: Starting task 47.0 in stage 5.0 (TID 251, slq1, partition 47,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:37 INFO scheduler.TaskSetManager: Finished task 44.0 in stage 5.0 (TID 248) in 1401 ms on slq1 (46/200)
16/04/13 00:17:37 INFO scheduler.TaskSetManager: Starting task 48.0 in stage 5.0 (TID 252, slq3, partition 48,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:37 INFO scheduler.TaskSetManager: Finished task 46.0 in stage 5.0 (TID 250) in 667 ms on slq3 (47/200)
16/04/13 00:17:37 INFO scheduler.TaskSetManager: Starting task 49.0 in stage 5.0 (TID 253, slq3, partition 49,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:37 INFO scheduler.TaskSetManager: Finished task 48.0 in stage 5.0 (TID 252) in 460 ms on slq3 (48/200)
16/04/13 00:17:38 INFO scheduler.TaskSetManager: Starting task 50.0 in stage 5.0 (TID 254, slq1, partition 50,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:38 INFO scheduler.TaskSetManager: Finished task 47.0 in stage 5.0 (TID 251) in 1035 ms on slq1 (49/200)
16/04/13 00:17:38 INFO scheduler.TaskSetManager: Starting task 51.0 in stage 5.0 (TID 255, slq3, partition 51,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:38 INFO scheduler.TaskSetManager: Finished task 49.0 in stage 5.0 (TID 253) in 721 ms on slq3 (50/200)
16/04/13 00:17:39 INFO scheduler.TaskSetManager: Starting task 52.0 in stage 5.0 (TID 256, slq3, partition 52,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:39 INFO scheduler.TaskSetManager: Finished task 51.0 in stage 5.0 (TID 255) in 624 ms on slq3 (51/200)
16/04/13 00:17:39 INFO scheduler.TaskSetManager: Starting task 53.0 in stage 5.0 (TID 257, slq3, partition 53,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:39 INFO scheduler.TaskSetManager: Finished task 52.0 in stage 5.0 (TID 256) in 625 ms on slq3 (52/200)
16/04/13 00:17:40 INFO scheduler.TaskSetManager: Starting task 54.0 in stage 5.0 (TID 258, slq1, partition 54,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:40 INFO scheduler.TaskSetManager: Finished task 50.0 in stage 5.0 (TID 254) in 2032 ms on slq1 (53/200)
16/04/13 00:17:40 INFO scheduler.TaskSetManager: Starting task 55.0 in stage 5.0 (TID 259, slq3, partition 55,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:40 INFO scheduler.TaskSetManager: Finished task 53.0 in stage 5.0 (TID 257) in 828 ms on slq3 (54/200)
16/04/13 00:17:41 INFO scheduler.TaskSetManager: Starting task 56.0 in stage 5.0 (TID 260, slq3, partition 56,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:41 INFO scheduler.TaskSetManager: Finished task 55.0 in stage 5.0 (TID 259) in 581 ms on slq3 (55/200)
16/04/13 00:17:41 INFO scheduler.TaskSetManager: Starting task 57.0 in stage 5.0 (TID 261, slq1, partition 57,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:41 INFO scheduler.TaskSetManager: Finished task 54.0 in stage 5.0 (TID 258) in 1162 ms on slq1 (56/200)
16/04/13 00:17:41 INFO scheduler.TaskSetManager: Starting task 58.0 in stage 5.0 (TID 262, slq3, partition 58,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:41 INFO scheduler.TaskSetManager: Finished task 56.0 in stage 5.0 (TID 260) in 633 ms on slq3 (57/200)
16/04/13 00:17:42 INFO scheduler.TaskSetManager: Starting task 59.0 in stage 5.0 (TID 263, slq3, partition 59,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:42 INFO scheduler.TaskSetManager: Finished task 58.0 in stage 5.0 (TID 262) in 590 ms on slq3 (58/200)
16/04/13 00:17:42 INFO scheduler.TaskSetManager: Starting task 60.0 in stage 5.0 (TID 264, slq1, partition 60,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:42 INFO scheduler.TaskSetManager: Finished task 57.0 in stage 5.0 (TID 261) in 1149 ms on slq1 (59/200)
16/04/13 00:17:43 INFO scheduler.TaskSetManager: Starting task 61.0 in stage 5.0 (TID 265, slq3, partition 61,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:43 INFO scheduler.TaskSetManager: Finished task 59.0 in stage 5.0 (TID 263) in 578 ms on slq3 (60/200)
16/04/13 00:17:43 INFO scheduler.TaskSetManager: Starting task 62.0 in stage 5.0 (TID 266, slq3, partition 62,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:43 INFO scheduler.TaskSetManager: Finished task 61.0 in stage 5.0 (TID 265) in 781 ms on slq3 (61/200)
16/04/13 00:17:44 INFO scheduler.TaskSetManager: Starting task 63.0 in stage 5.0 (TID 267, slq1, partition 63,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:44 INFO scheduler.TaskSetManager: Finished task 60.0 in stage 5.0 (TID 264) in 1341 ms on slq1 (62/200)
16/04/13 00:17:44 INFO scheduler.TaskSetManager: Starting task 64.0 in stage 5.0 (TID 268, slq3, partition 64,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:44 INFO scheduler.TaskSetManager: Finished task 62.0 in stage 5.0 (TID 266) in 702 ms on slq3 (63/200)
16/04/13 00:17:45 INFO scheduler.TaskSetManager: Starting task 65.0 in stage 5.0 (TID 269, slq3, partition 65,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:45 INFO scheduler.TaskSetManager: Finished task 64.0 in stage 5.0 (TID 268) in 600 ms on slq3 (64/200)
16/04/13 00:17:45 INFO scheduler.TaskSetManager: Starting task 66.0 in stage 5.0 (TID 270, slq1, partition 66,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:45 INFO scheduler.TaskSetManager: Finished task 63.0 in stage 5.0 (TID 267) in 1554 ms on slq1 (65/200)
16/04/13 00:17:45 INFO scheduler.TaskSetManager: Starting task 67.0 in stage 5.0 (TID 271, slq3, partition 67,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:45 INFO scheduler.TaskSetManager: Finished task 65.0 in stage 5.0 (TID 269) in 803 ms on slq3 (66/200)
16/04/13 00:17:46 INFO scheduler.TaskSetManager: Starting task 68.0 in stage 5.0 (TID 272, slq3, partition 68,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:46 INFO scheduler.TaskSetManager: Finished task 67.0 in stage 5.0 (TID 271) in 781 ms on slq3 (67/200)
16/04/13 00:17:47 INFO scheduler.TaskSetManager: Starting task 69.0 in stage 5.0 (TID 273, slq1, partition 69,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:47 INFO scheduler.TaskSetManager: Finished task 66.0 in stage 5.0 (TID 270) in 1609 ms on slq1 (68/200)
16/04/13 00:17:47 INFO scheduler.TaskSetManager: Starting task 70.0 in stage 5.0 (TID 274, slq3, partition 70,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:47 INFO scheduler.TaskSetManager: Finished task 68.0 in stage 5.0 (TID 272) in 713 ms on slq3 (69/200)
16/04/13 00:17:48 INFO scheduler.TaskSetManager: Starting task 71.0 in stage 5.0 (TID 275, slq3, partition 71,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:48 INFO scheduler.TaskSetManager: Finished task 70.0 in stage 5.0 (TID 274) in 698 ms on slq3 (70/200)
16/04/13 00:17:48 INFO scheduler.TaskSetManager: Starting task 72.0 in stage 5.0 (TID 276, slq1, partition 72,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:48 INFO scheduler.TaskSetManager: Finished task 69.0 in stage 5.0 (TID 273) in 1230 ms on slq1 (71/200)
16/04/13 00:17:48 INFO scheduler.TaskSetManager: Starting task 73.0 in stage 5.0 (TID 277, slq3, partition 73,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:48 INFO scheduler.TaskSetManager: Finished task 71.0 in stage 5.0 (TID 275) in 454 ms on slq3 (72/200)
16/04/13 00:17:49 INFO scheduler.TaskSetManager: Starting task 74.0 in stage 5.0 (TID 278, slq3, partition 74,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:49 INFO scheduler.TaskSetManager: Finished task 73.0 in stage 5.0 (TID 277) in 435 ms on slq3 (73/200)
16/04/13 00:17:49 INFO scheduler.TaskSetManager: Starting task 75.0 in stage 5.0 (TID 279, slq1, partition 75,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:49 INFO scheduler.TaskSetManager: Finished task 72.0 in stage 5.0 (TID 276) in 1091 ms on slq1 (74/200)
16/04/13 00:17:49 INFO scheduler.TaskSetManager: Starting task 76.0 in stage 5.0 (TID 280, slq3, partition 76,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:49 INFO scheduler.TaskSetManager: Finished task 74.0 in stage 5.0 (TID 278) in 542 ms on slq3 (75/200)
16/04/13 00:17:50 INFO scheduler.TaskSetManager: Starting task 77.0 in stage 5.0 (TID 281, slq3, partition 77,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:50 INFO scheduler.TaskSetManager: Finished task 76.0 in stage 5.0 (TID 280) in 617 ms on slq3 (76/200)
16/04/13 00:17:50 INFO scheduler.TaskSetManager: Starting task 78.0 in stage 5.0 (TID 282, slq3, partition 78,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:50 INFO scheduler.TaskSetManager: Finished task 77.0 in stage 5.0 (TID 281) in 703 ms on slq3 (77/200)
16/04/13 00:17:51 INFO scheduler.TaskSetManager: Starting task 79.0 in stage 5.0 (TID 283, slq1, partition 79,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:51 INFO scheduler.TaskSetManager: Finished task 75.0 in stage 5.0 (TID 279) in 1678 ms on slq1 (78/200)
16/04/13 00:17:51 INFO scheduler.TaskSetManager: Starting task 80.0 in stage 5.0 (TID 284, slq3, partition 80,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:51 INFO scheduler.TaskSetManager: Finished task 78.0 in stage 5.0 (TID 282) in 598 ms on slq3 (79/200)
16/04/13 00:17:51 INFO scheduler.TaskSetManager: Starting task 81.0 in stage 5.0 (TID 285, slq3, partition 81,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:51 INFO scheduler.TaskSetManager: Finished task 80.0 in stage 5.0 (TID 284) in 394 ms on slq3 (80/200)
16/04/13 00:17:52 INFO scheduler.TaskSetManager: Starting task 82.0 in stage 5.0 (TID 286, slq1, partition 82,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:52 INFO scheduler.TaskSetManager: Finished task 79.0 in stage 5.0 (TID 283) in 1590 ms on slq1 (81/200)
16/04/13 00:17:52 INFO scheduler.TaskSetManager: Starting task 83.0 in stage 5.0 (TID 287, slq3, partition 83,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:52 INFO scheduler.TaskSetManager: Finished task 81.0 in stage 5.0 (TID 285) in 970 ms on slq3 (82/200)
16/04/13 00:17:53 INFO scheduler.TaskSetManager: Starting task 84.0 in stage 5.0 (TID 288, slq3, partition 84,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:53 INFO scheduler.TaskSetManager: Finished task 83.0 in stage 5.0 (TID 287) in 761 ms on slq3 (83/200)
16/04/13 00:17:54 INFO scheduler.TaskSetManager: Starting task 85.0 in stage 5.0 (TID 289, slq3, partition 85,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:54 INFO scheduler.TaskSetManager: Finished task 84.0 in stage 5.0 (TID 288) in 904 ms on slq3 (84/200)
16/04/13 00:17:54 INFO scheduler.TaskSetManager: Starting task 86.0 in stage 5.0 (TID 290, slq1, partition 86,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:54 INFO scheduler.TaskSetManager: Finished task 82.0 in stage 5.0 (TID 286) in 1812 ms on slq1 (85/200)
16/04/13 00:17:55 INFO scheduler.TaskSetManager: Starting task 87.0 in stage 5.0 (TID 291, slq3, partition 87,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:55 INFO scheduler.TaskSetManager: Finished task 85.0 in stage 5.0 (TID 289) in 633 ms on slq3 (86/200)
16/04/13 00:17:55 INFO scheduler.TaskSetManager: Starting task 88.0 in stage 5.0 (TID 292, slq3, partition 88,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:55 INFO scheduler.TaskSetManager: Finished task 87.0 in stage 5.0 (TID 291) in 483 ms on slq3 (87/200)
16/04/13 00:17:55 INFO scheduler.TaskSetManager: Starting task 89.0 in stage 5.0 (TID 293, slq1, partition 89,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:55 INFO scheduler.TaskSetManager: Finished task 86.0 in stage 5.0 (TID 290) in 1369 ms on slq1 (88/200)
16/04/13 00:17:56 INFO scheduler.TaskSetManager: Starting task 90.0 in stage 5.0 (TID 294, slq3, partition 90,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:56 INFO scheduler.TaskSetManager: Finished task 88.0 in stage 5.0 (TID 292) in 766 ms on slq3 (89/200)
16/04/13 00:17:57 INFO scheduler.TaskSetManager: Starting task 91.0 in stage 5.0 (TID 295, slq3, partition 91,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:57 INFO scheduler.TaskSetManager: Finished task 90.0 in stage 5.0 (TID 294) in 666 ms on slq3 (90/200)
16/04/13 00:17:57 INFO scheduler.TaskSetManager: Starting task 92.0 in stage 5.0 (TID 296, slq1, partition 92,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:57 INFO scheduler.TaskSetManager: Finished task 89.0 in stage 5.0 (TID 293) in 1369 ms on slq1 (91/200)
16/04/13 00:17:57 INFO scheduler.TaskSetManager: Starting task 93.0 in stage 5.0 (TID 297, slq3, partition 93,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:57 INFO scheduler.TaskSetManager: Finished task 91.0 in stage 5.0 (TID 295) in 650 ms on slq3 (92/200)
16/04/13 00:17:58 INFO scheduler.TaskSetManager: Starting task 94.0 in stage 5.0 (TID 298, slq3, partition 94,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:58 INFO scheduler.TaskSetManager: Finished task 93.0 in stage 5.0 (TID 297) in 548 ms on slq3 (93/200)
16/04/13 00:17:58 INFO scheduler.TaskSetManager: Starting task 95.0 in stage 5.0 (TID 299, slq1, partition 95,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:58 INFO scheduler.TaskSetManager: Finished task 92.0 in stage 5.0 (TID 296) in 1522 ms on slq1 (94/200)
16/04/13 00:17:58 INFO scheduler.TaskSetManager: Starting task 96.0 in stage 5.0 (TID 300, slq3, partition 96,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:58 INFO scheduler.TaskSetManager: Finished task 94.0 in stage 5.0 (TID 298) in 780 ms on slq3 (95/200)
16/04/13 00:17:59 INFO scheduler.TaskSetManager: Starting task 97.0 in stage 5.0 (TID 301, slq3, partition 97,NODE_LOCAL, 2070 bytes)
16/04/13 00:17:59 INFO scheduler.TaskSetManager: Finished task 96.0 in stage 5.0 (TID 300) in 441 ms on slq3 (96/200)
16/04/13 00:18:00 INFO scheduler.TaskSetManager: Starting task 98.0 in stage 5.0 (TID 302, slq1, partition 98,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:00 INFO scheduler.TaskSetManager: Finished task 95.0 in stage 5.0 (TID 299) in 1224 ms on slq1 (97/200)
16/04/13 00:18:00 INFO scheduler.TaskSetManager: Starting task 99.0 in stage 5.0 (TID 303, slq3, partition 99,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:00 INFO scheduler.TaskSetManager: Finished task 97.0 in stage 5.0 (TID 301) in 749 ms on slq3 (98/200)
16/04/13 00:18:00 INFO scheduler.TaskSetManager: Starting task 100.0 in stage 5.0 (TID 304, slq3, partition 100,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:00 INFO scheduler.TaskSetManager: Finished task 99.0 in stage 5.0 (TID 303) in 485 ms on slq3 (99/200)
16/04/13 00:18:01 INFO scheduler.TaskSetManager: Starting task 101.0 in stage 5.0 (TID 305, slq1, partition 101,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:01 INFO scheduler.TaskSetManager: Finished task 98.0 in stage 5.0 (TID 302) in 1030 ms on slq1 (100/200)
16/04/13 00:18:01 INFO scheduler.TaskSetManager: Starting task 102.0 in stage 5.0 (TID 306, slq3, partition 102,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:01 INFO scheduler.TaskSetManager: Finished task 100.0 in stage 5.0 (TID 304) in 670 ms on slq3 (101/200)
16/04/13 00:18:01 INFO scheduler.TaskSetManager: Starting task 103.0 in stage 5.0 (TID 307, slq3, partition 103,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:01 INFO scheduler.TaskSetManager: Finished task 102.0 in stage 5.0 (TID 306) in 606 ms on slq3 (102/200)
16/04/13 00:18:02 INFO scheduler.TaskSetManager: Starting task 104.0 in stage 5.0 (TID 308, slq3, partition 104,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:02 INFO scheduler.TaskSetManager: Finished task 103.0 in stage 5.0 (TID 307) in 649 ms on slq3 (103/200)
16/04/13 00:18:02 INFO scheduler.TaskSetManager: Starting task 105.0 in stage 5.0 (TID 309, slq1, partition 105,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:02 INFO scheduler.TaskSetManager: Finished task 101.0 in stage 5.0 (TID 305) in 1561 ms on slq1 (104/200)
16/04/13 00:18:03 INFO scheduler.TaskSetManager: Starting task 106.0 in stage 5.0 (TID 310, slq3, partition 106,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:03 INFO scheduler.TaskSetManager: Finished task 104.0 in stage 5.0 (TID 308) in 633 ms on slq3 (105/200)
16/04/13 00:18:03 INFO scheduler.TaskSetManager: Starting task 107.0 in stage 5.0 (TID 311, slq3, partition 107,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:03 INFO scheduler.TaskSetManager: Finished task 106.0 in stage 5.0 (TID 310) in 694 ms on slq3 (106/200)
16/04/13 00:18:04 INFO scheduler.TaskSetManager: Starting task 108.0 in stage 5.0 (TID 312, slq1, partition 108,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:04 INFO scheduler.TaskSetManager: Finished task 105.0 in stage 5.0 (TID 309) in 1572 ms on slq1 (107/200)
16/04/13 00:18:04 INFO scheduler.TaskSetManager: Starting task 109.0 in stage 5.0 (TID 313, slq3, partition 109,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:04 INFO scheduler.TaskSetManager: Finished task 107.0 in stage 5.0 (TID 311) in 774 ms on slq3 (108/200)
16/04/13 00:18:05 INFO scheduler.TaskSetManager: Starting task 110.0 in stage 5.0 (TID 314, slq3, partition 110,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:05 INFO scheduler.TaskSetManager: Finished task 109.0 in stage 5.0 (TID 313) in 499 ms on slq3 (109/200)
16/04/13 00:18:05 INFO scheduler.TaskSetManager: Starting task 111.0 in stage 5.0 (TID 315, slq1, partition 111,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:05 INFO scheduler.TaskSetManager: Finished task 108.0 in stage 5.0 (TID 312) in 1628 ms on slq1 (110/200)
16/04/13 00:18:06 INFO scheduler.TaskSetManager: Starting task 112.0 in stage 5.0 (TID 316, slq3, partition 112,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:06 INFO scheduler.TaskSetManager: Finished task 110.0 in stage 5.0 (TID 314) in 839 ms on slq3 (111/200)
16/04/13 00:18:06 INFO scheduler.TaskSetManager: Starting task 113.0 in stage 5.0 (TID 317, slq3, partition 113,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:06 INFO scheduler.TaskSetManager: Finished task 112.0 in stage 5.0 (TID 316) in 547 ms on slq3 (112/200)
16/04/13 00:18:07 INFO scheduler.TaskSetManager: Starting task 114.0 in stage 5.0 (TID 318, slq3, partition 114,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:07 INFO scheduler.TaskSetManager: Finished task 113.0 in stage 5.0 (TID 317) in 601 ms on slq3 (113/200)
16/04/13 00:18:07 INFO scheduler.TaskSetManager: Starting task 115.0 in stage 5.0 (TID 319, slq1, partition 115,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:07 INFO scheduler.TaskSetManager: Finished task 111.0 in stage 5.0 (TID 315) in 1527 ms on slq1 (114/200)
16/04/13 00:18:07 INFO scheduler.TaskSetManager: Starting task 116.0 in stage 5.0 (TID 320, slq3, partition 116,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:07 INFO scheduler.TaskSetManager: Finished task 114.0 in stage 5.0 (TID 318) in 532 ms on slq3 (115/200)
16/04/13 00:18:08 INFO scheduler.TaskSetManager: Starting task 117.0 in stage 5.0 (TID 321, slq3, partition 117,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:08 INFO scheduler.TaskSetManager: Finished task 116.0 in stage 5.0 (TID 320) in 548 ms on slq3 (116/200)
16/04/13 00:18:08 INFO scheduler.TaskSetManager: Starting task 118.0 in stage 5.0 (TID 322, slq1, partition 118,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:08 INFO scheduler.TaskSetManager: Finished task 115.0 in stage 5.0 (TID 319) in 1465 ms on slq1 (117/200)
16/04/13 00:18:09 INFO scheduler.TaskSetManager: Starting task 119.0 in stage 5.0 (TID 323, slq3, partition 119,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:09 INFO scheduler.TaskSetManager: Finished task 117.0 in stage 5.0 (TID 321) in 820 ms on slq3 (118/200)
16/04/13 00:18:09 INFO scheduler.TaskSetManager: Starting task 120.0 in stage 5.0 (TID 324, slq3, partition 120,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:09 INFO scheduler.TaskSetManager: Finished task 119.0 in stage 5.0 (TID 323) in 433 ms on slq3 (119/200)
16/04/13 00:18:09 INFO scheduler.TaskSetManager: Starting task 121.0 in stage 5.0 (TID 325, slq1, partition 121,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:09 INFO scheduler.TaskSetManager: Finished task 118.0 in stage 5.0 (TID 322) in 885 ms on slq1 (120/200)
16/04/13 00:18:10 INFO scheduler.TaskSetManager: Starting task 122.0 in stage 5.0 (TID 326, slq3, partition 122,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:10 INFO scheduler.TaskSetManager: Finished task 120.0 in stage 5.0 (TID 324) in 539 ms on slq3 (121/200)
16/04/13 00:18:10 INFO scheduler.TaskSetManager: Starting task 123.0 in stage 5.0 (TID 327, slq3, partition 123,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:10 INFO scheduler.TaskSetManager: Finished task 122.0 in stage 5.0 (TID 326) in 466 ms on slq3 (122/200)
16/04/13 00:18:10 INFO scheduler.TaskSetManager: Starting task 124.0 in stage 5.0 (TID 328, slq1, partition 124,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:10 INFO scheduler.TaskSetManager: Finished task 121.0 in stage 5.0 (TID 325) in 982 ms on slq1 (123/200)
16/04/13 00:18:11 INFO scheduler.TaskSetManager: Starting task 125.0 in stage 5.0 (TID 329, slq3, partition 125,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:11 INFO scheduler.TaskSetManager: Finished task 123.0 in stage 5.0 (TID 327) in 565 ms on slq3 (124/200)
16/04/13 00:18:11 INFO scheduler.TaskSetManager: Starting task 126.0 in stage 5.0 (TID 330, slq3, partition 126,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:11 INFO scheduler.TaskSetManager: Finished task 125.0 in stage 5.0 (TID 329) in 659 ms on slq3 (125/200)
16/04/13 00:18:11 INFO scheduler.TaskSetManager: Starting task 127.0 in stage 5.0 (TID 331, slq1, partition 127,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:11 INFO scheduler.TaskSetManager: Finished task 124.0 in stage 5.0 (TID 328) in 1037 ms on slq1 (126/200)
16/04/13 00:18:12 INFO scheduler.TaskSetManager: Starting task 128.0 in stage 5.0 (TID 332, slq3, partition 128,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:12 INFO scheduler.TaskSetManager: Finished task 126.0 in stage 5.0 (TID 330) in 773 ms on slq3 (127/200)
16/04/13 00:18:12 INFO scheduler.TaskSetManager: Starting task 129.0 in stage 5.0 (TID 333, slq1, partition 129,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:13 INFO scheduler.TaskSetManager: Finished task 127.0 in stage 5.0 (TID 331) in 1314 ms on slq1 (128/200)
16/04/13 00:18:13 INFO scheduler.TaskSetManager: Starting task 130.0 in stage 5.0 (TID 334, slq3, partition 130,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:13 INFO scheduler.TaskSetManager: Finished task 128.0 in stage 5.0 (TID 332) in 871 ms on slq3 (129/200)
16/04/13 00:18:13 INFO scheduler.TaskSetManager: Starting task 131.0 in stage 5.0 (TID 335, slq3, partition 131,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:13 INFO scheduler.TaskSetManager: Finished task 130.0 in stage 5.0 (TID 334) in 668 ms on slq3 (130/200)
16/04/13 00:18:14 INFO scheduler.TaskSetManager: Starting task 132.0 in stage 5.0 (TID 336, slq1, partition 132,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:14 INFO scheduler.TaskSetManager: Finished task 129.0 in stage 5.0 (TID 333) in 1324 ms on slq1 (131/200)
16/04/13 00:18:14 INFO scheduler.TaskSetManager: Starting task 133.0 in stage 5.0 (TID 337, slq3, partition 133,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:14 INFO scheduler.TaskSetManager: Finished task 131.0 in stage 5.0 (TID 335) in 629 ms on slq3 (132/200)
16/04/13 00:18:15 INFO scheduler.TaskSetManager: Starting task 134.0 in stage 5.0 (TID 338, slq3, partition 134,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:15 INFO scheduler.TaskSetManager: Finished task 133.0 in stage 5.0 (TID 337) in 500 ms on slq3 (133/200)
16/04/13 00:18:15 INFO scheduler.TaskSetManager: Starting task 135.0 in stage 5.0 (TID 339, slq3, partition 135,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:15 INFO scheduler.TaskSetManager: Finished task 134.0 in stage 5.0 (TID 338) in 381 ms on slq3 (134/200)
16/04/13 00:18:15 INFO scheduler.TaskSetManager: Starting task 136.0 in stage 5.0 (TID 340, slq1, partition 136,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:15 INFO scheduler.TaskSetManager: Finished task 132.0 in stage 5.0 (TID 336) in 1225 ms on slq1 (135/200)
16/04/13 00:18:15 INFO scheduler.TaskSetManager: Starting task 137.0 in stage 5.0 (TID 341, slq3, partition 137,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:15 INFO scheduler.TaskSetManager: Finished task 135.0 in stage 5.0 (TID 339) in 421 ms on slq3 (136/200)
16/04/13 00:18:16 INFO scheduler.TaskSetManager: Starting task 138.0 in stage 5.0 (TID 342, slq3, partition 138,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:16 INFO scheduler.TaskSetManager: Finished task 137.0 in stage 5.0 (TID 341) in 461 ms on slq3 (137/200)
16/04/13 00:18:17 INFO scheduler.TaskSetManager: Starting task 139.0 in stage 5.0 (TID 343, slq1, partition 139,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:17 INFO scheduler.TaskSetManager: Finished task 136.0 in stage 5.0 (TID 340) in 1571 ms on slq1 (138/200)
16/04/13 00:18:17 INFO scheduler.TaskSetManager: Starting task 140.0 in stage 5.0 (TID 344, slq3, partition 140,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:17 INFO scheduler.TaskSetManager: Finished task 138.0 in stage 5.0 (TID 342) in 1158 ms on slq3 (139/200)
16/04/13 00:18:18 INFO scheduler.TaskSetManager: Starting task 141.0 in stage 5.0 (TID 345, slq3, partition 141,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:18 INFO scheduler.TaskSetManager: Finished task 140.0 in stage 5.0 (TID 344) in 691 ms on slq3 (140/200)
16/04/13 00:18:18 INFO scheduler.TaskSetManager: Starting task 142.0 in stage 5.0 (TID 346, slq1, partition 142,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:18 INFO scheduler.TaskSetManager: Finished task 139.0 in stage 5.0 (TID 343) in 1212 ms on slq1 (141/200)
16/04/13 00:18:18 INFO scheduler.TaskSetManager: Starting task 143.0 in stage 5.0 (TID 347, slq3, partition 143,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:18 INFO scheduler.TaskSetManager: Finished task 141.0 in stage 5.0 (TID 345) in 707 ms on slq3 (142/200)
16/04/13 00:18:19 INFO scheduler.TaskSetManager: Starting task 144.0 in stage 5.0 (TID 348, slq1, partition 144,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:19 INFO scheduler.TaskSetManager: Finished task 142.0 in stage 5.0 (TID 346) in 930 ms on slq1 (143/200)
16/04/13 00:18:19 INFO scheduler.TaskSetManager: Starting task 145.0 in stage 5.0 (TID 349, slq3, partition 145,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:19 INFO scheduler.TaskSetManager: Finished task 143.0 in stage 5.0 (TID 347) in 513 ms on slq3 (144/200)
16/04/13 00:18:19 INFO scheduler.TaskSetManager: Starting task 146.0 in stage 5.0 (TID 350, slq3, partition 146,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:19 INFO scheduler.TaskSetManager: Finished task 145.0 in stage 5.0 (TID 349) in 439 ms on slq3 (145/200)
16/04/13 00:18:20 INFO scheduler.TaskSetManager: Starting task 147.0 in stage 5.0 (TID 351, slq1, partition 147,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:20 INFO scheduler.TaskSetManager: Finished task 144.0 in stage 5.0 (TID 348) in 859 ms on slq1 (146/200)
16/04/13 00:18:20 INFO scheduler.TaskSetManager: Starting task 148.0 in stage 5.0 (TID 352, slq3, partition 148,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:20 INFO scheduler.TaskSetManager: Finished task 146.0 in stage 5.0 (TID 350) in 473 ms on slq3 (147/200)
16/04/13 00:18:20 INFO scheduler.TaskSetManager: Starting task 149.0 in stage 5.0 (TID 353, slq3, partition 149,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:20 INFO scheduler.TaskSetManager: Finished task 148.0 in stage 5.0 (TID 352) in 526 ms on slq3 (148/200)
16/04/13 00:18:21 INFO scheduler.TaskSetManager: Starting task 150.0 in stage 5.0 (TID 354, slq1, partition 150,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:21 INFO scheduler.TaskSetManager: Finished task 147.0 in stage 5.0 (TID 351) in 944 ms on slq1 (149/200)
16/04/13 00:18:21 INFO scheduler.TaskSetManager: Starting task 151.0 in stage 5.0 (TID 355, slq3, partition 151,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:21 INFO scheduler.TaskSetManager: Finished task 149.0 in stage 5.0 (TID 353) in 491 ms on slq3 (150/200)
16/04/13 00:18:22 INFO scheduler.TaskSetManager: Starting task 152.0 in stage 5.0 (TID 356, slq3, partition 152,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:22 INFO scheduler.TaskSetManager: Finished task 151.0 in stage 5.0 (TID 355) in 710 ms on slq3 (151/200)
16/04/13 00:18:22 INFO scheduler.TaskSetManager: Starting task 153.0 in stage 5.0 (TID 357, slq1, partition 153,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:22 INFO scheduler.TaskSetManager: Finished task 150.0 in stage 5.0 (TID 354) in 1194 ms on slq1 (152/200)
16/04/13 00:18:22 INFO scheduler.TaskSetManager: Starting task 154.0 in stage 5.0 (TID 358, slq3, partition 154,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:22 INFO scheduler.TaskSetManager: Finished task 152.0 in stage 5.0 (TID 356) in 508 ms on slq3 (153/200)
16/04/13 00:18:23 INFO scheduler.TaskSetManager: Starting task 155.0 in stage 5.0 (TID 359, slq3, partition 155,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:23 INFO scheduler.TaskSetManager: Finished task 154.0 in stage 5.0 (TID 358) in 575 ms on slq3 (154/200)
16/04/13 00:18:23 INFO scheduler.TaskSetManager: Starting task 156.0 in stage 5.0 (TID 360, slq1, partition 156,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:23 INFO scheduler.TaskSetManager: Finished task 153.0 in stage 5.0 (TID 357) in 1033 ms on slq1 (155/200)
16/04/13 00:18:23 INFO scheduler.TaskSetManager: Starting task 157.0 in stage 5.0 (TID 361, slq3, partition 157,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:23 INFO scheduler.TaskSetManager: Finished task 155.0 in stage 5.0 (TID 359) in 474 ms on slq3 (156/200)
16/04/13 00:18:23 INFO scheduler.TaskSetManager: Starting task 158.0 in stage 5.0 (TID 362, slq1, partition 158,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:23 INFO scheduler.TaskSetManager: Finished task 156.0 in stage 5.0 (TID 360) in 728 ms on slq1 (157/200)
16/04/13 00:18:24 INFO scheduler.TaskSetManager: Starting task 159.0 in stage 5.0 (TID 363, slq3, partition 159,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:24 INFO scheduler.TaskSetManager: Finished task 157.0 in stage 5.0 (TID 361) in 587 ms on slq3 (158/200)
16/04/13 00:18:24 INFO scheduler.TaskSetManager: Starting task 160.0 in stage 5.0 (TID 364, slq3, partition 160,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:24 INFO scheduler.TaskSetManager: Finished task 159.0 in stage 5.0 (TID 363) in 597 ms on slq3 (159/200)
16/04/13 00:18:24 INFO scheduler.TaskSetManager: Starting task 161.0 in stage 5.0 (TID 365, slq1, partition 161,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:24 INFO scheduler.TaskSetManager: Finished task 158.0 in stage 5.0 (TID 362) in 964 ms on slq1 (160/200)
16/04/13 00:18:25 INFO scheduler.TaskSetManager: Starting task 162.0 in stage 5.0 (TID 366, slq3, partition 162,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:25 INFO scheduler.TaskSetManager: Finished task 160.0 in stage 5.0 (TID 364) in 600 ms on slq3 (161/200)
16/04/13 00:18:25 INFO scheduler.TaskSetManager: Starting task 163.0 in stage 5.0 (TID 367, slq1, partition 163,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:25 INFO scheduler.TaskSetManager: Finished task 161.0 in stage 5.0 (TID 365) in 909 ms on slq1 (162/200)
16/04/13 00:18:25 INFO scheduler.TaskSetManager: Starting task 164.0 in stage 5.0 (TID 368, slq3, partition 164,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:25 INFO scheduler.TaskSetManager: Finished task 162.0 in stage 5.0 (TID 366) in 520 ms on slq3 (163/200)
16/04/13 00:18:26 INFO scheduler.TaskSetManager: Starting task 165.0 in stage 5.0 (TID 369, slq3, partition 165,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:26 INFO scheduler.TaskSetManager: Finished task 164.0 in stage 5.0 (TID 368) in 502 ms on slq3 (164/200)
16/04/13 00:18:26 INFO scheduler.TaskSetManager: Starting task 166.0 in stage 5.0 (TID 370, slq1, partition 166,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:26 INFO scheduler.TaskSetManager: Finished task 163.0 in stage 5.0 (TID 367) in 777 ms on slq1 (165/200)
16/04/13 00:18:26 INFO scheduler.TaskSetManager: Starting task 167.0 in stage 5.0 (TID 371, slq3, partition 167,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:26 INFO scheduler.TaskSetManager: Finished task 165.0 in stage 5.0 (TID 369) in 483 ms on slq3 (166/200)
16/04/13 00:18:27 INFO scheduler.TaskSetManager: Starting task 168.0 in stage 5.0 (TID 372, slq1, partition 168,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:27 INFO scheduler.TaskSetManager: Finished task 166.0 in stage 5.0 (TID 370) in 627 ms on slq1 (167/200)
16/04/13 00:18:27 INFO scheduler.TaskSetManager: Starting task 169.0 in stage 5.0 (TID 373, slq3, partition 169,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:27 INFO scheduler.TaskSetManager: Finished task 167.0 in stage 5.0 (TID 371) in 616 ms on slq3 (168/200)
16/04/13 00:18:28 INFO scheduler.TaskSetManager: Starting task 170.0 in stage 5.0 (TID 374, slq3, partition 170,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:28 INFO scheduler.TaskSetManager: Finished task 169.0 in stage 5.0 (TID 373) in 700 ms on slq3 (169/200)
16/04/13 00:18:28 INFO scheduler.TaskSetManager: Starting task 171.0 in stage 5.0 (TID 375, slq1, partition 171,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:28 INFO scheduler.TaskSetManager: Finished task 168.0 in stage 5.0 (TID 372) in 1092 ms on slq1 (170/200)
16/04/13 00:18:28 INFO scheduler.TaskSetManager: Starting task 172.0 in stage 5.0 (TID 376, slq3, partition 172,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:28 INFO scheduler.TaskSetManager: Finished task 170.0 in stage 5.0 (TID 374) in 803 ms on slq3 (171/200)
16/04/13 00:18:29 INFO scheduler.TaskSetManager: Starting task 173.0 in stage 5.0 (TID 377, slq1, partition 173,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:29 INFO scheduler.TaskSetManager: Finished task 171.0 in stage 5.0 (TID 375) in 997 ms on slq1 (172/200)
16/04/13 00:18:29 INFO scheduler.TaskSetManager: Starting task 174.0 in stage 5.0 (TID 378, slq3, partition 174,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:29 INFO scheduler.TaskSetManager: Finished task 172.0 in stage 5.0 (TID 376) in 793 ms on slq3 (173/200)
16/04/13 00:18:30 INFO scheduler.TaskSetManager: Starting task 175.0 in stage 5.0 (TID 379, slq1, partition 175,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:30 INFO scheduler.TaskSetManager: Finished task 173.0 in stage 5.0 (TID 377) in 1016 ms on slq1 (174/200)
16/04/13 00:18:30 INFO scheduler.TaskSetManager: Starting task 176.0 in stage 5.0 (TID 380, slq3, partition 176,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:30 INFO scheduler.TaskSetManager: Finished task 174.0 in stage 5.0 (TID 378) in 774 ms on slq3 (175/200)
16/04/13 00:18:31 INFO scheduler.TaskSetManager: Starting task 177.0 in stage 5.0 (TID 381, slq3, partition 177,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:31 INFO scheduler.TaskSetManager: Finished task 176.0 in stage 5.0 (TID 380) in 557 ms on slq3 (176/200)
16/04/13 00:18:31 INFO scheduler.TaskSetManager: Starting task 178.0 in stage 5.0 (TID 382, slq1, partition 178,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:31 INFO scheduler.TaskSetManager: Finished task 175.0 in stage 5.0 (TID 379) in 1089 ms on slq1 (177/200)
16/04/13 00:18:31 INFO scheduler.TaskSetManager: Starting task 179.0 in stage 5.0 (TID 383, slq3, partition 179,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:31 INFO scheduler.TaskSetManager: Finished task 177.0 in stage 5.0 (TID 381) in 651 ms on slq3 (178/200)
16/04/13 00:18:32 INFO scheduler.TaskSetManager: Starting task 180.0 in stage 5.0 (TID 384, slq3, partition 180,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:32 INFO scheduler.TaskSetManager: Finished task 179.0 in stage 5.0 (TID 383) in 615 ms on slq3 (179/200)
16/04/13 00:18:32 INFO scheduler.TaskSetManager: Starting task 181.0 in stage 5.0 (TID 385, slq1, partition 181,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:32 INFO scheduler.TaskSetManager: Finished task 178.0 in stage 5.0 (TID 382) in 1054 ms on slq1 (180/200)
16/04/13 00:18:32 INFO scheduler.TaskSetManager: Starting task 182.0 in stage 5.0 (TID 386, slq3, partition 182,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:32 INFO scheduler.TaskSetManager: Finished task 180.0 in stage 5.0 (TID 384) in 589 ms on slq3 (181/200)
16/04/13 00:18:33 INFO scheduler.TaskSetManager: Starting task 183.0 in stage 5.0 (TID 387, slq1, partition 183,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:33 INFO scheduler.TaskSetManager: Finished task 181.0 in stage 5.0 (TID 385) in 1089 ms on slq1 (182/200)
16/04/13 00:18:33 INFO scheduler.TaskSetManager: Starting task 184.0 in stage 5.0 (TID 388, slq3, partition 184,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:33 INFO scheduler.TaskSetManager: Finished task 182.0 in stage 5.0 (TID 386) in 857 ms on slq3 (183/200)
16/04/13 00:18:34 INFO scheduler.TaskSetManager: Starting task 185.0 in stage 5.0 (TID 389, slq3, partition 185,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:34 INFO scheduler.TaskSetManager: Finished task 184.0 in stage 5.0 (TID 388) in 513 ms on slq3 (184/200)
16/04/13 00:18:34 INFO scheduler.TaskSetManager: Starting task 186.0 in stage 5.0 (TID 390, slq1, partition 186,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:34 INFO scheduler.TaskSetManager: Finished task 183.0 in stage 5.0 (TID 387) in 1183 ms on slq1 (185/200)
16/04/13 00:18:35 INFO scheduler.TaskSetManager: Starting task 187.0 in stage 5.0 (TID 391, slq3, partition 187,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:35 INFO scheduler.TaskSetManager: Finished task 185.0 in stage 5.0 (TID 389) in 717 ms on slq3 (186/200)
16/04/13 00:18:35 INFO scheduler.TaskSetManager: Starting task 188.0 in stage 5.0 (TID 392, slq3, partition 188,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:35 INFO scheduler.TaskSetManager: Finished task 187.0 in stage 5.0 (TID 391) in 534 ms on slq3 (187/200)
16/04/13 00:18:35 INFO scheduler.TaskSetManager: Starting task 189.0 in stage 5.0 (TID 393, slq1, partition 189,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:35 INFO scheduler.TaskSetManager: Finished task 186.0 in stage 5.0 (TID 390) in 1217 ms on slq1 (188/200)
16/04/13 00:18:36 INFO scheduler.TaskSetManager: Starting task 190.0 in stage 5.0 (TID 394, slq3, partition 190,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:36 INFO scheduler.TaskSetManager: Finished task 188.0 in stage 5.0 (TID 392) in 671 ms on slq3 (189/200)
16/04/13 00:18:36 INFO scheduler.TaskSetManager: Starting task 191.0 in stage 5.0 (TID 395, slq3, partition 191,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:36 INFO scheduler.TaskSetManager: Finished task 190.0 in stage 5.0 (TID 394) in 468 ms on slq3 (190/200)
16/04/13 00:18:36 INFO scheduler.TaskSetManager: Starting task 192.0 in stage 5.0 (TID 396, slq1, partition 192,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:36 INFO scheduler.TaskSetManager: Finished task 189.0 in stage 5.0 (TID 393) in 952 ms on slq1 (191/200)
16/04/13 00:18:37 INFO scheduler.TaskSetManager: Starting task 193.0 in stage 5.0 (TID 397, slq3, partition 193,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:37 INFO scheduler.TaskSetManager: Finished task 191.0 in stage 5.0 (TID 395) in 508 ms on slq3 (192/200)
16/04/13 00:18:37 INFO scheduler.TaskSetManager: Starting task 194.0 in stage 5.0 (TID 398, slq1, partition 194,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:37 INFO scheduler.TaskSetManager: Finished task 192.0 in stage 5.0 (TID 396) in 595 ms on slq1 (193/200)
16/04/13 00:18:37 INFO scheduler.TaskSetManager: Starting task 195.0 in stage 5.0 (TID 399, slq3, partition 195,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:37 INFO scheduler.TaskSetManager: Finished task 193.0 in stage 5.0 (TID 397) in 444 ms on slq3 (194/200)
16/04/13 00:18:38 INFO scheduler.TaskSetManager: Starting task 196.0 in stage 5.0 (TID 400, slq3, partition 196,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:38 INFO scheduler.TaskSetManager: Finished task 195.0 in stage 5.0 (TID 399) in 488 ms on slq3 (195/200)
16/04/13 00:18:38 INFO scheduler.TaskSetManager: Starting task 197.0 in stage 5.0 (TID 401, slq1, partition 197,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:38 INFO scheduler.TaskSetManager: Finished task 194.0 in stage 5.0 (TID 398) in 728 ms on slq1 (196/200)
16/04/13 00:18:38 INFO scheduler.TaskSetManager: Starting task 198.0 in stage 5.0 (TID 402, slq3, partition 198,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:38 INFO scheduler.TaskSetManager: Finished task 196.0 in stage 5.0 (TID 400) in 589 ms on slq3 (197/200)
16/04/13 00:18:39 INFO scheduler.TaskSetManager: Starting task 199.0 in stage 5.0 (TID 403, slq1, partition 199,NODE_LOCAL, 2070 bytes)
16/04/13 00:18:39 INFO scheduler.TaskSetManager: Finished task 197.0 in stage 5.0 (TID 401) in 956 ms on slq1 (198/200)
16/04/13 00:18:39 INFO scheduler.TaskSetManager: Finished task 198.0 in stage 5.0 (TID 402) in 731 ms on slq3 (199/200)
16/04/13 00:18:39 INFO scheduler.TaskSetManager: Finished task 199.0 in stage 5.0 (TID 403) in 785 ms on slq1 (200/200)
16/04/13 00:18:39 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 5.0, whose tasks have all completed, from pool
16/04/13 00:18:39 INFO scheduler.DAGScheduler: ResultStage 5 (saveAsTable at SparkSQLWindowFunctionOps.scala:56) finished in 113.222 s
16/04/13 00:18:39 INFO scheduler.DAGScheduler: Job 2 finished: saveAsTable at SparkSQLWindowFunctionOps.scala:56, took 164.524630 s
16/04/13 00:18:51 INFO hadoop.ParquetFileReader: Initiating action with parallelism: 5
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
16/04/13 00:19:00 INFO datasources.DefaultWriterContainer: Job job_201604130015_0000 committed.
16/04/13 00:19:00 INFO parquet.ParquetRelation: Listing hdfs://slq1:9000/user/hive/warehouse/hive.db/sortedresultscores on driver
16/04/13 00:19:01 INFO parquet.ParquetRelation: Listing hdfs://slq1:9000/user/hive/warehouse/hive.db/sortedresultscores on driver
16/04/13 00:19:02 INFO parquet.ParquetRelation: Listing hdfs://slq1:9000/user/hive/warehouse/hive.db/sortedresultscores on driver
16/04/13 00:19:02 INFO hive.HiveContext$$anon$2: Persisting data source relation `sortedResultScores` with a single input path into Hive metastore in Hive compatible format. Input path: hdfs://slq1:9000/user/hive/warehouse/hive.db/sortedresultscores.
16/04/13 00:19:05 INFO spark.SparkContext: Invoking stop() from shutdown hook
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/static/sql,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/SQL/execution/json,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/SQL/execution,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/SQL/json,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/SQL,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/metrics/json,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/stage/kill,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/api,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/static,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/executors/threadDump/json,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/executors/threadDump,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/executors/json,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/executors,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/environment/json,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/environment,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/storage/rdd/json,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/storage/rdd,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/storage/json,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/storage,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/pool/json,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/pool,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/stage/json,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/stage,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/json,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/jobs/job/json,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/jobs/job,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/jobs/json,null}
16/04/13 00:19:06 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/jobs,null}
16/04/13 00:19:06 INFO ui.SparkUI: Stopped Spark web UI at http://192.168.1.121:4040
16/04/13 00:19:06 INFO cluster.SparkDeploySchedulerBackend: Shutting down all executors
16/04/13 00:19:06 INFO cluster.SparkDeploySchedulerBackend: Asking each executor to shut down
16/04/13 00:19:07 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
16/04/13 00:19:07 INFO storage.MemoryStore: MemoryStore cleared
16/04/13 00:19:07 INFO storage.BlockManager: BlockManager stopped
16/04/13 00:19:07 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
16/04/13 00:19:08 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
16/04/13 00:19:08 INFO spark.SparkContext: Successfully stopped SparkContext
16/04/13 00:19:08 INFO util.ShutdownHookManager: Shutdown hook called
16/04/13 00:19:08 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-103237c0-b116-4a08-b441-8ea4b3de3e99/httpd-dc709bf3-3487-4c46-9a7e-d2cb3ded5212
16/04/13 00:19:08 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-689fadbe-768f-4099-a0c6-92a55d3f68bc
16/04/13 00:19:08 INFO remote.RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/13 00:19:08 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/04/13 00:19:08 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-103237c0-b116-4a08-b441-8ea4b3de3e99
以上内容是王家林老师DT大数据梦工厂《 IMF传奇行动》第71课的学习笔记。
王家林老师是Spark、Flink、Docker、Android技术中国区布道师。Spark亚太研究院院长和首席专家，DT大数据梦工厂创始人，Android软硬整合源码级专家，英语发音魔术师，健身狂热爱好者。
微信公众账号：DT_Spark
电话：18610086859
QQ:1740415547
微信号：18610086859
新浪微博：ilovepains