Spark SQL and DataFrame for Spark 1.3

Spark SQL and DataFrame Guide

启动Start Shell

[jifeng@feng03 spark-1.3.0-bin-hadoop2.4]$ ./bin/spark-shell 
Spark assembly has been built with Hive, including Datanucleus jars on classpath
log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
15/03/14 10:54:39 INFO SecurityManager: Changing view acls to: jifeng
15/03/14 10:54:39 INFO SecurityManager: Changing modify acls to: jifeng
15/03/14 10:54:39 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(jifeng); users with modify permissions: Set(jifeng)
15/03/14 10:54:39 INFO HttpServer: Starting HTTP Server
15/03/14 10:54:39 INFO Server: jetty-8.y.z-SNAPSHOT
15/03/14 10:54:39 INFO AbstractConnector: Started [email protected]:35435
15/03/14 10:54:39 INFO Utils: Successfully started service 'HTTP class server' on port 35435.
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 1.3.0
      /_/

Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_45)
Type in expressions to have them evaluated.
Type :help for more information.
15/03/14 10:54:42 INFO SparkContext: Running Spark version 1.3.0
15/03/14 10:54:43 INFO SecurityManager: Changing view acls to: jifeng
15/03/14 10:54:43 INFO SecurityManager: Changing modify acls to: jifeng
15/03/14 10:54:43 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(jifeng); users with modify permissions: Set(jifeng)
15/03/14 10:54:43 INFO Slf4jLogger: Slf4jLogger started
15/03/14 10:54:43 INFO Remoting: Starting remoting
15/03/14 10:54:43 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriver@feng03:56764]
15/03/14 10:54:43 INFO Utils: Successfully started service 'sparkDriver' on port 56764.
15/03/14 10:54:43 INFO SparkEnv: Registering MapOutputTracker
15/03/14 10:54:43 INFO SparkEnv: Registering BlockManagerMaster
15/03/14 10:54:43 INFO DiskBlockManager: Created local directory at /tmp/spark-25749b16-5dcf-4728-8c4d-7ade07ff9d97/blockmgr-db96ec22-6fcc-4ccc-b179-79363fe9928c
15/03/14 10:54:43 INFO MemoryStore: MemoryStore started with capacity 265.4 MB
15/03/14 10:54:43 INFO HttpFileServer: HTTP File server directory is /tmp/spark-625015d6-d9ee-4c68-a33e-40cf8caed924/httpd-87a3be78-c156-41d3-8620-285f3cea0221
15/03/14 10:54:43 INFO HttpServer: Starting HTTP Server
15/03/14 10:54:43 INFO Server: jetty-8.y.z-SNAPSHOT
15/03/14 10:54:43 INFO AbstractConnector: Started [email protected]:39473
15/03/14 10:54:43 INFO Utils: Successfully started service 'HTTP file server' on port 39473.
15/03/14 10:54:43 INFO SparkEnv: Registering OutputCommitCoordinator
15/03/14 10:54:43 INFO Server: jetty-8.y.z-SNAPSHOT
15/03/14 10:54:43 INFO AbstractConnector: Started [email protected]:4040
15/03/14 10:54:43 INFO Utils: Successfully started service 'SparkUI' on port 4040.
15/03/14 10:54:43 INFO SparkUI: Started SparkUI at http://feng03:4040
15/03/14 10:54:44 INFO Executor: Starting executor ID  on host localhost
15/03/14 10:54:44 INFO Executor: Using REPL class URI: http://10.6.3.202:35435
15/03/14 10:54:44 INFO AkkaUtils: Connecting to HeartbeatReceiver: akka.tcp://sparkDriver@feng03:56764/user/HeartbeatReceiver
15/03/14 10:54:44 INFO NettyBlockTransferService: Server created on 42855
15/03/14 10:54:44 INFO BlockManagerMaster: Trying to register BlockManager
15/03/14 10:54:44 INFO BlockManagerMasterActor: Registering block manager localhost:42855 with 265.4 MB RAM, BlockManagerId(, localhost, 42855)
15/03/14 10:54:44 INFO BlockManagerMaster: Registered BlockManager
15/03/14 10:54:44 INFO SparkILoop: Created spark context..
Spark context available as sc.
15/03/14 10:54:44 INFO SparkILoop: Created sql context (with Hive support)..
SQL context available as sqlContext.

scala> 

Creating DataFrames

val sc: SparkContext // An existing SparkContext.
val sqlContext = new org.apache.spark.sql.SQLContext(sc)

val df = sqlContext.jsonFile("examples/src/main/resources/people.json")

// Displays the content of the DataFrame to stdout
df.show()

scala> val sqlContext = new org.apache.spark.sql.SQLContext(sc)
sqlContext: org.apache.spark.sql.SQLContext = org.apache.spark.sql.SQLContext@692d5cf2

scala> val df = sqlContext.jsonFile("examples/src/main/resources/people.json")
15/03/14 11:04:37 INFO MemoryStore: ensureFreeSpace(163705) called with curMem=0, maxMem=278302556
15/03/14 11:04:37 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 159.9 KB, free 265.3 MB)
15/03/14 11:04:37 INFO MemoryStore: ensureFreeSpace(22692) called with curMem=163705, maxMem=278302556
15/03/14 11:04:37 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 22.2 KB, free 265.2 MB)
15/03/14 11:04:37 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on localhost:59982 (size: 22.2 KB, free: 265.4 MB)
15/03/14 11:04:37 INFO BlockManagerMaster: Updated info of block broadcast_0_piece0
15/03/14 11:04:37 INFO SparkContext: Created broadcast 0 from textFile at JSONRelation.scala:98
15/03/14 11:04:37 INFO FileInputFormat: Total input paths to process : 1
15/03/14 11:04:37 INFO SparkContext: Starting job: reduce at JsonRDD.scala:51
15/03/14 11:04:37 INFO DAGScheduler: Got job 0 (reduce at JsonRDD.scala:51) with 2 output partitions (allowLocal=false)
15/03/14 11:04:37 INFO DAGScheduler: Final stage: Stage 0(reduce at JsonRDD.scala:51)
15/03/14 11:04:37 INFO DAGScheduler: Parents of final stage: List()
15/03/14 11:04:37 INFO DAGScheduler: Missing parents: List()
15/03/14 11:04:37 INFO DAGScheduler: Submitting Stage 0 (MapPartitionsRDD[3] at map at JsonRDD.scala:51), which has no missing parents
15/03/14 11:04:37 INFO MemoryStore: ensureFreeSpace(3200) called with curMem=186397, maxMem=278302556
15/03/14 11:04:37 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 3.1 KB, free 265.2 MB)
15/03/14 11:04:37 INFO MemoryStore: ensureFreeSpace(2279) called with curMem=189597, maxMem=278302556
15/03/14 11:04:37 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 2.2 KB, free 265.2 MB)
15/03/14 11:04:37 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on localhost:59982 (size: 2.2 KB, free: 265.4 MB)
15/03/14 11:04:37 INFO BlockManagerMaster: Updated info of block broadcast_1_piece0
15/03/14 11:04:37 INFO SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:839
15/03/14 11:04:37 INFO DAGScheduler: Submitting 2 missing tasks from Stage 0 (MapPartitionsRDD[3] at map at JsonRDD.scala:51)
15/03/14 11:04:37 INFO TaskSchedulerImpl: Adding task set 0.0 with 2 tasks
15/03/14 11:04:37 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, localhost, PROCESS_LOCAL, 1354 bytes)
15/03/14 11:04:37 INFO TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, localhost, PROCESS_LOCAL, 1354 bytes)
15/03/14 11:04:37 INFO Executor: Running task 0.0 in stage 0.0 (TID 0)
15/03/14 11:04:37 INFO Executor: Running task 1.0 in stage 0.0 (TID 1)
15/03/14 11:04:37 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:36+37
15/03/14 11:04:37 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:0+36
15/03/14 11:04:37 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
15/03/14 11:04:37 INFO deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
15/03/14 11:04:37 INFO deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap
15/03/14 11:04:37 INFO deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
15/03/14 11:04:37 INFO deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id
15/03/14 11:04:37 INFO Executor: Finished task 1.0 in stage 0.0 (TID 1). 2092 bytes result sent to driver
15/03/14 11:04:37 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 2092 bytes result sent to driver
15/03/14 11:04:37 INFO TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 221 ms on localhost (1/2)
15/03/14 11:04:37 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 238 ms on localhost (2/2)
15/03/14 11:04:37 INFO DAGScheduler: Stage 0 (reduce at JsonRDD.scala:51) finished in 0.252 s
15/03/14 11:04:37 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool 
15/03/14 11:04:37 INFO DAGScheduler: Job 0 finished: reduce at JsonRDD.scala:51, took 0.361336 s
df: org.apache.spark.sql.DataFrame = [age: bigint, name: string]

scala> df.show()
15/03/14 11:04:45 INFO MemoryStore: ensureFreeSpace(81443) called with curMem=191876, maxMem=278302556
15/03/14 11:04:45 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 79.5 KB, free 265.1 MB)
15/03/14 11:04:45 INFO MemoryStore: ensureFreeSpace(31262) called with curMem=273319, maxMem=278302556
15/03/14 11:04:45 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 30.5 KB, free 265.1 MB)
15/03/14 11:04:45 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on localhost:59982 (size: 30.5 KB, free: 265.4 MB)
15/03/14 11:04:45 INFO BlockManagerMaster: Updated info of block broadcast_2_piece0
15/03/14 11:04:45 INFO SparkContext: Created broadcast 2 from textFile at JSONRelation.scala:98
15/03/14 11:04:45 INFO FileInputFormat: Total input paths to process : 1
15/03/14 11:04:45 INFO SparkContext: Starting job: runJob at SparkPlan.scala:121
15/03/14 11:04:45 INFO DAGScheduler: Got job 1 (runJob at SparkPlan.scala:121) with 1 output partitions (allowLocal=false)
15/03/14 11:04:45 INFO DAGScheduler: Final stage: Stage 1(runJob at SparkPlan.scala:121)
15/03/14 11:04:45 INFO DAGScheduler: Parents of final stage: List()
15/03/14 11:04:45 INFO DAGScheduler: Missing parents: List()
15/03/14 11:04:45 INFO DAGScheduler: Submitting Stage 1 (MapPartitionsRDD[8] at map at SparkPlan.scala:96), which has no missing parents
15/03/14 11:04:45 INFO MemoryStore: ensureFreeSpace(4048) called with curMem=304581, maxMem=278302556
15/03/14 11:04:45 INFO MemoryStore: Block broadcast_3 stored as values in memory (estimated size 4.0 KB, free 265.1 MB)
15/03/14 11:04:45 INFO MemoryStore: ensureFreeSpace(2764) called with curMem=308629, maxMem=278302556
15/03/14 11:04:45 INFO MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 2.7 KB, free 265.1 MB)
15/03/14 11:04:45 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on localhost:59982 (size: 2.7 KB, free: 265.4 MB)
15/03/14 11:04:45 INFO BlockManagerMaster: Updated info of block broadcast_3_piece0
15/03/14 11:04:45 INFO SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:839
15/03/14 11:04:45 INFO DAGScheduler: Submitting 1 missing tasks from Stage 1 (MapPartitionsRDD[8] at map at SparkPlan.scala:96)
15/03/14 11:04:45 INFO TaskSchedulerImpl: Adding task set 1.0 with 1 tasks
15/03/14 11:04:45 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 2, localhost, PROCESS_LOCAL, 1354 bytes)
15/03/14 11:04:45 INFO Executor: Running task 0.0 in stage 1.0 (TID 2)
15/03/14 11:04:45 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:0+36
15/03/14 11:04:45 INFO Executor: Finished task 0.0 in stage 1.0 (TID 2). 2055 bytes result sent to driver
15/03/14 11:04:45 INFO DAGScheduler: Stage 1 (runJob at SparkPlan.scala:121) finished in 0.026 s
15/03/14 11:04:45 INFO TaskSetManager: Finished task 0.0 in stage 1.0 (TID 2) in 25 ms on localhost (1/1)
15/03/14 11:04:45 INFO TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool 
15/03/14 11:04:45 INFO DAGScheduler: Job 1 finished: runJob at SparkPlan.scala:121, took 0.038548 s
15/03/14 11:04:45 INFO SparkContext: Starting job: runJob at SparkPlan.scala:121
15/03/14 11:04:45 INFO DAGScheduler: Got job 2 (runJob at SparkPlan.scala:121) with 1 output partitions (allowLocal=false)
15/03/14 11:04:45 INFO DAGScheduler: Final stage: Stage 2(runJob at SparkPlan.scala:121)
15/03/14 11:04:45 INFO DAGScheduler: Parents of final stage: List()
15/03/14 11:04:45 INFO DAGScheduler: Missing parents: List()
15/03/14 11:04:45 INFO DAGScheduler: Submitting Stage 2 (MapPartitionsRDD[8] at map at SparkPlan.scala:96), which has no missing parents
15/03/14 11:04:45 INFO MemoryStore: ensureFreeSpace(4048) called with curMem=311393, maxMem=278302556
15/03/14 11:04:45 INFO MemoryStore: Block broadcast_4 stored as values in memory (estimated size 4.0 KB, free 265.1 MB)
15/03/14 11:04:45 INFO MemoryStore: ensureFreeSpace(2764) called with curMem=315441, maxMem=278302556
15/03/14 11:04:45 INFO MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 2.7 KB, free 265.1 MB)
15/03/14 11:04:45 INFO BlockManagerInfo: Added broadcast_4_piece0 in memory on localhost:59982 (size: 2.7 KB, free: 265.4 MB)
15/03/14 11:04:45 INFO BlockManagerMaster: Updated info of block broadcast_4_piece0
15/03/14 11:04:45 INFO SparkContext: Created broadcast 4 from broadcast at DAGScheduler.scala:839
15/03/14 11:04:45 INFO DAGScheduler: Submitting 1 missing tasks from Stage 2 (MapPartitionsRDD[8] at map at SparkPlan.scala:96)
15/03/14 11:04:45 INFO TaskSchedulerImpl: Adding task set 2.0 with 1 tasks
15/03/14 11:04:45 INFO TaskSetManager: Starting task 0.0 in stage 2.0 (TID 3, localhost, PROCESS_LOCAL, 1354 bytes)
15/03/14 11:04:45 INFO Executor: Running task 0.0 in stage 2.0 (TID 3)
15/03/14 11:04:45 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:36+37
15/03/14 11:04:45 INFO Executor: Finished task 0.0 in stage 2.0 (TID 3). 2030 bytes result sent to driver
15/03/14 11:04:45 INFO TaskSetManager: Finished task 0.0 in stage 2.0 (TID 3) in 15 ms on localhost (1/1)
15/03/14 11:04:45 INFO DAGScheduler: Stage 2 (runJob at SparkPlan.scala:121) finished in 0.015 s
15/03/14 11:04:45 INFO TaskSchedulerImpl: Removed TaskSet 2.0, whose tasks have all completed, from pool 
15/03/14 11:04:45 INFO DAGScheduler: Job 2 finished: runJob at SparkPlan.scala:121, took 0.025898 s
age  name   
null Michael
30   Andy   
19   Justin 

scala> 

DataFrame Operations

val sc: SparkContext // An existing SparkContext.
val sqlContext = new org.apache.spark.sql.SQLContext(sc)

// Create the DataFrame
val df = sqlContext.jsonFile("examples/src/main/resources/people.json")

// Show the content of the DataFrame
df.show()
// age  name   
// null Michael
// 30   Andy   
// 19   Justin 

// Print the schema in a tree format
df.printSchema()
// root
// |-- age: long (nullable = true)
// |-- name: string (nullable = true)

// Select only the "name" column
df.select("name").show()
// name   
// Michael
// Andy   
// Justin 

// Select everybody, but increment the age by 1
df.select("name", df("age") + 1).show()
// name    (age + 1)
// Michael null     
// Andy    31       
// Justin  20       

// Select people older than 21
df.filter(df("name") > 21).show()
// age name
// 30  Andy

// Count people by age
df.groupBy("age").count().show()
// age  count
// null 1
// 19   1
// 30   1

scala> df.printSchema()
root
 |-- age: long (nullable = true)
 |-- name: string (nullable = true)


scala> df.select("name").show()
15/03/14 11:06:25 INFO MemoryStore: ensureFreeSpace(215230) called with curMem=318205, maxMem=278302556
15/03/14 11:06:25 INFO MemoryStore: Block broadcast_5 stored as values in memory (estimated size 210.2 KB, free 264.9 MB)
15/03/14 11:06:25 INFO MemoryStore: ensureFreeSpace(31262) called with curMem=533435, maxMem=278302556
15/03/14 11:06:25 INFO MemoryStore: Block broadcast_5_piece0 stored as bytes in memory (estimated size 30.5 KB, free 264.9 MB)
15/03/14 11:06:25 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on localhost:59982 (size: 30.5 KB, free: 265.3 MB)
15/03/14 11:06:25 INFO BlockManagerMaster: Updated info of block broadcast_5_piece0
15/03/14 11:06:25 INFO SparkContext: Created broadcast 5 from textFile at JSONRelation.scala:98
15/03/14 11:06:25 INFO FileInputFormat: Total input paths to process : 1
15/03/14 11:06:25 INFO SparkContext: Starting job: runJob at SparkPlan.scala:121
15/03/14 11:06:25 INFO DAGScheduler: Got job 3 (runJob at SparkPlan.scala:121) with 1 output partitions (allowLocal=false)
15/03/14 11:06:25 INFO DAGScheduler: Final stage: Stage 3(runJob at SparkPlan.scala:121)
15/03/14 11:06:25 INFO DAGScheduler: Parents of final stage: List()
15/03/14 11:06:25 INFO DAGScheduler: Missing parents: List()
15/03/14 11:06:25 INFO DAGScheduler: Submitting Stage 3 (MapPartitionsRDD[14] at map at SparkPlan.scala:96), which has no missing parents
15/03/14 11:06:25 INFO MemoryStore: ensureFreeSpace(5048) called with curMem=564697, maxMem=278302556
15/03/14 11:06:25 INFO MemoryStore: Block broadcast_6 stored as values in memory (estimated size 4.9 KB, free 264.9 MB)
15/03/14 11:06:25 INFO MemoryStore: ensureFreeSpace(3461) called with curMem=569745, maxMem=278302556
15/03/14 11:06:25 INFO MemoryStore: Block broadcast_6_piece0 stored as bytes in memory (estimated size 3.4 KB, free 264.9 MB)
15/03/14 11:06:25 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on localhost:59982 (size: 3.4 KB, free: 265.3 MB)
15/03/14 11:06:25 INFO BlockManagerMaster: Updated info of block broadcast_6_piece0
15/03/14 11:06:25 INFO SparkContext: Created broadcast 6 from broadcast at DAGScheduler.scala:839
15/03/14 11:06:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 3 (MapPartitionsRDD[14] at map at SparkPlan.scala:96)
15/03/14 11:06:25 INFO TaskSchedulerImpl: Adding task set 3.0 with 1 tasks
15/03/14 11:06:25 INFO TaskSetManager: Starting task 0.0 in stage 3.0 (TID 4, localhost, PROCESS_LOCAL, 1354 bytes)
15/03/14 11:06:25 INFO Executor: Running task 0.0 in stage 3.0 (TID 4)
15/03/14 11:06:25 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:0+36
15/03/14 11:06:25 INFO Executor: Finished task 0.0 in stage 3.0 (TID 4). 1973 bytes result sent to driver
15/03/14 11:06:25 INFO DAGScheduler: Stage 3 (runJob at SparkPlan.scala:121) finished in 0.021 s
15/03/14 11:06:25 INFO TaskSetManager: Finished task 0.0 in stage 3.0 (TID 4) in 20 ms on localhost (1/1)
15/03/14 11:06:25 INFO TaskSchedulerImpl: Removed TaskSet 3.0, whose tasks have all completed, from pool 
15/03/14 11:06:25 INFO DAGScheduler: Job 3 finished: runJob at SparkPlan.scala:121, took 0.032207 s
15/03/14 11:06:25 INFO SparkContext: Starting job: runJob at SparkPlan.scala:121
15/03/14 11:06:25 INFO DAGScheduler: Got job 4 (runJob at SparkPlan.scala:121) with 1 output partitions (allowLocal=false)
15/03/14 11:06:25 INFO DAGScheduler: Final stage: Stage 4(runJob at SparkPlan.scala:121)
15/03/14 11:06:25 INFO DAGScheduler: Parents of final stage: List()
15/03/14 11:06:25 INFO DAGScheduler: Missing parents: List()
15/03/14 11:06:25 INFO DAGScheduler: Submitting Stage 4 (MapPartitionsRDD[14] at map at SparkPlan.scala:96), which has no missing parents
15/03/14 11:06:25 INFO MemoryStore: ensureFreeSpace(5048) called with curMem=573206, maxMem=278302556
15/03/14 11:06:25 INFO MemoryStore: Block broadcast_7 stored as values in memory (estimated size 4.9 KB, free 264.9 MB)
15/03/14 11:06:25 INFO MemoryStore: ensureFreeSpace(3461) called with curMem=578254, maxMem=278302556
15/03/14 11:06:25 INFO MemoryStore: Block broadcast_7_piece0 stored as bytes in memory (estimated size 3.4 KB, free 264.9 MB)
15/03/14 11:06:25 INFO BlockManagerInfo: Added broadcast_7_piece0 in memory on localhost:59982 (size: 3.4 KB, free: 265.3 MB)
15/03/14 11:06:25 INFO BlockManagerMaster: Updated info of block broadcast_7_piece0
15/03/14 11:06:25 INFO SparkContext: Created broadcast 7 from broadcast at DAGScheduler.scala:839
15/03/14 11:06:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 4 (MapPartitionsRDD[14] at map at SparkPlan.scala:96)
15/03/14 11:06:25 INFO TaskSchedulerImpl: Adding task set 4.0 with 1 tasks
15/03/14 11:06:25 INFO TaskSetManager: Starting task 0.0 in stage 4.0 (TID 5, localhost, PROCESS_LOCAL, 1354 bytes)
15/03/14 11:06:25 INFO Executor: Running task 0.0 in stage 4.0 (TID 5)
15/03/14 11:06:25 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:36+37
15/03/14 11:06:25 INFO Executor: Finished task 0.0 in stage 4.0 (TID 5). 1949 bytes result sent to driver
15/03/14 11:06:25 INFO TaskSetManager: Finished task 0.0 in stage 4.0 (TID 5) in 15 ms on localhost (1/1)
15/03/14 11:06:25 INFO DAGScheduler: Stage 4 (runJob at SparkPlan.scala:121) finished in 0.015 s
15/03/14 11:06:25 INFO TaskSchedulerImpl: Removed TaskSet 4.0, whose tasks have all completed, from pool 
15/03/14 11:06:25 INFO DAGScheduler: Job 4 finished: runJob at SparkPlan.scala:121, took 0.025134 s
name   
Michael
Andy   
Justin 

scala> df.select("name", df("age") + 1).show()
:26: error: overloaded method value select with alternatives:
  (col: String,cols: String*)org.apache.spark.sql.DataFrame 
  (cols: org.apache.spark.sql.Column*)org.apache.spark.sql.DataFrame
 cannot be applied to (String, org.apache.spark.sql.Column)
              df.select("name", df("age") + 1).show()
                 ^

scala> df.select("name",df("age") + 1).show()
:26: error: overloaded method value select with alternatives:
  (col: String,cols: String*)org.apache.spark.sql.DataFrame 
  (cols: org.apache.spark.sql.Column*)org.apache.spark.sql.DataFrame
 cannot be applied to (String, org.apache.spark.sql.Column)
              df.select("name",df("age") + 1).show()
                 ^

scala> df.filter(df("name") > 21).show()
15/03/14 11:08:51 INFO MemoryStore: ensureFreeSpace(215230) called with curMem=581715, maxMem=278302556
15/03/14 11:08:51 INFO MemoryStore: Block broadcast_8 stored as values in memory (estimated size 210.2 KB, free 264.6 MB)
15/03/14 11:08:51 INFO MemoryStore: ensureFreeSpace(31262) called with curMem=796945, maxMem=278302556
15/03/14 11:08:51 INFO MemoryStore: Block broadcast_8_piece0 stored as bytes in memory (estimated size 30.5 KB, free 264.6 MB)
15/03/14 11:08:51 INFO BlockManagerInfo: Added broadcast_8_piece0 in memory on localhost:59982 (size: 30.5 KB, free: 265.3 MB)
15/03/14 11:08:51 INFO BlockManagerMaster: Updated info of block broadcast_8_piece0
15/03/14 11:08:51 INFO SparkContext: Created broadcast 8 from textFile at JSONRelation.scala:98
15/03/14 11:08:51 INFO FileInputFormat: Total input paths to process : 1
15/03/14 11:08:51 INFO SparkContext: Starting job: runJob at SparkPlan.scala:121
15/03/14 11:08:51 INFO DAGScheduler: Got job 5 (runJob at SparkPlan.scala:121) with 1 output partitions (allowLocal=false)
15/03/14 11:08:51 INFO DAGScheduler: Final stage: Stage 5(runJob at SparkPlan.scala:121)
15/03/14 11:08:51 INFO DAGScheduler: Parents of final stage: List()
15/03/14 11:08:51 INFO DAGScheduler: Missing parents: List()
15/03/14 11:08:51 INFO DAGScheduler: Submitting Stage 5 (MapPartitionsRDD[20] at map at SparkPlan.scala:96), which has no missing parents
15/03/14 11:08:51 INFO MemoryStore: ensureFreeSpace(5456) called with curMem=828207, maxMem=278302556
15/03/14 11:08:51 INFO MemoryStore: Block broadcast_9 stored as values in memory (estimated size 5.3 KB, free 264.6 MB)
15/03/14 11:08:51 INFO MemoryStore: ensureFreeSpace(3728) called with curMem=833663, maxMem=278302556
15/03/14 11:08:51 INFO MemoryStore: Block broadcast_9_piece0 stored as bytes in memory (estimated size 3.6 KB, free 264.6 MB)
15/03/14 11:08:51 INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on localhost:59982 (size: 3.6 KB, free: 265.3 MB)
15/03/14 11:08:51 INFO BlockManagerMaster: Updated info of block broadcast_9_piece0
15/03/14 11:08:51 INFO SparkContext: Created broadcast 9 from broadcast at DAGScheduler.scala:839
15/03/14 11:08:51 INFO DAGScheduler: Submitting 1 missing tasks from Stage 5 (MapPartitionsRDD[20] at map at SparkPlan.scala:96)
15/03/14 11:08:51 INFO TaskSchedulerImpl: Adding task set 5.0 with 1 tasks
15/03/14 11:08:51 INFO TaskSetManager: Starting task 0.0 in stage 5.0 (TID 6, localhost, PROCESS_LOCAL, 1354 bytes)
15/03/14 11:08:51 INFO Executor: Running task 0.0 in stage 5.0 (TID 6)
15/03/14 11:08:51 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:0+36
15/03/14 11:08:51 INFO Executor: Finished task 0.0 in stage 5.0 (TID 6). 1800 bytes result sent to driver
15/03/14 11:08:51 INFO TaskSetManager: Finished task 0.0 in stage 5.0 (TID 6) in 17 ms on localhost (1/1)
15/03/14 11:08:51 INFO DAGScheduler: Stage 5 (runJob at SparkPlan.scala:121) finished in 0.018 s
15/03/14 11:08:51 INFO TaskSchedulerImpl: Removed TaskSet 5.0, whose tasks have all completed, from pool 
15/03/14 11:08:51 INFO DAGScheduler: Job 5 finished: runJob at SparkPlan.scala:121, took 0.029776 s
15/03/14 11:08:51 INFO SparkContext: Starting job: runJob at SparkPlan.scala:121
15/03/14 11:08:51 INFO DAGScheduler: Got job 6 (runJob at SparkPlan.scala:121) with 1 output partitions (allowLocal=false)
15/03/14 11:08:51 INFO DAGScheduler: Final stage: Stage 6(runJob at SparkPlan.scala:121)
15/03/14 11:08:51 INFO DAGScheduler: Parents of final stage: List()
15/03/14 11:08:51 INFO DAGScheduler: Missing parents: List()
15/03/14 11:08:51 INFO DAGScheduler: Submitting Stage 6 (MapPartitionsRDD[20] at map at SparkPlan.scala:96), which has no missing parents
15/03/14 11:08:51 INFO MemoryStore: ensureFreeSpace(5456) called with curMem=837391, maxMem=278302556
15/03/14 11:08:51 INFO MemoryStore: Block broadcast_10 stored as values in memory (estimated size 5.3 KB, free 264.6 MB)
15/03/14 11:08:51 INFO MemoryStore: ensureFreeSpace(3728) called with curMem=842847, maxMem=278302556
15/03/14 11:08:51 INFO MemoryStore: Block broadcast_10_piece0 stored as bytes in memory (estimated size 3.6 KB, free 264.6 MB)
15/03/14 11:08:51 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on localhost:59982 (size: 3.6 KB, free: 265.3 MB)
15/03/14 11:08:51 INFO BlockManagerMaster: Updated info of block broadcast_10_piece0
15/03/14 11:08:51 INFO SparkContext: Created broadcast 10 from broadcast at DAGScheduler.scala:839
15/03/14 11:08:51 INFO DAGScheduler: Submitting 1 missing tasks from Stage 6 (MapPartitionsRDD[20] at map at SparkPlan.scala:96)
15/03/14 11:08:51 INFO TaskSchedulerImpl: Adding task set 6.0 with 1 tasks
15/03/14 11:08:51 INFO TaskSetManager: Starting task 0.0 in stage 6.0 (TID 7, localhost, PROCESS_LOCAL, 1354 bytes)
15/03/14 11:08:51 INFO Executor: Running task 0.0 in stage 6.0 (TID 7)
15/03/14 11:08:51 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:36+37
15/03/14 11:08:51 INFO Executor: Finished task 0.0 in stage 6.0 (TID 7). 1800 bytes result sent to driver
15/03/14 11:08:51 INFO TaskSetManager: Finished task 0.0 in stage 6.0 (TID 7) in 15 ms on localhost (1/1)
15/03/14 11:08:51 INFO TaskSchedulerImpl: Removed TaskSet 6.0, whose tasks have all completed, from pool 
15/03/14 11:08:51 INFO DAGScheduler: Stage 6 (runJob at SparkPlan.scala:121) finished in 0.015 s
15/03/14 11:08:51 INFO DAGScheduler: Job 6 finished: runJob at SparkPlan.scala:121, took 0.026000 s
age name

scala> df.groupBy("age").count().show()
15/03/14 11:15:49 INFO MemoryStore: ensureFreeSpace(215230) called with curMem=846575, maxMem=278302556
15/03/14 11:15:49 INFO MemoryStore: Block broadcast_11 stored as values in memory (estimated size 210.2 KB, free 264.4 MB)
15/03/14 11:15:49 INFO MemoryStore: ensureFreeSpace(31262) called with curMem=1061805, maxMem=278302556
15/03/14 11:15:49 INFO MemoryStore: Block broadcast_11_piece0 stored as bytes in memory (estimated size 30.5 KB, free 264.4 MB)
15/03/14 11:15:49 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on localhost:59982 (size: 30.5 KB, free: 265.2 MB)
15/03/14 11:15:49 INFO BlockManagerMaster: Updated info of block broadcast_11_piece0
15/03/14 11:15:49 INFO SparkContext: Created broadcast 11 from textFile at JSONRelation.scala:98
15/03/14 11:15:49 INFO SparkContext: Starting job: runJob at SparkPlan.scala:121
15/03/14 11:15:49 INFO FileInputFormat: Total input paths to process : 1
15/03/14 11:15:49 INFO DAGScheduler: Registering RDD 27 (mapPartitions at Exchange.scala:64)
15/03/14 11:15:49 INFO DAGScheduler: Got job 7 (runJob at SparkPlan.scala:121) with 1 output partitions (allowLocal=false)
15/03/14 11:15:49 INFO DAGScheduler: Final stage: Stage 8(runJob at SparkPlan.scala:121)
15/03/14 11:15:49 INFO DAGScheduler: Parents of final stage: List(Stage 7)
15/03/14 11:15:49 INFO DAGScheduler: Missing parents: List(Stage 7)
15/03/14 11:15:49 INFO DAGScheduler: Submitting Stage 7 (MapPartitionsRDD[27] at mapPartitions at Exchange.scala:64), which has no missing parents
15/03/14 11:15:49 INFO MemoryStore: ensureFreeSpace(8952) called with curMem=1093067, maxMem=278302556
15/03/14 11:15:49 INFO MemoryStore: Block broadcast_12 stored as values in memory (estimated size 8.7 KB, free 264.4 MB)
15/03/14 11:15:49 INFO MemoryStore: ensureFreeSpace(5822) called with curMem=1102019, maxMem=278302556
15/03/14 11:15:49 INFO MemoryStore: Block broadcast_12_piece0 stored as bytes in memory (estimated size 5.7 KB, free 264.4 MB)
15/03/14 11:15:49 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on localhost:59982 (size: 5.7 KB, free: 265.2 MB)
15/03/14 11:15:49 INFO BlockManagerMaster: Updated info of block broadcast_12_piece0
15/03/14 11:15:49 INFO SparkContext: Created broadcast 12 from broadcast at DAGScheduler.scala:839
15/03/14 11:15:49 INFO DAGScheduler: Submitting 2 missing tasks from Stage 7 (MapPartitionsRDD[27] at mapPartitions at Exchange.scala:64)
15/03/14 11:15:49 INFO TaskSchedulerImpl: Adding task set 7.0 with 2 tasks
15/03/14 11:15:49 INFO TaskSetManager: Starting task 0.0 in stage 7.0 (TID 8, localhost, PROCESS_LOCAL, 1343 bytes)
15/03/14 11:15:49 INFO TaskSetManager: Starting task 1.0 in stage 7.0 (TID 9, localhost, PROCESS_LOCAL, 1343 bytes)
15/03/14 11:15:49 INFO Executor: Running task 0.0 in stage 7.0 (TID 8)
15/03/14 11:15:49 INFO Executor: Running task 1.0 in stage 7.0 (TID 9)
15/03/14 11:15:49 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:0+36
15/03/14 11:15:49 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:36+37
15/03/14 11:15:50 INFO Executor: Finished task 0.0 in stage 7.0 (TID 8). 2205 bytes result sent to driver
15/03/14 11:15:50 INFO Executor: Finished task 1.0 in stage 7.0 (TID 9). 2205 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Finished task 0.0 in stage 7.0 (TID 8) in 851 ms on localhost (1/2)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 1.0 in stage 7.0 (TID 9) in 850 ms on localhost (2/2)
15/03/14 11:15:50 INFO TaskSchedulerImpl: Removed TaskSet 7.0, whose tasks have all completed, from pool 
15/03/14 11:15:50 INFO DAGScheduler: Stage 7 (mapPartitions at Exchange.scala:64) finished in 0.852 s
15/03/14 11:15:50 INFO DAGScheduler: looking for newly runnable stages
15/03/14 11:15:50 INFO DAGScheduler: running: Set()
15/03/14 11:15:50 INFO DAGScheduler: waiting: Set(Stage 8)
15/03/14 11:15:50 INFO DAGScheduler: failed: Set()
15/03/14 11:15:50 INFO DAGScheduler: Missing parents for Stage 8: List()
15/03/14 11:15:50 INFO DAGScheduler: Submitting Stage 8 (MapPartitionsRDD[31] at map at SparkPlan.scala:96), which is now runnable
15/03/14 11:15:50 INFO MemoryStore: ensureFreeSpace(9736) called with curMem=1107841, maxMem=278302556
15/03/14 11:15:50 INFO MemoryStore: Block broadcast_13 stored as values in memory (estimated size 9.5 KB, free 264.3 MB)
15/03/14 11:15:50 INFO MemoryStore: ensureFreeSpace(6476) called with curMem=1117577, maxMem=278302556
15/03/14 11:15:50 INFO MemoryStore: Block broadcast_13_piece0 stored as bytes in memory (estimated size 6.3 KB, free 264.3 MB)
15/03/14 11:15:50 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on localhost:59982 (size: 6.3 KB, free: 265.2 MB)
15/03/14 11:15:50 INFO BlockManagerMaster: Updated info of block broadcast_13_piece0
15/03/14 11:15:50 INFO SparkContext: Created broadcast 13 from broadcast at DAGScheduler.scala:839
15/03/14 11:15:50 INFO DAGScheduler: Submitting 1 missing tasks from Stage 8 (MapPartitionsRDD[31] at map at SparkPlan.scala:96)
15/03/14 11:15:50 INFO TaskSchedulerImpl: Adding task set 8.0 with 1 tasks
15/03/14 11:15:50 INFO TaskSetManager: Starting task 0.0 in stage 8.0 (TID 10, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 0.0 in stage 8.0 (TID 10)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms
15/03/14 11:15:50 INFO Executor: Finished task 0.0 in stage 8.0 (TID 10). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Finished task 0.0 in stage 8.0 (TID 10) in 43 ms on localhost (1/1)
15/03/14 11:15:50 INFO DAGScheduler: Stage 8 (runJob at SparkPlan.scala:121) finished in 0.043 s
15/03/14 11:15:50 INFO TaskSchedulerImpl: Removed TaskSet 8.0, whose tasks have all completed, from pool 
15/03/14 11:15:50 INFO DAGScheduler: Job 7 finished: runJob at SparkPlan.scala:121, took 1.001202 s
15/03/14 11:15:50 INFO SparkContext: Starting job: runJob at SparkPlan.scala:121
15/03/14 11:15:50 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 167 bytes
15/03/14 11:15:50 INFO DAGScheduler: Got job 8 (runJob at SparkPlan.scala:121) with 199 output partitions (allowLocal=false)
15/03/14 11:15:50 INFO DAGScheduler: Final stage: Stage 10(runJob at SparkPlan.scala:121)
15/03/14 11:15:50 INFO DAGScheduler: Parents of final stage: List(Stage 9)
15/03/14 11:15:50 INFO DAGScheduler: Missing parents: List()
15/03/14 11:15:50 INFO DAGScheduler: Submitting Stage 10 (MapPartitionsRDD[31] at map at SparkPlan.scala:96), which has no missing parents
15/03/14 11:15:50 INFO MemoryStore: ensureFreeSpace(9736) called with curMem=1124053, maxMem=278302556
15/03/14 11:15:50 INFO MemoryStore: Block broadcast_14 stored as values in memory (estimated size 9.5 KB, free 264.3 MB)
15/03/14 11:15:50 INFO MemoryStore: ensureFreeSpace(6476) called with curMem=1133789, maxMem=278302556
15/03/14 11:15:50 INFO MemoryStore: Block broadcast_14_piece0 stored as bytes in memory (estimated size 6.3 KB, free 264.3 MB)
15/03/14 11:15:50 INFO BlockManagerInfo: Added broadcast_14_piece0 in memory on localhost:59982 (size: 6.3 KB, free: 265.2 MB)
15/03/14 11:15:50 INFO BlockManagerMaster: Updated info of block broadcast_14_piece0
15/03/14 11:15:50 INFO SparkContext: Created broadcast 14 from broadcast at DAGScheduler.scala:839
15/03/14 11:15:50 INFO DAGScheduler: Submitting 199 missing tasks from Stage 10 (MapPartitionsRDD[31] at map at SparkPlan.scala:96)
15/03/14 11:15:50 INFO TaskSchedulerImpl: Adding task set 10.0 with 199 tasks
15/03/14 11:15:50 INFO TaskSetManager: Starting task 0.0 in stage 10.0 (TID 11, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Starting task 1.0 in stage 10.0 (TID 12, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Starting task 2.0 in stage 10.0 (TID 13, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Starting task 3.0 in stage 10.0 (TID 14, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 1.0 in stage 10.0 (TID 12)
15/03/14 11:15:50 INFO Executor: Running task 0.0 in stage 10.0 (TID 11)
15/03/14 11:15:50 INFO Executor: Running task 2.0 in stage 10.0 (TID 13)
15/03/14 11:15:50 INFO Executor: Running task 3.0 in stage 10.0 (TID 14)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO Executor: Finished task 1.0 in stage 10.0 (TID 12). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 4.0 in stage 10.0 (TID 15, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 11 ms
15/03/14 11:15:50 INFO Executor: Running task 4.0 in stage 10.0 (TID 15)
15/03/14 11:15:50 INFO Executor: Finished task 3.0 in stage 10.0 (TID 14). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 5.0 in stage 10.0 (TID 16, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 5.0 in stage 10.0 (TID 16)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 1.0 in stage 10.0 (TID 12) in 30 ms on localhost (1/199)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 3.0 in stage 10.0 (TID 14) in 30 ms on localhost (2/199)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO Executor: Finished task 4.0 in stage 10.0 (TID 15). 894 bytes result sent to driver
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 22 ms
15/03/14 11:15:50 INFO TaskSetManager: Starting task 6.0 in stage 10.0 (TID 17, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 6.0 in stage 10.0 (TID 17)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO Executor: Finished task 5.0 in stage 10.0 (TID 16). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 7.0 in stage 10.0 (TID 18, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 7.0 in stage 10.0 (TID 18)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO Executor: Finished task 6.0 in stage 10.0 (TID 17). 894 bytes result sent to driver
15/03/14 11:15:50 INFO Executor: Finished task 2.0 in stage 10.0 (TID 13). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 8.0 in stage 10.0 (TID 19, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 8.0 in stage 10.0 (TID 19)
15/03/14 11:15:50 INFO TaskSetManager: Starting task 9.0 in stage 10.0 (TID 20, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 9.0 in stage 10.0 (TID 20)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:50 INFO Executor: Finished task 7.0 in stage 10.0 (TID 18). 894 bytes result sent to driver
15/03/14 11:15:50 INFO Executor: Finished task 9.0 in stage 10.0 (TID 20). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 10.0 in stage 10.0 (TID 21, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Starting task 11.0 in stage 10.0 (TID 22, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 10.0 in stage 10.0 (TID 21)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 6.0 in stage 10.0 (TID 17) in 23 ms on localhost (3/199)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 2.0 in stage 10.0 (TID 13) in 60 ms on localhost (4/199)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 7.0 in stage 10.0 (TID 18) in 19 ms on localhost (5/199)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 9.0 in stage 10.0 (TID 20) in 13 ms on localhost (6/199)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 5.0 in stage 10.0 (TID 16) in 36 ms on localhost (7/199)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 4.0 in stage 10.0 (TID 15) in 38 ms on localhost (8/199)
15/03/14 11:15:50 INFO Executor: Running task 11.0 in stage 10.0 (TID 22)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO Executor: Finished task 10.0 in stage 10.0 (TID 21). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 12.0 in stage 10.0 (TID 23, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 10.0 in stage 10.0 (TID 21) in 13 ms on localhost (9/199)
15/03/14 11:15:50 INFO Executor: Running task 12.0 in stage 10.0 (TID 23)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:50 INFO Executor: Finished task 8.0 in stage 10.0 (TID 19). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 13.0 in stage 10.0 (TID 24, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 8.0 in stage 10.0 (TID 19) in 32 ms on localhost (10/199)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO Executor: Running task 13.0 in stage 10.0 (TID 24)
15/03/14 11:15:50 INFO Executor: Finished task 12.0 in stage 10.0 (TID 23). 894 bytes result sent to driver
15/03/14 11:15:50 INFO Executor: Finished task 11.0 in stage 10.0 (TID 22). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 14.0 in stage 10.0 (TID 25, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 12.0 in stage 10.0 (TID 23) in 12 ms on localhost (11/199)
15/03/14 11:15:50 INFO TaskSetManager: Starting task 15.0 in stage 10.0 (TID 26, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 15.0 in stage 10.0 (TID 26)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 11.0 in stage 10.0 (TID 22) in 27 ms on localhost (12/199)
15/03/14 11:15:50 INFO Executor: Running task 14.0 in stage 10.0 (TID 25)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO Executor: Finished task 13.0 in stage 10.0 (TID 24). 894 bytes result sent to driver
15/03/14 11:15:50 INFO Executor: Finished task 15.0 in stage 10.0 (TID 26). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 16.0 in stage 10.0 (TID 27, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 13.0 in stage 10.0 (TID 24) in 16 ms on localhost (13/199)
15/03/14 11:15:50 INFO Executor: Running task 16.0 in stage 10.0 (TID 27)
15/03/14 11:15:50 INFO TaskSetManager: Starting task 17.0 in stage 10.0 (TID 28, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 15.0 in stage 10.0 (TID 26) in 11 ms on localhost (14/199)
15/03/14 11:15:50 INFO Executor: Running task 17.0 in stage 10.0 (TID 28)
15/03/14 11:15:50 INFO Executor: Finished task 14.0 in stage 10.0 (TID 25). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 18.0 in stage 10.0 (TID 29, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 18.0 in stage 10.0 (TID 29)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 14.0 in stage 10.0 (TID 25) in 15 ms on localhost (15/199)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO Executor: Finished task 16.0 in stage 10.0 (TID 27). 894 bytes result sent to driver
15/03/14 11:15:50 INFO Executor: Finished task 0.0 in stage 10.0 (TID 11). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 19.0 in stage 10.0 (TID 30, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Starting task 20.0 in stage 10.0 (TID 31, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Finished task 17.0 in stage 10.0 (TID 28). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Finished task 0.0 in stage 10.0 (TID 11) in 107 ms on localhost (16/199)
15/03/14 11:15:50 INFO Executor: Running task 20.0 in stage 10.0 (TID 31)
15/03/14 11:15:50 INFO TaskSetManager: Starting task 21.0 in stage 10.0 (TID 32, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 17.0 in stage 10.0 (TID 28) in 14 ms on localhost (17/199)
15/03/14 11:15:50 INFO Executor: Running task 21.0 in stage 10.0 (TID 32)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO Executor: Finished task 20.0 in stage 10.0 (TID 31). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 22.0 in stage 10.0 (TID 33, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 19.0 in stage 10.0 (TID 30)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 16.0 in stage 10.0 (TID 27) in 25 ms on localhost (18/199)
15/03/14 11:15:50 INFO Executor: Finished task 21.0 in stage 10.0 (TID 32). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Finished task 20.0 in stage 10.0 (TID 31) in 13 ms on localhost (19/199)
15/03/14 11:15:50 INFO Executor: Running task 22.0 in stage 10.0 (TID 33)
15/03/14 11:15:50 INFO TaskSetManager: Starting task 23.0 in stage 10.0 (TID 34, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 23.0 in stage 10.0 (TID 34)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 21.0 in stage 10.0 (TID 32) in 13 ms on localhost (20/199)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO Executor: Finished task 23.0 in stage 10.0 (TID 34). 894 bytes result sent to driver
15/03/14 11:15:50 INFO Executor: Finished task 22.0 in stage 10.0 (TID 33). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 24.0 in stage 10.0 (TID 35, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 24.0 in stage 10.0 (TID 35)
15/03/14 11:15:50 INFO TaskSetManager: Starting task 25.0 in stage 10.0 (TID 36, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 23.0 in stage 10.0 (TID 34) in 12 ms on localhost (21/199)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 22.0 in stage 10.0 (TID 33) in 13 ms on localhost (22/199)
15/03/14 11:15:50 INFO Executor: Running task 25.0 in stage 10.0 (TID 36)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO Executor: Finished task 24.0 in stage 10.0 (TID 35). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 26.0 in stage 10.0 (TID 37, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 24.0 in stage 10.0 (TID 35) in 12 ms on localhost (23/199)
15/03/14 11:15:50 INFO Executor: Running task 26.0 in stage 10.0 (TID 37)
15/03/14 11:15:50 INFO Executor: Finished task 25.0 in stage 10.0 (TID 36). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 27.0 in stage 10.0 (TID 38, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 27.0 in stage 10.0 (TID 38)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 25.0 in stage 10.0 (TID 36) in 23 ms on localhost (24/199)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO Executor: Finished task 26.0 in stage 10.0 (TID 37). 894 bytes result sent to driver
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO TaskSetManager: Starting task 28.0 in stage 10.0 (TID 39, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO TaskSetManager: Finished task 26.0 in stage 10.0 (TID 37) in 21 ms on localhost (25/199)
15/03/14 11:15:50 INFO Executor: Running task 28.0 in stage 10.0 (TID 39)
15/03/14 11:15:50 INFO Executor: Finished task 27.0 in stage 10.0 (TID 38). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 29.0 in stage 10.0 (TID 40, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 29.0 in stage 10.0 (TID 40)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 27.0 in stage 10.0 (TID 38) in 21 ms on localhost (26/199)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO Executor: Finished task 28.0 in stage 10.0 (TID 39). 894 bytes result sent to driver
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO TaskSetManager: Starting task 30.0 in stage 10.0 (TID 41, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 28.0 in stage 10.0 (TID 39) in 12 ms on localhost (27/199)
15/03/14 11:15:50 INFO Executor: Running task 30.0 in stage 10.0 (TID 41)
15/03/14 11:15:50 INFO Executor: Finished task 29.0 in stage 10.0 (TID 40). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 31.0 in stage 10.0 (TID 42, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 31.0 in stage 10.0 (TID 42)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 29.0 in stage 10.0 (TID 40) in 11 ms on localhost (28/199)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO Executor: Finished task 18.0 in stage 10.0 (TID 29). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 32.0 in stage 10.0 (TID 43, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 32.0 in stage 10.0 (TID 43)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 18.0 in stage 10.0 (TID 29) in 87 ms on localhost (29/199)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:50 INFO Executor: Finished task 32.0 in stage 10.0 (TID 43). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 33.0 in stage 10.0 (TID 44, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 32.0 in stage 10.0 (TID 43) in 12 ms on localhost (30/199)
15/03/14 11:15:50 INFO Executor: Finished task 31.0 in stage 10.0 (TID 42). 894 bytes result sent to driver
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO TaskSetManager: Starting task 34.0 in stage 10.0 (TID 45, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 31.0 in stage 10.0 (TID 42) in 23 ms on localhost (31/199)
15/03/14 11:15:50 INFO Executor: Finished task 30.0 in stage 10.0 (TID 41). 894 bytes result sent to driver
15/03/14 11:15:50 INFO Executor: Running task 33.0 in stage 10.0 (TID 44)
15/03/14 11:15:50 INFO Executor: Finished task 19.0 in stage 10.0 (TID 30). 894 bytes result sent to driver
15/03/14 11:15:50 INFO Executor: Running task 34.0 in stage 10.0 (TID 45)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:50 INFO Executor: Finished task 33.0 in stage 10.0 (TID 44). 894 bytes result sent to driver
15/03/14 11:15:50 INFO Executor: Finished task 34.0 in stage 10.0 (TID 45). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 35.0 in stage 10.0 (TID 46, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 30.0 in stage 10.0 (TID 41) in 41 ms on localhost (32/199)
15/03/14 11:15:50 INFO TaskSetManager: Starting task 36.0 in stage 10.0 (TID 47, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 19.0 in stage 10.0 (TID 30) in 106 ms on localhost (33/199)
15/03/14 11:15:50 INFO Executor: Running task 36.0 in stage 10.0 (TID 47)
15/03/14 11:15:50 INFO TaskSetManager: Starting task 37.0 in stage 10.0 (TID 48, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Starting task 38.0 in stage 10.0 (TID 49, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 35.0 in stage 10.0 (TID 46)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO TaskSetManager: Finished task 33.0 in stage 10.0 (TID 44) in 30 ms on localhost (34/199)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 34.0 in stage 10.0 (TID 45) in 29 ms on localhost (35/199)
15/03/14 11:15:50 INFO Executor: Finished task 36.0 in stage 10.0 (TID 47). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 39.0 in stage 10.0 (TID 50, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO Executor: Running task 38.0 in stage 10.0 (TID 49)
15/03/14 11:15:50 INFO Executor: Finished task 35.0 in stage 10.0 (TID 46). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Finished task 36.0 in stage 10.0 (TID 47) in 15 ms on localhost (36/199)
15/03/14 11:15:50 INFO Executor: Running task 39.0 in stage 10.0 (TID 50)
15/03/14 11:15:50 INFO TaskSetManager: Starting task 40.0 in stage 10.0 (TID 51, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 35.0 in stage 10.0 (TID 46) in 29 ms on localhost (37/199)
15/03/14 11:15:50 INFO Executor: Running task 40.0 in stage 10.0 (TID 51)
15/03/14 11:15:50 INFO Executor: Running task 37.0 in stage 10.0 (TID 48)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO Executor: Finished task 37.0 in stage 10.0 (TID 48). 894 bytes result sent to driver
15/03/14 11:15:50 INFO Executor: Finished task 39.0 in stage 10.0 (TID 50). 894 bytes result sent to driver
15/03/14 11:15:50 INFO TaskSetManager: Starting task 41.0 in stage 10.0 (TID 52, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Starting task 42.0 in stage 10.0 (TID 53, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 37.0 in stage 10.0 (TID 48) in 29 ms on localhost (38/199)
15/03/14 11:15:50 INFO TaskSetManager: Finished task 39.0 in stage 10.0 (TID 50) in 16 ms on localhost (39/199)
15/03/14 11:15:50 INFO Executor: Running task 41.0 in stage 10.0 (TID 52)
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:50 INFO Executor: Running task 42.0 in stage 10.0 (TID 53)
15/03/14 11:15:50 INFO Executor: Finished task 38.0 in stage 10.0 (TID 49). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 40.0 in stage 10.0 (TID 51). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 43.0 in stage 10.0 (TID 54, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 44.0 in stage 10.0 (TID 55, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 38.0 in stage 10.0 (TID 49) in 793 ms on localhost (40/199)
15/03/14 11:15:51 INFO Executor: Running task 44.0 in stage 10.0 (TID 55)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 40.0 in stage 10.0 (TID 51) in 778 ms on localhost (41/199)
15/03/14 11:15:51 INFO Executor: Running task 43.0 in stage 10.0 (TID 54)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 41.0 in stage 10.0 (TID 52). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 45.0 in stage 10.0 (TID 56, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 41.0 in stage 10.0 (TID 52) in 778 ms on localhost (42/199)
15/03/14 11:15:51 INFO Executor: Running task 45.0 in stage 10.0 (TID 56)
15/03/14 11:15:51 INFO Executor: Finished task 43.0 in stage 10.0 (TID 54). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 42.0 in stage 10.0 (TID 53). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 46.0 in stage 10.0 (TID 57, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 46.0 in stage 10.0 (TID 57)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 43.0 in stage 10.0 (TID 54) in 16 ms on localhost (43/199)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 47.0 in stage 10.0 (TID 58, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 42.0 in stage 10.0 (TID 53) in 780 ms on localhost (44/199)
15/03/14 11:15:51 INFO Executor: Running task 47.0 in stage 10.0 (TID 58)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 45.0 in stage 10.0 (TID 56). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 46.0 in stage 10.0 (TID 57). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 48.0 in stage 10.0 (TID 59, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 49.0 in stage 10.0 (TID 60, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 45.0 in stage 10.0 (TID 56) in 18 ms on localhost (45/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 46.0 in stage 10.0 (TID 57) in 16 ms on localhost (46/199)
15/03/14 11:15:51 INFO Executor: Running task 49.0 in stage 10.0 (TID 60)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 44.0 in stage 10.0 (TID 55). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Running task 48.0 in stage 10.0 (TID 59)
15/03/14 11:15:51 INFO Executor: Finished task 47.0 in stage 10.0 (TID 58). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 50.0 in stage 10.0 (TID 61, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 44.0 in stage 10.0 (TID 55) in 37 ms on localhost (47/199)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO Executor: Running task 50.0 in stage 10.0 (TID 61)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 51.0 in stage 10.0 (TID 62, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 51.0 in stage 10.0 (TID 62)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 49.0 in stage 10.0 (TID 60). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 52.0 in stage 10.0 (TID 63, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 52.0 in stage 10.0 (TID 63)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO Executor: Finished task 51.0 in stage 10.0 (TID 62). 894 bytes result sent to driver
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO Executor: Finished task 50.0 in stage 10.0 (TID 61). 894 bytes result sent to driver
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO TaskSetManager: Finished task 47.0 in stage 10.0 (TID 58) in 40 ms on localhost (48/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 49.0 in stage 10.0 (TID 60) in 28 ms on localhost (49/199)
15/03/14 11:15:51 INFO Executor: Finished task 52.0 in stage 10.0 (TID 63). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 53.0 in stage 10.0 (TID 64, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 54.0 in stage 10.0 (TID 65, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Finished task 48.0 in stage 10.0 (TID 59). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Running task 54.0 in stage 10.0 (TID 65)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 55.0 in stage 10.0 (TID 66, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 53.0 in stage 10.0 (TID 64)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO TaskSetManager: Finished task 52.0 in stage 10.0 (TID 63) in 36 ms on localhost (50/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 50.0 in stage 10.0 (TID 61) in 44 ms on localhost (51/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 51.0 in stage 10.0 (TID 62) in 44 ms on localhost (52/199)
15/03/14 11:15:51 INFO Executor: Running task 55.0 in stage 10.0 (TID 66)
15/03/14 11:15:51 INFO Executor: Finished task 53.0 in stage 10.0 (TID 64). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 54.0 in stage 10.0 (TID 65). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 56.0 in stage 10.0 (TID 67, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 56.0 in stage 10.0 (TID 67)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 48.0 in stage 10.0 (TID 59) in 58 ms on localhost (53/199)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 57.0 in stage 10.0 (TID 68, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 58.0 in stage 10.0 (TID 69, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 53.0 in stage 10.0 (TID 64) in 33 ms on localhost (54/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 54.0 in stage 10.0 (TID 65) in 35 ms on localhost (55/199)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO Executor: Running task 58.0 in stage 10.0 (TID 69)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO Executor: Finished task 55.0 in stage 10.0 (TID 66). 894 bytes result sent to driver
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Running task 57.0 in stage 10.0 (TID 68)
15/03/14 11:15:51 INFO Executor: Finished task 56.0 in stage 10.0 (TID 67). 894 bytes result sent to driver
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO TaskSetManager: Starting task 59.0 in stage 10.0 (TID 70, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO TaskSetManager: Finished task 55.0 in stage 10.0 (TID 66) in 43 ms on localhost (56/199)
15/03/14 11:15:51 INFO Executor: Running task 59.0 in stage 10.0 (TID 70)
15/03/14 11:15:51 INFO Executor: Finished task 58.0 in stage 10.0 (TID 69). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 60.0 in stage 10.0 (TID 71, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 60.0 in stage 10.0 (TID 71)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 61.0 in stage 10.0 (TID 72, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 61.0 in stage 10.0 (TID 72)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO TaskSetManager: Finished task 58.0 in stage 10.0 (TID 69) in 145 ms on localhost (57/199)
15/03/14 11:15:51 INFO Executor: Finished task 59.0 in stage 10.0 (TID 70). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Finished task 56.0 in stage 10.0 (TID 67) in 150 ms on localhost (58/199)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO TaskSetManager: Starting task 62.0 in stage 10.0 (TID 73, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 62.0 in stage 10.0 (TID 73)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 59.0 in stage 10.0 (TID 70) in 138 ms on localhost (59/199)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms
15/03/14 11:15:51 INFO Executor: Finished task 60.0 in stage 10.0 (TID 71). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 63.0 in stage 10.0 (TID 74, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 63.0 in stage 10.0 (TID 74)
15/03/14 11:15:51 INFO Executor: Finished task 61.0 in stage 10.0 (TID 72). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 64.0 in stage 10.0 (TID 75, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Running task 64.0 in stage 10.0 (TID 75)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 61.0 in stage 10.0 (TID 72) in 35 ms on localhost (60/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 60.0 in stage 10.0 (TID 71) in 37 ms on localhost (61/199)
15/03/14 11:15:51 INFO Executor: Finished task 57.0 in stage 10.0 (TID 68). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 62.0 in stage 10.0 (TID 73). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 65.0 in stage 10.0 (TID 76, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 65.0 in stage 10.0 (TID 76)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 66.0 in stage 10.0 (TID 77, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 57.0 in stage 10.0 (TID 68) in 164 ms on localhost (62/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 62.0 in stage 10.0 (TID 73) in 16 ms on localhost (63/199)
15/03/14 11:15:51 INFO Executor: Running task 66.0 in stage 10.0 (TID 77)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 65.0 in stage 10.0 (TID 76). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 63.0 in stage 10.0 (TID 74). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 64.0 in stage 10.0 (TID 75). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 67.0 in stage 10.0 (TID 78, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO Executor: Running task 67.0 in stage 10.0 (TID 78)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 65.0 in stage 10.0 (TID 76) in 11 ms on localhost (64/199)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 68.0 in stage 10.0 (TID 79, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 68.0 in stage 10.0 (TID 79)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 63.0 in stage 10.0 (TID 74) in 19 ms on localhost (65/199)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 69.0 in stage 10.0 (TID 80, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 69.0 in stage 10.0 (TID 80)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 64.0 in stage 10.0 (TID 75) in 17 ms on localhost (66/199)
15/03/14 11:15:51 INFO Executor: Finished task 66.0 in stage 10.0 (TID 77). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 70.0 in stage 10.0 (TID 81, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 66.0 in stage 10.0 (TID 77) in 15 ms on localhost (67/199)
15/03/14 11:15:51 INFO Executor: Running task 70.0 in stage 10.0 (TID 81)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 69.0 in stage 10.0 (TID 80). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 71.0 in stage 10.0 (TID 82, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 71.0 in stage 10.0 (TID 82)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO TaskSetManager: Finished task 69.0 in stage 10.0 (TID 80) in 11 ms on localhost (68/199)
15/03/14 11:15:51 INFO Executor: Finished task 67.0 in stage 10.0 (TID 78). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 72.0 in stage 10.0 (TID 83, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 72.0 in stage 10.0 (TID 83)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 67.0 in stage 10.0 (TID 78) in 17 ms on localhost (69/199)
15/03/14 11:15:51 INFO Executor: Finished task 70.0 in stage 10.0 (TID 81). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 73.0 in stage 10.0 (TID 84, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 73.0 in stage 10.0 (TID 84)
15/03/14 11:15:51 INFO Executor: Finished task 68.0 in stage 10.0 (TID 79). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 74.0 in stage 10.0 (TID 85, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 68.0 in stage 10.0 (TID 79) in 18 ms on localhost (70/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 70.0 in stage 10.0 (TID 81) in 15 ms on localhost (71/199)
15/03/14 11:15:51 INFO Executor: Running task 74.0 in stage 10.0 (TID 85)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO Executor: Finished task 71.0 in stage 10.0 (TID 82). 894 bytes result sent to driver
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO TaskSetManager: Starting task 75.0 in stage 10.0 (TID 86, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 71.0 in stage 10.0 (TID 82) in 13 ms on localhost (72/199)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Running task 75.0 in stage 10.0 (TID 86)
15/03/14 11:15:51 INFO Executor: Finished task 72.0 in stage 10.0 (TID 83). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 73.0 in stage 10.0 (TID 84). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 74.0 in stage 10.0 (TID 85). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 76.0 in stage 10.0 (TID 87, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 76.0 in stage 10.0 (TID 87)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 77.0 in stage 10.0 (TID 88, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 74.0 in stage 10.0 (TID 85) in 121 ms on localhost (73/199)
15/03/14 11:15:51 INFO Executor: Running task 77.0 in stage 10.0 (TID 88)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 73.0 in stage 10.0 (TID 84) in 123 ms on localhost (74/199)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO Executor: Finished task 75.0 in stage 10.0 (TID 86). 894 bytes result sent to driver
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO TaskSetManager: Starting task 78.0 in stage 10.0 (TID 89, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO TaskSetManager: Starting task 79.0 in stage 10.0 (TID 90, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 79.0 in stage 10.0 (TID 90)
15/03/14 11:15:51 INFO Executor: Running task 78.0 in stage 10.0 (TID 89)
15/03/14 11:15:51 INFO Executor: Finished task 76.0 in stage 10.0 (TID 87). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 77.0 in stage 10.0 (TID 88). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Finished task 72.0 in stage 10.0 (TID 83) in 143 ms on localhost (75/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 75.0 in stage 10.0 (TID 86) in 133 ms on localhost (76/199)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 80.0 in stage 10.0 (TID 91, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 81.0 in stage 10.0 (TID 92, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 80.0 in stage 10.0 (TID 91)
15/03/14 11:15:51 INFO Executor: Running task 81.0 in stage 10.0 (TID 92)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 76.0 in stage 10.0 (TID 87) in 27 ms on localhost (77/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 77.0 in stage 10.0 (TID 88) in 26 ms on localhost (78/199)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 78.0 in stage 10.0 (TID 89). 894 bytes result sent to driver
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO TaskSetManager: Starting task 82.0 in stage 10.0 (TID 93, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Finished task 79.0 in stage 10.0 (TID 90). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Running task 82.0 in stage 10.0 (TID 93)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 78.0 in stage 10.0 (TID 89) in 16 ms on localhost (79/199)
15/03/14 11:15:51 INFO Executor: Finished task 80.0 in stage 10.0 (TID 91). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 83.0 in stage 10.0 (TID 94, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 83.0 in stage 10.0 (TID 94)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 84.0 in stage 10.0 (TID 95, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 79.0 in stage 10.0 (TID 90) in 19 ms on localhost (80/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 80.0 in stage 10.0 (TID 91) in 13 ms on localhost (81/199)
15/03/14 11:15:51 INFO Executor: Running task 84.0 in stage 10.0 (TID 95)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO Executor: Finished task 82.0 in stage 10.0 (TID 93). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 85.0 in stage 10.0 (TID 96, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 85.0 in stage 10.0 (TID 96)
15/03/14 11:15:51 INFO Executor: Finished task 83.0 in stage 10.0 (TID 94). 894 bytes result sent to driver
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO TaskSetManager: Starting task 86.0 in stage 10.0 (TID 97, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Finished task 84.0 in stage 10.0 (TID 95). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 87.0 in stage 10.0 (TID 98, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 86.0 in stage 10.0 (TID 97)
15/03/14 11:15:51 INFO Executor: Running task 87.0 in stage 10.0 (TID 98)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 82.0 in stage 10.0 (TID 93) in 17 ms on localhost (82/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 84.0 in stage 10.0 (TID 95) in 16 ms on localhost (83/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 83.0 in stage 10.0 (TID 94) in 16 ms on localhost (84/199)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 81.0 in stage 10.0 (TID 92). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 87.0 in stage 10.0 (TID 98). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 88.0 in stage 10.0 (TID 99, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 89.0 in stage 10.0 (TID 100, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 81.0 in stage 10.0 (TID 92) in 39 ms on localhost (85/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 87.0 in stage 10.0 (TID 98) in 16 ms on localhost (86/199)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 85.0 in stage 10.0 (TID 96). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 86.0 in stage 10.0 (TID 97). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Running task 89.0 in stage 10.0 (TID 100)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 90.0 in stage 10.0 (TID 101, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 91.0 in stage 10.0 (TID 102, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 91.0 in stage 10.0 (TID 102)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 85.0 in stage 10.0 (TID 96) in 27 ms on localhost (87/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 86.0 in stage 10.0 (TID 97) in 26 ms on localhost (88/199)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 89.0 in stage 10.0 (TID 100). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Running task 90.0 in stage 10.0 (TID 101)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 92.0 in stage 10.0 (TID 103, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 88.0 in stage 10.0 (TID 99)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 89.0 in stage 10.0 (TID 100) in 18 ms on localhost (89/199)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Running task 92.0 in stage 10.0 (TID 103)
15/03/14 11:15:51 INFO Executor: Finished task 88.0 in stage 10.0 (TID 99). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 93.0 in stage 10.0 (TID 104, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 88.0 in stage 10.0 (TID 99) in 29 ms on localhost (90/199)
15/03/14 11:15:51 INFO Executor: Running task 93.0 in stage 10.0 (TID 104)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 91.0 in stage 10.0 (TID 102). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 92.0 in stage 10.0 (TID 103). 894 bytes result sent to driver
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 11 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO TaskSetManager: Starting task 94.0 in stage 10.0 (TID 105, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 95.0 in stage 10.0 (TID 106, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 91.0 in stage 10.0 (TID 102) in 30 ms on localhost (91/199)
15/03/14 11:15:51 INFO Executor: Running task 95.0 in stage 10.0 (TID 106)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 92.0 in stage 10.0 (TID 103) in 20 ms on localhost (92/199)
15/03/14 11:15:51 INFO Executor: Finished task 90.0 in stage 10.0 (TID 101). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 93.0 in stage 10.0 (TID 104). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Running task 94.0 in stage 10.0 (TID 105)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 96.0 in stage 10.0 (TID 107, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 97.0 in stage 10.0 (TID 108, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 90.0 in stage 10.0 (TID 101) in 35 ms on localhost (93/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 93.0 in stage 10.0 (TID 104) in 15 ms on localhost (94/199)
15/03/14 11:15:51 INFO Executor: Running task 96.0 in stage 10.0 (TID 107)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 95.0 in stage 10.0 (TID 106). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 94.0 in stage 10.0 (TID 105). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Running task 97.0 in stage 10.0 (TID 108)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO TaskSetManager: Starting task 98.0 in stage 10.0 (TID 109, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 99.0 in stage 10.0 (TID 110, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 99.0 in stage 10.0 (TID 110)
15/03/14 11:15:51 INFO Executor: Finished task 96.0 in stage 10.0 (TID 107). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Finished task 95.0 in stage 10.0 (TID 106) in 18 ms on localhost (95/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 94.0 in stage 10.0 (TID 105) in 19 ms on localhost (96/199)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 100.0 in stage 10.0 (TID 111, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 98.0 in stage 10.0 (TID 109)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO Executor: Running task 100.0 in stage 10.0 (TID 111)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 97.0 in stage 10.0 (TID 108). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 100.0 in stage 10.0 (TID 111). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Finished task 96.0 in stage 10.0 (TID 107) in 18 ms on localhost (97/199)
15/03/14 11:15:51 INFO Executor: Finished task 98.0 in stage 10.0 (TID 109). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 101.0 in stage 10.0 (TID 112, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 101.0 in stage 10.0 (TID 112)
15/03/14 11:15:51 INFO Executor: Finished task 99.0 in stage 10.0 (TID 110). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Finished task 97.0 in stage 10.0 (TID 108) in 30 ms on localhost (98/199)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 102.0 in stage 10.0 (TID 113, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 100.0 in stage 10.0 (TID 111) in 14 ms on localhost (99/199)
15/03/14 11:15:51 INFO Executor: Running task 102.0 in stage 10.0 (TID 113)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 103.0 in stage 10.0 (TID 114, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 98.0 in stage 10.0 (TID 109) in 22 ms on localhost (100/199)
15/03/14 11:15:51 INFO Executor: Running task 103.0 in stage 10.0 (TID 114)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 104.0 in stage 10.0 (TID 115, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 104.0 in stage 10.0 (TID 115)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 101.0 in stage 10.0 (TID 112). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 105.0 in stage 10.0 (TID 116, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 105.0 in stage 10.0 (TID 116)
15/03/14 11:15:51 INFO Executor: Finished task 102.0 in stage 10.0 (TID 113). 894 bytes result sent to driver
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO TaskSetManager: Starting task 106.0 in stage 10.0 (TID 117, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Finished task 103.0 in stage 10.0 (TID 114). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Finished task 102.0 in stage 10.0 (TID 113) in 12 ms on localhost (101/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 101.0 in stage 10.0 (TID 112) in 13 ms on localhost (102/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 99.0 in stage 10.0 (TID 110) in 33 ms on localhost (103/199)
15/03/14 11:15:51 INFO Executor: Running task 106.0 in stage 10.0 (TID 117)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 107.0 in stage 10.0 (TID 118, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 103.0 in stage 10.0 (TID 114) in 13 ms on localhost (104/199)
15/03/14 11:15:51 INFO Executor: Running task 107.0 in stage 10.0 (TID 118)
15/03/14 11:15:51 INFO Executor: Finished task 104.0 in stage 10.0 (TID 115). 894 bytes result sent to driver
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO TaskSetManager: Starting task 108.0 in stage 10.0 (TID 119, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 108.0 in stage 10.0 (TID 119)
15/03/14 11:15:51 INFO Executor: Finished task 105.0 in stage 10.0 (TID 116). 894 bytes result sent to driver
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO TaskSetManager: Starting task 109.0 in stage 10.0 (TID 120, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 109.0 in stage 10.0 (TID 120)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO Executor: Finished task 106.0 in stage 10.0 (TID 117). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Finished task 104.0 in stage 10.0 (TID 115) in 18 ms on localhost (105/199)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 39 ms
15/03/14 11:15:51 INFO TaskSetManager: Starting task 110.0 in stage 10.0 (TID 121, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 35 ms
15/03/14 11:15:51 INFO Executor: Running task 110.0 in stage 10.0 (TID 121)
15/03/14 11:15:51 INFO Executor: Finished task 107.0 in stage 10.0 (TID 118). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 108.0 in stage 10.0 (TID 119). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Finished task 106.0 in stage 10.0 (TID 117) in 51 ms on localhost (106/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 105.0 in stage 10.0 (TID 116) in 55 ms on localhost (107/199)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 111.0 in stage 10.0 (TID 122, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 107.0 in stage 10.0 (TID 118) in 52 ms on localhost (108/199)
15/03/14 11:15:51 INFO Executor: Running task 111.0 in stage 10.0 (TID 122)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 109.0 in stage 10.0 (TID 120). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 112.0 in stage 10.0 (TID 123, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 112.0 in stage 10.0 (TID 123)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 108.0 in stage 10.0 (TID 119) in 55 ms on localhost (109/199)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 113.0 in stage 10.0 (TID 124, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Running task 113.0 in stage 10.0 (TID 124)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 109.0 in stage 10.0 (TID 120) in 54 ms on localhost (110/199)
15/03/14 11:15:51 INFO Executor: Finished task 111.0 in stage 10.0 (TID 122). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 114.0 in stage 10.0 (TID 125, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 111.0 in stage 10.0 (TID 122) in 12 ms on localhost (111/199)
15/03/14 11:15:51 INFO Executor: Running task 114.0 in stage 10.0 (TID 125)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 112.0 in stage 10.0 (TID 123). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 113.0 in stage 10.0 (TID 124). 894 bytes result sent to driver
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO TaskSetManager: Starting task 115.0 in stage 10.0 (TID 126, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 115.0 in stage 10.0 (TID 126)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 116.0 in stage 10.0 (TID 127, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 112.0 in stage 10.0 (TID 123) in 15 ms on localhost (112/199)
15/03/14 11:15:51 INFO Executor: Running task 116.0 in stage 10.0 (TID 127)
15/03/14 11:15:51 INFO Executor: Finished task 114.0 in stage 10.0 (TID 125). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 117.0 in stage 10.0 (TID 128, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 114.0 in stage 10.0 (TID 125) in 12 ms on localhost (113/199)
15/03/14 11:15:51 INFO Executor: Running task 117.0 in stage 10.0 (TID 128)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 115.0 in stage 10.0 (TID 126). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 116.0 in stage 10.0 (TID 127). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 118.0 in stage 10.0 (TID 129, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 113.0 in stage 10.0 (TID 124) in 24 ms on localhost (114/199)
15/03/14 11:15:51 INFO Executor: Running task 118.0 in stage 10.0 (TID 129)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 115.0 in stage 10.0 (TID 126) in 14 ms on localhost (115/199)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 119.0 in stage 10.0 (TID 130, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 119.0 in stage 10.0 (TID 130)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 116.0 in stage 10.0 (TID 127) in 15 ms on localhost (116/199)
15/03/14 11:15:51 INFO Executor: Finished task 117.0 in stage 10.0 (TID 128). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 120.0 in stage 10.0 (TID 131, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 120.0 in stage 10.0 (TID 131)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 117.0 in stage 10.0 (TID 128) in 15 ms on localhost (117/199)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO Executor: Finished task 118.0 in stage 10.0 (TID 129). 894 bytes result sent to driver
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 119.0 in stage 10.0 (TID 130). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 121.0 in stage 10.0 (TID 132, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 121.0 in stage 10.0 (TID 132)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 118.0 in stage 10.0 (TID 129) in 13 ms on localhost (118/199)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 122.0 in stage 10.0 (TID 133, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 122.0 in stage 10.0 (TID 133)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 119.0 in stage 10.0 (TID 130) in 12 ms on localhost (119/199)
15/03/14 11:15:51 INFO Executor: Finished task 120.0 in stage 10.0 (TID 131). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 123.0 in stage 10.0 (TID 134, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 123.0 in stage 10.0 (TID 134)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 120.0 in stage 10.0 (TID 131) in 10 ms on localhost (120/199)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 122.0 in stage 10.0 (TID 133). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 124.0 in stage 10.0 (TID 135, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 124.0 in stage 10.0 (TID 135)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 122.0 in stage 10.0 (TID 133) in 10 ms on localhost (121/199)
15/03/14 11:15:51 INFO Executor: Finished task 121.0 in stage 10.0 (TID 132). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 125.0 in stage 10.0 (TID 136, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 125.0 in stage 10.0 (TID 136)
15/03/14 11:15:51 INFO Executor: Finished task 123.0 in stage 10.0 (TID 134). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 126.0 in stage 10.0 (TID 137, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 121.0 in stage 10.0 (TID 132) in 14 ms on localhost (122/199)
15/03/14 11:15:51 INFO Executor: Running task 126.0 in stage 10.0 (TID 137)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 123.0 in stage 10.0 (TID 134) in 12 ms on localhost (123/199)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 124.0 in stage 10.0 (TID 135). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 110.0 in stage 10.0 (TID 121). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 125.0 in stage 10.0 (TID 136). 894 bytes result sent to driver
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO TaskSetManager: Starting task 127.0 in stage 10.0 (TID 138, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 110 ms
15/03/14 11:15:51 INFO Executor: Running task 127.0 in stage 10.0 (TID 138)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 128.0 in stage 10.0 (TID 139, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 128.0 in stage 10.0 (TID 139)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 129.0 in stage 10.0 (TID 140, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 129.0 in stage 10.0 (TID 140)
15/03/14 11:15:51 INFO Executor: Finished task 126.0 in stage 10.0 (TID 137). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 130.0 in stage 10.0 (TID 141, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 110.0 in stage 10.0 (TID 121) in 186 ms on localhost (124/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 124.0 in stage 10.0 (TID 135) in 127 ms on localhost (125/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 125.0 in stage 10.0 (TID 136) in 126 ms on localhost (126/199)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 126.0 in stage 10.0 (TID 137) in 127 ms on localhost (127/199)
15/03/14 11:15:51 INFO Executor: Running task 130.0 in stage 10.0 (TID 141)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 129.0 in stage 10.0 (TID 140). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 131.0 in stage 10.0 (TID 142, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 129.0 in stage 10.0 (TID 140) in 14 ms on localhost (128/199)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO Executor: Finished task 130.0 in stage 10.0 (TID 141). 894 bytes result sent to driver
15/03/14 11:15:51 INFO Executor: Finished task 128.0 in stage 10.0 (TID 139). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 132.0 in stage 10.0 (TID 143, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 130.0 in stage 10.0 (TID 141) in 14 ms on localhost (129/199)
15/03/14 11:15:51 INFO TaskSetManager: Starting task 133.0 in stage 10.0 (TID 144, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO Executor: Running task 133.0 in stage 10.0 (TID 144)
15/03/14 11:15:51 INFO Executor: Running task 131.0 in stage 10.0 (TID 142)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 128.0 in stage 10.0 (TID 139) in 21 ms on localhost (130/199)
15/03/14 11:15:51 INFO Executor: Running task 132.0 in stage 10.0 (TID 143)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO Executor: Finished task 131.0 in stage 10.0 (TID 142). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 134.0 in stage 10.0 (TID 145, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 134.0 in stage 10.0 (TID 145)
15/03/14 11:15:51 INFO Executor: Finished task 132.0 in stage 10.0 (TID 143). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 135.0 in stage 10.0 (TID 146, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 131.0 in stage 10.0 (TID 142) in 22 ms on localhost (131/199)
15/03/14 11:15:51 INFO Executor: Running task 135.0 in stage 10.0 (TID 146)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 132.0 in stage 10.0 (TID 143) in 18 ms on localhost (132/199)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO Executor: Finished task 134.0 in stage 10.0 (TID 145). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 136.0 in stage 10.0 (TID 147, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 136.0 in stage 10.0 (TID 147)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 134.0 in stage 10.0 (TID 145) in 13 ms on localhost (133/199)
15/03/14 11:15:51 INFO Executor: Finished task 135.0 in stage 10.0 (TID 146). 894 bytes result sent to driver
15/03/14 11:15:51 INFO TaskSetManager: Starting task 137.0 in stage 10.0 (TID 148, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:51 INFO Executor: Running task 137.0 in stage 10.0 (TID 148)
15/03/14 11:15:51 INFO TaskSetManager: Finished task 135.0 in stage 10.0 (TID 146) in 15 ms on localhost (134/199)
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO Executor: Finished task 137.0 in stage 10.0 (TID 148). 894 bytes result sent to driver
15/03/14 11:15:52 INFO Executor: Finished task 133.0 in stage 10.0 (TID 144). 894 bytes result sent to driver
15/03/14 11:15:52 INFO Executor: Finished task 136.0 in stage 10.0 (TID 147). 894 bytes result sent to driver
15/03/14 11:15:52 INFO Executor: Finished task 127.0 in stage 10.0 (TID 138). 894 bytes result sent to driver
15/03/14 11:15:52 INFO TaskSetManager: Starting task 138.0 in stage 10.0 (TID 149, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO Executor: Running task 138.0 in stage 10.0 (TID 149)
15/03/14 11:15:52 INFO TaskSetManager: Starting task 139.0 in stage 10.0 (TID 150, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO Executor: Running task 139.0 in stage 10.0 (TID 150)
15/03/14 11:15:52 INFO TaskSetManager: Starting task 140.0 in stage 10.0 (TID 151, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO Executor: Running task 140.0 in stage 10.0 (TID 151)
15/03/14 11:15:52 INFO TaskSetManager: Starting task 141.0 in stage 10.0 (TID 152, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 137.0 in stage 10.0 (TID 148) in 220 ms on localhost (135/199)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 133.0 in stage 10.0 (TID 144) in 250 ms on localhost (136/199)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 136.0 in stage 10.0 (TID 147) in 228 ms on localhost (137/199)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 127.0 in stage 10.0 (TID 138) in 278 ms on localhost (138/199)
15/03/14 11:15:52 INFO Executor: Running task 141.0 in stage 10.0 (TID 152)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO Executor: Finished task 140.0 in stage 10.0 (TID 151). 894 bytes result sent to driver
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO Executor: Finished task 138.0 in stage 10.0 (TID 149). 894 bytes result sent to driver
15/03/14 11:15:52 INFO TaskSetManager: Starting task 142.0 in stage 10.0 (TID 153, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO TaskSetManager: Starting task 143.0 in stage 10.0 (TID 154, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO Executor: Running task 143.0 in stage 10.0 (TID 154)
15/03/14 11:15:52 INFO Executor: Running task 142.0 in stage 10.0 (TID 153)
15/03/14 11:15:52 INFO Executor: Finished task 141.0 in stage 10.0 (TID 152). 894 bytes result sent to driver
15/03/14 11:15:52 INFO TaskSetManager: Finished task 140.0 in stage 10.0 (TID 151) in 22 ms on localhost (139/199)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 138.0 in stage 10.0 (TID 149) in 27 ms on localhost (140/199)
15/03/14 11:15:52 INFO TaskSetManager: Starting task 144.0 in stage 10.0 (TID 155, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO TaskSetManager: Finished task 141.0 in stage 10.0 (TID 152) in 25 ms on localhost (141/199)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO Executor: Finished task 143.0 in stage 10.0 (TID 154). 894 bytes result sent to driver
15/03/14 11:15:52 INFO Executor: Finished task 142.0 in stage 10.0 (TID 153). 894 bytes result sent to driver
15/03/14 11:15:52 INFO Executor: Finished task 139.0 in stage 10.0 (TID 150). 894 bytes result sent to driver
15/03/14 11:15:52 INFO TaskSetManager: Starting task 145.0 in stage 10.0 (TID 156, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO Executor: Running task 144.0 in stage 10.0 (TID 155)
15/03/14 11:15:52 INFO TaskSetManager: Starting task 146.0 in stage 10.0 (TID 157, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO TaskSetManager: Starting task 147.0 in stage 10.0 (TID 158, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO Executor: Running task 147.0 in stage 10.0 (TID 158)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 143.0 in stage 10.0 (TID 154) in 14 ms on localhost (142/199)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 142.0 in stage 10.0 (TID 153) in 16 ms on localhost (143/199)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 139.0 in stage 10.0 (TID 150) in 36 ms on localhost (144/199)
15/03/14 11:15:52 INFO Executor: Running task 146.0 in stage 10.0 (TID 157)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:52 INFO Executor: Finished task 144.0 in stage 10.0 (TID 155). 894 bytes result sent to driver
15/03/14 11:15:52 INFO Executor: Running task 145.0 in stage 10.0 (TID 156)
15/03/14 11:15:52 INFO Executor: Finished task 146.0 in stage 10.0 (TID 157). 894 bytes result sent to driver
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO Executor: Finished task 147.0 in stage 10.0 (TID 158). 894 bytes result sent to driver
15/03/14 11:15:52 INFO Executor: Finished task 145.0 in stage 10.0 (TID 156). 894 bytes result sent to driver
15/03/14 11:15:52 INFO TaskSetManager: Starting task 148.0 in stage 10.0 (TID 159, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO Executor: Running task 148.0 in stage 10.0 (TID 159)
15/03/14 11:15:52 INFO TaskSetManager: Starting task 149.0 in stage 10.0 (TID 160, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO Executor: Running task 149.0 in stage 10.0 (TID 160)
15/03/14 11:15:52 INFO TaskSetManager: Starting task 150.0 in stage 10.0 (TID 161, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO Executor: Running task 150.0 in stage 10.0 (TID 161)
15/03/14 11:15:52 INFO TaskSetManager: Starting task 151.0 in stage 10.0 (TID 162, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO Executor: Running task 151.0 in stage 10.0 (TID 162)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 144.0 in stage 10.0 (TID 155) in 33 ms on localhost (145/199)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 146.0 in stage 10.0 (TID 157) in 28 ms on localhost (146/199)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 147.0 in stage 10.0 (TID 158) in 28 ms on localhost (147/199)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 145.0 in stage 10.0 (TID 156) in 30 ms on localhost (148/199)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO Executor: Finished task 150.0 in stage 10.0 (TID 161). 894 bytes result sent to driver
15/03/14 11:15:52 INFO TaskSetManager: Starting task 152.0 in stage 10.0 (TID 163, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO Executor: Finished task 151.0 in stage 10.0 (TID 162). 894 bytes result sent to driver
15/03/14 11:15:52 INFO TaskSetManager: Finished task 150.0 in stage 10.0 (TID 161) in 11 ms on localhost (149/199)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO Executor: Running task 152.0 in stage 10.0 (TID 163)
15/03/14 11:15:52 INFO TaskSetManager: Starting task 153.0 in stage 10.0 (TID 164, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 151.0 in stage 10.0 (TID 162) in 18 ms on localhost (150/199)
15/03/14 11:15:52 INFO Executor: Running task 153.0 in stage 10.0 (TID 164)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms
15/03/14 11:15:52 INFO Executor: Finished task 148.0 in stage 10.0 (TID 159). 894 bytes result sent to driver
15/03/14 11:15:52 INFO TaskSetManager: Starting task 154.0 in stage 10.0 (TID 165, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:52 INFO TaskSetManager: Finished task 148.0 in stage 10.0 (TID 159) in 28 ms on localhost (151/199)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO Executor: Running task 154.0 in stage 10.0 (TID 165)
15/03/14 11:15:52 INFO Executor: Finished task 152.0 in stage 10.0 (TID 163). 894 bytes result sent to driver
15/03/14 11:15:52 INFO Executor: Finished task 153.0 in stage 10.0 (TID 164). 894 bytes result sent to driver
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:52 INFO TaskSetManager: Starting task 155.0 in stage 10.0 (TID 166, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO TaskSetManager: Starting task 156.0 in stage 10.0 (TID 167, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 153.0 in stage 10.0 (TID 164) in 12 ms on localhost (152/199)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 152.0 in stage 10.0 (TID 163) in 22 ms on localhost (153/199)
15/03/14 11:15:52 INFO Executor: Finished task 149.0 in stage 10.0 (TID 160). 894 bytes result sent to driver
15/03/14 11:15:52 INFO Executor: Running task 156.0 in stage 10.0 (TID 167)
15/03/14 11:15:52 INFO TaskSetManager: Starting task 157.0 in stage 10.0 (TID 168, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 149.0 in stage 10.0 (TID 160) in 35 ms on localhost (154/199)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO Executor: Running task 157.0 in stage 10.0 (TID 168)
15/03/14 11:15:52 INFO Executor: Finished task 154.0 in stage 10.0 (TID 165). 894 bytes result sent to driver
15/03/14 11:15:52 INFO TaskSetManager: Starting task 158.0 in stage 10.0 (TID 169, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO TaskSetManager: Finished task 154.0 in stage 10.0 (TID 165) in 22 ms on localhost (155/199)
15/03/14 11:15:52 INFO Executor: Running task 158.0 in stage 10.0 (TID 169)
15/03/14 11:15:52 INFO Executor: Finished task 157.0 in stage 10.0 (TID 168). 894 bytes result sent to driver
15/03/14 11:15:52 INFO TaskSetManager: Starting task 159.0 in stage 10.0 (TID 170, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 157.0 in stage 10.0 (TID 168) in 18 ms on localhost (156/199)
15/03/14 11:15:52 INFO Executor: Running task 159.0 in stage 10.0 (TID 170)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO Executor: Running task 155.0 in stage 10.0 (TID 166)
15/03/14 11:15:52 INFO Executor: Finished task 158.0 in stage 10.0 (TID 169). 894 bytes result sent to driver
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 20 ms
15/03/14 11:15:52 INFO Executor: Finished task 159.0 in stage 10.0 (TID 170). 894 bytes result sent to driver
15/03/14 11:15:52 INFO Executor: Finished task 156.0 in stage 10.0 (TID 167). 894 bytes result sent to driver
15/03/14 11:15:52 INFO TaskSetManager: Starting task 160.0 in stage 10.0 (TID 171, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO Executor: Running task 160.0 in stage 10.0 (TID 171)
15/03/14 11:15:52 INFO TaskSetManager: Starting task 161.0 in stage 10.0 (TID 172, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO TaskSetManager: Starting task 162.0 in stage 10.0 (TID 173, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO Executor: Running task 161.0 in stage 10.0 (TID 172)
15/03/14 11:15:52 INFO Executor: Finished task 155.0 in stage 10.0 (TID 166). 894 bytes result sent to driver
15/03/14 11:15:52 INFO TaskSetManager: Starting task 163.0 in stage 10.0 (TID 174, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO Executor: Running task 163.0 in stage 10.0 (TID 174)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:52 INFO TaskSetManager: Finished task 156.0 in stage 10.0 (TID 167) in 40 ms on localhost (157/199)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 155.0 in stage 10.0 (TID 166) in 42 ms on localhost (158/199)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 158.0 in stage 10.0 (TID 169) in 26 ms on localhost (159/199)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 159.0 in stage 10.0 (TID 170) in 22 ms on localhost (160/199)
15/03/14 11:15:52 INFO Executor: Running task 162.0 in stage 10.0 (TID 173)
15/03/14 11:15:52 INFO Executor: Finished task 161.0 in stage 10.0 (TID 172). 894 bytes result sent to driver
15/03/14 11:15:52 INFO TaskSetManager: Starting task 164.0 in stage 10.0 (TID 175, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO TaskSetManager: Finished task 161.0 in stage 10.0 (TID 172) in 16 ms on localhost (161/199)
15/03/14 11:15:52 INFO Executor: Running task 164.0 in stage 10.0 (TID 175)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO Executor: Finished task 163.0 in stage 10.0 (TID 174). 894 bytes result sent to driver
15/03/14 11:15:52 INFO TaskSetManager: Starting task 165.0 in stage 10.0 (TID 176, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO Executor: Finished task 162.0 in stage 10.0 (TID 173). 894 bytes result sent to driver
15/03/14 11:15:52 INFO TaskSetManager: Finished task 163.0 in stage 10.0 (TID 174) in 15 ms on localhost (162/199)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:52 INFO TaskSetManager: Starting task 166.0 in stage 10.0 (TID 177, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO Executor: Finished task 160.0 in stage 10.0 (TID 171). 894 bytes result sent to driver
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:52 INFO TaskSetManager: Starting task 167.0 in stage 10.0 (TID 178, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:52 INFO Executor: Running task 167.0 in stage 10.0 (TID 178)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 160.0 in stage 10.0 (TID 171) in 25 ms on localhost (163/199)
15/03/14 11:15:52 INFO TaskSetManager: Finished task 162.0 in stage 10.0 (TID 173) in 24 ms on localhost (164/199)
15/03/14 11:15:52 INFO Executor: Running task 166.0 in stage 10.0 (TID 177)
15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO Executor: Finished task 164.0 in stage 10.0 (TID 175). 894 bytes result sent to driver
15/03/14 11:15:53 INFO TaskSetManager: Starting task 168.0 in stage 10.0 (TID 179, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO Executor: Running task 165.0 in stage 10.0 (TID 176)
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO Executor: Finished task 167.0 in stage 10.0 (TID 178). 894 bytes result sent to driver
15/03/14 11:15:53 INFO Executor: Running task 168.0 in stage 10.0 (TID 179)
15/03/14 11:15:53 INFO TaskSetManager: Starting task 169.0 in stage 10.0 (TID 180, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 167.0 in stage 10.0 (TID 178) in 685 ms on localhost (165/199)
15/03/14 11:15:53 INFO Executor: Running task 169.0 in stage 10.0 (TID 180)
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO Executor: Finished task 166.0 in stage 10.0 (TID 177). 894 bytes result sent to driver
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO TaskSetManager: Starting task 170.0 in stage 10.0 (TID 181, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 166.0 in stage 10.0 (TID 177) in 696 ms on localhost (166/199)
15/03/14 11:15:53 INFO Executor: Running task 170.0 in stage 10.0 (TID 181)
15/03/14 11:15:53 INFO Executor: Finished task 169.0 in stage 10.0 (TID 180). 894 bytes result sent to driver
15/03/14 11:15:53 INFO Executor: Finished task 168.0 in stage 10.0 (TID 179). 1116 bytes result sent to driver
15/03/14 11:15:53 INFO TaskSetManager: Finished task 164.0 in stage 10.0 (TID 175) in 708 ms on localhost (167/199)
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:53 INFO Executor: Finished task 165.0 in stage 10.0 (TID 176). 894 bytes result sent to driver
15/03/14 11:15:53 INFO Executor: Finished task 170.0 in stage 10.0 (TID 181). 894 bytes result sent to driver
15/03/14 11:15:53 INFO TaskSetManager: Starting task 171.0 in stage 10.0 (TID 182, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO Executor: Running task 171.0 in stage 10.0 (TID 182)
15/03/14 11:15:53 INFO TaskSetManager: Starting task 172.0 in stage 10.0 (TID 183, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO Executor: Running task 172.0 in stage 10.0 (TID 183)
15/03/14 11:15:53 INFO TaskSetManager: Starting task 173.0 in stage 10.0 (TID 184, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO TaskSetManager: Starting task 174.0 in stage 10.0 (TID 185, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO Executor: Running task 174.0 in stage 10.0 (TID 185)
15/03/14 11:15:53 INFO Executor: Running task 173.0 in stage 10.0 (TID 184)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 169.0 in stage 10.0 (TID 180) in 26 ms on localhost (168/199)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 168.0 in stage 10.0 (TID 179) in 37 ms on localhost (169/199)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 165.0 in stage 10.0 (TID 176) in 720 ms on localhost (170/199)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 170.0 in stage 10.0 (TID 181) in 27 ms on localhost (171/199)
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO Executor: Finished task 171.0 in stage 10.0 (TID 182). 894 bytes result sent to driver
15/03/14 11:15:53 INFO Executor: Finished task 172.0 in stage 10.0 (TID 183). 894 bytes result sent to driver
15/03/14 11:15:53 INFO Executor: Finished task 174.0 in stage 10.0 (TID 185). 894 bytes result sent to driver
15/03/14 11:15:53 INFO TaskSetManager: Starting task 175.0 in stage 10.0 (TID 186, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO Executor: Running task 175.0 in stage 10.0 (TID 186)
15/03/14 11:15:53 INFO TaskSetManager: Starting task 176.0 in stage 10.0 (TID 187, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO TaskSetManager: Starting task 177.0 in stage 10.0 (TID 188, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO Executor: Running task 177.0 in stage 10.0 (TID 188)
15/03/14 11:15:53 INFO Executor: Running task 176.0 in stage 10.0 (TID 187)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 172.0 in stage 10.0 (TID 183) in 25 ms on localhost (172/199)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 171.0 in stage 10.0 (TID 182) in 28 ms on localhost (173/199)
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO Executor: Finished task 173.0 in stage 10.0 (TID 184). 894 bytes result sent to driver
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO TaskSetManager: Starting task 178.0 in stage 10.0 (TID 189, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 173.0 in stage 10.0 (TID 184) in 38 ms on localhost (174/199)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 174.0 in stage 10.0 (TID 185) in 38 ms on localhost (175/199)
15/03/14 11:15:53 INFO Executor: Running task 178.0 in stage 10.0 (TID 189)
15/03/14 11:15:53 INFO Executor: Finished task 176.0 in stage 10.0 (TID 187). 894 bytes result sent to driver
15/03/14 11:15:53 INFO Executor: Finished task 177.0 in stage 10.0 (TID 188). 894 bytes result sent to driver
15/03/14 11:15:53 INFO TaskSetManager: Starting task 179.0 in stage 10.0 (TID 190, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 176.0 in stage 10.0 (TID 187) in 23 ms on localhost (176/199)
15/03/14 11:15:53 INFO Executor: Running task 179.0 in stage 10.0 (TID 190)
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO TaskSetManager: Starting task 180.0 in stage 10.0 (TID 191, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO Executor: Running task 180.0 in stage 10.0 (TID 191)
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms
15/03/14 11:15:53 INFO Executor: Finished task 175.0 in stage 10.0 (TID 186). 894 bytes result sent to driver
15/03/14 11:15:53 INFO TaskSetManager: Starting task 181.0 in stage 10.0 (TID 192, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO TaskSetManager: Finished task 177.0 in stage 10.0 (TID 188) in 32 ms on localhost (177/199)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 175.0 in stage 10.0 (TID 186) in 36 ms on localhost (178/199)
15/03/14 11:15:53 INFO Executor: Running task 181.0 in stage 10.0 (TID 192)
15/03/14 11:15:53 INFO Executor: Finished task 179.0 in stage 10.0 (TID 190). 894 bytes result sent to driver
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO TaskSetManager: Starting task 182.0 in stage 10.0 (TID 193, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO Executor: Running task 182.0 in stage 10.0 (TID 193)
15/03/14 11:15:53 INFO Executor: Finished task 178.0 in stage 10.0 (TID 189). 894 bytes result sent to driver
15/03/14 11:15:53 INFO Executor: Finished task 180.0 in stage 10.0 (TID 191). 894 bytes result sent to driver
15/03/14 11:15:53 INFO TaskSetManager: Finished task 179.0 in stage 10.0 (TID 190) in 19 ms on localhost (179/199)
15/03/14 11:15:53 INFO TaskSetManager: Starting task 183.0 in stage 10.0 (TID 194, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO Executor: Running task 183.0 in stage 10.0 (TID 194)
15/03/14 11:15:53 INFO TaskSetManager: Starting task 184.0 in stage 10.0 (TID 195, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO TaskSetManager: Finished task 178.0 in stage 10.0 (TID 189) in 39 ms on localhost (180/199)
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:53 INFO TaskSetManager: Finished task 180.0 in stage 10.0 (TID 191) in 23 ms on localhost (181/199)
15/03/14 11:15:53 INFO Executor: Running task 184.0 in stage 10.0 (TID 195)
15/03/14 11:15:53 INFO Executor: Finished task 182.0 in stage 10.0 (TID 193). 894 bytes result sent to driver
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO Executor: Finished task 183.0 in stage 10.0 (TID 194). 894 bytes result sent to driver
15/03/14 11:15:53 INFO Executor: Finished task 181.0 in stage 10.0 (TID 192). 894 bytes result sent to driver
15/03/14 11:15:53 INFO TaskSetManager: Starting task 185.0 in stage 10.0 (TID 196, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO Executor: Finished task 184.0 in stage 10.0 (TID 195). 894 bytes result sent to driver
15/03/14 11:15:53 INFO TaskSetManager: Starting task 186.0 in stage 10.0 (TID 197, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO TaskSetManager: Starting task 187.0 in stage 10.0 (TID 198, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO Executor: Running task 186.0 in stage 10.0 (TID 197)
15/03/14 11:15:53 INFO Executor: Running task 187.0 in stage 10.0 (TID 198)
15/03/14 11:15:53 INFO TaskSetManager: Starting task 188.0 in stage 10.0 (TID 199, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 182.0 in stage 10.0 (TID 193) in 25 ms on localhost (182/199)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 181.0 in stage 10.0 (TID 192) in 31 ms on localhost (183/199)
15/03/14 11:15:53 INFO Executor: Running task 185.0 in stage 10.0 (TID 196)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 183.0 in stage 10.0 (TID 194) in 15 ms on localhost (184/199)
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO Executor: Finished task 187.0 in stage 10.0 (TID 198). 1129 bytes result sent to driver
15/03/14 11:15:53 INFO Executor: Finished task 185.0 in stage 10.0 (TID 196). 894 bytes result sent to driver
15/03/14 11:15:53 INFO Executor: Finished task 186.0 in stage 10.0 (TID 197). 894 bytes result sent to driver
15/03/14 11:15:53 INFO Executor: Running task 188.0 in stage 10.0 (TID 199)
15/03/14 11:15:53 INFO TaskSetManager: Starting task 189.0 in stage 10.0 (TID 200, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO TaskSetManager: Starting task 190.0 in stage 10.0 (TID 201, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO TaskSetManager: Starting task 191.0 in stage 10.0 (TID 202, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO Executor: Running task 191.0 in stage 10.0 (TID 202)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 187.0 in stage 10.0 (TID 198) in 15 ms on localhost (185/199)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 184.0 in stage 10.0 (TID 195) in 27 ms on localhost (186/199)
15/03/14 11:15:53 INFO Executor: Running task 189.0 in stage 10.0 (TID 200)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 186.0 in stage 10.0 (TID 197) in 16 ms on localhost (187/199)
15/03/14 11:15:53 INFO Executor: Running task 190.0 in stage 10.0 (TID 201)
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
15/03/14 11:15:53 INFO Executor: Finished task 191.0 in stage 10.0 (TID 202). 894 bytes result sent to driver
15/03/14 11:15:53 INFO Executor: Finished task 189.0 in stage 10.0 (TID 200). 894 bytes result sent to driver
15/03/14 11:15:53 INFO Executor: Finished task 190.0 in stage 10.0 (TID 201). 894 bytes result sent to driver
15/03/14 11:15:53 INFO TaskSetManager: Starting task 192.0 in stage 10.0 (TID 203, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO TaskSetManager: Starting task 193.0 in stage 10.0 (TID 204, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO TaskSetManager: Starting task 194.0 in stage 10.0 (TID 205, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO Executor: Running task 194.0 in stage 10.0 (TID 205)
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO Executor: Running task 193.0 in stage 10.0 (TID 204)
15/03/14 11:15:53 INFO Executor: Finished task 188.0 in stage 10.0 (TID 199). 894 bytes result sent to driver
15/03/14 11:15:53 INFO Executor: Running task 192.0 in stage 10.0 (TID 203)
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO TaskSetManager: Finished task 191.0 in stage 10.0 (TID 202) in 15 ms on localhost (188/199)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 190.0 in stage 10.0 (TID 201) in 23 ms on localhost (189/199)
15/03/14 11:15:53 INFO Executor: Finished task 194.0 in stage 10.0 (TID 205). 894 bytes result sent to driver
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO TaskSetManager: Finished task 185.0 in stage 10.0 (TID 196) in 40 ms on localhost (190/199)
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms
15/03/14 11:15:53 INFO TaskSetManager: Finished task 189.0 in stage 10.0 (TID 200) in 27 ms on localhost (191/199)
15/03/14 11:15:53 INFO Executor: Finished task 193.0 in stage 10.0 (TID 204). 894 bytes result sent to driver
15/03/14 11:15:53 INFO Executor: Finished task 192.0 in stage 10.0 (TID 203). 894 bytes result sent to driver
15/03/14 11:15:53 INFO TaskSetManager: Starting task 195.0 in stage 10.0 (TID 206, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO Executor: Running task 195.0 in stage 10.0 (TID 206)
15/03/14 11:15:53 INFO TaskSetManager: Starting task 196.0 in stage 10.0 (TID 207, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO TaskSetManager: Starting task 197.0 in stage 10.0 (TID 208, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO Executor: Running task 197.0 in stage 10.0 (TID 208)
15/03/14 11:15:53 INFO TaskSetManager: Starting task 198.0 in stage 10.0 (TID 209, localhost, PROCESS_LOCAL, 1056 bytes)
15/03/14 11:15:53 INFO Executor: Running task 198.0 in stage 10.0 (TID 209)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 188.0 in stage 10.0 (TID 199) in 45 ms on localhost (192/199)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 194.0 in stage 10.0 (TID 205) in 19 ms on localhost (193/199)
15/03/14 11:15:53 INFO Executor: Running task 196.0 in stage 10.0 (TID 207)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 192.0 in stage 10.0 (TID 203) in 20 ms on localhost (194/199)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 193.0 in stage 10.0 (TID 204) in 20 ms on localhost (195/199)
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO Executor: Finished task 195.0 in stage 10.0 (TID 206). 894 bytes result sent to driver
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
15/03/14 11:15:53 INFO TaskSetManager: Finished task 195.0 in stage 10.0 (TID 206) in 16 ms on localhost (196/199)
15/03/14 11:15:53 INFO Executor: Finished task 197.0 in stage 10.0 (TID 208). 894 bytes result sent to driver
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks
15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 9 ms
15/03/14 11:15:53 INFO TaskSetManager: Finished task 197.0 in stage 10.0 (TID 208) in 19 ms on localhost (197/199)
15/03/14 11:15:53 INFO Executor: Finished task 198.0 in stage 10.0 (TID 209). 1129 bytes result sent to driver
15/03/14 11:15:53 INFO Executor: Finished task 196.0 in stage 10.0 (TID 207). 894 bytes result sent to driver
15/03/14 11:15:53 INFO TaskSetManager: Finished task 198.0 in stage 10.0 (TID 209) in 23 ms on localhost (198/199)
15/03/14 11:15:53 INFO TaskSetManager: Finished task 196.0 in stage 10.0 (TID 207) in 28 ms on localhost (199/199)
15/03/14 11:15:53 INFO TaskSchedulerImpl: Removed TaskSet 10.0, whose tasks have all completed, from pool 
15/03/14 11:15:53 INFO DAGScheduler: Stage 10 (runJob at SparkPlan.scala:121) finished in 3.044 s
15/03/14 11:15:53 INFO DAGScheduler: Job 8 finished: runJob at SparkPlan.scala:121, took 3.095694 s
age  count
null 1    
19   1    
30   1