es recover

on log operations
2015-07-01 14:33:09,517 TRACE indices.recovery: [15] [yotta-20150629][23] no translog operations (id: [1435590963614]) to send to [14][uwra9okWTSOVARg5QqTbxA][centos-14][inet[/192.168.1.14:9300]]
2015-07-01 14:33:09,517 TRACE indices.recovery: [15] [yotta-20150629][23] sending final batch of [0][0b] (total: [0], id: [1435590963614]) translog operations to [14][uwra9okWTSOVARg5QqTbxA][centos-14][inet[/192.168.1.14:9300]]
2015-07-01 14:33:09,517 TRACE indices.recovery: [15] [yotta-20150629][23] recovery [phase2] to [14][uwra9okWTSOVARg5QqTbxA][centos-14][inet[/192.168.1.14:9300]]: took [213.7micros]
2015-07-01 14:33:09,517 TRACE indices.recovery: [15] [yotta-20150629][23] recovery [phase3] to [14][uwra9okWTSOVARg5QqTbxA][centos-14][inet[/192.168.1.14:9300]]: sending transaction log operations
2015-07-01 14:33:09,517 TRACE indices.recovery: [15] [yotta-20150629][23] no translog operations (id: [1435590963614]) to send to [14][uwra9okWTSOVARg5QqTbxA][centos-14][inet[/192.168.1.14:9300]]
2015-07-01 14:33:09,517 TRACE indices.recovery: [15] [yotta-20150629][23] sending final batch of [0][0b] (total: [0], id: [1435590963614]) translog operations to [14][uwra9okWTSOVARg5QqTbxA][centos-14][inet[/192.168.1.14:9300]]
2015-07-01 14:33:09,519 TRACE indices.recovery: [15] [yotta-20150629][23] recovery [phase3] to [14][uwra9okWTSOVARg5QqTbxA][centos-14][inet[/192.168.1.14:9300]]: took [2.1ms]
2015-07-01 14:33:09,536 DEBUG zen.publish: [15] received cluster state version 1208
2015-07-01 14:33:09,790 DEBUG zen.publish: [15] received cluster state version 1209
2015-07-01 14:34:29,185 DEBUG index.engine: [15] [yotta-20150629][9] [[yotta-20150629][9]] skipping check for 3x segments
2015-07-01 14:34:29,511 DEBUG index.shard: [15] [yotta-20150629][9] scheduling refresher every 30s
2015-07-01 14:34:29,524 TRACE indices.recovery: [15] [yotta-20150629][9] marking recovery from [16][8-B6KcSJRhqI91HTrlf_hw][centos-16][inet[/192.168.1.16:9300]] as done, id [50]
2015-07-01 14:34:29,525 DEBUG index.shard: [15] [yotta-20150629][9] state: [RECOVERING]->[POST_RECOVERY], reason [peer recovery done]
2015-07-01 14:34:29,525 DEBUG action.shard: [15] sending shard started for [yotta-20150629][9], node[L2qO9dkLSnuByemxYMdx-A], [R], s[INITIALIZING], indexUUID [FH4DRptCS6K3cpDV54Q9ww], reason [after recovery (replica) from node [[16][8-B6KcSJRhqI91HTrlf_hw][centos-16][inet[/192.168.1.16:9300]]]]
2015-07-01 14:34:29,525 TRACE indices.recovery: [15] [yotta-20150629][9] recovery completed from [16][8-B6KcSJRhqI91HTrlf_hw][centos-16][inet[/192.168.1.16:9300]], took[3.4m]
   phase1: recovered_files [293] with total_size of [14.8gb], took [3.4m], throttling_wait [0s]
         : reusing_files   [0] with total_size of [0b]
   phase2: start took [326ms]
         : recovered [0] transaction log operations, took [0s]
   phase3: recovered [0] transaction log operations, took [1ms]
2015-07-01 14:34:29,532 DEBUG zen.publish: [15] received cluster state version 1210
2015-07-01 14:34:29,538 DEBUG index.shard: [15] [yotta-20150629][9] state: [POST_RECOVERY]->[STARTED], reason [global state is [STARTED]]
2015-07-01 14:34:49,982 TRACE indices.recovery: [15] [yotta-20150629][1] recovery [phase1] to [16][8-B6KcSJRhqI91HTrlf_hw][centos-16][inet[/192.168.1.16:9300]]: took [3.7m]
2015-07-01 14:34:49,982 TRACE indices.recovery: [15] [yotta-20150629][1] recovery [phase2] to [16][8-B6KcSJRhqI91HTrlf_hw][centos-16][inet[/192.168.1.16:9300]]: start
2015-07-01 14:34:51,452 TRACE indices.recovery: [15] [yotta-20150629][1] recovery [phase2] to [16][8-B6KcSJRhqI91HTrlf_hw][centos-16][inet[/192.168.1.16:9300]]: start took [1.4s]
2015-07-01 14:34:51,452 TRACE indices.recovery: [15] [yotta-20150629][1] recovery [phase2] to [16][8-B6KcSJRhqI91HTrlf_hw][centos-16][inet[/192.168.1.16:9300]]: updating current mapping to master
2015-07-01 14:34:51,454 TRACE indices.recovery: [15] [yotta-20150629][1] recovery [phase2] to [16][8-B6KcSJRhqI91HTrlf_hw][centos-16][inet[/192.168.1.16:9300]]: sending transaction log operations
2015-07-01 14:34:51,454 TRACE indices.recovery: [15] [yotta-20150629][1] no translog operations (id: [1435591003054]) to send to [16][8-B6KcSJRhqI91HTrlf_hw][centos-16][inet[/192.168.1.16:9300]]
2015-07-01 14:34:51,454 TRACE indices.recovery: [15] [yotta-20150629][1] sending final batch of [0][0b] (total: [0], id: [1435591003054]) translog operations to [16][8-B6KcSJRhqI91HTrlf_hw][centos-16][inet[/192.168.1.16:9300]]
2015-07-01 14:34:51,455 TRACE indices.recovery: [15] [yotta-20150629][1] recovery [phase2] to [16][8-B6KcSJRhqI91HTrlf_hw][centos-16][inet[/192.168.1.16:9300]]: took [377.9micros]
2015-07-01 14:34:51,455 TRACE indices.recovery: [15] [yotta-20150629][1] recovery [phase3] to [16][8-B6KcSJRhqI91HTrlf_hw][centos-16][inet[/192.168.1.16:9300]]: sending transaction log operations
2015-07-01 14:34:51,455 TRACE indices.recovery: [15] [yotta-20150629][1] no translog operations (id: [1435591003054]) to send to [16][8-B6KcSJRhqI91HTrlf_hw][centos-16][inet[/192.168.1.16:9300]]
2015-07-01 14:34:51,455 TRACE indices.recovery: [15] [yotta-20150629][1] sending final batch of [0][0b] (total: [0], id: [1435591003054]) translog operations to [16][8-B6KcSJRhqI91HTrlf_hw][centos-16][inet[/192.168.1.16:9300]]
2015-07-01 14:34:51,457 TRACE indices.recovery: [15] [yotta-20150629][1] recovery [phase3] to [16][8-B6KcSJRhqI91HTrlf_hw][centos-16][inet[/192.168.1.16:9300]]: took [1.8ms]
2015-07-01 14:34:51,484 DEBUG zen.publish: [15] received cluster state version 1211