记一次Elastic-search translog EOF 问题修复

日志

[gather.log][[gather.log][0]] IndexShardRecoveryException[failed to recovery from gateway]; nested:       
EngineCreationFailureException[failed to recover from translog]; nested: EngineException[failed to recover 
from translog]; nested: ElasticsearchException[unexpected exception reading from translog snapshot of 
/usr/local/elasticsearch/data/elasticsearch/nodes/0/indices/gather.log/0/trans
log/translog-382.tlog]; nested: EOFException[read past EOF. pos [3932480] length: [4] end: [3932480]]; 

解决方法

（1）先使用ES自带的elasticsearch-translog删除修复translog

bin/elasticsearch-translog truncate -d 
/var/lib/elasticsearchdata/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/
Checking existing translog files
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!   WARNING: Elasticsearch MUST be stopped before running this tool   !
!                                                                     !
!   WARNING:    Documents inside of translog files will be lost       !
!                                                                     !
!   WARNING:          The following files will be DELETED!            !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-41.ckp
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-6.ckp
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-37.ckp
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-24.ckp
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-11.ckp

Continue and DELETE files? [y/N] y
Reading translog UUID information from Lucene commit from shard at           
[data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/index]
Translog Generation: 3
Translog UUID      : AxqC4rocTC6e0fwsljAh-Q
Removing existing translog files
Creating new empty checkpoint at     
[data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog.ckp]
Creating new empty translog at [data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog- 
3.tlog]
Done.

（2）重新索引

POST /_cluster/reroute
  {
  "commands": [
    {
  "allocate_stale_primary": {
    "index": "entrance_data",
    "shard": 0,
    "node": "node-003",
    "accept_data_loss" : true
  }
}
 ]
}

原文链接

https://www.elastic.co/guide/en/elasticsearch/reference/5.4/index-modules-translog.html#corrupt-translog-truncation