（九）elasticsearch常见问题总结

1：分片未分配:unassigned_shards

现象：GET _cluster/health看到unassigned_shards
分析出未分配的分片信息及未分配的原因：

curl -XGET "http://ip:port/_cat/shards?v&h=index,shard,prirep,state,node,unassigned.reason" > shards.txt
获取索引名：cat shards.txt | grep UNASSIGNED
获取具体的分片失败原因：curl -XGET "http://194.168.223.19:24100/_cluster/allocation/explain?pretty" -H 'Content-Type:application/json' -d '{"index":"indexName","shard":shardnum,"primary":false}'
shardnum:可由GET _cat/shards?v 查看获取，

解决根本目的是找到未分配的原因解决并分配
解决1：开启重分配命令：

curl -XPUT 'localhost:9200/_cluster/settings'-d
'{
      "transient":
  {
     "cluster.routing.allocation.enable" : "all" 
  }
}'

解决2：若上面方式仍然无法解决，可用下面：

curl -XPUT "http://194.168.223.20:24100/_cluster/settings?pretty&master_timeout=180s" -H 'Content-Type:application/json' -d '{"transient":{"cluster.routing.allocation.allow_rebalance":"indices_primaries_active"}}'
再执行：POST _cluster/reroute?retry_failed=true
执行完分片会移动分配：GET _cluster/health 查看unassigned_shards是否减少

2： elasticsearch exception:java.io.IOException: listener timeout after waiting for [300000] ms

分析可知是某索引的请求集中在某台服务器导致的，也就是分片不均衡

ElasticsearchException[Elasticsearch exception [type=es_rejected_execution_exception, reason=rejected execution of processing of [114475]
[indices:data/write/bulk[s][p]]: request: BulkShardRequest [[index][11]] containing [52] requests, target allocation id:sWzFdprSRRCQzyeBZGqttQ,
primary term: 1 on EsThreadPoolExecutor[name =ip/write, queue capacity = 1000, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@d44250d
[Running, pool size = 40,active threads = 40, queued tasks = 1075, completed tasks = 44022]]]]

1：解决

1：查看线程池情况：GET _cat/thread_pool?v
找出拒绝多的节点
2：查看大量写的索引的分片分配位置：GET _cat/shards?v
找出正写入的索引是否分片分布不均衡，多数在上面查找的服务器上
也可能正在写入的分片多大导致，是否大于50G的会严重影响写入速度
3：查看集群是否存在未分配的分片进行处理：见问题1
4：进行分片分配优化，设置索引的setting。
最主要参数：total_shards_per_node（单个节点上单个索引分配分片最多个数）

put index/setting
{
     
"index" : {
     
        "mapping" : {
     
          "total_fields" : {
     
            "limit" : "2000"
          }
        },
        "refresh_interval" : "180s",
        "number_of_replicas" : "1",
        "routing" : {
     
          "allocation" : {
     
            "total_shards_per_node" : "4"
          }
        },
        "merge" : {
     
          "scheduler" : {
     
            "max_thread_count" : "1"
          },
          "policy" : {
     
            "segments_per_tier" : "20",
            "max_merge_at_once" : "20"
          }
        },
        "max_slices_per_scroll" : "5000"
      }
  }