ElasticSearch查询操作最大sharding数目限制

ElasticSearch查询操作最大sharding数目限制

1. 问题说明

在ES中做一个大范围的查询时，经常会报这个错误：

{
    "error":{
        "root_cause":[
            {
                "type":"illegal_argument_exception",
                "reason":"Trying to query 1230 shards, which is over the limit of 1000. This limit exists because querying many shards at the same time can make the job of the coordinating node very CPU and/or memory intensive. It is usually a better idea to have a smaller number of larger shards. Update [action.search.shard_count.limit] to a greater value if you really want to query that many shards at the same time."
            }
        ],
        "type":"illegal_argument_exception",
        "reason":"Trying to query 1230 shards, which is over the limit of 1000. This limit exists because querying many shards at the same time can make the job of the coordinating node very CPU and/or memory intensive. It is usually a better idea to have a smaller number of larger shards. Update [action.search.shard_count.limit] to a greater value if you really want to query that many shards at the same time."
    }
}

主要意思就是说这个查询涉及到的分片太多，超过了限制，可以通过调节这个参数action.search.shard_count.limit来修改这个限制。

2. 参数说明

action.search.shard_count.limit这个参数主要用于限制一次操作过多的分片，防止过多的占用内存和CPU资源。

相关的参数还有这些：
- index.max_result_window：限制一次查询出的数据条数
- TODO

ref

https://discuss.elastic.co/t/too-many-shards/59668