查询ES报错429 circuit_breaking_exception，“reason“:“[parent] Data too large, data for \[＜http_request\＞\]

查询ES报错：429 Too Many Requests；circuit_breaking_exception，”reason”:”[parent] Data too large, data for []“

问题：ES查询报错：429 Too Many Requests；circuit_breaking_exception，“reason”:“[parent] Data too large, data for []”
原因：ES查询缓存占用内存过大，超过阈值（默认70%），查询请求拒绝。
解决：1.清除es缓存 2.修改缓存阈值限制

• 修改ES查询缓存占用比例限制

PUT /_cluster/settings
{
  "persistent": {
    "indices.breaker.total.limit": "95%"
  }
} 

• 清理filed data cache缓存

curl -XPOST 'http://localhost:9200/fdns/_cache/clear?fielddata=true'

• 具体报错日志

2023-02-07 14:43:09.473 INFO  -- [ttp-nio-45560-exec-9] nV01ResFullTextQueryController        getResCascade: 102 :根据条件获取资源====query/allRes参数为：start getResCascade param=={"accurateParams":{"resTypeId":"MOD_BUSI_SYS"},"includeSubRegionRes":false,"includeSubSysRes":false,"mateParams":{},"mustExists":[],"needExt":false,"notExists":[],"pageNo":1,"pageSize":99999,"rangeParams":[],"sortMap":{"resName":"DESC"},"sortStr":"\"resName.keyword\":{\"order\":\"desc\"}","userAcc":"root"}
2023-02-07 14:43:09.475 ERROR -- [ttp-nio-45560-exec-9] ra.cmdb.common.util.es.ES7Util   searchTotalAndData: 502 :查询数据失败：org.elasticsearch.client.ResponseException: method [POST], host [http://BIT-5:45070], URI [/bit-matrix2124_cmdb_trans_index/_search], status line [HTTP/1.1 429 Too Many Requests]
{"error":{"root_cause":[{"type":"circuit_breaking_exception","reason":"[parent] Data too large, data for [] would be [26573653640/24.7gb], which is larger than the limit of [26521423052/24.6gb], real usage: [26573653264/24.7gb], new bytes reserved: [376/376b], usages [request=504/504b, fielddata=43375615/41.3mb, in_flight_requests=376/376b, accounting=296499042/282.7mb]","bytes_wanted":26573653640,"bytes_limit":26521423052,"durability":"PERMANENT"}],"type":"circuit_breaking_exception","reason":"[parent] Data too large, data for [] would be [26573653640/24.7gb], which is larger than the limit of [26521423052/24.6gb], real usage: [26573653264/24.7gb], new bytes reserved: [376/376b], usages [request=504/504b, fielddata=43375615/41.3mb, in_flight_requests=376/376b, accounting=296499042/282.7mb]","bytes_wanted":26573653640,"bytes_limit":26521423052,"durability":"PERMANENT"},"status":429}
	at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:347)
	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:313)
	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:288)
	at com.ultra.cmdb.common.util.es.ES7Util.doQuery(ES7Util.java:239)
	at com.ultra.cmdb.common.util.es.ES7Util.searchTotalAndData(ES7Util.java:481)
	at com.ultra.cmdb.dao.cmdb.EsDaoEs7Impl.getPageNodeByQueryJson(EsDaoEs7Impl.java:235)
	at com.ultra.cmdb.service.es.impl.ESServiceImpl.getPageNodeByQueryJson(ESServiceImpl.java:1514)
	at com.ultra.cmdb.service.es.impl.ESServiceImpl.queryResByUserIdV2(ESServiceImpl.java:831)
	at com.ultra.cmdb.service.es.impl.ESServiceImpl.getResListByParam(ESServiceImpl.java:1179)
	at com.ultra.cmdb.service.resource.ResFullTextQueryServiceImpl.getResCascade(ResFullTextQueryServiceImpl.java:378)
	at com.ultra.cmdb.service.resource.ResFullTextQueryServiceImpl$$FastClassBySpringCGLIB$$35469f9.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
	at com.ultra.cmdb.service.resource.ResFullTextQueryServiceImpl$$EnhancerBySpringCGLIB$$cea20b7e.getResCascade(<generated>)
	at com.ultra.cmdb.web.controller.open.v01.OpenV01ResFullTextQueryController.getResCascade(OpenV01ResFullTextQueryController.java:106)
	at sun.reflect.GeneratedMethodAccessor816.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:878)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:792)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:681)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at com.ultra.cmdb.xss.XssFilter.doFilter(XssFilter.java:48)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at com.ultrapower.msa.sdk.oauth.MsaAuthenticationFilter.doFilter(MsaAuthenticationFilter.java:77)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:890)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1789)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:745)

2023-02-07 14:43:09.475 INFO  -- [ttp-nio-45560-exec-9] tra.cmdb.dao.cmdb.EsDaoEs7Impl tPageNodeByQueryJson: 237 :get page node by query json es query json index:bit-matrix2124_cmdb_trans_index, cost:1, queryStr:{"track_total_hits":true,"query":{"bool":{"must":[{ "match_phrase" : { "resTypeId.keyword" : "MOD_BUSI_SYS" }}   ]  } },"from":0,"size":99999 ,"sort": {"resName.keyword":{"order":"desc"}}}
2023-02-07 14:43:09.475 INFO  -- [ttp-nio-45560-exec-9] ce.ResFullTextQueryServiceImpl        getResCascade: 380 :esSvs.getResListByParam cost : 1 ms
2023-02-07 14:43:09.475 INFO  -- [ttp-nio-45560-exec-9] nV01ResFullTextQueryController        getResCascade: 114 :根据条件获取资源===query/allRes结果为: end getResCascade total rows:null, cost:2

参考
Elasticsearch circuit_breaking_exception, Data too large异常案例

Kibana查询ES报错:[circuit_breaking_exception] [parent] Data too large

Elasticsearch-circuit_breaking_exception [parent] Data too large, data for [＜http_request＞]

Elasticsearch circuit_breaking_exception, Data too large异常案例