Failed to invoke the method getXXX in the service Tried 3 times of the providers [xxxx dubbo填坑
首先大的背景是这样的，公司的测试服务器是过年前才调好的。经过一番测试没有任何问题。用的框架是dubbo+zookeeper。过完春节回来，改了一个需求，用jenkins发版后。测试发现，dubbo总是提示消费端无法调用服务端的服务。异常信息如下：
异常信息
com.alibaba.dubbo.rpc.RpcException: Failed to invoke the method releasePaper in the service com.lenovo.lls.exam.service.ExamInationPaperService. Tried 3 times of the providers [127.0.0.1:20880] (1/1) from the registry 127.0.0.1:2181 on the consumer 127.0.0.1 using the dubbo version 2.5.3. Last error is: Invoke remote method timeout. method: releasePaper, provider: dubbo://127.0.0.1:20880/com.lenovo.lls.exam.service.ExamInationPaperService?anyhost=true&application=lls-web&check=false&default.timeout=5000&dubbo=2.5.3&interface=com.lenovo.lls.exam.service.ExamInationPaperService&methods=findByidPaper,toTalCountPaper,updatePaper,findByPaper,delPaper,releasePaper,insertExamInationPaper&monitor=dubbo%3A%2F%2F10.103.241.2%3A2181%2Fcom.alibaba.dubbo.registry.RegistryService%3Fapplication%3Dlls-web%26dubbo%3D2.5.3%26pid%3D5827%26protocol%3Dregistry%26refer%3Ddubbo%253D2.5.3%2526interface%253Dcom.alibaba.dubbo.monitor.MonitorService%2526pid%253D5827%2526timestamp%253D1550543049434%26registry%3Dzookeeper%26timestamp%3D1550543049434&pid=5827&side=consumer×tamp=1550543049425, cause: Waiting server-side response timeout. start time: 2019-02-19 10:25:51.091, end time: 2019-02-19 10:25:56.092, client elapsed: 0 ms, server elapsed: 5001 ms, timeout: 5000 ms, request: Request [id=772, version=2.0.0, twoway=true, event=false, broken=false, data=RpcInvocation [methodName=releasePaper, parameterTypes=[class com.lenovo.lls.exam.model.ExamInation], arguments=[com.lenovo.lls.exam.model.ExamInation@2cff2bfb], attachments={path=com.lenovo.lls.exam.service.ExamInationPaperService, interface=com.lenovo.lls.exam.service.ExamInationPaperService, version=0.0.0, timeout=5000}]], channel: /127.0.0.1:39895 -> /127.0.0.1:20880
at com.alibaba.dubbo.rpc.cluster.support.FailoverClusterInvoker.doInvoke(FailoverClusterInvoker.java:101)
at com.alibaba.dubbo.rpc.cluster.support.AbstractClusterInvoker.invoke(AbstractClusterInvoker.java:227)
at com.alibaba.dubbo.rpc.cluster.support.wrapper.MockClusterInvoker.invoke(MockClusterInvoker.java:72)
at com.alibaba.dubbo.rpc.proxy.InvokerInvocationHandler.invoke(InvokerInvocationHandler.java:52)
at com.alibaba.dubbo.common.bytecode.proxy14.releasePaper(proxy14.java)
at com.lenovo.lls.exam.controller.ExamInationPaperController.releasePaper(ExamInationPaperController.java:204)
at com.lenovo.lls.exam.controller.ExamInationPaperController$$FastClassBySpringCGLIB$$9c7530a3.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
at com.lenovo.lls.acpect.LlsProcessAspect.checkLogin(LlsProcessAspect.java:58)
at sun.reflect.GeneratedMethodAccessor288.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
at com.lenovo.lls.exam.controller.ExamInationPaperController$$EnhancerBySpringCGLIB$$162b4a0b.releasePaper()
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:648)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:209)
at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:244)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:94)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1104)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1519)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1475)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.alibaba.dubbo.remoting.TimeoutException: Waiting server-side response timeout. start time: 2019-02-19 10:25:51.091, end time: 2019-02-19 10:25:56.092, client elapsed: 0 ms, server elapsed: 5001 ms, timeout: 5000 ms, request: Request [id=772, version=2.0.0, twoway=true, event=false, broken=false, data=RpcInvocation [methodName=releasePaper, parameterTypes=[class com.lenovo.lls.exam.model.ExamInation], arguments=[com.lenovo.lls.exam.model.ExamInation@2cff2bfb], attachments={path=com.lenovo.lls.exam.service.ExamInationPaperService, interface=com.lenovo.lls.exam.service.ExamInationPaperService, version=0.0.0, timeout=5000}]], channel: /127.0.0.1:39895 -> /127.0.0.1:20880
at com.alibaba.dubbo.remoting.exchange.support.DefaultFuture.get(DefaultFuture.java:107)
at com.alibaba.dubbo.remoting.exchange.support.DefaultFuture.get(DefaultFuture.java:84)
at com.alibaba.dubbo.rpc.protocol.dubbo.DubboInvoker.doInvoke(DubboInvoker.java:96)
at com.alibaba.dubbo.rpc.protocol.AbstractInvoker.invoke(AbstractInvoker.java:144)
at com.alibaba.dubbo.monitor.support.MonitorFilter.invoke(MonitorFilter.java:65)
at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:91)
at com.alibaba.dubbo.rpc.protocol.dubbo.filter.FutureFilter.invoke(FutureFilter.java:53)
at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:91)
at com.alibaba.dubbo.rpc.filter.ConsumerContextFilter.invoke(ConsumerContextFilter.java:48)
at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:91)
at com.alibaba.dubbo.rpc.listener.ListenerInvokerWrapper.invoke(ListenerInvokerWrapper.java:74)
at com.alibaba.dubbo.rpc.protocol.InvokerWrapper.invoke(InvokerWrapper.java:53)
at com.alibaba.dubbo.rpc.cluster.support.FailoverClusterInvoker.doInvoke(FailoverClusterInvoker.java:77)
... 67 more 如上述代码所示：dubbo的消费者总是无法调用服务端的服务，并尝试了3次自动终止尝试。
常规排错
常规的解决办法如下：
主要就是检查配置文件，并查看服务是否成功注册
1.将服务的连接时间修改的久一点
2.检查客户端接口和服务端接口的实现类是否一一对应
3.传输的参数过大，导致服务端返回给客户端的时候出现问题
经检查不是
4.返回的数据，没有进行序列化，导致结果返回不了给客户端
经检查是序列化的
5.进入zookeeper的控制台，通过ls命令查看服务是否注册成功
经检查服务是注册成功的
我把能用的常规方法用了一遍，依旧没有奏效
非常规解决
从根源去解决。
首先消费端只是提示调用服务超时。于是我去找服务端，服务端也没有报错。有关查询的操作也都是没有问题的。但是插入却不行。于是我定位到是数据库的问题。
找到测试数据库服务器df -h 我的天。数据库服务器磁盘使用量为100%。好了，问题就出在了这里。是数据库服务器没有了容量，难怪只有查询没问题，插入却不行。由于数据库的插入不行，所以导致service层服务拿不到返回值，以至于dubbo消费端提示服务端连接超时。
从服务器删除多余的log文件，释放磁盘空间后。问题解决。
这个问题解决了三天，一直在想是dubbo，zookeeper的问题。我把dubbo和zookeeper全部重新安装配置后，都没有解决。真的是把所有常规的方法都用了。绞尽脑汁也没想到会是这里出了问题。
所以以后解决问题还是不要只看问题的表面，要往问题的本质去考虑。