springcloud 在Hystrix中出现拒绝服务的熔断异常

 

java.util.concurrent.RejectedExecutionException: Task 
java.util.concurrent.FutureTask@3f54147b rejected from 
java.util.concurrent.ThreadPoolExecutor@55407c43[Running, pool size = 10, active threads = 10, queued tasks = 0, completed tasks = 2638]

 

根据文档资料得到，在默认设置下，maxQueueSize的长度为-1，则使用了SynchronousQueue

 * maxQueueSize BlockingQueue的最大队列大小，设为-1使用SynchronousQueue，否则使用LinkedBlockingQueue，默认值：-1

    翻hystrix线程池的代码，得到如果是-1的情况下，hystrix是不进行判断队列的长度的，此时，直接交由线程池去处理。

    //HystrixThreadPool.java
    @Override
        public boolean isQueueSpaceAvailable() {
            if (queueSize <= 0) {
                // we don't have a queue so we won't look for space but instead
                // let the thread-pool reject or not
                return true;
            } else {
                return threadPool.getQueue().size() < properties.queueSizeRejectionThreshold().get();
            }
        }

 通过异常，我们得知，产生的原因是因为队列无法加入任务被reject了，当我们设置-1的时候，此时的队列用的实现是SynchronousQueue（0），意味着，线程池不会进行等待，如果对应的任务满了coreSize,则直接被默认的策略reject掉。

（hystrix使用的拒绝策略是满了直接拒绝）

   //HystrixConcurrencyStrategy.java
     /**
     * Factory method to provide instance of {@code BlockingQueue} used for each {@link ThreadPoolExecutor} as constructed in {@link #getThreadPool}.
     * 

     * Note: The maxQueueSize value is provided so any type of queue can be used but typically an implementation such as {@link SynchronousQueue} without a queue (just a handoff) is preferred as
     * queueing is an anti-pattern to be purposefully avoided for latency tolerance reasons.
     * 

     * Default Implementation
     * 

     * Implementation returns {@link SynchronousQueue} when maxQueueSize <= 0 or {@link LinkedBlockingQueue} when maxQueueSize > 0.
     * 
     * @param maxQueueSize
     *            The max size of the queue requested via properties (or system default if no properties set).
     * @return instance of {@code BlockingQueue}
     */
    public BlockingQueue getBlockingQueue(int maxQueueSize) {
        /*
         * We are using SynchronousQueue if maxQueueSize <= 0 (meaning a queue is not wanted).
         * 

         * SynchronousQueue will do a handoff from calling thread to worker thread and not allow queuing which is what we want.
         * 

         * Queuing results in added latency and would only occur when the thread-pool is full at which point there are latency issues
         * and rejecting is the preferred solution.
         */
        if (maxQueueSize <= 0) {
            return new SynchronousQueue();
        } else {
            return new LinkedBlockingQueue(maxQueueSize);
        }
    }

通过上述，解决这个异常的问题通过两条路，一个是修改线程池的等待队列，一个是加大工作线程数（不建议改coreSize,应该通过修改最大线程数来变更，避免有常驻线程过多）。

1、同时设置

1、同时设置队列长度跟拒绝长度
maxQueueSize: 1000 #BlockingQueue的最大队列数，默认值-1 
queueSizeRejectionThreshold: 800 #即使maxQueueSize没有达到，达到queueSizeRejectionThreshold该值后，请求也会被拒绝，默认值5

2、同时设置最多工作线程数以及allowMaximumSizeToDivergeFromCoreSize 
maximumSize: 100
allowMaximumSizeToDivergeFromCoreSize: true