spring-boot-redis-starter源码分析

spring-boot-redis-starter源码分析

RedisAutoConfiguration自动配置类，redis链接工厂以及哨兵等监听

1. RedisAutoConfiguration里面主要做JedisConnectionFactory的装配工作：
   根据配置的sentinel或者cluster进行不同的装配，加载配置等。

@Bean
@ConditionalOnMissingBean(RedisConnectionFactory.class)
public JedisConnectionFactory redisConnectionFactory()
    throws UnknownHostException {
  return applyProperties(createJedisConnectionFactory());
}

2. JedisConnectionFactory实现了InitializingBean，在装载Bean之后调用afterPropertiesSet方法进行池JedisSentinelPool的创建

public void afterPropertiesSet() {
  if (shardInfo == null) {
    shardInfo = new JedisShardInfo(hostName, port);

    if (StringUtils.hasLength(password)) {
      shardInfo.setPassword(password);
    }

    if (timeout > 0) {
      setTimeoutOn(shardInfo, timeout);
    }
  }

  if (usePool && clusterConfig == null) {
    this.pool = createPool();
  }

  if (clusterConfig != null) {
    this.cluster = createCluster();
  }
}

这里主要是pool的创建，pool中主要存储的是Jedis对象，主要执行Jedis的管理工作，为后续连接的租借、归还进行一定的处理.

3. JedisSentinelPool中首先initSentinels会根据sentinel的配置获取到主master节点的信息,遍历每个sentinel节点执行sentinel get-master-addr-by-name mymaster命令获取master(其中一个返回master即可)

for (String sentinel : sentinels) {
      final HostAndPort hap = HostAndPort.parseString(sentinel);

      log.fine("Connecting to Sentinel " + hap);

      Jedis jedis = null;
      try {
        jedis = new Jedis(hap.getHost(), hap.getPort());

        List<String> masterAddr = jedis.sentinelGetMasterAddrByName(masterName);

        // connected to sentinel...
        sentinelAvailable = true;

        if (masterAddr == null || masterAddr.size() != 2) {
          log.warning("Can not get master addr, master name: " + masterName + ". Sentinel: " + hap
              + ".");
          continue;
        }

        master = toHostAndPort(masterAddr);
        log.fine("Found Redis master at " + master);
        break;
      } catch (JedisException e) {
        log.warning("Cannot get master address from sentinel running @ " + hap + ". Reason: " + e
            + ". Trying next one.");
      } finally {
        if (jedis != null) {
          jedis.close();
        }
      }
    }

4. 开启一个Daemon后台线程订阅+switch-master的消息，当master切换后收到sentinel发送的切换通知重新初始化连接pool。

for (String sentinel : sentinels) {
      final HostAndPort hap = HostAndPort.parseString(sentinel);
      MasterListener masterListener = new MasterListener(masterName, hap.getHost(), hap.getPort());
      // whether MasterListener threads are alive or not, process can be stopped
      masterListener.setDaemon(true);
      masterListeners.add(masterListener);
      masterListener.start();
    }

5. MasterListener的作用就是对每个哨兵进行订阅消息，当收到通知后改变currentHostMaster，进行切换

MasterListener
@Override
    public void run() {
      running.set(true);
      while (running.get()) {
        j = new Jedis(host, port);
        try {
          // double check that it is not being shutdown
          if (!running.get()) {
            break;
          }
          j.subscribe(new JedisPubSub() {
            @Override
            public void onMessage(String channel, String message) {
              log.fine("Sentinel " + host + ":" + port + " published: " + message + ".");
              String[] switchMasterMsg = message.split(" ");
              if (switchMasterMsg.length > 3) {
                if (masterName.equals(switchMasterMsg[0])) {
                  initPool(toHostAndPort(Arrays.asList(switchMasterMsg[3], switchMasterMsg[4])));
                } else {
                  log.fine("Ignoring message on +switch-master for master name "
                      + switchMasterMsg[0] + ", our master name is " + masterName);
                }
              } else {
                log.severe("Invalid message received on Sentinel " + host + ":" + port
                    + " on channel +switch-master: " + message);
              }
            }
          }, "+switch-master");
        } catch (JedisConnectionException e) {
          if (running.get()) {
            log.log(Level.SEVERE, "Lost connection to Sentinel at " + host + ":" + port
                + ". Sleeping 5000ms and retrying.", e);
            try {
              Thread.sleep(subscribeRetryWaitTimeMillis);
            } catch (InterruptedException e1) {
              log.log(Level.SEVERE, "Sleep interrupted: ", e1);
            }
          } else {
            log.fine("Unsubscribing from Sentinel at " + host + ":" + port);
          }
        } finally {
          j.close();
        }
      }
    }

6. 根据上面步骤获取到的master执行initPool(master)初始化redis的连接池，保存当前的master节点信息至currentHostMaster(供maser改变时判断)，创建JedisFactory工厂

private void initPool(HostAndPort master) {
  if (!master.equals(currentHostMaster)) {
    currentHostMaster = master;
    if (factory == null) {
      factory = new JedisFactory(master.getHost(), master.getPort(), connectionTimeout,
          soTimeout, password, database, clientName, false, null, null, null);
      initPool(poolConfig, factory);
    } else {
      factory.setHostAndPort(currentHostMaster);
      // 当master改变时，会把pool里面所有的链接清空，重新建立，注意此时仅仅清理的是空闲的链接
      internalPool.clear();
    }
    log.info("Created JedisPool to master at " + master);
  }
}

7. 初始化链接池，创建内部池对象,创建空闲队列

public void initPool(final GenericObjectPoolConfig poolConfig, PooledObjectFactory<T> factory) {

  if (this.internalPool != null) {
    try {
      closeInternalPool();
    } catch (Exception e) {
    }
  }
  this.internalPool = new GenericObjectPool<T>(factory, poolConfig);
}

RedisTemplate

设置key和value的序列化方式、执行redis基本操作、封装ValueOperations,ListOperations,SetOperations,ZSetOperations等执行类、设置key的过期时间、获取绑定key的ValueOperations,ListOperations,SetOperations,ZSetOperations操作.

1. 封装redis基本操作类

方法	说明
opsForHash()	HashOperations相关操作
opsForList()	ListOperations相关操作
opsForSet()	SetOperations相关操作
opsForValue()	ValueOperations相关操作

2. 绑定key的操作
   对于上述的操作，可以使用其相关的boundHashOps(K key)获取绑定key的HashOperations，则此时所有的操作均可以不用指定hashkey即可进行相应的hash操作了.

3. 指定序列化方式
   setKeySerializer(RedisSerializer serializer):
   setHashKeySerializer(RedisSerializer hashKeySerializer)
   setValueSerializer(RedisSerializer serializer)
   setHashValueSerializer(RedisSerializer hashValueSerializer)
   目前常用的序列化方式有四种：
   StringRedisSerializer、JdkSerializationRedisSerializer、JacksonJsonRedisSerializer、Jackson2JsonRedisSerializer。
   RedisTemplate中默认使用的是JdkSerializationRedisSerializer

4. exec操作:根据不同的场景获取redis连接，使用RedisCallback回调进行相应的操作，这里exec里面主要是连接的获取以及释放。

public <T> T execute(RedisCallback<T> action, boolean exposeConnection, boolean pipeline) {
		Assert.isTrue(initialized, "template not initialized; call afterPropertiesSet() before using it");
		Assert.notNull(action, "Callback object must not be null");

		RedisConnectionFactory factory = getConnectionFactory();
		RedisConnection conn = null;
		try {

			if (enableTransactionSupport) {
				// only bind resources in case of potential transaction synchronization
				conn = RedisConnectionUtils.bindConnection(factory, enableTransactionSupport);
			} else {
				conn = RedisConnectionUtils.getConnection(factory);
			}

			boolean existingConnection = TransactionSynchronizationManager.hasResource(factory);

			RedisConnection connToUse = preProcessConnection(conn, existingConnection);

			boolean pipelineStatus = connToUse.isPipelined();
			if (pipeline && !pipelineStatus) {
				connToUse.openPipeline();
			}

			RedisConnection connToExpose = (exposeConnection ? connToUse : createRedisConnectionProxy(connToUse));
			T result = action.doInRedis(connToExpose);

			// close pipeline
			if (pipeline && !pipelineStatus) {
				connToUse.closePipeline();
			}

			// TODO: any other connection processing?
			return postProcessResult(result, connToUse, existingConnection);
		} finally {
			RedisConnectionUtils.releaseConnection(conn, factory);
		}

执行操作时的链接建立过程

1. RedisConnectionUtils.getConnection(factory)
   使用工具类获取连接，其中factory为JedisConnectionFactory，为在autoconfigration中生成的bean

public static RedisConnection doGetConnection(RedisConnectionFactory factory, boolean allowCreate, boolean bind,
			boolean enableTransactionSupport) {
        ...
        // 创建连接
		RedisConnection conn = factory.getConnection();
        ...
		return conn;
	}

2. JedisConnectionFactory连接工厂中创建redis的连接,其中redis的链接创建基于jedis实例，因此需要创建一个Jedis实例

public RedisConnection getConnection() {

  if (cluster != null) {
    return getClusterConnection();
  }

  Jedis jedis = fetchJedisConnector();
  JedisConnection connection = (usePool ? new JedisConnection(jedis, pool, dbIndex, clientName)
      : new JedisConnection(jedis, null, dbIndex, clientName));
  connection.setConvertPipelineAndTxResults(convertPipelineAndTxResults);
  return postProcessConnection(connection);
}

3. jedis创建过程，当使用连接池时，jedis对象统一在Pool池中管理并创建以及销毁

// 创建jedis实例，通过jedis实例创建JedisConnection
protected Jedis fetchJedisConnector() {
  try {

    if (usePool && pool != null) {
      return pool.getResource();  
    }

    Jedis jedis = new Jedis(getShardInfo());
    // force initialization (see Jedis issue #82)
    jedis.connect();

    potentiallySetClientName(jedis);
    return jedis;
  } catch (Exception ex) {
    throw new RedisConnectionFailureException("Cannot get Jedis connection", ex);
  }
}

上面中连接的获取主要是通过pool.getResource()获取的，其中pool池对象也是在redis自动配置的时候随工厂类创建的。另外pool.getResource()中主要调用的是Pool中的borrowObject方法。 Pool基于commons-pool2的第三方类库，主要使用到了池化技术，其中Pool中泛型为Jedis，标识池中存储的对象为Jedis
commons-pool2池化技术

commons-pool2池化技术

Common-pool2由三大模块组成：ObjectPool、PooledObject、PooledObjectFactory。

1. ObjectPool：提供所有对象的存取管理
2. PooledObject：池化的对象，是对对象的一个包装，加上了对象的一些其他信息，包括对象的状态（已用、空闲），对象的创建时间等
3. PooledObjectFactory：工厂类，负责池化对象的创建，对象的初始化，对象状态的销毁和对象状态的验证
4. ObjectPool会持有PooledObjectFactory，将具体的对象的创建、初始化、销毁等任务交给它处理，其操作对象是PooledObject，即具体的Object的包装类

GenericObjectPool.borrowObject()大体思路如下

首先明确几个重要的属性：

idleObjects：LinkedBlockingDeque类型，队列中存储的是空闲的Jedis对象；
allObjects:Map类型，和当前池关联的所有对象,Map的大小少于或等于maxActive的值。

1. 从LinkedBlockingDeque中pollFirst p = idleObjects.pollFirst();
2. 当p为空,即未取到空闲的Jedis对象时，则创建 p = create();

private PooledObject<T> create() throws Exception {
    int localMaxTotal = getMaxTotal();
    if (localMaxTotal < 0) {
        localMaxTotal = Integer.MAX_VALUE;
    }
    // 创建池对象
    Boolean create = null;
    while (create == null) {
        synchronized (makeObjectCountLock) {
            // 创建时，数量先 +1，判断是否超过最大数量
            final long newCreateCount = createCount.incrementAndGet();
            if (newCreateCount > localMaxTotal) {
                // 当前池中的数量已经足够，此刻不允许创建，先将刚才的+1恢复
                createCount.decrementAndGet();
                if (makeObjectCount == 0) {
                    // 当前池中数量已满并且当前线程中无创建时，此时直接返回null
                    create = Boolean.FALSE;
                } else {
                  // 当前池中数量已满并且当前线程中存在创建时，此时需要进行等待其他线程执行完成            
                    makeObjectCountLock.wait();
                }
            } else {
                // The pool is not at capacity. Create a new object.
                makeObjectCount++;
                create = Boolean.TRUE;
            }
        }
    }
    if (!create.booleanValue()) {
        return null;
    }
    final PooledObject<T> p;
    try {
        p = factory.makeObject();
    } catch (final Exception e) {
        createCount.decrementAndGet();
        throw e;
    } finally {
        synchronized (makeObjectCountLock) {
            makeObjectCount--;
            makeObjectCountLock.notifyAll();
        }
    }
    final AbandonedConfig ac = this.abandonedConfig;
    if (ac != null && ac.getLogAbandoned()) {
        p.setLogAbandoned(true);
    }
    createdCount.incrementAndGet();
    allObjects.put(new IdentityWrapper<T>(p.getObject()), p);
    return p;
}

3. 无法创建时，即上述函数返回为null，此时去判断blockWhenExhausted的值(默认为true)，如果为false并且p==null时，此时无法获取Jedis连接，则抛出异常Pool exhausted；

// 当pool中的资源耗尽时是否阻塞，默认为true
if (blockWhenExhausted) {
    if (p == null) {
        if (borrowMaxWaitMillis < 0) {
            //当配置MaxWait为-1，此时会一直取直到从空闲队列中取出
            p = idleObjects.takeFirst();
        } else {
            //否则从空闲队列中获取，直到给定的时间MaxWait达到
            p = idleObjects.pollFirst(borrowMaxWaitMillis,
                    TimeUnit.MILLISECONDS);
        }
    }
    if (p == null) {
        // 在空闲队列中获取连接时，超过了等到时间，抛出该异常
        throw new NoSuchElementException(
                "Timeout waiting for idle object");
    }
} else {
    if (p == null) {
        //pool耗尽不阻塞时，直接抛出Pool exhausted异常
        throw new NoSuchElementException("Pool exhausted");
    }
}

4. 对上述获取到的池对象进行分配操作：主要是修改该对象的状态，记录租借时间以及使用时间。尝试激活对象,JedisFactory激活对象的操作竟然只是判断其Jedis中的db是否选择正确。

if (!p.allocate()) {
    p = null;
}

factory.activateObject(p);

5. 判断上述获取的池对象是否需要进行验证

// 判断租借对象时是否需要测试(默认为false)；判断新创建的事都需要测试(默认为false)
if (p != null && (getTestOnBorrow() || create && getTestOnCreate())) {
    boolean validate = false;
    Throwable validationThrowable = null;
    try {
        validate = factory.validateObject(p);
    } catch (final Throwable t) {
        PoolUtils.checkRethrow(t);
        validationThrowable = t;
    }
    if (!validate) {
        try {
            destroy(p);
            destroyedByBorrowValidationCount.incrementAndGet();
        } catch (final Exception e) {
            // Ignore - validation failure is more important
        }
        p = null;
        if (create) {
            final NoSuchElementException nsee = new NoSuchElementException(
                    "Unable to validate object");
            nsee.initCause(validationThrowable);
            throw nsee;
        }
    }
}

验证的方式也是比较简单，判断Jedis中的host、port是否一致，并且执行ping命令后是否返回内容为PONG

public boolean validateObject(PooledObject<Jedis> pooledJedis) {
  final BinaryJedis jedis = pooledJedis.getObject();
  try {
    HostAndPort hostAndPort = this.hostAndPort.get();
    String connectionHost = jedis.getClient().getHost();
    int connectionPort = jedis.getClient().getPort();
    return hostAndPort.getHost().equals(connectionHost)
        && hostAndPort.getPort() == connectionPort && jedis.isConnected()
        && jedis.ping().equals("PONG");
  } catch (final Exception e) {
    return false;
  }
}

Redis链接释放过程

RedisTemplate在exec完相关的操作之后会主动将连接释放，其释放过程如下：

public <T> T execute(RedisCallback<T> action, boolean exposeConnection, boolean pipeline) {
  ......
  } finally {
    RedisConnectionUtils.releaseConnection(conn, factory);
  }

1. 调用工具类RedisConnectionUtils中的releaseConnection方法，该方法中会根据该连接是否为事务进行一顿操作：

if (isConnectionTransactional(conn, factory)
    && TransactionSynchronizationManager.isCurrentTransactionReadOnly()) {
  unbindConnection(factory);
} else if (!isConnectionTransactional(conn, factory)) {
  if (log.isDebugEnabled()) {
    log.debug("Closing Redis Connection");
  }
  conn.close();
}

这里分析下非事务操作的情况，连接关闭主要是使用了JedisConnection中的close方法

2. JedisConnection中的close方法,有如下几种判断:支持pool(pool!=null)时，连接不断开，直接归还到pool中;判断当前连接是否broken(在执行操作时是否抛出了异常),如果为broken的话，会使用 java pool.returnBrokenResource(jedis)(去激活，从idleObjects、allObjects中移除该对象)方法返回; 如果pool已经为null,则直接断开该jedis

if (pool != null) {
  if (broken) {
    pool.returnBrokenResource(jedis);
  } else {
    try {
       // 归回conn时，默认将该连接的操作db设置为db0
      if (dbIndex > 0) {
        jedis.select(0);
      }
      return;
    } catch (Exception ex) {
      throw convertJedisAccessException(ex);
    } finally {
      // 调用jedis中的close方法关闭连接
      jedis.close();
    }
  }
}

3. Jedis的close方法，如果client没有broken的话,(connection连接没有出现任何异常),则使用Pool中的returnResource方法归还Jedis连接

public void close() {
  if (dataSource != null) {
    if (client.isBroken()) {
      this.dataSource.returnBrokenResource(this);
    } else {
      this.dataSource.returnResource(this);
    }
  } else {
    client.close();
  }
}

4. 一路调用，最终归还池对象的方法还是在中，GenericObjectPool.returnObject归还对象,任然是在common-pool2中实现，
5. 首先根据归还的对象名称从allObjects的Map中获取到该对象；
6. 判断该对象的状态，如果不是已经分配，则说明该对象已经被返回,抛出异常;
7. 根据配置判断归还时是否进行测试，默认不测试；需要测试而且测试不成功时销毁
8. 将对该对象执行passive操作，其实什么也没做
9. 执行deallocate操作，其实就是修改该对象的state为IDLE，并记录lastReturnTime
10. 判断该对象是销毁还是放到空闲队列中，根据maxIdle的值判断，如果该值比空闲队列的size小，则此时应该destroy(idleObjects、allObjects中remove)，否则add到idleObjects中供下次租借

public void returnObject(final T obj) {
    // 根据对象名称在allObjects获取到该对象
    final PooledObject<T> p = allObjects.get(new IdentityWrapper<T>(obj));
    if (p == null) {
        if (!isAbandonedConfig()) {
            throw new IllegalStateException(
                    "Returned object not currently part of this pool");
        }
        return; // Object was abandoned and removed
    }

    synchronized(p) {
        final PooledObjectState state = p.getState();
        if (state != PooledObjectState.ALLOCATED) {
            throw new IllegalStateException(
                    "Object has already been returned to this pool or is invalid");
        }
        p.markReturning(); // Keep from being marked abandoned
    }

    final long activeTime = p.getActiveTimeMillis();

    if (getTestOnReturn()) {
        if (!factory.validateObject(p)) {
            try {
                destroy(p);
            } catch (final Exception e) {
                swallowException(e);
            }
            try {
                ensureIdle(1, false);
            } catch (final Exception e) {
                swallowException(e);
            }
            updateStatsReturn(activeTime);
            return;
        }
    }

    try {
        factory.passivateObject(p);
    } catch (final Exception e1) {
        swallowException(e1);
        try {
            destroy(p);
        } catch (final Exception e) {
            swallowException(e);
        }
        try {
            ensureIdle(1, false);
        } catch (final Exception e) {
            swallowException(e);
        }
        updateStatsReturn(activeTime);
        return;
    }

    if (!p.deallocate()) {
        throw new IllegalStateException(
                "Object has already been returned to this pool or is invalid");
    }

    final int maxIdleSave = getMaxIdle();
    if (isClosed() || maxIdleSave > -1 && maxIdleSave <= idleObjects.size()) {
        try {
            destroy(p);
        } catch (final Exception e) {
            swallowException(e);
        }
    } else {
        if (getLifo()) {
            idleObjects.addFirst(p);
        } else {
            idleObjects.addLast(p);
        }
        if (isClosed()) {
            // Pool closed while object was being added to idle objects.
            // Make sure the returned object is destroyed rather than left
            // in the idle object pool (which would effectively be a leak)
            clear();
        }
    }
    updateStatsReturn(activeTime);
}

几个重要的redis操作

RedisConnectionFactory

1. getConnection

提供一个稳定的连接与redis交互，在redistemplate执行命令时，也是通过该方法获取的connection，两种取到获取，一种是在idleObject中获取，一种是create。

2. getClusterConnection

获取一个稳定的连接与redis集群交互：

RedisClusterConnection extends RedisConnection, RedisClusterCommands:

String ping(RedisClusterNode node);
void bgReWriteAof(RedisClusterNode node);
void bgSave(RedisClusterNode node);
Long lastSave(RedisClusterNode node);
void save(RedisClusterNode node);
Long dbSize(RedisClusterNode node);
void flushDb(RedisClusterNode node);
void flushAll(RedisClusterNode node);
Properties info(RedisClusterNode node);
Set<byte[]> keys(RedisClusterNode node, byte[] pattern);
byte[] randomKey(RedisClusterNode node);
List<String> getConfig(RedisClusterNode node, String pattern);
void setConfig(RedisClusterNode node, String param, String value);
void resetConfigStats(RedisClusterNode node);
Long time(RedisClusterNode node);
public List<RedisClientInfo> getClientList(RedisClusterNode node);

RedisClusterCommands:

3. getSentinelConnection

获取一个稳定的连接与redis哨兵交互：

RedisSentinelConnection extends RedisSentinelCommands, Closeable:

boolean isOpen();

RedisSentinelCommands:
void failover(NamedNode master);
Collection<RedisServer> masters();
Collection<RedisServer> slaves(NamedNode master);
void remove(NamedNode master);
void monitor(RedisServer master);

如从哨兵获取master操作时可以采用redisTemplate.getConnectionFactory().getSentinelConnection().masters();方式获取。