聊聊JedisFactory
序
本文主要研究一下JedisFactory
PooledObjectFactory
org/apache/commons/pool2/PooledObjectFactory.java
/**
* An interface defining life-cycle methods for instances to be served by an
* {@link ObjectPool}.
*
* By contract, when an {@link ObjectPool} delegates to a
* {@link PooledObjectFactory},
*
*
* -
* {@link #makeObject} is called whenever a new instance is needed.
*
* -
* {@link #activateObject} is invoked on every instance that has been
* {@link #passivateObject passivated} before it is
* {@link ObjectPool#borrowObject borrowed} from the pool.
*
* -
* {@link #validateObject} may be invoked on {@link #activateObject activated}
* instances to make sure they can be {@link ObjectPool#borrowObject borrowed}
* from the pool. {@link #validateObject} may also be used to
* test an instance being {@link ObjectPool#returnObject returned} to the pool
* before it is {@link #passivateObject passivated}. It will only be invoked
* on an activated instance.
*
* -
* {@link #passivateObject} is invoked on every instance when it is returned
* to the pool.
*
* -
* {@link #destroyObject} is invoked on every instance when it is being
* "dropped" from the pool (whether due to the response from
* {@link #validateObject}, or for reasons specific to the pool
* implementation.) There is no guarantee that the instance being destroyed
* will be considered active, passive or in a generally consistent state.
*
*
* {@link PooledObjectFactory} must be thread-safe. The only promise
* an {@link ObjectPool} makes is that the same instance of an object will not
* be passed to more than one method of a {@code PoolableObjectFactory}
* at a time.
*
* While clients of a {@link KeyedObjectPool} borrow and return instances of
* the underlying value type {@code V}, the factory methods act on instances of
* {@link PooledObject PooledObject<V>}. These are the object wrappers that
* pools use to track and maintain state information about the objects that
* they manage.
*
*
* @param Type of element managed in this factory.
*
* @see ObjectPool
*
* @since 2.0
*/
public interface PooledObjectFactory {
/**
* Creates an instance that can be served by the pool and wrap it in a
* {@link PooledObject} to be managed by the pool.
*
* @return a {@code PooledObject} wrapping an instance that can be served by the pool
*
* @throws Exception if there is a problem creating a new instance,
* this will be propagated to the code requesting an object.
*/
PooledObject makeObject() throws Exception;
/**
* Destroys an instance no longer needed by the pool.
*
* It is important for implementations of this method to be aware that there
* is no guarantee about what state {@code obj} will be in and the
* implementation should be prepared to handle unexpected errors.
*
*
* Also, an implementation must take in to consideration that instances lost
* to the garbage collector may never be destroyed.
*
*
* @param p a {@code PooledObject} wrapping the instance to be destroyed
*
* @throws Exception should be avoided as it may be swallowed by
* the pool implementation.
*
* @see #validateObject
* @see ObjectPool#invalidateObject
*/
void destroyObject(PooledObject p) throws Exception;
/**
* Ensures that the instance is safe to be returned by the pool.
*
* @param p a {@code PooledObject} wrapping the instance to be validated
*
* @return {@code false} if {@code obj} is not valid and should
* be dropped from the pool, {@code true} otherwise.
*/
boolean validateObject(PooledObject p);
/**
* Reinitializes an instance to be returned by the pool.
*
* @param p a {@code PooledObject} wrapping the instance to be activated
*
* @throws Exception if there is a problem activating {@code obj},
* this exception may be swallowed by the pool.
*
* @see #destroyObject
*/
void activateObject(PooledObject p) throws Exception;
/**
* Uninitializes an instance to be returned to the idle object pool.
*
* @param p a {@code PooledObject} wrapping the instance to be passivated
*
* @throws Exception if there is a problem passivating {@code obj},
* this exception may be swallowed by the pool.
*
* @see #destroyObject
*/
void passivateObject(PooledObject p) throws Exception;
}
PooledObjectFactory接口定义了makeObject、destroyObject、validateObject、activateObject、passivateObject方法
JedisFactory
redis/clients/jedis/JedisFactory.java
class JedisFactory implements PooledObjectFactory {
private final AtomicReference hostAndPort = new AtomicReference();
private final int connectionTimeout;
private final int soTimeout;
private final String user;
private final String password;
private final int database;
private final String clientName;
private final boolean ssl;
private final SSLSocketFactory sslSocketFactory;
private final SSLParameters sslParameters;
private final HostnameVerifier hostnameVerifier;
//......
@Override
public void activateObject(PooledObject pooledJedis) throws Exception {
final BinaryJedis jedis = pooledJedis.getObject();
if (jedis.getDB() != database) {
jedis.select(database);
}
}
@Override
public void destroyObject(PooledObject pooledJedis) throws Exception {
final BinaryJedis jedis = pooledJedis.getObject();
if (jedis.isConnected()) {
try {
try {
jedis.quit();
} catch (Exception e) {
}
jedis.disconnect();
} catch (Exception e) {
}
}
}
@Override
public PooledObject makeObject() throws Exception {
final HostAndPort hp = this.hostAndPort.get();
final Jedis jedis = new Jedis(hp.getHost(), hp.getPort(), connectionTimeout, soTimeout,
ssl, sslSocketFactory, sslParameters, hostnameVerifier);
try {
jedis.connect();
if (user != null) {
jedis.auth(user, password);
} else if (password != null) {
jedis.auth(password);
}
if (database != 0) {
jedis.select(database);
}
if (clientName != null) {
jedis.clientSetname(clientName);
}
} catch (JedisException je) {
jedis.close();
throw je;
}
return new DefaultPooledObject<>(jedis);
}
@Override
public void passivateObject(PooledObject pooledJedis) throws Exception {
// TODO maybe should select db 0? Not sure right now.
}
@Override
public boolean validateObject(PooledObject pooledJedis) {
final BinaryJedis jedis = pooledJedis.getObject();
try {
HostAndPort hostAndPort = this.hostAndPort.get();
String connectionHost = jedis.getClient().getHost();
int connectionPort = jedis.getClient().getPort();
return hostAndPort.getHost().equals(connectionHost)
&& hostAndPort.getPort() == connectionPort && jedis.isConnected()
&& jedis.ping().equals("PONG");
} catch (final Exception e) {
return false;
}
}
}
JedisFactory实现了PooledObjectFactory接口，其泛型为Jedis；其makeObject方法主要是创建Jedis并connect，auth、还有select db；其destroyObject主要是执行quit及disconnect方法；其validateObject方法执行ping命令；其activateObject方法判断db是否变化，有变化则重新select db；其passivateObject目前为空实现
borrowObject
/**
* Borrows an object from the pool using the specific waiting time which only
* applies if {@link #getBlockWhenExhausted()} is true.
*
* If there is one or more idle instance available in the pool, then an
* idle instance will be selected based on the value of {@link #getLifo()},
* activated and returned. If activation fails, or {@link #getTestOnBorrow()
* testOnBorrow} is set to {@code true} and validation fails, the
* instance is destroyed and the next available instance is examined. This
* continues until either a valid instance is returned or there are no more
* idle instances available.
*
*
* If there are no idle instances available in the pool, behavior depends on
* the {@link #getMaxTotal() maxTotal}, (if applicable)
* {@link #getBlockWhenExhausted()} and the value passed in to the
* {@code borrowMaxWaitMillis} parameter. If the number of instances
* checked out from the pool is less than {@code maxTotal,} a new
* instance is created, activated and (if applicable) validated and returned
* to the caller. If validation fails, a {@code NoSuchElementException}
* is thrown.
*
*
* If the pool is exhausted (no available idle instances and no capacity to
* create new ones), this method will either block (if
* {@link #getBlockWhenExhausted()} is true) or throw a
* {@code NoSuchElementException} (if
* {@link #getBlockWhenExhausted()} is false). The length of time that this
* method will block when {@link #getBlockWhenExhausted()} is true is
* determined by the value passed in to the {@code borrowMaxWaitMillis}
* parameter.
*
*
* When the pool is exhausted, multiple calling threads may be
* simultaneously blocked waiting for instances to become available. A
* "fairness" algorithm has been implemented to ensure that threads receive
* available instances in request arrival order.
*
*
* @param borrowMaxWaitMillis The time to wait in milliseconds for an object
* to become available
*
* @return object instance from the pool
*
* @throws NoSuchElementException if an instance cannot be returned
*
* @throws Exception if an object instance cannot be returned due to an
* error
*/
public T borrowObject(final long borrowMaxWaitMillis) throws Exception {
assertOpen();
final AbandonedConfig ac = this.abandonedConfig;
if (ac != null && ac.getRemoveAbandonedOnBorrow() &&
(getNumIdle() < 2) &&
(getNumActive() > getMaxTotal() - 3) ) {
removeAbandoned(ac);
}
PooledObject p = null;
// Get local copy of current config so it is consistent for entire
// method execution
final boolean blockWhenExhausted = getBlockWhenExhausted();
boolean create;
final long waitTime = System.currentTimeMillis();
while (p == null) {
create = false;
p = idleObjects.pollFirst();
if (p == null) {
p = create();
if (p != null) {
create = true;
}
}
if (blockWhenExhausted) {
if (p == null) {
if (borrowMaxWaitMillis < 0) {
p = idleObjects.takeFirst();
} else {
p = idleObjects.pollFirst(borrowMaxWaitMillis,
TimeUnit.MILLISECONDS);
}
}
if (p == null) {
throw new NoSuchElementException(
"Timeout waiting for idle object");
}
} else {
if (p == null) {
throw new NoSuchElementException("Pool exhausted");
}
}
if (!p.allocate()) {
p = null;
}
if (p != null) {
try {
factory.activateObject(p);
} catch (final Exception e) {
try {
destroy(p);
} catch (final Exception e1) {
// Ignore - activation failure is more important
}
p = null;
if (create) {
final NoSuchElementException nsee = new NoSuchElementException(
"Unable to activate object");
nsee.initCause(e);
throw nsee;
}
}
if (p != null && getTestOnBorrow()) {
boolean validate = false;
Throwable validationThrowable = null;
try {
validate = factory.validateObject(p);
} catch (final Throwable t) {
PoolUtils.checkRethrow(t);
validationThrowable = t;
}
if (!validate) {
try {
destroy(p);
destroyedByBorrowValidationCount.incrementAndGet();
} catch (final Exception e) {
// Ignore - validation failure is more important
}
p = null;
if (create) {
final NoSuchElementException nsee = new NoSuchElementException(
"Unable to validate object");
nsee.initCause(validationThrowable);
throw nsee;
}
}
}
}
}
updateStatsBorrow(p, System.currentTimeMillis() - waitTime);
return p.getObject();
}
borrowObject方法在从idleObjects.pollFirst获取到的jedis不为null的时候会执行activateObject(
这里db没有变化，相等于空操作)，然后如果testOnBorrow为true则执行validateObject(执行ping命令)
evict
org/apache/commons/pool2/impl/GenericObjectPool.java
/**
* {@inheritDoc}
*
* Successive activations of this method examine objects in sequence,
* cycling through objects in oldest-to-youngest order.
*
*/
@Override
public void evict() throws Exception {
assertOpen();
if (idleObjects.size() > 0) {
PooledObject underTest = null;
final EvictionPolicy evictionPolicy = getEvictionPolicy();
synchronized (evictionLock) {
final EvictionConfig evictionConfig = new EvictionConfig(
getMinEvictableIdleTimeMillis(),
getSoftMinEvictableIdleTimeMillis(),
getMinIdle());
final boolean testWhileIdle = getTestWhileIdle();
for (int i = 0, m = getNumTests(); i < m; i++) {
if (evictionIterator == null || !evictionIterator.hasNext()) {
evictionIterator = new EvictionIterator(idleObjects);
}
if (!evictionIterator.hasNext()) {
// Pool exhausted, nothing to do here
return;
}
try {
underTest = evictionIterator.next();
} catch (final NoSuchElementException nsee) {
// Object was borrowed in another thread
// Don't count this as an eviction test so reduce i;
i--;
evictionIterator = null;
continue;
}
if (!underTest.startEvictionTest()) {
// Object was borrowed in another thread
// Don't count this as an eviction test so reduce i;
i--;
continue;
}
// User provided eviction policy could throw all sorts of
// crazy exceptions. Protect against such an exception
// killing the eviction thread.
boolean evict;
try {
evict = evictionPolicy.evict(evictionConfig, underTest,
idleObjects.size());
} catch (final Throwable t) {
// Slightly convoluted as SwallowedExceptionListener
// uses Exception rather than Throwable
PoolUtils.checkRethrow(t);
swallowException(new Exception(t));
// Don't evict on error conditions
evict = false;
}
if (evict) {
destroy(underTest);
destroyedByEvictorCount.incrementAndGet();
} else {
if (testWhileIdle) {
boolean active = false;
try {
factory.activateObject(underTest);
active = true;
} catch (final Exception e) {
destroy(underTest);
destroyedByEvictorCount.incrementAndGet();
}
if (active) {
if (!factory.validateObject(underTest)) {
destroy(underTest);
destroyedByEvictorCount.incrementAndGet();
} else {
try {
factory.passivateObject(underTest);
} catch (final Exception e) {
destroy(underTest);
destroyedByEvictorCount.incrementAndGet();
}
}
}
}
if (!underTest.endEvictionTest(idleObjects)) {
// TODO - May need to add code here once additional
// states are used
}
}
}
}
}
final AbandonedConfig ac = this.abandonedConfig;
if (ac != null && ac.getRemoveAbandonedOnMaintenance()) {
removeAbandoned(ac);
}
}
evict方法在evict为false且testWhileIdle为true的时候，会执行activateObject方法，然后再执行validateObject方法
小结
JedisFactory实现了PooledObjectFactory接口，其泛型为Jedis；其makeObject方法主要是创建Jedis并connect，auth、还有select db；其destroyObject主要是执行quit及disconnect方法；其validateObject方法执行ping命令；其activateObject方法判断db是否变化，有变化则重新select db；其passivateObject目前为空实现
目前borrowObject方法在从idleObjects.pollFirst获取到的jedis不为null的时候会执行activateObject(
这里db没有变化，相等于空操作)，然后如果testOnBorrow为true则执行validateObject(执行ping命令)
evict方法在evict为false且testWhileIdle为true的时候，会执行activateObject方法，然后再执行validateObject方法
doc
- 一次访问Redis延时高问题排查与总结
- 实战总结｜一次访问Redis延时高问题排查与总结（续）