memcached的multiGet解析
@Test
public void testSetMc() {
logger.debug("test set value to mc");
for(int i = 1; i <= 20; i++) {
memCachedClient.set("test-"+i, "test value " + i);
}
}
mc配置了8个服务，就是说有8个节点，以上代码设置20个缓存，key都是以“test-”作为前缀的，但缓存会分散在8个服务器上[set(key, value)]会以key的hash值算出存放节点。
@Test
public void testMultiGet() {
String keyPrefix = "test-";
String[] keys = new String[20];
for(int i = 0; i < 20; i++) {
keys[i] = keyPrefix + (i+1);
}
Object[] values = memCachedClient.getMultiArray(keys);
if(values != null) {
for(Object o : values) {
logger.debug("value: {}", o);
}
}
}
//实际从mc服务器获取命令
节点1 => get test-5 test-10 test-18
节点2 => get test-1 test-9 test-14
节点3 => get test-8 test-13
节点4 => get test-6 test-11 test-19
节点5 => get test-3 test-16
节点6 => get test-4 test-17 test-20
节点7 => get test-2 test-15
节点8 => get test-7 test-12
从上面getMulti可以看出要那20个值需要连接8次mc服务器，验证过客户端是串行的，就是说节点1连接获取数据返回后在连接节点2，依次类推，可想而知如果mc服务器的节点在增加呢，每次获取的数量不是20是200或者更多呢？
1 -> loadMulti before: java.nio.HeapByteBuffer[pos=111 lim=8192 cap=8192]
ret(3): {test-10=test value 10, test-5=test value 5, test-18=test value 18}
1 -> loadMulti after: java.nio.HeapByteBuffer[pos=0 lim=8192 cap=8192]
2 -> loadMulti before: java.nio.HeapByteBuffer[pos=109 lim=8192 cap=8192]
ret(6): {test-9=test value 9, test-10=test value 10, test-5=test value 5, test-14=test value 14, test-1=test value 1, test-18=test value 18}
2 -> loadMulti after: java.nio.HeapByteBuffer[pos=0 lim=8192 cap=8192]
3 -> loadMulti before: java.nio.HeapByteBuffer[pos=75 lim=8192 cap=8192]
ret(8): {test-9=test value 9, test-10=test value 10, test-8=test value 8, test-5=test value 5, test-13=test value 13, test-14=test value 14, test-1=test value 1, test-18=test value 18}
3 -> loadMulti after: java.nio.HeapByteBuffer[pos=0 lim=8192 cap=8192]
4 -> loadMulti before: java.nio.HeapByteBuffer[pos=111 lim=8192 cap=8192]
ret(11): {test-9=test value 9, test-10=test value 10, test-11=test value 11, test-8=test value 8, test-5=test value 5, test-6=test value 6, test-13=test value 13, test-14=test value 14, test-1=test value 1, test-19=test value 19, test-18=test value 18}
4 -> loadMulti after: java.nio.HeapByteBuffer[pos=0 lim=8192 cap=8192]
5 -> loadMulti before: java.nio.HeapByteBuffer[pos=75 lim=8192 cap=8192]
ret(13): {test-9=test value 9, test-10=test value 10, test-11=test value 11, test-8=test value 8, test-5=test value 5, test-6=test value 6, test-13=test value 13, test-3=test value 3, test-14=test value 14, test-1=test value 1, test-19=test value 19, test-18=test value 18, test-16=test value 16}
5 -> loadMulti after: java.nio.HeapByteBuffer[pos=0 lim=8192 cap=8192]
6 -> loadMulti before: java.nio.HeapByteBuffer[pos=111 lim=8192 cap=8192]
ret(16): {test-9=test value 9, test-10=test value 10, test-11=test value 11, test-8=test value 8, test-5=test value 5, test-6=test value 6, test-13=test value 13, test-3=test value 3, test-14=test value 14, test-20=test value 20, test-4=test value 4, test-1=test value 1, test-19=test value 19, test-18=test value 18, test-17=test value 17, test-16=test value 16}
6 -> loadMulti after: java.nio.HeapByteBuffer[pos=0 lim=8192 cap=8192]
7 -> loadMulti before: java.nio.HeapByteBuffer[pos=75 lim=8192 cap=8192]
ret(18): {test-9=test value 9, test-10=test value 10, test-11=test value 11, test-8=test value 8, test-5=test value 5, test-6=test value 6, test-13=test value 13, test-3=test value 3, test-14=test value 14, test-15=test value 15, test-20=test value 20, test-4=test value 4, test-1=test value 1, test-2=test value 2, test-19=test value 19, test-18=test value 18, test-17=test value 17, test-16=test value 16}
7 -> loadMulti after: java.nio.HeapByteBuffer[pos=0 lim=8192 cap=8192]
8 -> loadMulti before: java.nio.HeapByteBuffer[pos=75 lim=8192 cap=8192]
ret(20): {test-9=test value 9, test-7=test value 7, test-10=test value 10, test-11=test value 11, test-8=test value 8, test-12=test value 12, test-5=test value 5, test-6=test value 6, test-13=test value 13, test-3=test value 3, test-14=test value 14, test-15=test value 15, test-20=test value 20, test-4=test value 4, test-1=test value 1, test-2=test value 2, test-19=test value 19, test-18=test value 18, test-17=test value 17, test-16=test value 16}
8 -> loadMulti after: java.nio.HeapByteBuffer[pos=0 lim=8192 cap=8192]
这样看来如果mc数据量大的话增加节点不一定是解决问题，原先只需要一次连接的，现在要连接很多次，光网络io就可能是个新的问题，而且这个客户端又是串行连接返回的。这就是著名的mutiGet无底洞了。