memcache 一致性hash算法研究

问题背景

1、在解决memcache的分布式存储的时候,客户端需要选择存储节点;
2、常规取模运算的hash算法:集群中机器上下线之后,命中率急剧下降,缓存需要重新建立,瞬间会给DB带来极高的系统负载;
3、一致性hash算法:集群中机器上下线之后,能保持较高的命中率。

设计核心点

1、cache迁移少(提高命中率)
2、key分布均匀(负载均衡)

算法解读

基本思想

不仅仅key进行hash,机器节点本身也进行hash

1、取0~2^32-1的环
2、算机器的hash,划分区间
3、存取时,算key的hash,选择区间的机器

图解参见链接: https://blog.csdn.net/lihao21/article/details/54193868

虚拟节点

一致性hash解决命中率问题,虚拟节点解决key的分布均匀问题(负载均衡)

1、使用一般的hash运算,服务器的映射地点的分布非常不均匀;
2、使用虚拟节点的思想,为每个物理节点(服务器)在圆环上分配100~200个点。
3、这样就能抑制分布不均匀,最大限度地减小服务器增减时的缓存重新分布。

源码解读(采用一致性hash时,即定义枚举型变量Locator为CONSISTENT时)

服务器对于hash区间的割分

KetamaNodeLocator类中

//机器划分hash区间,记录在本类的ketamaNodes成员属性中
protected void setKetamaNodes(List nodes) {
    TreeMap newNodeMap =
            new TreeMap();
    int numReps = config.getNodeRepetitions();
    int nodeCount = nodes.size();
    int totalWeight = 0;

    if (isWeightedKetama) {
        for (MemcachedNode node : nodes) {
            totalWeight += weights.get(node.getSocketAddress());
        }
    }

    for (MemcachedNode node : nodes) {
      if (isWeightedKetama) {
          //带权重的一致性hash算法节点划分@3@
          int thisWeight = weights.get(node.getSocketAddress());
          float percent = (float)thisWeight / (float)totalWeight;
          int pointerPerServer = (int)((Math.floor((float)(percent * (float)config.getNodeRepetitions() / 4 * (float)nodeCount + 0.0000000001))) * 4);
          for (int i = 0; i < pointerPerServer / 4; i++) {
              for(long position : ketamaNodePositionsAtIteration(node, i)) {
                  newNodeMap.put(position, node);
                  getLogger().debug("Adding node %s with weight %s in position %d", node, thisWeight, position);
              }
          }
      } else {
          // Ketama does some special work with md5 where it reuses chunks.
          // Check to be backwards compatible, the hash algorithm does not
          // matter for Ketama, just the placement should always be done using
          // MD5
          //一致性hash算法节点划分@2@
          if (hashAlg == DefaultHashAlgorithm.KETAMA_HASH) {
              for (int i = 0; i < numReps / 4; i++) {
                  for(long position : ketamaNodePositionsAtIteration(node, i)) {
                    newNodeMap.put(position, node);
                    getLogger().debug("Adding node %s in position %d", node, position);
                  }
              }
          } else {
            //非一致性hash算法节点划分@1@
              for (int i = 0; i < numReps; i++) {
                  newNodeMap.put(hashAlg.hash(config.getKeyForNode(node, i)), node);
              }
          }
      }
    }
    assert newNodeMap.size() == numReps * nodes.size();
    ketamaNodes = newNodeMap;
  }

@1@:非一致性hash算法节点划分
1、重复numReps次,每次都是计算一个虚拟节点的hash结果

2、计算每个虚拟节点进行hash运算的key值:config.getKeyForNode(node, i)
KetamaNodeKeyFormatter类中

public String getKeyForNode(MemcachedNode node, int repetition) {
        // Carrried over from the DefaultKetamaNodeLocatorConfiguration:
        // Internal Using the internal map retrieve the socket addresses
        // for given nodes.
        // I'm aware that this code is inherently thread-unsafe as
        // I'm using a HashMap implementation of the map, but the worst
        // case ( I believe) is we're slightly in-efficient when
        // a node has never been seen before concurrently on two different
        // threads, so it the socketaddress will be requested multiple times!
        // all other cases should be as fast as possible.
        String nodeKey = nodeKeys.get(node);
        if (nodeKey == null) {
            switch(this.format) {
                case LIBMEMCACHED:
                    InetSocketAddress address = (InetSocketAddress)node.getSocketAddress();
                    nodeKey = address.getHostName();
                    if (address.getPort() != 11211) {
                        nodeKey += ":" + address.getPort();
                    }
                    break;
                case SPYMEMCACHED:
                    nodeKey = String.valueOf(node.getSocketAddress());
                    if (nodeKey.startsWith("/")) {
                        nodeKey = nodeKey.substring(1);
                    }
                    break;
                default:
                    assert false;
            }
            nodeKeys.put(node, nodeKey);
        }
        return nodeKey + "-" + repetition;
    }

计算结果就是"SocketAddress-虚拟节点重复数"(10.10.1.118:11411-0)

3、根据key计算出虚拟节点的hash结果 DefaultHashAlgorithm类中

/**
* Compute the hash for the given key.
*
* @return a positive integer hash
*/
public long hash(final String k) {
    long rv = 0;
    int len = k.length();
    switch (this) {
    case NATIVE_HASH:
        rv = k.hashCode();
        break;
    case CRC_HASH:
        // return (crc32(shift) >> 16) & 0x7fff;
        CRC32 crc32 = new CRC32();
        crc32.update(KeyUtil.getKeyBytes(k));
        rv = (crc32.getValue() >> 16) & 0x7fff;
        break;
    case FNV1_64_HASH:
        // Thanks to [email protected] for the pointer
        rv = FNV_64_INIT;
        for (int i = 0; i < len; i++) {
        rv *= FNV_64_PRIME;
        rv ^= k.charAt(i);
        }
        break;
    case FNV1A_64_HASH:
        rv = FNV_64_INIT;
        for (int i = 0; i < len; i++) {
        rv ^= k.charAt(i);
        rv *= FNV_64_PRIME;
        }
        break;
    case FNV1_32_HASH:
        rv = FNV_32_INIT;
        for (int i = 0; i < len; i++) {
        rv *= FNV_32_PRIME;
        rv ^= k.charAt(i);
        }
        break;
    case FNV1A_32_HASH:
        rv = FNV_32_INIT;
        for (int i = 0; i < len; i++) {
        rv ^= k.charAt(i);
        rv *= FNV_32_PRIME;
        }
        break;
    case KETAMA_HASH:
        byte[] bKey = computeMd5(k);
        rv = ((long) (bKey[3] & 0xFF) << 24)
          | ((long) (bKey[2] & 0xFF) << 16)
          | ((long) (bKey[1] & 0xFF) << 8)
          | (bKey[0] & 0xFF);
        break;
    default:
        assert false;
    }
    return rv & 0xffffffffL; /* Truncate to 32-bits */
}

@2@一致性hash算法节点划分
1、对于每个节点要算出numReps个虚拟节点,每组算出四个,一共算numReps/4组;

2、算一组四个虚拟节点:ketamaNodePositionsAtIteration(node, i)
KetamaNodeLocator类中

private List ketamaNodePositionsAtIteration(MemcachedNode node, int iteration) {
    List positions = new ArrayList();
    byte[] digest = DefaultHashAlgorithm.computeMd5(config.getKeyForNode(node, iteration));
    for (int h = 0; h < 4; h++) {
      Long k = ((long) (digest[3 + h * 4] & 0xFF) << 24)
          | ((long) (digest[2 + h * 4] & 0xFF) << 16)
          | ((long) (digest[1 + h * 4] & 0xFF) << 8)
          | (digest[h * 4] & 0xFF);
      positions.add(k);
    }
    return positions;
}

config.getKeyForNode(node, iteration)还是取出"SocketAddress-虚拟节点重复数"(10.10.1.118:11411-0)作为key值

DefaultHashAlgorithm类中computeMd5方法用key算出md5值digest字节数组

digest字节数组是16位的,每四位一取,倒序拼接成32位的二进制数【(long) (digest[3 + h * 4] & 0xFF) << 24是将相邻四个字节中的最后一位以二进制左移32位】,作为虚拟节点最终在hash结果集合内的落点,一组就算出四个虚拟节点的位置

@3@带权重的一致性hash算法节点划分 1、从weights中获得每个节点的权重(如果是带权重的节点,之前初始化的时候会设置这里的值),算出每个节点权重所占的百分比;

2、根据百分比算出每个节点应该有的虚拟节点数(因为floor的关系,这里算出来的虚拟节点数可能不等于重复率乘上节点数量,一般会比这个值小一点,原权重比例也会有所偏差,可以以权重2、3、5,重复率13,节点数3进行测算)

3、对于每个节点的虚拟节点,按照4个一组进行计算计算虚拟节点的hash结果ketamaNodePositionsAtIteration(node, i),和不带权重的ketama计算方式是一样的

客户端对于key值的计算

KetamaNodeLocator类中

public MemcachedNode getPrimary(final String k) {
    MemcachedNode rv = getNodeForKey(hashAlg.hash(k));
    assert rv != null : "Found no node for key " + k;
    return rv;
}

1、根据存储对象的key计算出hash结果hashAlg.hash(k),与计算服务器的虚拟节点hash结果用的是一个方法

2、根据算得的hash坐标在环中找到应该存储的服务器节点getNodeForKey(long hash)

MemcachedNode getNodeForKey(long hash) {
    final MemcachedNode rv;
    if (!ketamaNodes.containsKey(hash)) {
        // Java 1.6 adds a ceilingKey method, but I'm still stuck in 1.5
        // in a lot of places, so I'm doing this myself.
        SortedMap<Long, MemcachedNode> tailMap = getKetamaNodes().tailMap(hash);
        if (tailMap.isEmpty()) {
            hash = getKetamaNodes().firstKey();
        } else {
            hash = tailMap.firstKey();
        }
    }
    rv = getKetamaNodes().get(hash);
    return rv;
}

判断hash坐标是否正好坐落在服务器虚拟节点上,在则直接获得服务器节点ip

不在,获取hash坐标之后的弧,如果之后的弧上没有服务器节点ip了,说明应该绕完整个环的尾部,去找环的第一个节点,获得服务器节点ip

如果弧上还有服务器节点ip的话,则选取第一个ip作为应该存储的服务器节点ip。

测试结果对比

不同算法结果对比

步骤:
1、将不同的算法枚举值作为参数传入,创建不同的memcache客户端,创建过程根据不同的hash算法计算生成不同的服务器hash结果集合(存储在对应客户端的MemcachedConnection的NodeLocator中)

2、获得不同算法产生的NodeLocator

3、实验标本使用线上环境某个长时间段内的大量数据进行测试,计算对应key值hash之后最终选择的服务器,统计对于大量数据每个算法的耗时,以及每个服务器命中数的均匀性。

4、进行三次实验对比

测试结果:

1st:
FNV1A_64_HASH      costs:1184ms,      number:2387858      perdata:0.49584188004479324ns.
10.10.1.118:11411 : 555973 , 23.28%
10.16.69.133:11411 : 533208 , 22.33%
10.16.69.135:11411 : 355425 , 14.88%
10.16.6.184:11411 : 235848 , 9.88%
10.10.1.228:11411 : 346293 , 14.50%
10.10.1.208:11411 : 361111 , 15.12%
variance:0.33143952290634493

FNV1_32_HASH      costs:1174ms,      number:2387858      perdata:0.49165402632819877ns.
10.10.1.118:11411 : 489055 , 20.48%
10.16.69.133:11411 : 395165 , 16.55%
10.16.69.135:11411 : 259880 , 10.88%
10.16.6.184:11411 : 232323 , 9.73%
10.10.1.228:11411 : 277434 , 11.62%
10.10.1.208:11411 : 734001 , 30.74%
variance:0.6438545073606985

KETAMA_HASH      costs:2352ms,      number:2387858      perdata:0.9849831941430353ns.
10.10.1.118:11411 : 411977 , 17.25%
10.16.69.133:11411 : 449525 , 18.83%
10.16.6.184:11411 : 387824 , 16.24%
10.16.69.135:11411 : 346523 , 14.51%
10.10.1.228:11411 : 382838 , 16.03%
10.10.1.208:11411 : 409171 , 17.14%
variance:0.12852730715084365

NATIVE_HASH      costs:1024ms,      number:2387858      perdata:0.42883622057928067ns.
10.10.1.118:11411 : 293716 , 12.30%
10.16.69.133:11411 : 209344 , 8.77%
10.16.69.135:11411 : 694618 , 29.09%
10.16.6.184:11411 : 362735 , 15.19%
10.10.1.228:11411 : 191147 , 8.00%
10.10.1.208:11411 : 636298 , 26.65%
variance:0.7987988531108471

CRC_HASH      costs:1733ms,      number:2387858      perdata:0.7257550490858334ns.
10.10.1.118:11411 : 347616 , 14.56%
10.16.69.133:11411 : 327404 , 13.71%
10.16.6.184:11411 : 459595 , 19.25%
10.16.69.135:11411 : 481764 , 20.18%
10.10.1.228:11411 : 395621 , 16.57%
10.10.1.208:11411 : 375858 , 15.74%
variance:0.1661475366273886

FNV1_64_HASH      costs:971ms,      number:2387858      perdata:0.4066405958813296ns.
10.10.1.118:11411 : 248007 , 10.39%
10.16.69.133:11411 : 324288 , 13.58%
10.16.6.184:11411 : 1051464 , 44.03%
10.16.69.135:11411 : 239652 , 10.04%
10.10.1.228:11411 : 126840 , 5.31%
10.10.1.208:11411 : 397607 , 16.65%
variance:1.7291444612562392

FNV1A_32_HASH      costs:1578ms,      number:2387858      perdata:0.660843316478618ns.
10.10.1.118:11411 : 312106 , 13.07%
10.16.69.133:11411 : 253577 , 10.62%
10.16.6.184:11411 : 644748 , 27.00%
10.16.69.135:11411 : 421664 , 17.66%
10.10.1.228:11411 : 323920 , 13.57%
10.10.1.208:11411 : 431843 , 18.08%
variance:0.3926374779645682



2nd:
FNV1A_64_HASH      costs:1169ms,      number:2387858      perdata:0.4895600994699015ns.
10.10.1.118:11411 : 555973 , 23.28%
10.16.69.133:11411 : 533208 , 22.33%
10.16.69.135:11411 : 355425 , 14.88%
10.16.6.184:11411 : 235848 , 9.88%
10.10.1.228:11411 : 346293 , 14.50%
10.10.1.208:11411 : 361111 , 15.12%
variance:0.33143952290634493

FNV1_32_HASH      costs:1205ms,      number:2387858      perdata:0.5046363728496418ns.
10.10.1.118:11411 : 489055 , 20.48%
10.16.69.133:11411 : 395165 , 16.55%
10.16.69.135:11411 : 259880 , 10.88%
10.16.6.184:11411 : 232323 , 9.73%
10.10.1.228:11411 : 277434 , 11.62%
10.10.1.208:11411 : 734001 , 30.74%
variance:0.6438545073606985

KETAMA_HASH      costs:2314ms,      number:2387858      perdata:0.9690693500199761ns.
10.10.1.118:11411 : 411977 , 17.25%
10.16.69.133:11411 : 449525 , 18.83%
10.16.6.184:11411 : 387824 , 16.24%
10.16.69.135:11411 : 346523 , 14.51%
10.10.1.228:11411 : 382838 , 16.03%
10.10.1.208:11411 : 409171 , 17.14%
variance:0.12852730715084365

NATIVE_HASH      costs:987ms,      number:2387858      perdata:0.4133411618278809ns.
10.10.1.118:11411 : 293716 , 12.30%
10.16.69.133:11411 : 209344 , 8.77%
10.16.69.135:11411 : 694618 , 29.09%
10.16.6.184:11411 : 362735 , 15.19%
10.10.1.228:11411 : 191147 , 8.00%
10.10.1.208:11411 : 636298 , 26.65%
variance:0.7987988531108471

CRC_HASH      costs:1718ms,      number:2387858      perdata:0.7194732685109416ns.
10.10.1.118:11411 : 347616 , 14.56%
10.16.69.133:11411 : 327404 , 13.71%
10.16.6.184:11411 : 459595 , 19.25%
10.16.69.135:11411 : 481764 , 20.18%
10.10.1.228:11411 : 395621 , 16.57%
10.10.1.208:11411 : 375858 , 15.74%
variance:0.1661475366273886

FNV1_64_HASH      costs:1006ms,      number:2387858      perdata:0.42129808388941054ns.
10.10.1.118:11411 : 248007 , 10.39%
10.16.69.133:11411 : 324288 , 13.58%
10.16.6.184:11411 : 1051464 , 44.03%
10.16.69.135:11411 : 239652 , 10.04%
10.10.1.228:11411 : 126840 , 5.31%
10.10.1.208:11411 : 397607 , 16.65%
variance:1.7291444612562392

FNV1A_32_HASH      costs:1528ms,      number:2387858      perdata:0.6399040478956454ns.
10.10.1.118:11411 : 312106 , 13.07%
10.16.69.133:11411 : 253577 , 10.62%
10.16.6.184:11411 : 644748 , 27.00%
10.16.69.135:11411 : 421664 , 17.66%
10.10.1.228:11411 : 323920 , 13.57%
10.10.1.208:11411 : 431843 , 18.08%
variance:0.3926374779645682



3rd
FNV1A_64_HASH      costs:1085ms,      number:2387858      perdata:0.45438212825050733ns.
10.10.1.118:11411 : 555973 , 23.28%
10.16.69.133:11411 : 533208 , 22.33%
10.16.69.135:11411 : 355425 , 14.88%
10.16.6.184:11411 : 235848 , 9.88%
10.10.1.228:11411 : 346293 , 14.50%
10.10.1.208:11411 : 361111 , 15.12%
variance:0.33143952290634493

FNV1_32_HASH      costs:1199ms,      number:2387858      perdata:0.502123660619685ns.
10.10.1.118:11411 : 489055 , 20.48%
10.16.69.133:11411 : 395165 , 16.55%
10.16.69.135:11411 : 259880 , 10.88%
10.16.6.184:11411 : 232323 , 9.73%
10.10.1.228:11411 : 277434 , 11.62%
10.10.1.208:11411 : 734001 , 30.74%
variance:0.6438545073606985

KETAMA_HASH      costs:2551ms,      number:2387858      perdata:1.0683214831032668ns.
10.10.1.118:11411 : 411977 , 17.25%
10.16.69.133:11411 : 449525 , 18.83%
10.16.6.184:11411 : 387824 , 16.24%
10.16.69.135:11411 : 346523 , 14.51%
10.10.1.228:11411 : 382838 , 16.03%
10.10.1.208:11411 : 409171 , 17.14%
variance:0.12852730715084365

NATIVE_HASH      costs:1050ms,      number:2387858      perdata:0.43972464024242647ns.
10.10.1.118:11411 : 293716 , 12.30%
10.16.69.133:11411 : 209344 , 8.77%
10.16.69.135:11411 : 694618 , 29.09%
10.16.6.184:11411 : 362735 , 15.19%
10.10.1.228:11411 : 191147 , 8.00%
10.10.1.208:11411 : 636298 , 26.65%
variance:0.7987988531108471

CRC_HASH      costs:1806ms,      number:2387858      perdata:0.7563263812169736ns.
10.10.1.118:11411 : 347616 , 14.56%
10.16.69.133:11411 : 327404 , 13.71%
10.16.6.184:11411 : 459595 , 19.25%
10.16.69.135:11411 : 481764 , 20.18%
10.10.1.228:11411 : 395621 , 16.57%
10.10.1.208:11411 : 375858 , 15.74%
variance:0.1661475366273886

FNV1_64_HASH      costs:1027ms,      number:2387858      perdata:0.43009257669425904ns.
10.10.1.118:11411 : 248007 , 10.39%
10.16.69.133:11411 : 324288 , 13.58%
10.16.6.184:11411 : 1051464 , 44.03%
10.16.69.135:11411 : 239652 , 10.04%
10.10.1.228:11411 : 126840 , 5.31%
10.10.1.208:11411 : 397607 , 16.65%
variance:1.7291444612562392

FNV1A_32_HASH      costs:1546ms,      number:2387858      perdata:0.6474421845855155ns.
10.10.1.118:11411 : 312106 , 13.07%
10.16.69.133:11411 : 253577 , 10.62%
10.16.6.184:11411 : 644748 , 27.00%
10.16.69.135:11411 : 421664 , 17.66%
10.10.1.228:11411 : 323920 , 13.57%
10.10.1.208:11411 : 431843 , 18.08%
variance:0.3926374779645682


说明:每段开头是采用的hash算法的名称,number表示处理的总数据量,costs表示总耗时,perdata是计算得到的对于每个数据的平均处理耗时,下面是罗列的每个服务器对应的ip、命中数和命中占比,variance表示占比计算得到的方差(乘了100倍处理)。

实验结论:
1、对于同一标本,三次实验采用各算法,每个服务器的命中占比是不变的,其中KETAMA_HASH个服务器命中占比比较均匀,其次表现较好的是CRC_HASH,其他的算法命中占比相差比较大;

2、对于同一标本,三次实验采用各算法,所耗时间略有波动,综合来说,NATIVE_HASH和FNV1_64_HASH耗时较少,KETAMA_HASH耗时较多,约是前两种算法的两倍,其他算法居于中间。

增减客户端个数结果对比

步骤:基本步骤和上述相同,本次设置不同的服务器台数,分别设置服务器台数为2,4,6台,将结果进行纵向对比

测试结果:

6台:
FNV1A_64_HASH      costs:1085ms,      number:2387858      perdata:0.45438212825050733ns.
10.10.1.118:11411 : 555973 , 23.28%
10.16.69.133:11411 : 533208 , 22.33%
10.16.69.135:11411 : 355425 , 14.88%
10.16.6.184:11411 : 235848 , 9.88%
10.10.1.228:11411 : 346293 , 14.50%
10.10.1.208:11411 : 361111 , 15.12%
variance:0.33143952290634493

FNV1_32_HASH      costs:1199ms,      number:2387858      perdata:0.502123660619685ns.
10.10.1.118:11411 : 489055 , 20.48%
10.16.69.133:11411 : 395165 , 16.55%
10.16.69.135:11411 : 259880 , 10.88%
10.16.6.184:11411 : 232323 , 9.73%
10.10.1.228:11411 : 277434 , 11.62%
10.10.1.208:11411 : 734001 , 30.74%
variance:0.6438545073606985

KETAMA_HASH      costs:2551ms,      number:2387858      perdata:1.0683214831032668ns.
10.10.1.118:11411 : 411977 , 17.25%
10.16.69.133:11411 : 449525 , 18.83%
10.16.6.184:11411 : 387824 , 16.24%
10.16.69.135:11411 : 346523 , 14.51%
10.10.1.228:11411 : 382838 , 16.03%
10.10.1.208:11411 : 409171 , 17.14%
variance:0.12852730715084365

NATIVE_HASH      costs:1050ms,      number:2387858      perdata:0.43972464024242647ns.
10.10.1.118:11411 : 293716 , 12.30%
10.16.69.133:11411 : 209344 , 8.77%
10.16.69.135:11411 : 694618 , 29.09%
10.16.6.184:11411 : 362735 , 15.19%
10.10.1.228:11411 : 191147 , 8.00%
10.10.1.208:11411 : 636298 , 26.65%
variance:0.7987988531108471

CRC_HASH      costs:1806ms,      number:2387858      perdata:0.7563263812169736ns.
10.10.1.118:11411 : 347616 , 14.56%
10.16.69.133:11411 : 327404 , 13.71%
10.16.6.184:11411 : 459595 , 19.25%
10.16.69.135:11411 : 481764 , 20.18%
10.10.1.228:11411 : 395621 , 16.57%
10.10.1.208:11411 : 375858 , 15.74%
variance:0.1661475366273886

FNV1_64_HASH      costs:1027ms,      number:2387858      perdata:0.43009257669425904ns.
10.10.1.118:11411 : 248007 , 10.39%
10.16.69.133:11411 : 324288 , 13.58%
10.16.6.184:11411 : 1051464 , 44.03%
10.16.69.135:11411 : 239652 , 10.04%
10.10.1.228:11411 : 126840 , 5.31%
10.10.1.208:11411 : 397607 , 16.65%
variance:1.7291444612562392

FNV1A_32_HASH      costs:1546ms,      number:2387858      perdata:0.6474421845855155ns.
10.10.1.118:11411 : 312106 , 13.07%
10.16.69.133:11411 : 253577 , 10.62%
10.16.6.184:11411 : 644748 , 27.00%
10.16.69.135:11411 : 421664 , 17.66%
10.10.1.228:11411 : 323920 , 13.57%
10.10.1.208:11411 : 431843 , 18.08%
variance:0.3926374779645682



4台:
FNV1A_64_HASH      costs:1147ms,      number:2387858      perdata:0.48034682129339346ns.
10.10.1.118:11411 : 936837 , 39.23%
10.16.6.184:11411 : 713990 , 29.90%
10.10.1.228:11411 : 375920 , 15.74%
10.10.1.208:11411 : 361111 , 15.12%
variance:1.274646297334262

FNV1_32_HASH      costs:1074ms,      number:2387858      perdata:0.44977548916225335ns.
10.10.1.118:11411 : 809789 , 33.91%
10.16.6.184:11411 : 529418 , 22.17%
10.10.1.228:11411 : 314650 , 13.18%
10.10.1.208:11411 : 734001 , 30.74%
variance:0.9003891935351122

KETAMA_HASH      costs:2217ms,      number:2387858      perdata:0.928447168969009ns.
10.10.1.118:11411 : 653133 , 27.35%
10.16.6.184:11411 : 621488 , 26.03%
10.10.1.228:11411 : 551835 , 23.11%
10.10.1.208:11411 : 561402 , 23.51%
variance:0.2809445444425362

NATIVE_HASH      costs:947ms,      number:2387858      perdata:0.39658974696150273ns.
10.10.1.118:11411 : 312854 , 13.10%
10.16.6.184:11411 : 884987 , 37.06%
10.10.1.228:11411 : 191147 , 8.00%
10.10.1.208:11411 : 998870 , 41.83%
variance:2.3979443120759343

CRC_HASH      costs:1574ms,      number:2387858      perdata:0.6591681749919802ns.
10.10.1.118:11411 : 411390 , 17.23%
10.16.6.184:11411 : 845242 , 35.40%
10.10.1.228:11411 : 564908 , 23.66%
10.10.1.208:11411 : 566318 , 23.72%
variance:0.679887523513382

FNV1_64_HASH      costs:976ms,      number:2387858      perdata:0.40873452273962685ns.
10.10.1.118:11411 : 248007 , 10.39%
10.16.6.184:11411 : 1270549 , 53.21%
10.10.1.228:11411 : 471695 , 19.75%
10.10.1.208:11411 : 397607 , 16.65%
variance:3.016301833895297

FNV1A_32_HASH      costs:1326ms,      number:2387858      perdata:0.5553094028204357ns.
10.10.1.118:11411 : 413146 , 17.30%
10.16.6.184:11411 : 1056172 , 44.23%
10.10.1.228:11411 : 443434 , 18.57%
10.10.1.208:11411 : 475106 , 19.90%
variance:1.4911808957900026



2台:
FNV1A_64_HASH      costs:1202ms,      number:2387858      perdata:0.5033800167346635ns.
10.10.1.118:11411 : 1937413 , 81.14%
10.10.1.208:11411 : 450445 , 18.86%
variance:18.694518863029643

FNV1_32_HASH      costs:1126ms,      number:2387858      perdata:0.471552328488545ns.
10.10.1.118:11411 : 809789 , 33.91%
10.10.1.208:11411 : 1578069 , 66.09%
variance:11.587986880910444

KETAMA_HASH      costs:2202ms,      number:2387858      perdata:0.9221653883941173ns.
10.10.1.118:11411 : 1334362 , 55.88%
10.10.1.208:11411 : 1053496 , 44.12%
variance:9.345876737253043

NATIVE_HASH      costs:1062ms,      number:2387858      perdata:0.44475006470233996ns.
10.10.1.118:11411 : 1073300 , 44.95%
10.10.1.208:11411 : 1314558 , 55.05%
variance:9.255203403228546

CRC_HASH      costs:1577ms,      number:2387858      perdata:0.6604245311069586ns.
10.10.1.118:11411 : 959036 , 40.16%
10.10.1.208:11411 : 1428822 , 59.84%
variance:9.96766081685872

FNV1_64_HASH      costs:1057ms,      number:2387858      perdata:0.4426561378440426ns.
10.10.1.118:11411 : 712144 , 29.82%
10.10.1.208:11411 : 1675714 , 70.18%
variance:13.070891761052566

FNV1A_32_HASH      costs:1317ms,      number:2387858      perdata:0.5515403344755007ns.
10.10.1.118:11411 : 1023129 , 42.85%
10.10.1.208:11411 : 1364729 , 57.15%
variance:9.511633224617519

实验结论:
1、不同的服务器台数对于命中率的均匀性是有影响的;

2、KETAMA_HASH在各数量台数服务器的情况下表现的命中均匀性都比较好;

3、CRC_HASH和FNV1A_32_HASH在某些情况下均匀性较好,某些情况下较差,具有波动性;

4、其他hash算法在各种情况下表现得均匀性都很差。

不同客户端ip对于均匀性的影响

步骤:基本步骤和上述相同,本次设置服务器台数为4台,三次实验服务器ip不同,对比三次结果服务器命中率均匀性的不同 实验结果:

FNV1A_64_HASH      costs:1092ms,      number:2387858      perdata:0.45731362585212354ns.
10.10.1.118:11411 : 936837 , 39.23%
10.16.6.184:11411 : 713990 , 29.90%
10.10.1.228:11411 : 375920 , 15.74%
10.10.1.208:11411 : 361111 , 15.12%
variance:1.274646297334262

FNV1_32_HASH      costs:1213ms,      number:2387858      perdata:0.5079866558229175ns.
10.10.1.118:11411 : 809789 , 33.91%
10.16.6.184:11411 : 529418 , 22.17%
10.10.1.228:11411 : 314650 , 13.18%
10.10.1.208:11411 : 734001 , 30.74%
variance:0.9003891935351122

KETAMA_HASH      costs:2372ms,      number:2387858      perdata:0.9933589015762244ns.
10.10.1.118:11411 : 653133 , 27.35%
10.16.6.184:11411 : 621488 , 26.03%
10.10.1.228:11411 : 551835 , 23.11%
10.10.1.208:11411 : 561402 , 23.51%
variance:0.2809445444425362

NATIVE_HASH      costs:1142ms,      number:2387858      perdata:0.4782528944350962ns.
10.10.1.118:11411 : 312854 , 13.10%
10.16.6.184:11411 : 884987 , 37.06%
10.10.1.228:11411 : 191147 , 8.00%
10.10.1.208:11411 : 998870 , 41.83%
variance:2.3979443120759343

CRC_HASH      costs:1722ms,      number:2387858      perdata:0.7211484099975795ns.
10.10.1.118:11411 : 411390 , 17.23%
10.16.6.184:11411 : 845242 , 35.40%
10.10.1.228:11411 : 564908 , 23.66%
10.10.1.208:11411 : 566318 , 23.72%
variance:0.679887523513382

FNV1_64_HASH      costs:1094ms,      number:2387858      perdata:0.45815119659544246ns.
10.10.1.118:11411 : 248007 , 10.39%
10.16.6.184:11411 : 1270549 , 53.21%
10.10.1.228:11411 : 471695 , 19.75%
10.10.1.208:11411 : 397607 , 16.65%
variance:3.016301833895297

FNV1A_32_HASH      costs:1443ms,      number:2387858      perdata:0.6043072913045918ns.
10.10.1.118:11411 : 413146 , 17.30%
10.16.6.184:11411 : 1056172 , 44.23%
10.10.1.228:11411 : 443434 , 18.57%
10.10.1.208:11411 : 475106 , 19.90%
variance:1.4911808957900026






FNV1A_64_HASH      costs:1059ms,      number:2387858      perdata:0.4434937085873616ns.
10.10.1.118:11411 : 555973 , 23.28%
10.16.69.133:11411 : 995377 , 41.68%
10.16.69.135:11411 : 386063 , 16.17%
10.10.1.208:11411 : 450445 , 18.86%
variance:1.2424831477658953

FNV1_32_HASH      costs:1172ms,      number:2387858      perdata:0.49081645558487985ns.
10.10.1.118:11411 : 489055 , 20.48%
10.16.69.133:11411 : 752254 , 31.50%
10.16.69.135:11411 : 382095 , 16.00%
10.10.1.208:11411 : 764454 , 32.01%
variance:0.7322146248064811

KETAMA_HASH      costs:2288ms,      number:2387858      perdata:0.9581809303568303ns.
10.10.1.118:11411 : 672247 , 28.15%
10.16.69.133:11411 : 599286 , 25.10%
10.16.69.135:11411 : 525914 , 22.02%
10.10.1.208:11411 : 590411 , 24.73%
variance:0.29719492884167864

NATIVE_HASH      costs:999ms,      number:2387858      perdata:0.41836658628779433ns.
10.10.1.118:11411 : 605520 , 25.36%
10.16.69.133:11411 : 380544 , 15.94%
10.16.69.135:11411 : 694618 , 29.09%
10.10.1.208:11411 : 707176 , 29.62%
variance:0.5507515972060639

CRC_HASH      costs:1719ms,      number:2387858      perdata:0.719892053882601ns.
10.10.1.118:11411 : 638712 , 26.75%
10.16.69.133:11411 : 499041 , 20.90%
10.16.69.135:11411 : 707826 , 29.64%
10.10.1.208:11411 : 542279 , 22.71%
variance:0.3666839176114445

FNV1_64_HASH      costs:936ms,      number:2387858      perdata:0.39198310787324875ns.
10.10.1.118:11411 : 263803 , 11.05%
10.16.69.133:11411 : 427774 , 17.91%
10.16.69.135:11411 : 716761 , 30.02%
10.10.1.208:11411 : 979520 , 41.02%
variance:1.5667703845361418

FNV1A_32_HASH      costs:1390ms,      number:2387858      perdata:0.5821116666066407ns.
10.10.1.118:11411 : 428731 , 17.95%
10.16.69.133:11411 : 796957 , 33.38%
10.16.69.135:11411 : 618340 , 25.90%
10.10.1.208:11411 : 543830 , 22.77%
variance:0.5638433094805416








FNV1A_64_HASH      costs:1030ms,      number:2387858      perdata:0.4313489328092374ns.
10.16.69.133:11411 : 533208 , 22.33%
10.16.6.184:11411 : 1144370 , 47.92%
10.16.69.135:11411 : 363987 , 15.24%
10.10.1.228:11411 : 346293 , 14.50%
variance:2.0951521038180405

FNV1_32_HASH      costs:1035ms,      number:2387858      perdata:0.43344285966753465ns.
10.16.69.133:11411 : 395165 , 16.55%
10.16.69.135:11411 : 760587 , 31.85%
10.16.6.184:11411 : 232323 , 9.73%
10.10.1.228:11411 : 999783 , 41.87%
variance:1.8403633087057236

KETAMA_HASH      costs:2251ms,      number:2387858      perdata:0.9426858716054305ns.
10.16.69.133:11411 : 664844 , 27.84%
10.16.6.184:11411 : 634944 , 26.59%
10.16.69.135:11411 : 530538 , 22.22%
10.10.1.228:11411 : 557532 , 23.35%
variance:0.3026909579301235

NATIVE_HASH      costs:936ms,      number:2387858      perdata:0.39198310787324875ns.
10.16.69.133:11411 : 209344 , 8.77%
10.16.69.135:11411 : 985967 , 41.29%
10.16.6.184:11411 : 362735 , 15.19%
10.10.1.228:11411 : 829812 , 34.75%
variance:2.0505247156316835

CRC_HASH      costs:1637ms,      number:2387858      perdata:0.6855516534065259ns.
10.16.69.133:11411 : 537661 , 22.52%
10.16.6.184:11411 : 581728 , 24.36%
10.16.69.135:11411 : 511612 , 21.43%
10.10.1.228:11411 : 756857 , 31.70%
variance:0.41047254512820197

FNV1_64_HASH      costs:973ms,      number:2387858      perdata:0.4074781666246486ns.
10.16.69.133:11411 : 607950 , 25.46%
10.16.6.184:11411 : 1144065 , 47.91%
10.16.69.135:11411 : 239652 , 10.04%
10.10.1.228:11411 : 396191 , 16.59%
variance:2.299425018737725

FNV1A_32_HASH      costs:1335ms,      number:2387858      perdata:0.5590784711653708ns.
10.16.69.133:11411 : 557717 , 23.36%
10.16.6.184:11411 : 729101 , 30.53%
10.16.69.135:11411 : 653616 , 27.37%
10.10.1.228:11411 : 447424 , 18.74%
variance:0.44542802226614386

实验结论:
1、不同的服务器ip对于每个算法的服务器分布均匀性都有影响;

2、NATIVE_HASH受服务器ip的影响特别大;

3、KETAMA_HASH受服务器ip影响较小,且能保持良好的均匀性,CRC_HASH在某些情况下均匀性较好,某些情况下较差;

4、其他hash算法在各种情况下表现得均匀性都很差。

设置不同虚拟节点倍数

步骤:
1、控制虚拟节点倍数的是numReps变量,它在KetamaNodeLocatorConfiguration接口中定义,用getNodeRepetitions()方法获得;
2、KetamaNodeLocatorConfiguration会在ConnectionFactoryBuilder进行build的时候,在createLocator方法中构造KetamaNodeLocator时被传入KetamaNodeLocator中;
3、memcache默认使用的ConnectionFactoryBuilder会在build的时候将默认的DefaultKetamaNodeLocatorConfiguration载入KetamaNodeLocator,他的numReps默认为160且不可修改

private final int numReps = 160;

4、自己编写一个ConnectionFactoryBuilder,在build的时候,重写createLocator方法时,传入自己的MyKetamaNodeLocatorConfiguration,在自己的MyKetamaNodeLocatorConfiguration中重新设置numReps的值为160,320,480,640,1600,3200,对比实验结果。

测试结果:

FNV1A_64_HASH
160
FNV1A_64_HASH      costs:1176ms,      number:2387858      perdata:0.49249159707151763ns.
10.10.1.118:11411 : 555973 , 23.28%
10.16.69.133:11411 : 533208 , 22.33%
10.16.69.135:11411 : 355425 , 14.88%
10.16.6.184:11411 : 235848 , 9.88%
10.10.1.228:11411 : 346293 , 14.50%
10.10.1.208:11411 : 361111 , 15.12%
variance:0.33143952290634493

320
FNV1A_64_HASH      costs:1362ms,      number:2387858      perdata:0.570385676200176ns.
10.10.1.118:11411 : 441835 , 18.50%
10.16.69.133:11411 : 480306 , 20.11%
10.16.69.135:11411 : 355425 , 14.88%
10.16.6.184:11411 : 318742 , 13.35%
10.10.1.228:11411 : 431662 , 18.08%
10.10.1.208:11411 : 359888 , 15.07%
variance:0.16774724292381052

480
FNV1A_64_HASH      costs:1241ms,      number:2387858      perdata:0.5197126462293822ns.
10.10.1.118:11411 : 193664 , 8.11%
10.16.69.133:11411 : 479463 , 20.08%
10.16.69.135:11411 : 355425 , 14.88%
10.16.6.184:11411 : 373414 , 15.64%
10.10.1.228:11411 : 432505 , 18.11%
10.10.1.208:11411 : 553387 , 23.18%
variance:0.3336763701976884

640
FNV1A_64_HASH      costs:1379ms,      number:2387858      perdata:0.5775050275183867ns.
10.10.1.118:11411 : 373886 , 15.66%
10.16.69.133:11411 : 479463 , 20.08%
10.16.6.184:11411 : 427466 , 17.90%
10.16.69.135:11411 : 382256 , 16.01%
10.10.1.228:11411 : 432505 , 18.11%
10.10.1.208:11411 : 292282 , 12.24%
variance:0.17161969978829905

1600
FNV1A_64_HASH      costs:1766ms,      number:2387858      perdata:0.7395749663505954ns.
10.10.1.118:11411 : 545111 , 22.83%
10.16.69.133:11411 : 613142 , 25.68%
10.16.6.184:11411 : 289263 , 12.11%
10.16.69.135:11411 : 346545 , 14.51%
10.10.1.228:11411 : 446566 , 18.70%
10.10.1.208:11411 : 147231 , 6.17%
variance:0.5426740049801704

3200
FNV1A_64_HASH      costs:2211ms,      number:2387858      perdata:0.9259344567390523ns.
10.10.1.118:11411 : 455113 , 19.06%
10.16.69.133:11411 : 346512 , 14.51%
10.16.69.135:11411 : 443590 , 18.58%
10.16.6.184:11411 : 389935 , 16.33%
10.10.1.228:11411 : 410865 , 17.21%
10.10.1.208:11411 : 341843 , 14.32%
variance:0.14436196587267802





FNV1_32_HASH 
160
FNV1_32_HASH      costs:1205ms,      number:2387858      perdata:0.5046363728496418ns.
10.10.1.118:11411 : 489055 , 20.48%
10.16.69.133:11411 : 395165 , 16.55%
10.16.69.135:11411 : 259880 , 10.88%
10.16.6.184:11411 : 232323 , 9.73%
10.10.1.228:11411 : 277434 , 11.62%
10.10.1.208:11411 : 734001 , 30.74%
variance:0.6438545073606985

320
FNV1_32_HASH      costs:1389ms,      number:2387858      perdata:0.5816928812349813ns.
10.10.1.118:11411 : 489184 , 20.49%
10.16.69.133:11411 : 333188 , 13.95%
10.16.69.135:11411 : 362610 , 15.19%
10.16.6.184:11411 : 483796 , 20.26%
10.10.1.228:11411 : 350105 , 14.66%
10.10.1.208:11411 : 368975 , 15.45%
variance:0.1820378975835861

480
FNV1_32_HASH      costs:1506ms,      number:2387858      perdata:0.6306907697191374ns.
10.10.1.118:11411 : 473926 , 19.85%
10.16.69.133:11411 : 319241 , 13.37%
10.16.69.135:11411 : 414314 , 17.35%
10.16.6.184:11411 : 465959 , 19.51%
10.10.1.228:11411 : 335353 , 14.04%
10.10.1.208:11411 : 379065 , 15.87%
variance:0.17289055882258353

640
FNV1_32_HASH      costs:1552ms,      number:2387858      perdata:0.6499548968154722ns.
10.10.1.118:11411 : 482243 , 20.20%
10.16.69.133:11411 : 318593 , 13.34%
10.16.69.135:11411 : 445108 , 18.64%
10.16.6.184:11411 : 477512 , 20.00%
10.10.1.228:11411 : 320861 , 13.44%
10.10.1.208:11411 : 343541 , 14.39%
variance:0.20131515448144954

1600
FNV1_32_HASH      costs:1945ms,      number:2387858      perdata:0.8145375478776377ns.
10.10.1.118:11411 : 392205 , 16.42%
10.16.69.133:11411 : 460475 , 19.28%
10.16.69.135:11411 : 329994 , 13.82%
10.16.6.184:11411 : 576403 , 24.14%
10.10.1.228:11411 : 278081 , 11.65%
10.10.1.208:11411 : 350700 , 14.69%
variance:0.2777437217142079

3200
FNV1_32_HASH      costs:2156ms,      number:2387858      perdata:0.9029012612977824ns.
10.10.1.118:11411 : 409141 , 17.13%
10.16.69.133:11411 : 539964 , 22.61%
10.16.6.184:11411 : 280652 , 11.75%
10.16.69.135:11411 : 363105 , 15.21%
10.10.1.228:11411 : 348210 , 14.58%
10.10.1.208:11411 : 446786 , 18.71%
variance:0.22839794624712736







KETAMA_HASH      
160
KETAMA_HASH      costs:2311ms,      number:2387858      perdata:0.9678129939049976ns.
10.10.1.118:11411 : 411977 , 17.25%
10.16.69.133:11411 : 449525 , 18.83%
10.16.6.184:11411 : 387824 , 16.24%
10.16.69.135:11411 : 346523 , 14.51%
10.10.1.228:11411 : 382838 , 16.03%
10.10.1.208:11411 : 409171 , 17.14%
variance:0.12852730715084365

320
KETAMA_HASH      costs:2499ms,      number:2387858      perdata:1.046544643776975ns.
10.10.1.118:11411 : 391581 , 16.40%
10.16.69.133:11411 : 438704 , 18.37%
10.16.6.184:11411 : 390527 , 16.35%
10.16.69.135:11411 : 348138 , 14.58%
10.10.1.228:11411 : 396903 , 16.62%
10.10.1.208:11411 : 422005 , 17.67%
variance:0.1251928291794245

480
KETAMA_HASH      costs:2526ms,      number:2387858      perdata:1.0578518488117803ns.
10.10.1.118:11411 : 388526 , 16.27%
10.16.69.133:11411 : 416766 , 17.45%
10.16.6.184:11411 : 420646 , 17.62%
10.16.69.135:11411 : 368347 , 15.43%
10.10.1.228:11411 : 383291 , 16.05%
10.10.1.208:11411 : 410282 , 17.18%
variance:0.11754543766219773

640
KETAMA_HASH      costs:2587ms,      number:2387858      perdata:1.0833977564830068ns.
10.10.1.118:11411 : 409084 , 17.13%
10.16.69.133:11411 : 393467 , 16.48%
10.16.6.184:11411 : 404729 , 16.95%
10.16.69.135:11411 : 377004 , 15.79%
10.10.1.228:11411 : 398191 , 16.68%
10.10.1.208:11411 : 405383 , 16.98%
variance:0.11311062154023681

1600
KETAMA_HASH      costs:2674ms,      number:2387858      perdata:1.1198320838173794ns.
10.10.1.118:11411 : 393722 , 16.49%
10.16.69.133:11411 : 405244 , 16.97%
10.16.6.184:11411 : 389027 , 16.29%
10.16.69.135:11411 : 396296 , 16.60%
10.10.1.228:11411 : 415356 , 17.39%
10.10.1.208:11411 : 388213 , 16.26%
variance:0.11272230273956219

3200
KETAMA_HASH      costs:2927ms,      number:2387858      perdata:1.2257847828472213ns.
10.10.1.118:11411 : 389331 , 16.30%
10.16.69.133:11411 : 397212 , 16.63%
10.16.69.135:11411 : 400242 , 16.76%
10.16.6.184:11411 : 397344 , 16.64%
10.10.1.228:11411 : 404854 , 16.95%
10.10.1.208:11411 : 398875 , 16.70%
variance:0.11148809000778596






NATIVE_HASH      
160
NATIVE_HASH      costs:990ms,      number:2387858      perdata:0.4145975179428592ns.
10.10.1.118:11411 : 293716 , 12.30%
10.16.69.133:11411 : 209344 , 8.77%
10.16.69.135:11411 : 694618 , 29.09%
10.16.6.184:11411 : 362735 , 15.19%
10.10.1.228:11411 : 191147 , 8.00%
10.10.1.208:11411 : 636298 , 26.65%
variance:0.7987988531108471

320
NATIVE_HASH      costs:1036ms,      number:2387858      perdata:0.4338616450391941ns.
10.10.1.118:11411 : 293718 , 12.30%
10.16.69.133:11411 : 209343 , 8.77%
10.16.69.135:11411 : 694617 , 29.09%
10.16.6.184:11411 : 362735 , 15.19%
10.10.1.228:11411 : 191146 , 8.00%
10.10.1.208:11411 : 636299 , 26.65%
variance:0.7987996050576887

480
NATIVE_HASH      costs:1042ms,      number:2387858      perdata:0.43637435726915086ns.
10.10.1.118:11411 : 293719 , 12.30%
10.16.69.133:11411 : 209342 , 8.77%
10.16.69.135:11411 : 694616 , 29.09%
10.16.6.184:11411 : 362735 , 15.19%
10.10.1.228:11411 : 191146 , 8.00%
10.10.1.208:11411 : 636300 , 26.65%
variance:0.7987997573996342


640
NATIVE_HASH      costs:1042ms,      number:2387858      perdata:0.43637435726915086ns.
10.10.1.118:11411 : 293720 , 12.30%
10.16.69.133:11411 : 209342 , 8.77%
10.16.69.135:11411 : 694616 , 29.09%
10.16.6.184:11411 : 362735 , 15.19%
10.10.1.228:11411 : 191145 , 8.00%
10.10.1.208:11411 : 636300 , 26.65%
variance:0.7988003570512985

1600
NATIVE_HASH      costs:1097ms,      number:2387858      perdata:0.4594075527104208ns.
10.10.1.118:11411 : 300948 , 12.60%
10.16.69.133:11411 : 209342 , 8.77%
10.16.69.135:11411 : 587836 , 24.62%
10.16.6.184:11411 : 375217 , 15.71%
10.10.1.228:11411 : 171437 , 7.18%
10.10.1.208:11411 : 743078 , 31.12%
variance:0.8476458026756032

3200
NATIVE_HASH      costs:1132ms,      number:2387858      perdata:0.4740650407185017ns.
10.10.1.118:11411 : 300980 , 12.60%
10.16.69.133:11411 : 209342 , 8.77%
10.16.69.135:11411 : 587799 , 24.62%
10.16.6.184:11411 : 375214 , 15.71%
10.10.1.228:11411 : 171408 , 7.18%
10.10.1.208:11411 : 743115 , 31.12%
variance:0.8477000496670247









CRC_HASH      
160
CRC_HASH      costs:1770ms,      number:2387858      perdata:0.7412501078372332ns.
10.10.1.118:11411 : 347616 , 14.56%
10.16.69.133:11411 : 327404 , 13.71%
10.16.6.184:11411 : 459595 , 19.25%
10.16.69.135:11411 : 481764 , 20.18%
10.10.1.228:11411 : 395621 , 16.57%
10.10.1.208:11411 : 375858 , 15.74%
variance:0.1661475366273886

320
CRC_HASH      costs:1843ms,      number:2387858      perdata:0.7718214399683734ns.
10.10.1.118:11411 : 291374 , 12.20%
10.16.69.133:11411 : 387281 , 16.22%
10.16.6.184:11411 : 462735 , 19.38%
10.16.69.135:11411 : 371289 , 15.55%
10.10.1.228:11411 : 489835 , 20.51%
10.10.1.208:11411 : 385344 , 16.14%
variance:0.1841338067695997

480
CRC_HASH      costs:1873ms,      number:2387858      perdata:0.784385001118157ns.
10.10.1.118:11411 : 375365 , 15.72%
10.16.69.133:11411 : 362789 , 15.19%
10.16.6.184:11411 : 568127 , 23.79%
10.16.69.135:11411 : 324662 , 13.60%
10.10.1.228:11411 : 410299 , 17.18%
10.10.1.208:11411 : 346616 , 14.52%
variance:0.22471542586997548


640
CRC_HASH      costs:1904ms,      number:2387858      perdata:0.7973673476396ns.
10.10.1.118:11411 : 416794 , 17.45%
10.16.69.133:11411 : 400187 , 16.76%
10.16.6.184:11411 : 539187 , 22.58%
10.16.69.135:11411 : 365325 , 15.30%
10.10.1.228:11411 : 373834 , 15.66%
10.10.1.208:11411 : 292531 , 12.25%
variance:0.20776696947943804

1600
CRC_HASH      costs:1923ms,      number:2387858      perdata:0.8053242697011297ns.
10.10.1.118:11411 : 281105 , 11.77%
10.16.69.133:11411 : 365290 , 15.30%
10.16.69.135:11411 : 441715 , 18.50%
10.16.6.184:11411 : 466691 , 19.54%
10.10.1.228:11411 : 456081 , 19.10%
10.10.1.208:11411 : 376976 , 15.79%
variance:0.18471055021331423

3200
CRC_HASH      costs:1904ms,      number:2387858      perdata:0.7973673476396ns.
10.10.1.118:11411 : 282749 , 11.84%
10.16.69.133:11411 : 435415 , 18.23%
10.16.69.135:11411 : 450655 , 18.87%
10.16.6.184:11411 : 486627 , 20.38%
10.10.1.228:11411 : 372277 , 15.59%
10.10.1.208:11411 : 360135 , 15.08%
variance:0.19121762886993007







FNV1_64_HASH      
160
FNV1_64_HASH      costs:969ms,      number:2387858      perdata:0.4058030251380107ns.
10.10.1.118:11411 : 248007 , 10.39%
10.16.69.133:11411 : 324288 , 13.58%
10.16.6.184:11411 : 1051464 , 44.03%
10.16.69.135:11411 : 239652 , 10.04%
10.10.1.228:11411 : 126840 , 5.31%
10.10.1.208:11411 : 397607 , 16.65%
variance:1.7291444612562392

320
FNV1_64_HASH      costs:1028ms,      number:2387858      perdata:0.4305113620659185ns.
10.10.1.118:11411 : 248327 , 10.40%
10.16.69.133:11411 : 323947 , 13.57%
10.16.6.184:11411 : 1051144 , 44.02%
10.16.69.135:11411 : 239652 , 10.04%
10.10.1.228:11411 : 126841 , 5.31%
10.10.1.208:11411 : 397947 , 16.67%
variance:1.7277872629051703

480
FNV1_64_HASH      costs:1055ms,      number:2387858      perdata:0.44181856710072376ns.
10.10.1.118:11411 : 248327 , 10.40%
10.16.69.133:11411 : 324255 , 13.58%
10.16.6.184:11411 : 1051365 , 44.03%
10.16.69.135:11411 : 239652 , 10.04%
10.10.1.228:11411 : 126841 , 5.31%
10.10.1.208:11411 : 397418 , 16.64%
variance:1.7284991720338077

640
FNV1_64_HASH      costs:1028ms,      number:2387858      perdata:0.4305113620659185ns.
10.10.1.118:11411 : 248327 , 10.40%
10.16.69.133:11411 : 324377 , 13.58%
10.16.6.184:11411 : 1051245 , 44.02%
10.16.69.135:11411 : 239652 , 10.04%
10.10.1.228:11411 : 126841 , 5.31%
10.10.1.208:11411 : 397416 , 16.64%
variance:1.7279883165546042

1600
FNV1_64_HASH      costs:1045ms,      number:2387858      perdata:0.43763071338412923ns.
10.10.1.118:11411 : 308326 , 12.91%
10.16.69.133:11411 : 365291 , 15.30%
10.16.6.184:11411 : 263191 , 11.02%
10.16.69.135:11411 : 504782 , 21.14%
10.10.1.228:11411 : 206764 , 8.66%
10.10.1.208:11411 : 739504 , 30.97%
variance:0.6719899505883513

3200
FNV1_64_HASH      costs:1185ms,      number:2387858      perdata:0.49626066541645275ns.
10.10.1.118:11411 : 132475 , 5.55%
10.16.69.133:11411 : 509299 , 21.33%
10.16.6.184:11411 : 131879 , 5.52%
10.16.69.135:11411 : 566719 , 23.73%
10.10.1.228:11411 : 202817 , 8.49%
10.10.1.208:11411 : 844669 , 35.37%
variance:1.338157043584133



FNV1A_32_HASH      
160
FNV1A_32_HASH      costs:1507ms,      number:2387858      perdata:0.6311095550907968ns.
10.10.1.118:11411 : 312106 , 13.07%
10.16.69.133:11411 : 253577 , 10.62%
10.16.6.184:11411 : 644748 , 27.00%
10.16.69.135:11411 : 421664 , 17.66%
10.10.1.228:11411 : 323920 , 13.57%
10.10.1.208:11411 : 431843 , 18.08%
variance:0.3926374779645682

320
FNV1A_32_HASH      costs:1761ms,      number:2387858      perdata:0.7374810394922982ns.
10.10.1.118:11411 : 400635 , 16.78%
10.16.69.133:11411 : 368357 , 15.43%
10.16.6.184:11411 : 362826 , 15.19%
10.16.69.135:11411 : 287307 , 12.03%
10.10.1.228:11411 : 410149 , 17.18%
10.10.1.208:11411 : 558584 , 23.39%
variance:0.228939842705781

480
FNV1A_32_HASH      costs:1806ms,      number:2387858      perdata:0.7563263812169736ns.
10.10.1.118:11411 : 397470 , 16.65%
10.16.69.133:11411 : 369449 , 15.47%
10.16.69.135:11411 : 337365 , 14.13%
10.16.6.184:11411 : 342005 , 14.32%
10.10.1.228:11411 : 391804 , 16.41%
10.10.1.208:11411 : 549765 , 23.02%
variance:0.20084339336306026

640
FNV1A_32_HASH      costs:1812ms,      number:2387858      perdata:0.7588390934469303ns.
10.10.1.118:11411 : 332503 , 13.92%
10.16.69.133:11411 : 420161 , 17.60%
10.16.69.135:11411 : 354662 , 14.85%
10.16.6.184:11411 : 346627 , 14.52%
10.10.1.228:11411 : 462565 , 19.37%
10.10.1.208:11411 : 471340 , 19.74%
variance:0.16619754052235786

1600
FNV1A_32_HASH      costs:1994ms,      number:2387858      perdata:0.8350580310889508ns.
10.10.1.118:11411 : 407301 , 17.06%
10.16.69.133:11411 : 381842 , 15.99%
10.16.69.135:11411 : 335819 , 14.06%
10.16.6.184:11411 : 458970 , 19.22%
10.10.1.228:11411 : 425412 , 17.82%
10.10.1.208:11411 : 378514 , 15.85%
variance:0.13760105340499912

3200
FNV1A_32_HASH      costs:2194ms,      number:2387858      perdata:0.9188151054208415ns.
10.10.1.118:11411 : 399804 , 16.74%
10.16.69.133:11411 : 397910 , 16.66%
10.16.6.184:11411 : 373879 , 15.66%
10.16.69.135:11411 : 404753 , 16.95%
10.10.1.228:11411 : 392987 , 16.46%
10.10.1.208:11411 : 418525 , 17.53%
variance:0.1142594682506669


对比结论:
1、KETAMA_HASH算法在各种情况下都有很好的均匀分布,并且倍数越高越均匀;

2、NATIVE_HASH算法不受节点倍数影响;

3、FNV1_64_HASH算法不受节点倍数影响,但在超过某段范围之后,分布均匀性会改变;

4、FNV1_32_HASH 和 FNV1A_32_HASH随着节点倍数增加,均匀性会变好;

5、FNV1A_64_HASH 和 CRC_HASH 随节点倍数增加波动性很大;

算法原理(160倍率、640适用)

NATIVE_HASH

1、hash是取字符串的hashcode,
2、这样虚拟节点的最后hash落位基本都在一起,虚拟节点穿插分布的作用并没有起到,相同节点的虚拟节点基本都分布在一起,相当于没有用到虚拟节点,
3、最终服务器区间的分布也不均匀。

CRC_HASH

1、取CRC32并右移16位;
2、会发生虚拟节点落位冲突的情况,后面的将前面的直接覆盖(CRC32本身就有碰撞概率);
3、会散乱分布。

相关链接:
http://xdxd.love/2016/01/14/CRC32算法详解和使用场景/

https://blog.csdn.net/yunhua_lee/article/details/42775039

http://preshing.com/20110504/hash-collision-probabilities/

http://www.backplane.com/matt/crc64.html

FNV HASH算法:

FNV1_64_HASH:虚拟节点集中分布,无冲突覆盖

FNV1A_64_HASH:虚拟节点略有分散但是还是聚集分布,无冲突覆盖

FNV1_32_HASH:虚拟节点集中分布,无冲突覆盖

FNV1A_32_HASH:虚拟节点略有打乱,无冲突覆盖

https://blog.csdn.net/fall221/article/details/8291381

KETAMA_HASH

1、根据key取MD5得到16位字符数组,

2、将16位字符数组相邻4位倒序拼装成hash,一次得到四个虚拟节点hash,

3、最终虚拟节点hash分布穿插错落且分布区间比较均匀,无冲突覆盖。

问题与思考

原来的hash算法是将集群中的ip对应上一个点,每个机器ip对应上hash结果的一个点(比如8台机器,进行模7运算,将0-7对应的hash结果分别分配给8台机器),这样其实可以减少节点(比如第七台机器下线了,分配的hash值为6,则可以让第八台机器负担流量,接收6,7的hash结果),但是增加节点的话分配不够了(比如增加了一台机器9,但是hash结果只有八个,不能区分出流量分给他,则必须重新定义hash的模数)
一致性hash是将hash结果集取得很大(0~2^32),然后将机器ip也取模,最后落在结果集上,将结果集划分成区间,按照区间分配给各个机器,这样机器减少时还是一样,只是新负担的机器接管了区间,而在增加的时候,因为是划分了区间,所以可以对区间再进行划分,从原区间中划分出来一部分分给新机器就可以了(其实就是新机器算一次ip的hash,将上一个节点的ip hash结果到新节点的ip hash结果这段区间分给新机器)

客户端没有存储key和host的map,而是存储了服务器虚拟节点的区间分布情况,存储在ketamaNodeLocator类中的ketamaNodes中,这是一个TreeMap结构,每次存储时,都根据要存储的key计算hash,然后在ketamaNodes中找到对应应该存储的节点。

你可能感兴趣的:(日常工作)