一致性hash算法简介
首先为什么需要一致性hash算法？
因为传统的hash算法，对于将数据映射到具体的结点确实有用，如key % N，key是数据的key，N的机器结点数。但是如果在集群中增加或者删除机器的时候，所有的映射都无效了，需要重新映射，代价很高。
算法的原理在《大数据》一文中又提到过，很简单。算法的描述如下：
先构造一个长度为的整数环（这个环被称为一致性Hash环），根据结点名称的Hash值（其分布为[]）将服务器结点放置到这个Hash环中，然后根据数据的Key值计算得到其Hash值（其分布为[]），接着在Hash环上顺时针查找距离这个Key值的Hash值最近的服务器结点，完成Key到服务器的映射查找。
数据结构的选取
1.解决方案一：排序+List
不好用，排序最快的时间复杂度为O(nlogn)
2.解决方案二：遍历+List
时间复杂度能压缩到O(n)，貌似还不错。
3.解决方案三：二叉查找树（红黑树）
如果不使用List，用二叉查找树，特别是红黑树，查询的时间复杂度能到O(logn)，非常快了。理由如下：
- 红黑树主要存储有序的数据，并且效率很高，所以可以存储服务器结点的Hash值。
- JDK里面提供了红黑树的实现TreeMap和TreeSet
另外，以TreeMap为例，TreeMap本身提供了tailMap(K fromKey)方法，该方法会返回一个比fromKey大的值的map子集合。
Hash值重新计算
关于Hash算法的选取，是不能用String的hashCode()方法的，因为产生的范围不是这个范围，所以自己重新实现。
一致性Hash算法的实现版本1：不带虚拟结点
使用一致性Hash算法，尽管增强系统的伸缩性，但是可能导致负载分布不均匀，解决方法是用虚拟结点代替真实结点。
下面是不带虚拟结点的实现：
package com.xushu;
import java.util.SortedMap;
import java.util.TreeMap;
/**
* 不带虚拟节点的一致性Hash算法
*/
public class ConsistentHashingWithoutVirtualNode {
/**
* 待添加入Hash环的服务器列表
*/
private static String[] servers = {"192.168.0.0:111", "192.168.0.1:111", "192.168.0.2:111",
"192.168.0.3:111", "192.168.0.4:111"};
/**
* key表示服务器的hash值，value表示服务器的名称
*/
private static SortedMap sortedMap = new TreeMap();
/**
* 程序初始化，将所有的服务器防区sortedMap中
*/
static{
for(int i = 0; i < servers.length; i++){
int hash = getHash(servers[i]);
System.out.println("[" + servers[i] + "]加入集合中, 其Hash值为" + hash);
sortedMap.put(hash, servers[i]);
}
System.out.println();
}
/**
* 使用FNV1_32_HASH算法计算服务器的Hash值，这里不使用重写hashCode的方法，最终效果没区别
*/
private static int getHash(String str){
final int p = 16777619;
int hash = (int)2166136261L;
for (int i = 0; i < str.length(); i++)
hash = (hash ^ str.charAt(i)) * p;
hash += hash << 13;
hash ^= hash >> 7;
hash += hash << 3;
hash ^= hash >> 17;
hash += hash << 5;
// 如果算出来的值为负数则取其绝对值
if (hash < 0)
hash = Math.abs(hash);
return hash;
}
/**
* 得到应当路由到的结点，也就是算出数据应该放到哪个server
*/
private static String getServer(String node){
//得到待路由的结点的Hash值
int hash = getHash(node);
//得到大于该Hash值的所有Map
SortedMap subMap = sortedMap.tailMap(hash);
// 第一个Key就是顺时针过去离node最近的那个结点
Integer integer = subMap.firstKey();
//返回对应的服务器名称
return subMap.get(integer);
}
public static void main(String[] args)
{
String[] nodes = {"127.0.0.1:1111", "221.226.0.1:2222", "10.211.0.1:3333"};
for (int i = 0; i < nodes.length; i++)
System.out.println("[" + nodes[i] + "]的hash值为" +
getHash(nodes[i]) + ", 被路由到结点[" + getServer(nodes[i]) + "]");
}
}
运行结果如下：
使用虚拟结点改善一致性Hash算法
上面的一致性Hash算法实现，可以在很大程度上解决很多分布式环境下不好的路由算法导致系统伸缩性差的问题，但是会带来另外一个问题：负载不均。
比如Hash环上有A，B，C三个服务器结点，分别有100个请求会被路由到相应的服务器上。现在在A与B之间增加一个结点D，这导致了原来会路由到B上的部分节点被路由到了D上，这样A、C上被路由到的请求明显多于B、D上的，原来三个服务器节点上均衡的负载被打破了。某种程度上来说，这失去了负载均衡的意义，因为负载均衡的目的本身就是为了使得目标服务器均分所有的请求。
解决这个问题的办法是引入虚拟节点，其工作原理是：将一个物理节点拆分为多个虚拟节点，并且同一个物理节点的虚拟节点尽量均匀分布在Hash环上。采取这样的方式，就可以有效地解决增加或减少节点时候的负载不均衡的问题。
一致性Hash算法实现版本2：带虚拟节点
带虚拟结点需要考虑的问题是：
- 1.一个真实结点如何对应多个虚拟结点
- 2.虚拟结点找到后如何还原成真实结点
本文给的方法是，给每个真实结点后面根据虚拟节点加上后缀再取Hash值，比如"192.168.0.0:111"就把它变成"192.168.0.0:111&&VN0"到"192.168.0.0:111&&VN4"，VN就是Virtual Node的缩写，还原的时候只需要从头截取字符串到"&&"的位置就可以了。
下面是代码实现：
package com.xushu;
import java.util.LinkedList;
import java.util.List;
import java.util.SortedMap;
import java.util.TreeMap;
/**
* 带虚拟节点的一致性Hash算法
*/
public class ConsistentHashingWithVirtualNode
{
/**
* 待添加入Hash环的服务器列表
*/
private static String[] servers = {"192.168.0.0:111", "192.168.0.1:111", "192.168.0.2:111",
"192.168.0.3:111", "192.168.0.4:111"};
/**
* 真实结点列表,考虑到服务器上线、下线的场景，即添加、删除的场景会比较频繁，这里使用LinkedList会更好
*/
private static List realNodes = new LinkedList();
/**
* 虚拟节点，key表示虚拟节点的hash值，value表示虚拟节点的名称
*/
private static SortedMap virtualNodes =
new TreeMap();
/**
* 虚拟节点的数目，这里写死，为了演示需要，一个真实结点对应5个虚拟节点
*/
private static final int VIRTUAL_NODES = 5;
static
{
// 先把原始的服务器添加到真实结点列表中
for (int i = 0; i < servers.length; i++)
realNodes.add(servers[i]);
// 再添加虚拟节点，遍历LinkedList使用foreach循环效率会比较高
for (String str : realNodes)
{
for (int i = 0; i < VIRTUAL_NODES; i++)
{
String virtualNodeName = str + "&&VN" + String.valueOf(i);
int hash = getHash(virtualNodeName);
System.out.println("虚拟节点[" + virtualNodeName + "]被添加, hash值为" + hash);
//virtualNodes.put(hash, virtualNodeName);
// 其实，多个虚拟节点的 hash 值对应相同的真实节点就好了
virtualNodes.put(hash, str);
}
}
System.out.println();
}
/**
* 使用FNV1_32_HASH算法计算服务器的Hash值,这里不使用重写hashCode的方法，最终效果没区别
*/
private static int getHash(String str)
{
final int p = 16777619;
int hash = (int)2166136261L;
for (int i = 0; i < str.length(); i++)
hash = (hash ^ str.charAt(i)) * p;
hash += hash << 13;
hash ^= hash >> 7;
hash += hash << 3;
hash ^= hash >> 17;
hash += hash << 5;
// 如果算出来的值为负数则取其绝对值
if (hash < 0)
hash = Math.abs(hash);
return hash;
}
/**
* 得到应当路由到的结点
*/
private static String getServer(String node)
{
// 得到带路由的结点的Hash值
int hash = getHash(node);
// 得到大于该Hash值的所有Map
SortedMap subMap =
virtualNodes.tailMap(hash);
// 第一个Key就是顺时针过去离node最近的那个结点
Integer i = subMap.firstKey();
return subMap.get(i);
// 返回对应的虚拟节点名称，这里字符串稍微截取一下
//String virtualNode = subMap.get(i);
//return virtualNode.substring(0, virtualNode.indexOf("&&"));
}
public static void main(String[] args)
{
String[] nodes = {"127.0.0.1:1111", "221.226.0.1:2222", "10.211.0.1:3333"};
for (int i = 0; i < nodes.length; i++)
System.out.println("[" + nodes[i] + "]的hash值为" +
getHash(nodes[i]) + ", 被路由到结点[" + getServer(nodes[i]) + "]");
}
}
运行截图：
终极代码篇（番外篇）
主要是这位前辈的代码写的太好了，所以我要贴出来，我怕下次搞丢了。
Node.java
package com.meixianfeng.hash;
/**
* Created with IntelliJ IDEA.
* User: yfwangqing
* Date: 13-8-23
* Time: 下午3:27
* 节点的IP实现
*/
public class Node {
private String name;
private String ip;
public Node(String name, String ip) {
this.name = name;
this.ip = ip;
}
public Node(String ip) {
this.ip = ip;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getIp() {
return ip;
}
public void setIp(String ip) {
this.ip = ip;
}
@Override
public String toString() {
if (name != null && !"".equals(name)) {
return ip + "-" + name;
}
return ip;
}
@Override
public boolean equals(Object o) {
if (o == null) return false;
Node node = (Node) o;
if (node.getIp() == null && ip == null && node.getName() == null && name == null) return true;
if (name == null && node.getName() != null) return false;
if (ip == null && node.getIp() != null) return false;
assert ip != null;
assert name != null;
return name.equals(node.getName()) && ip.equals(node.getIp());
}
@Override
public int hashCode() {
int result = name != null ? name.hashCode() : 0;
result = 31 * result + (ip != null ? ip.hashCode() : 0);
return result;
}
}
ByteUtils.java
package com.meixianfeng.hash;
import java.io.UnsupportedEncodingException;
import java.nio.ByteBuffer;
import java.nio.charset.Charset;
/**
*Copyright [2009-2010] [dennis zhuang([email protected])]
*Licensed under the Apache License, Version 2.0 (the "License");
*you may not use this file except in compliance with the License.
*You may obtain a copy of the License at
* http://www.apache.org/licenses/LICENSE-2.0
*Unless required by applicable law or agreed to in writing,
*software distributed under the License is distributed on an "AS IS" BASIS,
*WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
*either express or implied. See the License for the specific language governing permissions and limitations under the License
*/
/**
* Utilities for byte process
*
* @author dennis
*/
public final class ByteUtils {
public static final String DEFAULT_CHARSET_NAME = "utf-8";
public static final Charset DEFAULT_CHARSET = Charset
.forName(DEFAULT_CHARSET_NAME);
/**
* if it is testing,check key argument even if use binary protocol. The user
* must never change this value at all.
*/
public static boolean testing;
private ByteUtils() {
}
public static boolean isNumber(String string) {
if (string == null || string.isEmpty()) {
return false;
}
int i = 0;
if (string.charAt(0) == '-') {
if (string.length() > 1) {
i++;
} else {
return false;
}
}
for (; i < string.length(); i++) {
if (!Character.isDigit(string.charAt(i))) {
return false;
}
}
return true;
}
public static final byte[] getBytes(String k) {
if (k == null || k.length() == 0) {
throw new IllegalArgumentException("Key must not be blank");
}
try {
return k.getBytes(DEFAULT_CHARSET_NAME);
} catch (UnsupportedEncodingException e) {
throw new RuntimeException(e);
}
}
private static int maxKeyLength = 250;
public static final int normalizeCapacity(int requestedCapacity) {
switch (requestedCapacity) {
case 0:
case 1 << 0:
case 1 << 1:
case 1 << 2:
case 1 << 3:
case 1 << 4:
case 1 << 5:
case 1 << 6:
case 1 << 7:
case 1 << 8:
case 1 << 9:
case 1 << 10:
case 1 << 11:
case 1 << 12:
case 1 << 13:
case 1 << 14:
case 1 << 15:
case 1 << 16:
case 1 << 17:
case 1 << 18:
case 1 << 19:
case 1 << 21:
case 1 << 22:
case 1 << 23:
case 1 << 24:
case 1 << 25:
case 1 << 26:
case 1 << 27:
case 1 << 28:
case 1 << 29:
case 1 << 30:
case Integer.MAX_VALUE:
return requestedCapacity;
}
int newCapacity = 1;
while (newCapacity < requestedCapacity) {
newCapacity <<= 1;
if (newCapacity < 0) {
return Integer.MAX_VALUE;
}
}
return newCapacity;
}
public static final boolean stepBuffer(ByteBuffer buffer, int remaining) {
if (buffer.remaining() >= remaining) {
buffer.position(buffer.position() + remaining);
return true;
} else {
return false;
}
}
public static String getString(byte[] bytes) {
try {
return new String(bytes, DEFAULT_CHARSET_NAME);
} catch (UnsupportedEncodingException e) {
throw new RuntimeException(e);
}
}
public static void byte2hex(byte b, StringBuffer buf) {
char[] hexChars = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9',
'A', 'B', 'C', 'D', 'E', 'F'};
int high = ((b & 0xf0) >> 4);
int low = (b & 0x0f);
buf.append(hexChars[high]);
buf.append(hexChars[low]);
}
public static void int2hex(int a, StringBuffer str) {
str.append(Integer.toHexString(a));
}
public static void short2hex(int a, StringBuffer str) {
str.append(Integer.toHexString(a));
}
public static void getBytes(long i, int index, byte[] buf) {
long q;
int r;
int pos = index;
byte sign = 0;
if (i < 0) {
sign = '-';
i = -i;
}
// Get 2 digits/iteration using longs until quotient fits into an int
while (i > Integer.MAX_VALUE) {
q = i / 100;
// really: r = i - (q * 100);
r = (int) (i - ((q << 6) + (q << 5) + (q << 2)));
i = q;
buf[--pos] = DigitOnes[r];
buf[--pos] = DigitTens[r];
}
// Get 2 digits/iteration using ints
int q2;
int i2 = (int) i;
while (i2 >= 65536) {
q2 = i2 / 100;
// really: r = i2 - (q * 100);
r = i2 - ((q2 << 6) + (q2 << 5) + (q2 << 2));
i2 = q2;
buf[--pos] = DigitOnes[r];
buf[--pos] = DigitTens[r];
}
// Fall thru to fast mode for smaller numbers
// assert(i2 <= 65536, i2);
for (; ; ) {
q2 = (i2 * 52429) >>> (16 + 3);
r = i2 - ((q2 << 3) + (q2 << 1)); // r = i2-(q2*10) ...
buf[--pos] = digits[r];
i2 = q2;
if (i2 == 0)
break;
}
if (sign != 0) {
buf[--pos] = sign;
}
}
/**
* Places characters representing the integer i into the character array
* buf. The characters are placed into the buffer backwards starting with
* the least significant digit at the specified index (exclusive), and
* working backwards from there.
*
* Will fail if i == Integer.MIN_VALUE
*/
static void getBytes(int i, int index, byte[] buf) {
int q, r;
int pos = index;
byte sign = 0;
if (i < 0) {
sign = '-';
i = -i;
}
// Generate two digits per iteration
while (i >= 65536) {
q = i / 100;
// really: r = i - (q * 100);
r = i - ((q << 6) + (q << 5) + (q << 2));
i = q;
buf[--pos] = DigitOnes[r];
buf[--pos] = DigitTens[r];
}
// Fall thru to fast mode for smaller numbers
// assert(i <= 65536, i);
for (; ; ) {
q = (i * 52429) >>> (16 + 3);
r = i - ((q << 3) + (q << 1)); // r = i-(q*10) ...
buf[--pos] = digits[r];
i = q;
if (i == 0)
break;
}
if (sign != 0) {
buf[--pos] = sign;
}
}
/**
* All possible chars for representing a number as a String
*/
final static byte[] digits = {'0', '1', '2', '3', '4', '5', '6', '7', '8',
'9', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l',
'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y',
'z'};
final static byte[] DigitTens = {'0', '0', '0', '0', '0', '0', '0', '0',
'0', '0', '1', '1', '1', '1', '1', '1', '1', '1', '1', '1', '2',
'2', '2', '2', '2', '2', '2', '2', '2', '2', '3', '3', '3', '3',
'3', '3', '3', '3', '3', '3', '4', '4', '4', '4', '4', '4', '4',
'4', '4', '4', '5', '5', '5', '5', '5', '5', '5', '5', '5', '5',
'6', '6', '6', '6', '6', '6', '6', '6', '6', '6', '7', '7', '7',
'7', '7', '7', '7', '7', '7', '7', '8', '8', '8', '8', '8', '8',
'8', '8', '8', '8', '9', '9', '9', '9', '9', '9', '9', '9', '9',
'9',};
final static byte[] DigitOnes = {'0', '1', '2', '3', '4', '5', '6', '7',
'8', '9', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '0',
'1', '2', '3', '4', '5', '6', '7', '8', '9', '0', '1', '2', '3',
'4', '5', '6', '7', '8', '9', '0', '1', '2', '3', '4', '5', '6',
'7', '8', '9', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9',
'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '0', '1', '2',
'3', '4', '5', '6', '7', '8', '9', '0', '1', '2', '3', '4', '5',
'6', '7', '8', '9', '0', '1', '2', '3', '4', '5', '6', '7', '8',
'9',};
final static int[] sizeTable = {9, 99, 999, 9999, 99999, 999999, 9999999,
99999999, 999999999, Integer.MAX_VALUE};
// Requires positive x
public static final int stringSize(int x) {
for (int i = 0; ; i++)
if (x <= sizeTable[i])
return i + 1;
}
// Requires positive x
public static final int stringSize(long x) {
long p = 10;
for (int i = 1; i < 19; i++) {
if (x < p)
return i;
p = 10 * p;
}
return 19;
}
final static int[] byte_len_array = new int[256];
static {
for (int i = Byte.MIN_VALUE; i <= Byte.MAX_VALUE; ++i) {
int size = (i < 0) ? stringSize(-i) + 1 : stringSize(i);
byte_len_array[i & 0xFF] = size;
}
}
}
HashAlgorithm.java
package com.meixianfeng.hash;
import java.io.UnsupportedEncodingException;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.zip.CRC32;
public enum HashAlgorithm {
/**
* Native hash (String.hashCode()).
*/
NATIVE_HASH,
/**
* CRC32_HASH as used by the perl API. This will be more consistent both
* across multiple API users as well as java versions, but is mostly likely
* significantly slower.
*/
CRC32_HASH,
/**
* FNV hashes are designed to be fast while maintaining a low collision
* rate. The FNV speed allows one to quickly hash lots of data while
* maintaining a reasonable collision rate.
*
* // * @see http://www.isthe.com/chongo/tech/comp/fnv/
* // * @see http://en.wikipedia.org/wiki/Fowler_Noll_Vo_hash
*/
FNV1_64_HASH,
/**
* Variation of FNV.
*/
FNV1A_64_HASH,
/**
* 32-bit FNV1.
*/
FNV1_32_HASH,
/**
* 32-bit FNV1a.
*/
FNV1A_32_HASH,
/**
* MD5-based hash algorithm used by ketama.
*/
KETAMA_HASH,
/**
* From mysql source
*/
MYSQL_HASH,
ELF_HASH,
RS_HASH,
/**
* From lua source,it is used for long key
*/
LUA_HASH,
/**
* MurMurHash算法，是非加密HASH算法，性能很高，
* 比传统的CRC32,MD5，SHA-1（这两个算法都是加密HASH算法，复杂度本身就很高，带来的性能上的损害也不可避免）
* 等HASH算法要快很多，这个算法的碰撞率很低.
* http://murmurhash.googlepages.com/
*/
MurMurHash,
/**
* The Jenkins One-at-a-time hash ,please see
* http://www.burtleburtle.net/bob/hash/doobs.html
*/
ONE_AT_A_TIME;
private static final long FNV_64_INIT = 0xcbf29ce484222325L;
private static final long FNV_64_PRIME = 0x100000001b3L;
private static final long FNV_32_INIT = 2166136261L;
private static final long FNV_32_PRIME = 16777619;
/**
* Compute the hash for the given key.
*
* @return a positive integer hash
*/
public long hash(final String k) {
long rv = 0;
switch (this) {
case NATIVE_HASH:
rv = k.hashCode();
break;
case CRC32_HASH:
// return (crc32(shift) >> 16) & 0x7fff;
CRC32 crc32 = new CRC32();
crc32.update(ByteUtils.getBytes(k));
rv = crc32.getValue() >> 16 & 0x7fff;
break;
case FNV1_64_HASH: {
// Thanks to [email protected] for the pointer
rv = FNV_64_INIT;
int len = k.length();
for (int i = 0; i < len; i++) {
rv *= FNV_64_PRIME;
rv ^= k.charAt(i);
}
}
break;
case MurMurHash:
ByteBuffer buf = ByteBuffer.wrap(k.getBytes());
int seed = 0x1234ABCD;
ByteOrder byteOrder = buf.order();
buf.order(ByteOrder.LITTLE_ENDIAN);
long m = 0xc6a4a7935bd1e995L;
int r = 47;
rv = seed ^ (buf.remaining() * m);
long ky;
while (buf.remaining() >= 8) {
ky = buf.getLong();
ky *= m;
ky ^= ky >>> r;
ky *= m;
rv ^= ky;
rv *= m;
}
if (buf.remaining() > 0) {
ByteBuffer finish = ByteBuffer.allocate(8).order(
ByteOrder.LITTLE_ENDIAN);
// for big-endian version, do this first:
// finish.position(8-buf.remaining());
finish.put(buf).rewind();
rv ^= finish.getLong();
rv *= m;
}
rv ^= rv >>> r;
rv *= m;
rv ^= rv >>> r;
buf.order(byteOrder);
break;
case FNV1A_64_HASH: {
rv = FNV_64_INIT;
int len = k.length();
for (int i = 0; i < len; i++) {
rv ^= k.charAt(i);
rv *= FNV_64_PRIME;
}
}
break;
case FNV1_32_HASH: {
rv = FNV_32_INIT;
int len = k.length();
for (int i = 0; i < len; i++) {
rv *= FNV_32_PRIME;
rv ^= k.charAt(i);
}
}
break;
case FNV1A_32_HASH: {
rv = FNV_32_INIT;
int len = k.length();
for (int i = 0; i < len; i++) {
rv ^= k.charAt(i);
rv *= FNV_32_PRIME;
}
}
break;
case KETAMA_HASH:
byte[] bKey = computeMd5(k);
rv = (long) (bKey[3] & 0xFF) << 24 | (long) (bKey[2] & 0xFF) << 16
| (long) (bKey[1] & 0xFF) << 8 | bKey[0] & 0xFF;
break;
case MYSQL_HASH:
int nr2 = 4;
for (int i = 0; i < k.length(); i++) {
rv ^= ((rv & 63) + nr2) * k.charAt(i) + (rv << 8);
nr2 += 3;
}
break;
case ELF_HASH:
long x = 0;
for (int i = 0; i < k.length(); i++) {
rv = (rv << 4) + k.charAt(i);
if ((x = rv & 0xF0000000L) != 0) {
rv ^= x >> 24;
rv &= ~x;
}
}
rv = rv & 0x7FFFFFFF;
break;
case RS_HASH:
long b = 378551;
long a = 63689;
for (int i = 0; i < k.length(); i++) {
rv = rv * a + k.charAt(i);
a *= b;
}
rv = rv & 0x7FFFFFFF;
break;
case LUA_HASH:
int step = (k.length() >> 5) + 1;
rv = k.length();
for (int len = k.length(); len >= step; len -= step) {
rv = rv ^ (rv << 5) + (rv >> 2) + k.charAt(len - 1);
}
case ONE_AT_A_TIME:
try {
int hash = 0;
for (byte bt : k.getBytes("utf-8")) {
hash += (bt & 0xFF);
hash += (hash << 10);
hash ^= (hash >>> 6);
}
hash += (hash << 3);
hash ^= (hash >>> 11);
hash += (hash << 15);
return hash;
} catch (UnsupportedEncodingException e) {
throw new IllegalStateException("Hash function error", e);
}
default:
assert false;
}
return rv & 0xffffffffL; /* Truncate to 32-bits */
}
/**
* Get the md5 of the given key.
*/
public static byte[] computeMd5(String k) {
MessageDigest md5;
try {
md5 = MessageDigest.getInstance("MD5");
} catch (NoSuchAlgorithmException e) {
throw new RuntimeException("MD5 not supported", e);
}
md5.reset();
md5.update(ByteUtils.getBytes(k));
return md5.digest();
}
// public static void main(String[] args) {
// HashAlgorithm alg=HashAlgorithm.LUA_HASH;
// long h=0;
// long start=System.currentTimeMillis();
// for(int i=0;i<100000;i++) {
// h=alg.hash("dddddd");
//// System.out.println(h);
// }
// System.out.println(System.currentTimeMillis()-start);
// }
}
ConsistantHash.java
package com.meixianfeng.hash;
import java.util.Set;
import java.util.TreeMap;
/**
* Created with IntelliJ IDEA.
* User: yfwangqing
* Date: 13-8-23
* Time: 上午9:54
* 控制中心
*/
public class ConsistantHash {
private int virtualNum = 100; //平均虚拟节点数
private HashAlgorithm alg = HashAlgorithm.KETAMA_HASH;//采用的HASH算法
private Set nodeSet; //节点列表
private final TreeMap();
private static class SingletonHolder {
private static ConsistantHash instance = new ConsistantHash();
}
private ConsistantHash() {
}
public static ConsistantHash getInstance() {
return SingletonHolder.instance;
}
/**
* 构建一致性HASH环
*/
public void buildHashCycle() {
if (nodeSet == null) return;
for (Node node : nodeSet) {
for (int i = 0; i < virtualNum; i++) {
long nodeKey = this.alg.hash(node.toString() + "-" + i);
nodeMap.put(nodeKey, node);
}
}
}
/**
* 沿环的顺时针找到虚拟节点
*
* @param key
* @return
*/
public Node findNodeByKey(String key) {
final Long hash = this.alg.hash(key);
Long target = hash;
if (!nodeMap.containsKey(hash)) {
target = nodeMap.ceilingKey(hash);
if (target == null && !nodeMap.isEmpty()) {
target = nodeMap.firstKey();
}
}
return nodeMap.get(target);
}
/**
* 设置每个节点的虚拟节点个数，该参数默认是100
*
* @param virtualNum 虚拟节点数
*/
public void setVirtualNum(int virtualNum) {
this.virtualNum = virtualNum;
}
/**
* 设置一致性HASH的算法，默认采用 KETAMA_HASH
* 对于一致性HASH而言选择的HASH算法首先要考虑发散度其次再考虑性能
*
* @param alg 具体支持的算法
* @see HashAlgorithm
*/
public void setAlg(HashAlgorithm alg) {
this.alg = alg;
}
/**
* 配置实际的节点，允许同一个IP上多个节点，但是应该用name区分开
*
* @param nodeList 节点列表
*/
public void setNodeList(Set nodeList) {
this.nodeSet = nodeList;
}
/**
* 获取环形HASH
*
* @return
*/
public TreeMap getNodeMap() {
return nodeMap;
}
}
参考资料
https://www.cnblogs.com/xrq730/p/5186728.html