Riak - 使用篇（1）

分布式高可用键值对数据库Riak - 使用篇（1）

请先参考Riak - 安装运维篇（1）安装部署并启动Riak集群（3个Node）。
Riak默认有两种端口，一种是protobuf端口，还有一种是HTTP Restful端口。
以前的Riak client java API会支持两种端口。最新的Riak client Java API作了很多改造，比如说利用netty4作为网络通信框架，简化了API代码，并且只支持Protobuf端口。理由如下：

1. 利用Protocol Buffers端口会快25%左右
2. HTTP接口不支持基于证书的认证
3. HTTP协议抽象不够详细

我们之后主要使用riak client2.0.5和Riak的protobuf端口进行开发使用Riak客户端，在某些情况下，会穿插一些Restful端口使用。因为某些功能在riak client2.0.5还未实现或者实现的不完整：

1. MapReduce
2. 二级索引查询
3. 列出所有keys
4. 列出所有buckets

1. 建立与Riak集群的连接

建立maven项目，添加如下依赖：

<properties>
    <!-- NIO framework -->
    <netty_version>5.0.0.Alpha2</netty_version>
    <!-- riak -->
    <riak-client_version>2.0.5</riak-client_version>
    <!-- Encoding -->
    <!-- fast json -->
    <jackson-core_version>2.7.3</jackson-core_version>
    <!-- log -->
    <slf4j_version>1.7.2</slf4j_version>
</properties>
<dependencies>
   <dependency>
       <groupId>com.basho.riak</groupId>
       <artifactId>riak-client</artifactId>
       <version>${riak-client_version}</version>
   </dependency>
   <dependency>
       <groupId>io.netty</groupId>
       <artifactId>netty-all</artifactId>
       <version>${netty_version}</version>
   </dependency>
   <dependency>
       <groupId>org.slf4j</groupId>
       <artifactId>slf4j-log4j12</artifactId>
       <version>${slf4j_version}</version>
   </dependency>
   <dependency>
       <groupId>com.fasterxml.jackson.core</groupId>
       <artifactId>jackson-core</artifactId>
       <version>${jackson-core_version}</version>
   </dependency>
</dependencies>

之后新建ClusterRiakClient类：

package io.timberwolf.cache;

import com.basho.riak.client.api.RiakClient;
import com.basho.riak.client.api.cap.Quorum;
import com.basho.riak.client.api.commands.kv.FetchValue;
import com.basho.riak.client.api.commands.kv.StoreValue;
import com.basho.riak.client.core.RiakCluster;
import com.basho.riak.client.core.RiakNode;
import com.basho.riak.client.core.query.Location;
import com.basho.riak.client.core.query.Namespace;

import java.net.UnknownHostException;
import java.util.LinkedList;
import java.util.concurrent.ExecutionException;

/** * The client class of Riak * * @author Hash Zhang * @version 0.0.0 * @see @https://github.com/basho/riak-java-client/wiki/RiakClient-%26-Cluster-Node-Builders-%28v2.0%29 */
//implements AutoCloseable是为了利用Java7的新特性，try-with-resources
public class ClusterRiakClient implements AutoCloseable{
    private final static int DEFAULTMAXCONNECTIONS = 50;
    private final static int DEFAULTMINCONNECTIONS = 10;
    private final static int RETRIES = 5;

    private int maxConnections = DEFAULTMAXCONNECTIONS;
    private int minConnections = DEFAULTMINCONNECTIONS;
    private int retries = DEFAULTMINCONNECTIONS;

    private RiakClient riakClient;

    /** * 构建Riak集群的客户端 * * @param hosts (hosts格式：127.0.0.1:10017,127.0.0.1:10027,127.0.0.1:10037) * @throws UnknownHostException */
    public ClusterRiakClient(String hosts) throws UnknownHostException {
        String[] addresses = hosts.split(",");
        //使用RiakNode Builder用于构建每个RiakNode
        RiakNode.Builder riakNodeBuilder = new RiakNode.Builder()
                .withMinConnections(minConnections)
                .withMaxConnections(maxConnections);
        //构建每个RiakNode并保存在list中
        LinkedList<RiakNode> nodes = new LinkedList<RiakNode>();
        for (int i = 0; i < addresses.length; i++) {
            int j = addresses[i].indexOf(":");
            nodes.add(riakNodeBuilder.withRemoteAddress(addresses[i].substring(0, j))
                    .withRemotePort(Integer.parseInt(addresses[i].substring(j + 1))).build());
        }
        //构建Riak集群并启动，注意，必须调用start()
        RiakCluster riakCliuster = RiakCluster.builder(nodes).withExecutionAttempts(retries).build();
        riakCliuster.start();
        riakClient = new RiakClient(riakCliuster);
    }
    public void close() throws Exception {
        riakClient.shutdown();
    }
}

将控制台（Console）日志级别设为debug，编写测试main：

public static void main(String[] args) throws Exception {
    ClusterRiakClient clusterRiakClient = new ClusterRiakClient("10.202.44.206:10017,10.202.44.206:10027,10.202.44.206:10037");
    clusterRiakClient.close();
}

运行，查看控制台输出

04-21 08:38:40.061   INFO [main] (RiakNode.java:282) -RiakNode started; 10.202.44.206:10017
04-21 08:38:40.147   INFO [main] (RiakNode.java:282) -RiakNode started; 10.202.44.206:10027
04-21 08:38:40.173   INFO [main] (RiakNode.java:282) -RiakNode started; 10.202.44.206:10037
04-21 08:38:40.174   INFO [main] (RiakCluster.java:142) -RiakCluster is starting.
04-21 08:38:40.175   INFO [main] (RiakCluster.java:149) -RiakCluster is shutting down.
04-21 08:38:40.677   INFO [pool-1-thread-2] (RiakCluster.java:428) -All operations have completed
04-21 08:38:40.677   INFO [pool-1-thread-2] (RiakNode.java:291) -RiakNode shutting down; 10.202.44.206:10017
04-21 08:38:40.677   INFO [pool-1-thread-1] (RiakCluster.java:416) -Retrier shutting down.
04-21 08:38:40.677   INFO [pool-1-thread-2] (DefaultNodeManager.java:159) -NodeManager removed node due to it shutting down; 10.202.44.206:10017
04-21 08:38:40.683   INFO [pool-1-thread-2] (RiakNode.java:291) -RiakNode shutting down; 10.202.44.206:10027
04-21 08:38:40.683   INFO [pool-1-thread-2] (DefaultNodeManager.java:159) -NodeManager removed node due to it shutting down; 10.202.44.206:10027
04-21 08:38:40.685   INFO [pool-1-thread-2] (RiakNode.java:291) -RiakNode shutting down; 10.202.44.206:10037
04-21 08:38:40.685   INFO [pool-1-thread-2] (DefaultNodeManager.java:159) -NodeManager removed node due to it shutting down; 10.202.44.206:10037
04-21 08:38:40.688   INFO [pool-1-thread-1] (RiakCluster.java:305) -RiakCluster has shut down

可以看出，Riak集群连接成功，并成功关闭连接。
如果日志级别为Debug，你可以看出，Riak客户端使用了Netty客户端连接的Riak集群
这里，Riakclient和RiakCluster还有实际的Riak集群之间的关系如下图所示：

2. 将值放在桶中

Riak是一种键值的存储方式，所以必须提供键值对才能保存数据。为了防止键冲突，Riak运用了桶的概念，用户可以将键放入不同的桶中（相当于键的namespace）

首先，我们建立一个简单的POJO类，保存快递员的基本信息。

/** * User POJO * * @author Hash Zhang * @version 0.0.0 */
public class User {
    private String username;
    private String password;

    public String getUsername() {
        return username;
    }

    public void setUsername(String username) {
        this.username = username;
    }

    public String getPassword() {
        return password;
    }

    public void setPassword(String password) {
        this.password = password;
    }
}

在ClusterRiakClient中添加如下方法：

/** * * 插入POJO对象 * * @param bucket 桶 * @param key 键 * @param value 值 * @throws ExecutionException * @throws InterruptedException * @throws JsonProcessingException */
public void set(String bucket, String key, Object value) throws ExecutionException, InterruptedException, JsonProcessingException {
    Location location = new Location(new Namespace(bucket), key);
    RiakObject riakObject = new RiakObject();
    riakObject.setValue(BinaryValue.create(OBJECT_MAPPER.writeValueAsBytes(value)));
    StoreValue sv = new StoreValue.Builder(riakObject).withLocation(location).build();
    StoreValue.Response svResponse = this.riakClient.execute(sv);
}

/** * 取得POJO对象 * * @param bucket 桶 * @param key 键 * @return FetchValue.Response * @throws ExecutionException * @throws InterruptedException */
public RiakObject get(String bucket, String key) throws ExecutionException, InterruptedException {
    Location location = new Location(new Namespace(bucket), key);
    FetchValue fv = new FetchValue.Builder(location).build();
    return this.riakClient.execute(fv).getValue(RiakObject.class);
}

Location通过Bucket和Key确定唯一位置。编写测试main

public static void main(String[] args) throws Exception {
    ClusterRiakClient clusterRiakClient = new ClusterRiakClient("10.202.44.206:10017,10.202.44.206:10027,10.202.44.206:10037");
    User user1 = new User();
    user1.setUsername("zhxhash").setPassword("123456");
    clusterRiakClient.set("users", "user1", user1);
    RiakObject riakObject = clusterRiakClient.get("users", "user1");
    User getUser1 = OBJECT_MAPPER.readValue(riakObject.getValue().getValue(),User.class);
    System.out.println(getUser1.getUsername()+"|"+getUser1.getPassword());
    clusterRiakClient.close();
}

结果：

04-21 09:31:03.213   INFO [main] (RiakNode.java:282) -RiakNode started; 10.202.44.206:10017
04-21 09:31:03.240   INFO [main] (RiakNode.java:282) -RiakNode started; 10.202.44.206:10027
04-21 09:31:03.264   INFO [main] (RiakNode.java:282) -RiakNode started; 10.202.44.206:10037
04-21 09:31:03.266   INFO [main] (RiakCluster.java:142) -RiakCluster is starting.
zhxhash|123456
04-21 09:31:04.048   INFO [main] (RiakCluster.java:149) -RiakCluster is shutting down.
04-21 09:31:04.549   INFO [pool-1-thread-2] (RiakCluster.java:428) -All operations have completed
04-21 09:31:04.549   INFO [pool-1-thread-2] (RiakNode.java:291) -RiakNode shutting down; 10.202.44.206:10017
04-21 09:31:04.549   INFO [pool-1-thread-2] (DefaultNodeManager.java:159) -NodeManager removed node due to it shutting down; 10.202.44.206:10017
04-21 09:31:04.555   INFO [pool-1-thread-2] (RiakNode.java:291) -RiakNode shutting down; 10.202.44.206:10027
04-21 09:31:04.556   INFO [pool-1-thread-2] (DefaultNodeManager.java:159) -NodeManager removed node due to it shutting down; 10.202.44.206:10027
04-21 09:31:04.558   INFO [pool-1-thread-2] (RiakNode.java:291) -RiakNode shutting down; 10.202.44.206:10037
04-21 09:31:04.558   INFO [pool-1-thread-2] (DefaultNodeManager.java:159) -NodeManager removed node due to it shutting down; 10.202.44.206:10037
04-21 09:31:04.562   INFO [pool-1-thread-2] (RiakCluster.java:305) -RiakCluster has shut down
04-21 09:31:04.564   INFO [pool-1-thread-1] (RiakCluster.java:416) -Retrier shutting down.

以上是比较简单的增加和查询的实现，还有异步实现的方式。

/** * 异步插入POJO对象 * * @param bucket 桶 * @param key 键 * @param value 值 * @return RiakFuture<StoreValue.Response, Location> * @throws JsonProcessingException */
public RiakFuture<StoreValue.Response, Location> asyncSet(String bucket, String key, Object value) throws JsonProcessingException {
    Location location = new Location(new Namespace(bucket), key);
    RiakObject riakObject = new RiakObject();
    riakObject.setValue(BinaryValue.create(OBJECT_MAPPER.writeValueAsBytes(value)));
    StoreValue sv = new StoreValue.Builder(riakObject).withLocation(location).build();
    return this.riakClient.executeAsync(sv);
}
/** * 异步取得POJO对象 * * @param bucket * @param key * @return RiakFuture<FetchValue.Response, Location> */
public RiakFuture<FetchValue.Response, Location> asyncGet(String bucket, String key) {
    Location location = new Location(new Namespace(bucket), key);
    FetchValue fv = new FetchValue.Builder(location).build();
    return this.riakClient.executeAsync(fv);
}

测试代码：

public static void main(String[] args) throws Exception {
    final ClusterRiakClient clusterRiakClient = new ClusterRiakClient("10.202.44.206:10017,10.202.44.206:10027,10.202.44.206:10037");
    final CountDownLatch countDownLatch = new CountDownLatch(1);
    Thread thread = new Thread() {
        @Override
        public void run() {
            User user1 = new User();
            user1.setUsername("zhxhash").setPassword("123456");
            RiakFuture<StoreValue.Response, Location> riakFuture = null;
            while(true) {
                try {
                    riakFuture = clusterRiakClient.asyncSet("users", "user1", user1);
                    riakFuture.await(100L, TimeUnit.MILLISECONDS);
                } catch (JsonProcessingException | InterruptedException e) {
                    e.printStackTrace();
                }
                if (riakFuture.isDone() && riakFuture.isSuccess())
                {
                    countDownLatch.countDown();
                    break;
                }
            }
        }
    };
    thread.start();
    countDownLatch.await();
    RiakFuture<FetchValue.Response, Location> riakFuture = null;
    while(true) {
        try {
            riakFuture = clusterRiakClient.asyncGet("users", "user1");
            riakFuture.await(100L, TimeUnit.MILLISECONDS);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        if (riakFuture.isDone() && riakFuture.isSuccess())
        {
            RiakObject riakObject = riakFuture.get().getValue(RiakObject.class);
            User getUser1 = OBJECT_MAPPER.readValue(riakObject.getValue().getValue(),User.class);
            System.out.println(getUser1.getUsername()+"|"+getUser1.getPassword());
            break;
        }
    }
    clusterRiakClient.close();
}

输出结果和之前同步的应该一样。
我们还可以删除某个键值对，同样的，有同步和异步两种实现：

/** * 删除一个键值对 * * @param bucket 桶 * @param key 键 * @throws ExecutionException * @throws InterruptedException */
public void remove(String bucket, String key) throws ExecutionException, InterruptedException {
    Location location = new Location(new Namespace(bucket), key);
    DeleteValue dv = new DeleteValue.Builder(location).build();
    this.riakClient.execute(dv);
}

/** * 异步删除一个键值对 * * @param bucket 桶 * @param key 键 * @return RiakFuture<Void, Location> */
public RiakFuture<Void, Location> asyncRemove(String bucket, String key){
    Location location = new Location(new Namespace(bucket), key);
    DeleteValue dv = new DeleteValue.Builder(location).build();
    return this.riakClient.executeAsync(dv);
}

测试：

public static void main(String[] args) throws Exception {
        ClusterRiakClient clusterRiakClient = new ClusterRiakClient("10.202.44.206:10017,10.202.44.206:10027,10.202.44.206:10037");
        clusterRiakClient.remove("users", "user1");
        RiakObject riakObject = clusterRiakClient.get("users", "user1");
        if (riakObject!=null) {
            User getUser1 = OBJECT_MAPPER.readValue(riakObject.getValue().getValue(), User.class);
            System.out.println(getUser1.getUsername() + "|" + getUser1.getPassword());
        } else{
            System.out.println("Not Found");
        }
        clusterRiakClient.close();
}

更新比较特殊，我们放到之后的章节去说
取得一个bucket下的所有键：

/** * 取得一个bucket下所有键 * * @param bucket 桶 * @return List<String> * @throws ExecutionException * @throws InterruptedException */
public List<String> getKeys(String bucket) throws ExecutionException, InterruptedException {
    Namespace ns = new Namespace(bucket);
    ListKeys lk = new ListKeys.Builder(ns).build();
    ListKeys.Response response = this.riakClient.execute(lk);
    List<String> keys = new LinkedList<>();
    for (Location l : response)
    {
        keys.add(l.getKeyAsString());
    }
    return keys;
}

<script type="text/javascript"> $(function () { $('pre.prettyprint code').each(function () { var lines = $(this).text().split('\n').length; var $numbering = $('<ul/>').addClass('pre-numbering').hide(); $(this).addClass('has-numbering').parent().append($numbering); for (i = 1; i <= lines; i++) { $numbering.append($('<li/>').text(i)); }; $numbering.fadeIn(1700); }); }); </script>