利用ycsb测试redis性能

1、redis的安装

在 http://redis.io/download上面下载了redis-2.8.3.tar.gz存放在/home/hs/program目录下

cd /home/hs/program
tar -zxvf redis-2.8.3.tar.gz
cd ./redis-2.8.3/
make 32bit
make test   //（用于测试安装是否成功）

结果显示:没有安装tcl或者需要比8.5新的版本的tcl（tool command language）。

在 http://www.linuxfromscratch.org/blfs/view/svn/general/tcl.html上面下载了tcl8.6.1-src.tar.gz存放在/home/hs/program目录下
tar -zxvf tcl8.6.1-src.tar.gz
cd ./tcl8.6.1/unix/
执行如下命令（参考上面的网址）：

./configure --prefix=/usr --without-tzdata --mandir=/usr/share/man
make
sed -e "s@^\(TCL_SRC_DIR='\).*@\1/usr/include'@"     -e "/TCL_B/s@='\(-L\)\?.*unix@='\1/usr/lib@" \
sed -e "s@^\(TCL_SRC_DIR='\).*@\1/usr/include'@" -e "/TCL_B/s@='\(-L\)\?.*unix@='\1/usr/lib@" -i tclConfig.sh
make test
sudo make install
sudo make install-private-headers
sudo ln -v -sf tclsh8.6 /usr/bin/tclsh
sudo chmod -v 755 /usr/lib/libtcl8.6.so

然后重新返回到/home/hs/program/redis-2.8.3

make distclean   //（清除上次编译信息）
make 32bit
make test

显示安装成功之后，就可以使用redis-server/redis-cli进行一些简单的测试。


2、jdk的安装

在 http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html上面
下载了jdk-7u45-linux-i586.tar.gz存放在/home/hs/program目录下

cd /home/hs/program
tar -zxvf jdk-7u45-linux-i586.tar.gz

得到jdk1.7.0_45


然后设置java的环境变量
sudo gedit /etc/profile

在最后加上了如下代码：

#set java environment
export JAVA_HOME=/home/hs/program/jdk1.7.0_45
export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$CLASSPATH
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH

然后. /etc/profile（使对/etc/profile里面的更改生效）


接着执行如下代码：java -version
能够得到如下结果：

java version "1.7.0_45"
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) Client VM (build 24.45-b08, mixed mode)

3、maven的安装

在 http://maven.apache.org/download.cgi上面下载了
apache-maven-3.1.1-bin.tar.gz也存放在/home/hs/program目录下

cd /home/hs/program
tar -zxvf apache-maven-3.1.1-bin.tar.gz -C /usr/local/

然后ls /usr/local得到如下结果：
apache-maven-3.1.1  bin  etc  games  include  lib  man  sbin  share  src

然后设置maven的环境变量
sudo gedit /etc/profile

在最后加上如下代码：

#set maven environment
export M2_HOME=/usr/local/apache-maven-3.1.1
export MAVEN_OPTS="-Xms256m -Xmx512m"
export PATH=$M2_HOME/bin:$PATH

执行 mvn -version也可以得到如下结果：

Apache Maven 3.1.1 (0728685237757ffbf44136acec0402957f723d9a; 2013-09-17 23:22:22+0800)
Maven home: /usr/local/apache-maven-3.1.1
Java version: 1.7.0_45, vendor: Oracle Corporation
Java home: /home/hs/program/jdk1.7.0_45/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "linux", version: "3.0.0-32-generic", arch: "i386", family: "unix"

4、YCSB的安装

在/home/hs/program目录下执行如下命令：

git clone git://github.com/brianfrankcooper/YCSB.git
cd YCSB

然后mvn clean package > build-result.dat（为了方便查看信息，我将输出重定向了）
接着去查看build-result.dat里面的内容就发现了好多错误，几个典型错误如下：
a、Downloading: http://snapshots.repository.codehaus.org/org/mortbay/jetty/jetty-util/6.1.5/jetty-util-6.1.5.jar
[INFO] Unable to find resource 'org.mortbay.jetty:jetty-util:jar:6.1.5' in repository codehaus.org (http://snapshots.repository.codehaus.org)

b、[INFO] Starting audit...
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:0: Missing package-info.java file.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:1: Line is longer than 80 characters.

c、/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:14: Line is longer than 80 characters.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:15: Line is longer than 80 characters.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:23: Missing a Javadoc comment.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:24: member def type at indentation level 8 not at correct indentation, 2
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:24:1: File contains tab characters (this is the first instance).
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:24:16: Variable 'str' must be private and have accessor methods.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:25: member def type at indentation level 8 not at correct indentation, 2
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:25:13: Variable 'off' must be private and have accessor methods.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:31: Line is longer than 80 characters.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:31: method def modifier at indentation level 8 not at correct indentation, 2
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:32: Line is longer than 80 characters.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:32: for at indentation level 15 not at correct indentation, 4
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:33: method def rcurly at indentation level 8 not at correct indentation, 2
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java:39: Line is longer than 80 characters.

d、/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/Client.java:75:25: '{' should be on the previous line.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/Client.java:76: if at indentation level 32 not at correct indentation, 8
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/Client.java:77: if lcurly at indentation level 32 not at correct indentation, 8
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/Client.java:77:33: '{' should be on the previous line.
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/Client.java:78: if child at indentation level 40 not at correct indentation, 10
/home/hs/program/YCSB/core/src/main/java/com/yahoo/ycsb/Client.java:79: if rcurly at indentation level 32 not at correct indentation, 8

e、[INFO] There are 444 checkstyle errors.
[WARNING] Unable to locate Source XRef to link to - DISABLED
[INFO] [resources:resources {execution: default-resources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/hs/program/YCSB/cassandra/src/main/resources

f、[INFO] [INFO] ------------------------------------------------------------------------
[ERROR] FATAL ERROR
[INFO] ------------------------------------------------------------------------
[INFO] null
[INFO] ------------------------------------------------------------------------
[INFO] Trace
java.lang.StackOverflowError
at sun.nio.cs.UTF_8.updatePositions(UTF_8.java:77)
at sun.nio.cs.UTF_8$Encoder.encodeArrayLoop(UTF_8.java:564)
at sun.nio.cs.UTF_8$Encoder.encodeLoop(UTF_8.java:619)
at java.nio.charset.CharsetEncoder.encode(CharsetEncoder.java:561)
at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:271)
at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
at java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
at java.io.PrintStream.write(PrintStream.java:526)
at java.io.PrintStream.print(PrintStream.java:669)
at org.codehaus.plexus.logging.console.ConsoleLogger.info(ConsoleLogger.java:40)
at org.codehaus.plexus.logging.AbstractLogger.info(AbstractLogger.java:56)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:478)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:481)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:481)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:481)
at org.codehaus.plexus.archiver.AbstractArchiver$1.hasNext(AbstractArchiver.java:481)

里面有好多的语法错误，这一点我很纳闷，git得到的源文件，我并没有做任何的修改

在github/ycsb上提问得到了别人的回答如下：
需要将YCSB的top目录下的pom.xml文件中的infinispan注释掉即可。


然后重新mvn clean package,得到如下结果：

[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] YCSB Root ............................................. SUCCESS [8.624s]
[INFO] Core YCSB ............................................. SUCCESS [21.317s]
[INFO] Cassandra DB Binding .................................. SUCCESS [13.588s]
[INFO] HBase DB Binding ...................................... SUCCESS [35.079s]
[INFO] Hypertable DB Binding ................................. SUCCESS [5.871s]
[INFO] DynamoDB DB Binding ................................... SUCCESS [7.593s]
[INFO] ElasticSearch Binding ................................. SUCCESS [18.131s]
[INFO] JDBC DB Binding ....................................... SUCCESS [36.329s]
[INFO] Mapkeeper DB Binding .................................. SUCCESS [11.661s]
[INFO] Mongo DB Binding ...................................... SUCCESS [8.293s]
[INFO] OrientDB Binding ...................................... SUCCESS [32.434s]
[INFO] Redis DB Binding ...................................... SUCCESS [7.217s]
[INFO] Voldemort DB Binding .................................. SUCCESS [15.579s]
[INFO] YCSB Release Distribution Builder ..................... SUCCESS [13.252s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3 minutes 56 seconds
[INFO] Finished at: Sat Dec 14 12:07:06 CST 2013
[INFO] Final Memory: 64M/205M

注：上面的一些语法还是有一些，比如 Line is longer than 80 characters.等等。但是文件最后面的BUILD SUCCESSFUL确实不争的事实。

但是在重新执行./bin/ycsb shell redis，又出现了新问题：

Exception in thread "main" java.lang.IllegalArgumentException: hostname can't be null
	at java.net.InetSocketAddress.checkHost(InetSocketAddress.java:149)
	at java.net.InetSocketAddress.(InetSocketAddress.java:216)
	at redis.clients.jedis.Connection.connect(Connection.java:119)
	at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:54)
	at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1657)
	at com.yahoo.ycsb.db.RedisClient.init(RedisClient.java:49)
	at com.yahoo.ycsb.CommandLine.main(CommandLine.java:210)

在ycsb的邮件列表中找到解决方法如下：
如果你是在同一台机器上跑ycsb和redis，并且都是在默认的配置下，那么你在ycsb上测redis时 需要加上参数：
-p redis.host=localhost -p redis.port=6379
重新执行./bin/ycsb shell redis -p redis.host=localhost -p redis.port=6379
得到结果如下：
Connected.
>

5、在确认与redis连接成功之后，就可以进行测试

1、load

./bin/ycsb load redis -P workloads/workloada -p redis.host=localhost -p redis.port=6379 > ./my-results/redis-load-a

结果如下：

YCSB Client 0.1
Command line: -db com.yahoo.ycsb.db.RedisClient -P workloads/workloada -p redis.host=localhost -p redis.port=6379 -load
[OVERALL], RunTime(ms), 752.0
[OVERALL], Throughput(ops/sec), 1329.787234042553
[INSERT], Operations, 1000
[INSERT], AverageLatency(us), 461.042
[INSERT], MinLatency(us), 216
[INSERT], MaxLatency(us), 59507
[INSERT], 95thPercentileLatency(ms), 0
[INSERT], 99thPercentileLatency(ms), 2
[INSERT], Return=0, 1000

2、run

./bin/ycsb run redis -P workloads/workloada -p redis.host=localhost -p redis.port=6379 > ./my-results/redis-run-a

结果如下：

YCSB Client 0.1
Command line: -db com.yahoo.ycsb.db.RedisClient -P workloads/workloada -p redis.host=localhost -p redis.port=6379 -t
[OVERALL], RunTime(ms), 291.0
[OVERALL], Throughput(ops/sec), 3436.426116838488
[UPDATE], Operations, 513
[UPDATE], AverageLatency(us), 179.85185185185185
[UPDATE], MinLatency(us), 99
[UPDATE], MaxLatency(us), 8500
[UPDATE], 95thPercentileLatency(ms), 0
[UPDATE], 99thPercentileLatency(ms), 0
[UPDATE], Return=0, 513
......
......
......


[READ], Operations, 487
[READ], AverageLatency(us), 252.36139630390144
[READ], MinLatency(us), 101
[READ], MaxLatency(us), 27355
[READ], 95thPercentileLatency(ms), 0
[READ], 99thPercentileLatency(ms), 0
[READ], Return=0, 487