MemcacheDB, Tokyo Tyrant, Redis performance test

I had tested the following key-value store for set() and get()

• MemcacheDB, use memcached client protocol.
• Tokyo Tyrant (Tokyo Cabinet), use memcached client protocol
• Redis, use JRedis Java client

1. Test environment

1.1 Hardware/OS

2 Linux boxes in a LAN, 1 server and 1 test client
Linux Centos 5.2 64bit
Intel(R) Xeon(R) CPU E5410  @ 2.33GHz (L2 cache: 6M), Quad-Core * 2
8G memory
SCSI disk (standalone disk, no other access)

1.2 Software version

db-4.7.25.tar.gz
libevent-1.4.11-stable.tar.gz
memcached-1.2.8.tar.gz
memcachedb-1.2.1-beta.tar.gz
redis-0.900_2.tar.gz
tokyocabinet-1.4.9.tar.gz
tokyotyrant-1.1.9.tar.gz

1.3 Configuration

Memcachedb startup parameter
Test 100 bytes
./memcachedb -H /data5/kvtest/bdb/data -d -p 11212 -m 2048 -N -L 8192
(Update: As mentioned by Steve, the 100-byte-test missed the -N paramter, so I added it and updated the data)
Test 20k bytes
./memcachedb -H /data5/kvtest/mcdb/data -d -p 11212 -b 21000 -N -m 2048

Tokyo Tyrant (Tokyo Cabinet) configuration
Use default Tokyo Tyrant sbin/ttservctl
use .tch database, hashtable database

ulimsiz=”256m”
sid=1
dbname=”$basedir/casket.tch#bnum=50000000″ # default 1M is not enough!
maxcon=”65536″
retval=0

Redis configuration
timeout 300
save 900 1
save 300 10
save 60 10000
# no maxmemory settings

1.4 Test client

Client in Java, JDK1.6.0, 16 threads
Use Memcached client java_memcached-release_2.0.1.jar
JRedis client for Redis test, another JDBC-Redis has poor performance.

2. Small data size test result

Test 1, 1-5,000,000 as key, 100 bytes string value, do set, then get test, all get test has result.
Request per second(mean)

Store	Write	Read
Memcached	55,989	50,974
Memcachedb	25,583	35,260
Tokyo Tyrant	42,988	46,238
Redis	85,765	71,708

Server Load Average

Store	Write	Read
Memcached	1.80, 1.53, 0.87	1.17, 1.16, 0.83
MemcacheDB	1.44, 0.93, 0.64	4.35, 1.94, 1.05
Tokyo Tyrant	3.70, 1.71, 1.14	2.98, 1.81, 1.26
Redis	1.06, 0.32, 0.18	1.56, 1.00, 0.54

3. Larger data size test result

Test 2, 1-500,000 as key, 20k bytes string value, do set, then get test, all get test has result.
Request per second(mean)
(Aug 13 Update: fixed a bug on get() that read non-exist key)
 

Store	Write	Read
Memcachedb	357	327
Tokyo Tyrant	3,501	257
Redis	1,542	957

4. Some notes about the test

When test Redis server, the memory goes up steadily, consumed all 8G and then use swap(and write speed slow down), after all memory and swap space is used, the client will get exceptions. So use Redis in a productive environment should limit to a small data size. It is another cache solution rather than a persistent storage. So compare Redis together with MemcacheDB/TC may not fair because Redis actually does not save data to disk during the test.

Tokyo cabinet and memcachedb are very stable during heavy load, use very little memory in set test and less than physical memory in get test.

MemcacheDB peformance is poor for write large data size(20k).

The call response time was not monitored in this test.