casandra 环境搭建

cassandra集群环境搭建

 

老实说，虽然cassandra在业界不是很受待见，但是但从使用上来看，cassandra还是挺简单的，本次测试环境仍然是UBUNTU，集群中最简单的使用了2个节点。

首先安装java，这个可以参考我之前的博客：Ubuntu 11.04上安装jdk1.6+oracle 11g笔记 。一系列各种JAVA_HOME什么的配置完之后，应该可以看到这个

~$ java -version
java version "1.8.0"
Java(TM) SE Runtime Environment (build 1.8.0-b132)
Java HotSpot(TM) 64-Bit Server VM (build 25.0-b70, mixed mode)

现在我用的版本是java 8。然后需要从cassandra官网上下载安装包，这个不再多说，我的版本是apache-cassandra-2.0.6-bin.tar.gz，ubuntu中好像没有cassandra的源。

然后是建立数据文件和log的目录并改变其所属用户和所属用户组，

XXXXX@XXXXX-asus:~$ sudo mkdir /var/lib/cassandra
XXXXX@XXXXX-asus:~$ sudo chown XXXXX /var/lib/cassandra/
XXXXX@XXXXX-asus:~$ sudo chgrp XXXXX /var/lib/cassandra/
XXXXX@XXXXX-asus:~$ sudo mkdir /var/log/cassandra
XXXXX@XXXXX-asus:~$ sudo chown XXXXX /var/log/cassandra/
XXXXX@XXXXX-asus:~$ sudo chgrp XXXXX /var/log/cassandra/
XXXXX@XXXXX-asus:~$ ll /var/lib/ | grep cas
drwxr-xr-x  2 XXXXX         XXXXX         4096  4月 13 11:09 cassandra/
XXXXX@XXXXX-asus:~$ ll /var/log/ | grep cas
drwxr-xr-x  2 XXXXX   XXXXX      4096  4月 13 11:09 cassandra/
XXXXX@XXXXX-asus:~$ 

当然，cassandra的数据和log文件是不是非要死板的放在/var/lib和/var/log下面呢？当然不是，有地方可以配置的，这里，为了便于实验，仍然采用默认配置。

解压cassandra的安装包后可以查看主要的配置文件，都在conf/目录下，conf/cassandra.yaml比较重要，其中需要着重注意的有以下一些配置项

cluster_name: 'TC01'
num_tokens: 256
seed_provider:
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
      parameters:
          - seeds: "192.168.2.103"
listen_address: 192.168.2.103
rpc_address: 192.168.2.103
endpoint_snitch: SimpleSnitch

cluster_name一定要修改，免得用默认的“Test Cluster”和他人冲突，所有有显示ip的地方都需要注意并修改，将127.0.0.1和localhost改为你的对外访问的ip，比如我的局域网ip就是192.168.2.103。

conf/cassandra.yaml中还有改动数据文件位置的选项，搜索/var/lib相关的选项修改即可，conf/log4j-server.properties中有改动log文件位置的选项，搜索/var/log相关的选项。这次实验中都没有修改这些选项。

然后启动cassandra，

XXXXX@XXXXX-asus:~/apache-cassandra-2.0.6$ cd bin/
XXXXX@XXXXX-asus:~/apache-cassandra-2.0.6/bin$ ./cassandra -f

用-f参数启动，可以看到cassandra启动超长的log，接着看一下状态

XXXXX@XXXXX-asus:~/apache-cassandra-2.0.6$ cd bin/
XXXXX@XXXXX-asus:~/apache-cassandra-2.0.6/bin$ ls
cassandra          cassandra-shuffle      debug-cql.bat     sstable2json      sstableloader.bat    sstablesplit.bat
cassandra.bat      cassandra-shuffle.bat  json2sstable      sstable2json.bat  sstablemetadata.bat  sstableupgrade
cassandra-cli      cqlsh                  json2sstable.bat  sstablekeys       sstablescrub         sstableupgrade.bat
cassandra-cli.bat  cqlsh.bat              nodetool          sstablekeys.bat   sstablescrub.bat     stop-server
cassandra.in.sh    debug-cql              nodetool.bat      sstableloader     sstablesplit
XXXXX@XXXXX-asus:~/apache-cassandra-2.0.6/bin$ ./nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address        Load       Tokens  Owns (effective)  Host ID                               Rack
UN  192.168.2.103  40.92 KB   256     100.0%            80a96105-819a-40f2-ac7c-489285017de1  rack1

现在cassandra已经启动了节点1，节点2在一台虚拟机上（ip:192.168.2.102），其配置与节点1非常类似，要改动配置里面的seeds那一项的ip为节点1的ip

cluster_name: 'TC01'
num_tokens: 256
seed_provider:
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
      parameters:
          - seeds: "192.168.2.103"
listen_address: 192.168.2.102
rpc_address: 192.168.2.102
endpoint_snitch: SimpleSnitch

以同样的方式启动节点2，就可以在节点1中看到节点2已经加入到集群。此时，节点1的log中可以看到以下信息

 WARN 17:36:15,477 Gossip stage has 1 pending tasks; skipping status check (no nodes will be marked down)
 INFO 17:36:16,358 Handshaking version with /192.168.2.102
 INFO 17:36:17,999 Node /192.168.2.102 is now part of the cluster
 INFO 17:36:18,496 Handshaking version with /192.168.2.102
 INFO 17:36:18,727 InetAddress /192.168.2.102 is now UP
 INFO 17:36:53,523 [Stream #24d92630-c2ef-11e3-9d13-81bcfc870408] Received streaming plan for Bootstrap
 INFO 17:36:54,336 [Stream #24d92630-c2ef-11e3-9d13-81bcfc870408] Session with /192.168.2.102 is complete
 INFO 17:36:54,586 [Stream #24d92630-c2ef-11e3-9d13-81bcfc870408] All sessions completed

如果用nodetool来看，可以看到以下信息

XXXXX@XXXXX-asus:~/apache-cassandra-2.0.6/bin$ ./nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address        Load       Tokens  Owns (effective)  Host ID                               Rack
UN  192.168.2.102  54.74 KB   256     100.0%            00782976-b020-43a7-8fa5-b7dee95e6f1e  rack1
UN  192.168.2.103  45.75 KB   256     100.0%            80a96105-819a-40f2-ac7c-489285017de1  rack1

很清楚的可以看到2个节点全部在线。接下去开始实验集群的数据同步，先登陆node1节点，建立必要的keyspace和column family

XXXXX@XXXXX-asus:~/apache-cassandra-2.0.6/bin$ ./cassandra-cli -h 192.168.2.103 -p 9160
Connected to: "TC01" on 192.168.2.103/9160
Welcome to Cassandra CLI version 2.0.6

The CLI is deprecated and will be removed in Cassandra 3.0.  Consider migrating to cqlsh.
CQL is fully backwards compatible with Thrift data; see http://www.datastax.com/dev/blog/thrift-to-cql3

Type 'help;' or '?' for help.
Type 'quit;' or 'exit;' to quit.

[default@unknown] create keyspace test;
fa857b42-d483-31a8-8ef0-b30251cf6f72
[default@unknown] use test;
Authenticated to keyspace: test
[default@test] create column family test_cf;
869e20c5-ab6f-349d-8466-a86454af18cc
[default@test] show schema;

WARNING: CQL3 tables are intentionally omitted from 'show schema' output.
See https://issues.apache.org/jira/browse/CASSANDRA-4377 for details.

create keyspace test
  with placement_strategy = 'NetworkTopologyStrategy'
  and strategy_options = {datacenter1 : 1}
  and durable_writes = true;

use test;

create column family test_cf
  with column_type = 'Standard'
  and comparator = 'BytesType'
  and default_validation_class = 'BytesType'
  and key_validation_class = 'BytesType'
  and read_repair_chance = 0.1
  and dclocal_read_repair_chance = 0.0
  and populate_io_cache_on_flush = false
  and gc_grace = 864000
  and min_compaction_threshold = 4
  and max_compaction_threshold = 32
  and replicate_on_write = true
  and compaction_strategy = 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
  and caching = 'KEYS_ONLY'
  and default_time_to_live = 0
  and speculative_retry = 'NONE'
  and compression_options = {'sstable_compression' : 'org.apache.cassandra.io.compress.LZ4Compressor'}
  and index_interval = 128;

但是在插入数据的时候会发生奇怪的错误，用ascii函数可以解决

[default@test] set test_cf['num']['num1']=1111;
org.apache.cassandra.serializers.MarshalException: cannot parse 'num1' as hex bytes
[default@test] set test_cf[ascii('num')][ascii('num1')]=1111;
Value inserted.
Elapsed time: 97 msec(s).

原因是因为列族中使用了BytesType，如果用UTF8Type就不会有错了，drop之前的列族，重建

[default@test] create column family testcf  with column_type = 'Standard'  and comparator = 'UTF8Type'  and default_validation_class = 'UTF8Type'  and key_validation_class = 'UTF8Type';
08656500-2f27-345f-bcf7-c58e225fb2ff
[default@test] show schema;

WARNING: CQL3 tables are intentionally omitted from 'show schema' output.
See https://issues.apache.org/jira/browse/CASSANDRA-4377 for details.

create keyspace test
  with placement_strategy = 'NetworkTopologyStrategy'
  and strategy_options = {datacenter1 : 1}
  and durable_writes = true;

use test;

create column family testcf
  with column_type = 'Standard'
  and comparator = 'UTF8Type'
  and default_validation_class = 'UTF8Type'
  and key_validation_class = 'UTF8Type'
  and read_repair_chance = 0.1
  and dclocal_read_repair_chance = 0.0
  and populate_io_cache_on_flush = false
  and gc_grace = 864000
  and min_compaction_threshold = 4
  and max_compaction_threshold = 32
  and replicate_on_write = true
  and compaction_strategy = 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
  and caching = 'KEYS_ONLY'
  and default_time_to_live = 0
  and speculative_retry = 'NONE'
  and compression_options = {'sstable_compression' : 'org.apache.cassandra.io.compress.LZ4Compressor'}
  and index_interval = 128;

测试插入数据和查询，以及和node2之间的数据同步。先看node1上的插入数据

XXXXX@XXXXX-asus:~/apache-cassandra-2.0.6/bin$ ./cassandra-cli -h 192.168.2.103 -p 9160
Connected to: "TC01" on 192.168.2.103/9160
Welcome to Cassandra CLI version 2.0.6

The CLI is deprecated and will be removed in Cassandra 3.0.  Consider migrating to cqlsh.
CQL is fully backwards compatible with Thrift data; see http://www.datastax.com/dev/blog/thrift-to-cql3

Type 'help;' or '?' for help.
Type 'quit;' or 'exit;' to quit.

[default@unknown] use test;
Authenticated to keyspace: test
[default@test] set testcf['num']['num1']=1111;
Value inserted.
Elapsed time: 66 msec(s).
[default@test] list testcf;
Using default limit of 100
Using default cell limit of 100
-------------------
RowKey: num
=> (name=num1, value=1111, timestamp=1397389549047000)

1 Row Returned.
Elapsed time: 823 msec(s).
[default@test] 

再看node2上的同步的结果

XXXXX@XXXXX-asus:~/apache-cassandra-2.0.6/bin$ ./cassandra-cli -h 192.168.2.102 -p 9160
Connected to: "TC01" on 192.168.2.102/9160
Welcome to Cassandra CLI version 2.0.6

The CLI is deprecated and will be removed in Cassandra 3.0.  Consider migrating to cqlsh.
CQL is fully backwards compatible with Thrift data; see http://www.datastax.com/dev/blog/thrift-to-cql3

Type 'help;' or '?' for help.
Type 'quit;' or 'exit;' to quit.

[default@unknown] use test;
Authenticated to keyspace: test
[default@test] list testcf;
Using default limit of 100
Using default cell limit of 100
-------------------
RowKey: num
=> (name=num1, value=1111, timestamp=1397389549047000)

1 Row Returned.
Elapsed time: 914 msec(s).
[default@test]