基于Kubernetes的框架和应用系列之五：在Kubernetes安装基于Ceph PVC的cassandra集群

一、前言

在数据库领域，针对不同的业务需求，有不同类型的数据库方案，比如：

• 关系型数据库，如Mysql、Oracle、DB2、Sqlserver
• 非关系型数据库
  • Key-Value数据库，如Memcached、Redis、MemcacheDB
  • Document-based数据库，如MongoDB、CouchDB
  • Column-based数据库，如Cassandra、HBase
  • 图形数据库，如Neo4J、InforGrid

下图是在DB排名网站https://db-engines.com/en/ranking的TOP20数据库：

可见Cassandra是列数据库中最热门的，Cassandra有如下一些特点：

• 极高的读写性能，写入操作只涉及到顺序写入和内存操作，具有很高的写性能；而进行读操作时，Cassandra实现数据分层存储，将热点数据放在Memtable和相对小的SSTable中，所以具有很高的读性能。
• 简单的部署结构，Cassandra是去中心化的P2P结构，所有节点完全一样没有单点问题

上图是但数据中心consistency level 是1的数据读取过程，节点10是Coordinator，负责和客户端交互，真正的数据有三个副本分别在1，3，6节点，由于consistency level 是1，所以只要从节点1读取数据并返回就可以了。更复杂的场景可以在这里找到分析。

本文讲述如何在Kubernetes部署带持久存储的高可用Cassandra集群。

转载自https://blog.csdn.net/cloudvtech

二、安装Cassandra集群

2.1 准备工作

创建namespace

kubectl create ns cassandra

创建Ceph StorageClass的access key

kubectl create ns cassandra
kubectl -n ceph get secrets/pvc-ceph-client-key -o json | jq '.metadata.namespace = "cassandra"' | kubectl create -f -
kubectl get secret -n cassandra
NAME                  TYPE                                  DATA      AGE
default-token-7ldwg   kubernetes.io/service-account-token   3         1m
pvc-ceph-client-key   kubernetes.io/rbd                     1         14s

2.2 部署文件

cassandra-service.yaml，提供集群访问接口

apiVersion: v1
kind: Service
metadata:
  labels:
    app: cassandra
  name: cassandra
  namespace: cassandra
spec:
  clusterIP: None
  ports:
    - port: 9042
  selector:
    app: cassandra

cassandra-statefulset.yaml，基于statefulset的9节点集群的部署

apiVersion: "apps/v1beta1"
kind: StatefulSet
metadata:
  name: cassandra
  namespace: cassandra
spec:
  serviceName: cassandra
  replicas: 9
  template:
    metadata:
      labels:
        app: cassandra
    spec:
      containers:
        - name: cassandra
          image: cassandra:3
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 7000
              name: intra-node
            - containerPort: 7001
              name: tls-intra-node
            - containerPort: 7199
              name: jmx
            - containerPort: 9042
              name: cql
          env:
            - name: CASSANDRA_SEEDS
              value: cassandra-0.cassandra.cassandra.svc.cluster.local # cassandra-0.cassandra.%%NAMESPACE%%.svc.cluster.local
            - name: MAX_HEAP_SIZE
              value: 256M
            - name: HEAP_NEWSIZE
              value: 100M
            - name: CASSANDRA_CLUSTER_NAME
              value: "Cassandra"
            - name: CASSANDRA_DC
              value: "DC1"
            - name: CASSANDRA_RACK
              value: "Rack1"
            - name: CASSANDRA_ENDPOINT_SNITCH
              value: GossipingPropertyFileSnitch
          volumeMounts:
            - name: cassandra-data
              mountPath: /var/lib/cassandra/data
  volumeClaimTemplates:
    - metadata:
        name: cassandra-data
        annotations:  # comment line if you want to use a StorageClass
          # or specify which StorageClass
          volume.beta.kubernetes.io/storage-class: "ceph-rbd"   # comment line if you
          # want to use a StorageClass or specify which StorageClass
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 100Gi

2.3 查看部署结果

所有的POD

[root@k8s-master-01 cassandra-ceph]# kubectl get pods -n cassandra
NAME          READY     STATUS    RESTARTS   AGE
cassandra-0   1/1       Running   0          8m
cassandra-1   1/1       Running   0          7m
cassandra-2   1/1       Running   6          7m
cassandra-3   1/1       Running   0          7m
cassandra-4   1/1       Running   1          7m
cassandra-5   1/1       Running   3          6m
cassandra-6   1/1       Running   4          6m
cassandra-7   1/1       Running   0          6m
cassandra-8   1/1       Running   1          5m

所有的PV

[root@k8s-master-01 cassandra-ceph]# kubectl get pv -n cassandra | grep cassandra
pvc-41daebbf-cd35-11e8-9135-fa163ebda1b8   100Gi      RWO            Delete           Bound         cassandra/cassandra-data-cassandra-2   ceph-rbd                 10m
pvc-4b1866b0-cd35-11e8-9135-fa163ebda1b8   100Gi      RWO            Delete           Bound         cassandra/cassandra-data-cassandra-3   ceph-rbd                 9m
pvc-59be4568-cd35-11e8-9135-fa163ebda1b8   100Gi      RWO            Delete           Bound         cassandra/cassandra-data-cassandra-4   ceph-rbd                 9m
pvc-67c8c038-cd35-11e8-9135-fa163ebda1b8   100Gi      RWO            Delete           Bound         cassandra/cassandra-data-cassandra-5   ceph-rbd                 8m
pvc-6e4e1b94-cd31-11e8-9135-fa163ebda1b8   100Gi      RWO            Delete           Bound         cassandra/cassandra-data-cassandra-0   ceph-rbd                 37m
pvc-74b73065-cd31-11e8-9135-fa163ebda1b8   100Gi      RWO            Delete           Bound         cassandra/cassandra-data-cassandra-1   ceph-rbd                 37m
pvc-7848907c-cd35-11e8-9135-fa163ebda1b8   100Gi      RWO            Delete           Bound         cassandra/cassandra-data-cassandra-6   ceph-rbd                 8m
pvc-7ef9c6bc-cd35-11e8-9135-fa163ebda1b8   100Gi      RWO            Delete           Bound         cassandra/cassandra-data-cassandra-7   ceph-rbd                 8m
pvc-99379af4-cd35-11e8-9135-fa163ebda1b8   100Gi      RWO            Delete           Bound         cassandra/cassandra-data-cassandra-8   ceph-rbd                 7m

相互之间数据交互关系

2.4 查看Cassandra集群状态

[root@k8s-master-01 cassandra-ceph]# kubectl exec -ti cassandra-0 -n cassandra -- nodetool status 
Datacenter: DC1
===============
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address         Load       Tokens       Owns (effective)  Host ID                               Rack
UN  10.244.44.214   193.95 KiB  256          20.9%             f81e3aad-5d13-49a3-a0d7-77a505833f42  Rack1
UN  10.244.89.150   114.71 KiB  256          22.0%             d3299f20-ae10-443f-97d0-25e5462833ca  Rack1
UN  10.244.154.215  82.01 KiB  256          22.7%             d1261c1b-8f6a-4f0a-ae0b-21541dd99f77  Rack1
UN  10.244.44.199   88.89 KiB  256          22.3%             62e2e592-9b80-47ec-98a4-6e8d909f1955  Rack1
UN  10.244.44.210   69.92 KiB  256          22.9%             ab0e1502-995b-41a4-a363-a9cf42fe6bb2  Rack1
UN  10.244.89.147   77.78 KiB  256          21.9%             ee1d368a-1599-49e7-8eb6-a16c7ef8dbf9  Rack1
UN  10.244.89.190   114.72 KiB  256          22.7%             28e27ba4-5ecd-4126-aec2-c3444c287442  Rack1
UN  10.244.154.223  69.93 KiB  256          23.0%             55d126e8-adfa-435b-a5d1-c2b0e074910b  Rack1
UN  10.244.154.207  108.62 KiB  256          21.6%             ff0f5f91-fc98-4a58-956d-336586e048fb  Rack1

转载自https://blog.csdn.net/cloudvtech