一起学习ETCD系列——运维操作之etcdctl使用
文章目录
-
- 概要
- 一、命令
- 二、实操
-
-
- 2.1、基本操作
- 2.2、watch
- 2.3、租约
- 2.4、分布式锁
- 2.5、角色
- 2.6、用户
- 2.7、认证
- 2.8、集群
-
概要
本文主要用来总结ETCD客户端ctcdctl的命令操作，在运维过程中可能常常用到的。
一、命令
etcd工具
etcdctl官方命令示例
[root@test etcd-v3.5.11]# ./etcdctl -h
NAME:
etcdctl - A simple command line client for etcd3.
USAGE:
etcdctl [flags]
VERSION:
3.5.11
API VERSION:
3.5
COMMANDS: #命令
alarm disarm Disarms all alarms #关闭告警
alarm list Lists all alarms #列出告警信息
auth disable Disables authentication #关闭认证
auth enable Enables authentication #开启认证
auth status Returns authentication status #查看认证状态
check datascale Check the memory usage of holding data for different workloads on a given server endpoint. #检查给定实例的内存使用情况
check perf Check the performance of the etcd cluster #检查etcd集群的性能
compaction Compacts the event history in etcd #压缩etcd的历史事件
defrag Defragments the storage of the etcd members with given endpoints #整理给定实例的存储碎片
del Removes the specified key or range of keys [key, range_end) #删除key
elect Observes and participates in leader election #加入leader选举
endpoint hashkv Prints the KV history hash for each endpoint in --endpoints #打印给定若干实例KV history hash
endpoint health Checks the healthiness of endpoints specified in `--endpoints` flag #检查给定若干实例健康
endpoint status Prints out the status of endpoints specified in `--endpoints` flag #打印给定若干实例状态
get Gets the key or a range of keys #获取key值
help Help about any command #打印命令帮助信息
lease grant Creates leases #创建一个租约
lease keep-alive Keeps leases alive (renew) #续约
lease list List all active leases #列出所有租约
lease revoke Revokes leases #撤销租约
lease timetolive Get lease information #获取租约详情，比如剩余有效期
lock Acquires a named lock #获取命名锁
make-mirror Makes a mirror at the destination etcd cluster #指定一个etcd集群作为镜像集群
member add Adds a member into the cluster #添加一个成员到etcd集群
member list Lists all members in the cluster #列出etcd集群全部成员
member promote Promotes a non-voting member in the cluster #将成员从Follower状态到Candidate状态
member remove Removes a member from the cluster #移除成员
member update Updates a member in the cluster #更新成员
move-leader Transfers leadership to another etcd cluster member. #指定成员成为Leader状态
put Puts the given key into the store #插入或更新key
role add Adds a new role #添加新角色
role delete Deletes a role #删除角色
role get Gets detailed information of a role #获取角色的详细信息
role grant-permission Grants a key to a role #授予角色对key的权限，比如读写
role list Lists all roles #列出全部的角色
role revoke-permission Revokes a key from a role #撤销角色对指定key的权限
snapshot restore Restores an etcd member snapshot to an etcd directory #恢复快照到指定数据目录
snapshot save Stores an etcd node backend snapshot to a given file #保存快照到指定文件
snapshot status [deprecated] Gets backend snapshot status of a given file #[弃用] 获取给定快照文件的状态
txn Txn processes all the requests in one transaction #开启事务
user add Adds a new user #添加账户
user delete Deletes a user #删除账户
user get Gets detailed information of a user #获取账户详情
user grant-role Grants a role to a user #指定账户角色
user list Lists all users #列出所有账户
user passwd Changes password of user #修改账户密码
user revoke-role Revokes a role from a user #撤销账户角色
version Prints the version of etcdctl #打印ectd当前版本
watch Watches events stream on keys or prefixes #监听一些key或共同前缀key的事件【】
OPTIONS: #命令的可选参数
--cacert="" verify certificates of TLS-enabled secure servers using this CA bundle #服务端使用https时，使用ca文件进行验证
--cert="" identify secure client using this TLS certificate file #指定HTTPS的TLS证书文件
--command-timeout=5s timeout for short running command (excluding dial timeout) #命令执行超时时间
--debug[=false] enable client-side debug logging #启用客户端调试日志，默认false
--dial-timeout=2s dial timeout for client connections #指定连接超时时间
-d, --discovery-srv="" domain name to query for SRV records describing cluster endpoints #用于查询描述集群端点的srv记录的域名
--discovery-srv-name="" service name to query when using DNS discovery #使用DNS发现时要查询的服务名称
--endpoints=[127.0.0.1:2379] gRPC endpoints #指定实例访问ip+端口
-h, --help[=false] help for etcdctl
--hex[=false] print byte strings as hex encoded strings #将字节字符串打印为十六进制编码字符串
--insecure-discovery[=true] accept insecure SRV records describing cluster endpoints #接受描述集群端点的不安全的srv记录
--insecure-skip-tls-verify[=false] skip server certificate verification (CAUTION: this option should be enabled only for testing purposes)#跳过服务器证书验证（注意：此选项应仅用于测试目的）
--insecure-transport[=true] disable transport security for client connections #客户端禁用安全传输
--keepalive-time=2s keepalive time for client connections #客户端连接的keepalive时间
--keepalive-timeout=6s keepalive timeout for client connections #客户端连接的keepalive超时时间
--key="" identify secure client using this TLS key file #使用此 TLS 密钥文件识别安全客户端
--password="" password for authentication (if this option is used, --user option shouldn't include password) #用于身份验证的密码（如果使用此选项，--user 选项不应包含密码）
--user="" username[:password] for authentication (prompt if password is not supplied)#用于身份验证（如果未提供密码则提示）
-w, --write-out="simple" set the output format (fields, json, protobuf, simple, table) #输出格式,默认simple，个人感觉table格式看着最舒服
二、实操
etcdctl [command] -h 可以查看具体命令的使用说明：
[root@test etcd-v3.5.11]# ./etcdctl lease -h
NAME:
lease - Lease related commands
USAGE:
etcdctl lease <subcommand> [flags]
API VERSION:
3.5
COMMANDS:
grant Creates leases
keep-alive Keeps leases alive (renew)
list List all active leases
revoke Revokes leases
timetolive Get lease information
OPTIONS:
-h, --help[=false] help for lease
[root@test etcd-v3.5.11]# ./etcdctl lease grant -h
NAME:
lease grant - Creates leases
USAGE:
etcdctl lease grant <ttl> [flags] #详细使用说明
OPTIONS:
-h, --help[=false] help for grant
2.1、基本操作
#插入key
[root@test etcd-v3.5.11]# ./etcdctl put /test/key1 value1
OK
#创建500s的租约，其id为43e58d2bb5a3102d
[root@test etcd-v3.5.11]# ./etcdctl lease grant 500
lease 43e58d2bb5a3102d granted with TTL(500s)
#获取租约详情
[root@test etcd-v3.5.11]# ./etcdctl lease timetolive 43e58d2bb5a3102d
lease 43e58d2bb5a3102d granted with TTL(500s), remaining(470s)
#插入key，并指定租约
[root@test etcd-v3.5.11]# ./etcdctl put /test/key2 value2 --lease=43e58d2bb5a3102d
OK
#获取
[root@test etcd-v3.5.11]# ./etcdctl get /test/key1
/test/key1
value1
[root@test etcd-v3.5.11]# ./etcdctl get /test/key2
/test/key2
value2
#删除
[root@test etcd-v3.5.11]# ./etcdctl del /test/key2
1
[root@test etcd-v3.5.11]# ./etcdctl get /test/key1 -w josn
{"header":{"cluster_id":11073961802084797819,"member_id":8518302759694844901,"revision":10769,"raft_term":15},"kvs":[{"key":"L3Rlc3Qva2V5MQ==","create_revision":10759,"mod_revision":10761,"version":3,"value":"dmFsdWUxNA=="}],"count":1}
#cluster_id：请求的etcd集群ID。
#member_id：请求的etcd节点ID。
#revision：etcd服务端当前全局数据版本号。对任一key的put或delete操作都会使revision自增1。revision=1是etcd的保留版本号，因此用户的key版本号将从2开始。
#raft_term：etcd当前raft主节点任期号。
#create_revision：当前key创建时全局数据版本号revision的值。
#mod_revision：当前key最后一次修改时全局数据版本号revision的值。
#version：当前key的数据版本号。key创建时version为1，对当前key进行put操作会使version自增1，将key删除后，重新创建，version又会从1开始计数。
2.2、watch
监听key，可以看到key的事件
[root@test etcd-v3.5.11]# ./etcdctl watch /test/key3
PUT
/test/key3
value3
PUT
/test/key3
value31
PUT
/test/key3
value33
DELETE
/test/key3
PUT
/test/key3
value35
操作key
[root@test etcd-v3.5.11]# ./etcdctl put /test/key3 value3
OK
[root@test etcd-v3.5.11]# ./etcdctl put /test/key3 value31
OK
[root@test etcd-v3.5.11]# ./etcdctl put /test/key3 value33
OK
[root@test etcd-v3.5.11]# ./etcdctl del /test/key3
1
[root@test etcd-v3.5.11]# ./etcdctl put /test/key3 value35
OK
2.3、租约
#创建500s的租约，其id为43e58d2bb5a3102d
[root@test etcd-v3.5.11]# ./etcdctl lease grant 500
lease 43e58d2bb5a3102d granted with TTL(500s)
#获取租约详情
[root@test etcd-v3.5.11]# ./etcdctl lease timetolive 43e58d2bb5a3102d
lease 43e58d2bb5a3102d granted with TTL(500s), remaining(470s)
#只续约一次
[root@test etcd-v3.5.11]# ./etcdctl lease keep-alive --once=true 43e58d2bb5a3102d
#一直自动续约
[root@test etcd-v3.5.11]# ./etcdctl lease keep-alive 43e58d2bb5a3102d
#删除租约
[root@test etcd-v3.5.11]# ./etcdctl lease revoke 43e58d2bb5a3102d
2.4、分布式锁
etcd中分布式锁的结构是：lock_key/lease_id。
实现过程（源码）如下：
- 准备
客户端连接 Etcd，以 /lock/mylock 为前缀创建全局唯一的 key，假设第一个客户端对应的 key=“/lock/mylock/UUID1”，第二个为 key=“/lock/mylock/UUID2”；客户端分别为自己的 key 创建租约 - Lease，租约的长度根据业务耗时确定，假设为 15s； - 创建租约
当一个客户端持有锁期间，其它客户端只能等待，为了避免等待期间租约失效，客户端需创建一个定时任务作为“心跳”进行续约。此外，如果持有锁期间客户端崩溃，心跳停止，key 将因租约到期而被删除，从而锁释放，避免死锁。 - 客户端写入自己全局唯一的 key
通过 PUT 操作，将步骤 1 中创建的 key 绑定租约写入 Etcd，根据 Etcd 的 Revision 机制，假设两个客户端 put 操作返回的 Revision 分别为 1、2，客户端需记录 Revision 用以接下来判断自己是否获得锁。 - 客户端判断是否获得锁
客户端以前缀 /lock/mylock 读取 keyValue 列表（keyValue 中带有 key 对应的 Revision），判断自己 key 的 Revision 是否为当前列表中最小的，如果是则认为获得锁；否则监听列表中前一个 Revision 比自己小的 key 的删除事件，一旦监听到删除事件或者因租约失效而删除的事件，则自己获得锁。 - 执行业务
获得锁后，操作共享资源，执行业务代码。 - 释放锁
完成业务流程后，释放锁,通过删除对应的key完成。
测试：
开三个窗口
#获取锁
[root@test etcd-v3.5.11]# ./etcdctl lock /lock/test --ttl=5
/lock/test/43e58d437ddbc834
#等待锁
[root@test etcd-v3.5.11]# ./etcdctl lock /lock/test --ttl=5
#watch监控/lock/test为前缀的key变化
[root@kdzl etcd-v3.5.11]# ./etcdctl watch /lock/test --prefix
PUT
/lock/test/43e58d437ddbc834
PUT
/lock/test/43e58d437ddbc838
2.5、角色
[root@test etcd-v3.5.11]# ./etcdctl role add book
Role book created
[root@test etcd-v3.5.11]# ./etcdctl role add man
Role man created
[root@test etcd-v3.5.11]# ./etcdctl role list
book
man
[root@test etcd-v3.5.11]# ./etcdctl role get man
Role man
KV Read:
KV Write:
#给key授权角色
#etcdctl role grant-permission [options] [endkey] [flags]
[root@test etcd-v3.5.11]# ./etcdctl role grant-permission man read /test
Role man updated
[root@test etcd-v3.5.11]# ./etcdctl role get man
Role man
KV Read:
/test
KV Write:
#移除key角色
[root@test etcd-v3.5.11]# ./etcdctl role revoke-permission man /test
Permission of key /test is revoked from role man
[root@test etcd-v3.5.11]# ./etcdctl role get man
Role man
KV Read:
KV Write:
2.6、用户
#创建用户
[root@test etcd-v3.5.11]# ./etcdctl user add root:root
User root created
[root@test etcd-v3.5.11]# ./etcdctl user add test --new-user-password=12345
User test created
[root@test etcd-v3.5.11]# ./etcdctl user add pick:25823
User pick created
#列出全部的用户
[root@test etcd-v3.5.11]# ./etcdctl user list
root
test
[root@test etcd-v3.5.11]# ./etcdctl user get test
User: test
Roles:
#给账号一个角色,一个账户可以有多个角色
[root@test etcd-v3.5.11]# ./etcdctl user grant-role test man
Role man is granted to user test
[root@test etcd-v3.5.11]# ./etcdctl user get test
User: test
Roles: man
#删除角色
[root@test etcd-v3.5.11]# ./etcdctl user del pick
User pick deleted
#撤销用户角色
[root@test etcd-v3.5.11]# ./etcdctl user revoke-role test man
Role man is revoked from user test
2.7、认证
启用auth需要一个root用户，并且root用户有root角色
[root@test etcd-v3.5.11]# ./etcdctl role add root
Role root created
[root@test etcd-v3.5.11]# ./etcdctl role grant-permission --prefix=true root readwrite /
Role root updated
[root@test etcd-v3.5.11]# ./etcdctl user grant-role root root
Role root is granted to user root
[root@test etcd-v3.5.11]# ./etcdctl auth enable
Authentication Enabled
[root@test etcd-v3.5.11]# ./etcdctl auth status --user=root:root
Authentication Status: true
AuthRevision: 15
换成 test 用户操作
[root@test etcd-v3.5.11]# ./etcdctl role grant-permission --prefix=true man read /test --user=root:root #对/test前缀的key有只读权限
Role man updated
[root@test etcd-v3.5.11]# ./etcdctl role get man --user=root:root
Role man
KV Read:
/test
KV Write:
[root@test etcd-v3.5.11]# ./etcdctl user grant-role test man --user=root:root
Role man is granted to user test
[root@test etcd-v3.5.11]# ./etcdctl get /test/key1 --user=test:12345
/test/key1
value14
[root@test etcd-v3.5.11]# ./etcdctl put /test/key5 value5 --user=test:12345
{"level":"warn","ts":"2024-01-25T23:38:54.753123+0800","logger":"etcd-client","caller":"[email protected]/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc000362e00/127.0.0.1:2379","attempt":0,"error":"rpc error: code = PermissionDenied desc = etcdserver: permission denied"}
Error: etcdserver: permission denied
可以看到权限生效
2.8、集群
1：查看每个实例状态,即检查etcd集群状态
[root@test etcd-v3.5.11]# ./etcdctl endpoint status -w table --endpoints"=127.0.0.1:2379,127.0.0.1:2369,127.0.0.1:2359"
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| 127.0.0.1:2379 | 7637173a60d743e5 | 3.5.11 | 21 MB | true | false | 15 | 10923 | 10923 | |
| 127.0.0.1:2369 | ce6b0efed2935560 | 3.5.11 | 21 MB | false | false | 15 | 10923 | 10923 | |
| 127.0.0.1:2359 | 6ea1523f71bf92f0 | 3.5.11 | 21 MB | false | false | 15 | 10923 | 10923 | |
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
2:查看健康状态
#TOOK表示心跳
[root@test etcd-v3.5.11]# ./etcdctl endpoint health --endpoints="127.0.0.1:2379,127.0.0.1:2369,127.0.0.1:2359" -w table
+----------------+--------+-------------+-------+
| ENDPOINT | HEALTH | TOOK | ERROR |
+----------------+--------+-------------+-------+
| 127.0.0.1:2369 | true | 16.178532ms | |
| 127.0.0.1:2379 | true | 17.826437ms | |
| 127.0.0.1:2359 | true | 19.841588ms | |
+----------------+--------+-------------+-------+
[root@test etcd-v3.5.11]# ./etcdctl endpoint hashkv --endpoints="127.0.0.1:2379,127.0.0.1:2369,127.0.0.1:2359" -w table
+----------------+------------+
| ENDPOINT | HASH |
+----------------+------------+
| 127.0.0.1:2379 | 2618695019 |
| 127.0.0.1:2369 | 2618695019 |
| 127.0.0.1:2359 | 2618695019 |
+----------------+------------+
3：切换集群leader
原来集群leader实例是127.0.0.1:2379,现在通过move-leader命令将leader切换成实例127.0.0.1:2359
[root@test etcd-v3.5.11]# ./etcdctl move-leader 6ea1523f71bf92f0 --endpoints="127.0.0.1:2379,127.0.0.1:2369,127.0.0.1:2359"
Leadership transferred from 7637173a60d743e5 to 6ea1523f71bf92f0
[root@test etcd-v3.5.11]# ./etcdctl endpoint status --endpoints="127.0.0.1:2379,127.0.0.1:2369,127.0.0.1:2359" -w table
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| 127.0.0.1:2379 | 7637173a60d743e5 | 3.5.11 | 21 MB | false | false | 19 | 10933 | 10933 | |
| 127.0.0.1:2369 | ce6b0efed2935560 | 3.5.11 | 21 MB | false | false | 19 | 10933 | 10933 | |
| 127.0.0.1:2359 | 6ea1523f71bf92f0 | 3.5.11 | 21 MB | true | false | 19 | 10933 | 10933 | |
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
4：查看集群成员
[root@test etcd-v3.5.11]# ./etcdctl member list -w table
+------------------+---------+--------+-----------------------+-----------------------+------------+
| ID | STATUS | NAME | PEER ADDRS | CLIENT ADDRS | IS LEARNER |
+------------------+---------+--------+-----------------------+-----------------------+------------+
| 6ea1523f71bf92f0 | started | etcd3 | http://127.0.0.1:2360 | http://127.0.0.1:2359 | false |
| 7637173a60d743e5 | started | etcd1 | http://127.0.0.1:2380 | http://127.0.0.1:2379 | false |
| ce6b0efed2935560 | started | etcd2 | http://127.0.0.1:2370 | http://127.0.0.1:2369 | false |
+------------------+---------+--------+-----------------------+-----------------------+------------+
5：添加成员（一个learner成员，不参与投票）
1）etcdctl member add 命令将新成员添加到集群
[root@testl etcd-v3.5.11]# ./etcdctl member add etcd04 --peer-urls="http://127.0.0.16:2350" --learner=true
Member 690b31e00158f43a added to cluster 99ae9d962d824d7b
ETCD_NAME="etcd4"
ETCD_INITIAL_CLUSTER="etcd4=http://127.0.0.1:2350,etcd3=http://127.0.0.1:2360,etcd1=http://127.0.0.1:2380,etcd2=http://127.0.0.1:2370"
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://127.0.0.1:2350"
ETCD_INITIAL_CLUSTER_STATE="existing"
#可以看到该成员尚未启动
[root@test etcd-v3.5.11]# ./etcdctl member list -w table
+------------------+-----------+--------+----------------------------+-----------------------+------------+
| ID | STATUS | NAME | PEER ADDRS | CLIENT ADDRS | IS LEARNER |
+------------------+-----------+--------+----------------------------+-----------------------+------------+
| 690b31e00158f43a | unstarted | | http://127.0.0.1:2350 | | true |
| 6ea1523f71bf92f0 | started | etcd3 | http://127.0.0.1:2360 | http://127.0.0.1:2359 | false |
| 7637173a60d743e5 | started | etcd1 | http://127.0.0.1:2380 | http://127.0.0.1:2379 | false |
| ce6b0efed2935560 | started | etcd2 | http://127.0.0.1:2370 | http://127.0.0.1:2369 | false |
+------------------+-----------+--------+----------------------------+-----------------------+------------+
2）启动新成员
回溯集群安装篇
配置文件如下：
mkdir -p /usr/local/etcd-v3.5.11/etcd4/data
vim /usr/local/etcd-v3.5.11/etcd4/conf.yml
name: etcd4
data-dir: /usr/local/etcd-v3.5.11/etcd4/data
initial-advertise-peer-urls: http://127.0.0.1:2350
listen-peer-urls: http://127.0.0.1:2350
listen-client-urls: http://172.20.101.222:2349,http://127.0.0.1:2349
advertise-client-urls: http://127.0.0.1:2349
initial-cluster-token: test-etcd-cluster
initial-cluster: etcd1=http://127.0.0.1:2380,etcd2=http://127.0.0.1:2370,etcd3=http://127.0.0.1:2360,etcd4=http://127.0.0.1:2350
initial-cluster-state: existing #注意，这里一定要标记为existing
/usr/local/etcd-v3.5.11/etcd --config-file /usr/local/etcd-v3.5.11/etcd4/conf.yml
可以看到已经成功加入集群
[root@test etcd-v3.5.11]# ./etcdctl member list -w table
+------------------+---------+--------+-----------------------+-----------------------+------------+
| ID | STATUS | NAME | PEER ADDRS | CLIENT ADDRS | IS LEARNER |
+------------------+---------+--------+-----------------------+-----------------------+------------+
| 6ea1523f71bf92f0 | started | etcd3 | http://127.0.0.1:2360 | http://127.0.0.1:2359 | false |
| 7637173a60d743e5 | started | etcd1 | http://127.0.0.1:2380 | http://127.0.0.1:2379 | false |
| c64a0c36c4a5869f | started | etcd4 | http://127.0.0.1:2350 | http://127.0.0.1:2349 | true |
| ce6b0efed2935560 | started | etcd2 | http://127.0.0.1:2370 | http://127.0.0.1:2369 | false |
+------------------+---------+--------+-----------------------+-----------------------+------------+
6：learner成员提升为follower成员
实例http://127.0.0.1:2350还是learner，没有投票权，用member promote命令提升下
[root@test etcd-v3.5.11]# ./etcdctl member promote c64a0c36c4a5869f
Member c64a0c36c4a5869f promoted in cluster 99ae9d962d824d7b
[root@test etcd-v3.5.11]# ./etcdctl member list -w table
+------------------+---------+--------+-----------------------+-----------------------+------------+
| ID | STATUS | NAME | PEER ADDRS | CLIENT ADDRS | IS LEARNER |
+------------------+---------+--------+-----------------------+-----------------------+------------+
| 6ea1523f71bf92f0 | started | etcd3 | http://127.0.0.1:2360 | http://127.0.0.1:2359 | false |
| 7637173a60d743e5 | started | etcd1 | http://127.0.0.1:2380 | http://127.0.0.1:2379 | false |
| c64a0c36c4a5869f | started | etcd4 | http://127.0.0.1:2350 | http://127.0.0.1:2349 | false | #可以看到已经不是learner了
| ce6b0efed2935560 | started | etcd2 | http://127.0.0.1:2370 | http://127.0.0.1:2369 | false |
+------------------+---------+--------+-----------------------+-----------------------+------------+