kafka学习笔记八kafka的权限管理之ACL权限控制

kafka的权限管理之ACL权限控制

1.broker acl configuration

2.producer acl write

3.consumer acl read

4.programming acl （consumer & producer）

 

一、配置 broker sasl(acl) configuration
1.1 分别复制server.properties为sasl-server.properties，修改sasl-server.properties增加如下配置

advertised.listeners=PLAINTEXT://120.26.198.248:9092，SASL_PLAINTEXT://120.26.198.248:9093

authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
security.inter.broker.protocol= SASL_PLAINTEXT
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN
super.users=User:admin

1.2分别在config下创建文件kafka_cluster_jaas.conf

配置如下

kafkaServer{
        org.apache.kafka.common.security.plain.PlainLoginModule required
        username="admin"
        password="admin"
        user_admin="admin"
        user_reader="reader"
        user_writer="writer";
};

1.3分别修改启动脚本

bin/kafka-server-start.sh

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS -Djava.security.auth.login.config=/usr/local/kafka/kafka_2.11-0.11.0.1/config/kafka_cl
uster_jaas.conf  kafka.Kafka "$@"

1.4启动kafka服务

[root@iZbp1c8mn5lner8nhvreblZ kafka_2.11-0.11.0.1]# bin/kafka-server-start.sh -daemon config/sasl-server.properties 

二、kafka ACL命令

2.1  查看topic的权限信息

[root@iZbp14ouog5ocoeakj39prZ kafka_2.11-0.11.0.1]# bin/kafka-acls.sh --list --authorizer-properties zookeeper.connect=10.174.32.122:2181,10.117.15.224:2181,10.168.96.248:9092
[2018-08-29 11:32:52,490] WARN Client session timed out, have not heard from server in 2011ms for sessionid 0x0 (org.apache.zookeeper.ClientCnxn)

2.2 创建name=acl-test的topic，并检查是否创建成功

[root@iZbp14ouog5ocoeakj39prZ kafka_2.11-0.11.0.1]# bin/kafka-topics.sh --create --topic acl-test --partitions 3 --replication-factor 3 --zookeeper 10.174.32.122:2181,10.117.15.224:2181,10.168.96.248:9092
[2018-08-29 11:43:11,837] WARN Client session timed out, have not heard from server in 10019ms for sessionid 0x0 (org.apache.zookeeper.ClientCnxn)
Created topic "acl-test".
[root@iZbp14ouog5ocoeakj39prZ kafka_2.11-0.11.0.1]# bin/kafka-topics.sh --describe --topic acl-test --zookeeper 10.174.32.122:2181,10.117.15.224:2181,10.168.96.248:9092                                    
Topic:acl-test  PartitionCount:3        ReplicationFactor:3     Configs:
        Topic: acl-test Partition: 0    Leader: 0       Replicas: 0,2,1 Isr: 0,2,1
        Topic: acl-test Partition: 1    Leader: 1       Replicas: 1,0,2 Isr: 1,0,2
        Topic: acl-test Partition: 2    Leader: 2       Replicas: 2,1,0 Isr: 2,1,0

2.3运行kafka-console.producer.sh 测试生产者，提示无法写入，原因是无授权用户，因为没有配置授权用户导致如下

[root@iZbp14ouog5ocoeakj39prZ kafka_2.11-0.11.0.1]# bin/kafka-console-producer.sh --topic acl-test --broker-list 10.174.32.122:9092,10.117.15.224:9092,10.168.96.198:9092
>hello^H^H^H^H
[2018-08-29 13:21:21,549] WARN Connection to node -1 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2018-08-29 13:21:21,552] WARN Connection to node -3 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

三、配置 producer acl（sasl）configuration

3.1创建config/producer_jaas.conf

[root@iZbp14ouog5ocoeakj39prZ config]# vim producer_jaas.conf
"producer_jaas.conf" [New File]                                                                                   0,0-1         All
KafkaClient{
        org.apache.kafka.common.security.plain.PlainLoginModule required
        username="writer"
        password="writer";
};
~    

3.2创建config/producer-jaas.properties

[root@iZbp14ouog5ocoeakj39prZ config]# vim producer-jaas.properties
"producer-jaas.properties" [New File]                                                                             0,0-1         All
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN

3.3修改bin/kafka-console-producer.sh文件

exec $(dirname $0)/kafka-run-class.sh kafka.tools.ConsoleProducer "$@"

为

exec $(dirname $0)/kafka-run-class.sh -Djava.security.auth.login.config=/usr/local/kafka/kafka_2.11-0.11.0.1/config/producer_jaas.co
nf  kafka.tools.ConsoleProducer "$@"

3.4 带上config/producer-jaas.properties测试向acl-test中发送数据

[root@iZbp14ouog5ocoeakj39prZ kafka_2.11-0.11.0.1]# bin/kafka-console-producer.sh --topic acl-test --broker-list 10.174.32.122:9010.117.15.224:9092,10.168.96.198:9092 --producer.config config/producer-jaas.properties
>aaaaaa
[2018-08-29 14:08:31,034] WARN Connection to node -3 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2018-08-29 14:08:31,039] WARN Connection to node -2 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

依然为未授权提示

3.5通过bin/kafka-acls.sh给user writer 赋予权限

[root@iZbp14ouog5ocoeakj39prZ kafka_2.11-0.11.0.1]# bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=10.174.32.122:2181,10.117.15.224:2181,10.168.96.198:2181 --add --allow-principal User:writer --operation Write --topic acl-test
Adding ACLs for resource `Topic:acl-test`: 
        User:writer has Allow permission for operations: Write from hosts: * 

Current ACLs for resource `Topic:acl-test`: 
        User:writer has Allow permission for operations: Write from hosts: * 

[root@iZbp14ouog5ocoeakj39prZ kafka_2.11-0.11.0.1]# 

再通过bin/kafka-console-producer.sh测试

[root@iZbp14ouog5ocoeakj39prZ kafka_2.11-0.11.0.1]# bin/kafka-console-producer.sh --topic acl-test --broker-list 120.55.125.58:9093,120.26.198.248:9093,121.40.200.37:9093 --producer.config config/producer-jaas.properties
>fdf
>dfdfdf
>

可以正常写入，请注意--broker-list 120.55.125.58:9093,120.26.198.248:9093,121.40.200.37:9093  ip为 sasl-server.properties中配置的listeners中配置的ip，端口号为9093

四、配置consumer acl configuration

4.1 创建consumer_jaas.conf

[root@iZbp14ouog5ocoeakj39prZ config]# vim consumer_jaas.conf 

"consumer_jaas.conf" 5L, 121C                                                                                     1,1           All
KafkaClient{
        org.apache.kafka.common.security.plain.PlainLoginModule required
        username="reader"
        password="reader";
};

4.2创建consumer-jaas.properties

[root@iZbp14ouog5ocoeakj39prZ config]# vim consumer-jaas.properties
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
group.id=hw-group

4.3修改kafka-console-consumer.sh 

将

exec $(dirname $0)/kafka-run-class.sh kafka.tools.ConsoleConsumer "$@"

修改为

exec $(dirname $0)/kafka-run-class.sh -Djava.security.auth.login.config=/usr/local/kafka/kafka_2.11-0.11.0.1/config/consumer_jaas.co
nf  kafka.tools.ConsoleConsumer "$@"

4.4对user reader进行bin/acls.sh赋权，给与topic acl-test的read权限 group.id hw-group的read权限，

[root@iZbp14ouog5ocoeakj39prZ kafka_2.11-0.11.0.1]# bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=10.174.32.122:2181,10.117.15.224:2181,10.168.96.198:2181 --add --allow-principal User:reader --operation Read --topic acl-test
Adding ACLs for resource `Topic:acl-test`: 
        User:reader has Allow permission for operations: Read from hosts: * 

Current ACLs for resource `Topic:acl-test`: 
        User:writer has Allow permission for operations: Write from hosts: *
        User:reader has Allow permission for operations: Read from hosts: * 

[root@iZbp14ouog5ocoeakj39prZ kafka_2.11-0.11.0.1]# bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=10.174.32.122:2181,10.117.15.224:2181,10.168.96.198:2181 --add --allow-principal User:reader --operation Read --group hw-group
Adding ACLs for resource `Group:hw-group`: 
        User:reader has Allow permission for operations: Read from hosts: * 

Current ACLs for resource `Group:hw-group`: 
        User:reader has Allow permission for operations: Read from hosts: * 

[root@iZbp14ouog5ocoeakj39prZ kafka_2.11-0.11.0.1]# 

4.5对 user reader进行测试读取数据

[root@iZbp14ouog5ocoeakj39prZ kafka_2.11-0.11.0.1]# bin/kafka-console-consumer.sh --topic acl-test --bootstrap-server 120.55.125.58:9093,120.26.198.248:9093,121.40.200.37:9093 --consumer.config config/consumer-jaas.properties --from-beginning
fdf
jfdsfdskfjdlkf
dfjkdsjfdsf
dfdfdf
kfkdsjfkjdslfjs
djfdjfdf
dfdfdsfd
fdksjfkdsjfds