kafka配置sasl/plain安全认证及librdkafka1.3.0消费者

kafka配置sasl/plain安全认证

1.SASL认证机制版本支持

SASL/GSSAPI (Kerberos) - starting at version 0.9.0.0
SASL/PLAIN - starting at version 0.10.0.0
SASL/SCRAM-SHA-256 and SASL/SCRAM-SHA-512 - starting at version 0.10.2.0

2.配置zookeeper(不配置也可用,暂时不需要)

1)修改zoo.cfg增加两行配置:

authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
requireClientAuthScheme=sasl

2)配置JAAS文件:conf目录下创建zk_server_jaas.conf(定义了需要链接到Zookeeper服务器的用户名和密码)

Server {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="admin"
password="admin-sec";
};

3)加入需要的包:(从kafka下拷贝)

kafka-clients-0.10.0.1.jar
lz4-1.3.0.jar
slf4j-api-1.7.21.jar
slf4j-log4j12-1.7.21.jar
snappy-java-1.1.2.6.jar

3)修改zkEnv.sh
最后一行添加

export SERVER_JVMFLAGS=" -Djava.security.auth.login.config=/usr/local/zookeeper/conf/zk_server_jaas.con"

4)启动Zookeeper

3.配置kafka服务端

1)kafka增加认证信息:conf/kafka_server_jaas.conf

创建JAAS文件:

KafkaServer {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="admin"
password="admin-sec"
user_admin="admin-sec"
user_producer="prod-sec"
user_consumer="cons-sec";
};

2)配置server.properties

listeners=SASL_PLAINTEXT://主机名称:9092
security.inter.broker.protocol=SASL_PLAINTEXT
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
allow.everyone.if.no.acl.found=true //当没有找到ACL配置时,允许所有的访问操作。

3)修改启动脚本 bin/kafka-server-start.sh
修改

exec $base_dir/kafka-run-class.sh kafka.Kafka "$@"

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS -Djava.security.auth.login.config=/usr/local/kafka/config/kafka_server_jaas.conf kafka.Kafka "$@"

4.配置kafka客户端

1)创建JAAS文件:

消费者:conf/kafka-consumer-jaas.conf

KafkaClient {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="consumer"
password="cons-sec";
};

生产者:conf/kafka-producer-jaas.conf

KafkaClient {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="producer"
password="prod-sec";
};

2)修改客户端配置信息:

分别在conf/producer.properties和conf/consumer.properties添加认证机制

security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
consumer.properties中额外加入分组配置
group.id=test-group

3)修改客户端脚本指定JAAS文件加载:

生产者bin/kafka-console-producer.sh:
修改

exec $(dirname $0)/kafka-run-class.sh kafka.tools.ConsoleProducer "$@"

exec $(dirname $0)/kafka-run-class.sh -Djava.security.auth.login.config=/usr/local/kafka/config/kafka-producer-jaas.conf kafka.tools.ConsoleProducer "$@"

消费者bin/kafka-console-consumer.sh:
修改

exec $(dirname $0)/kafka-run-class.sh kafka.tools.ConsoleConsumer "$@"

exec $(dirname $0)/kafka-run-class.sh -Djava.security.auth.login.config=/usr/local/kafka/config/kafka-consumer-jaas.conf kafka.tools.ConsoleConsumer "$@"

5.进行授权

1)创建主题 test
2)增加生产权限

./bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:producer --operation Write --topic test

3)配置消费权限

./bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:consumer --operation Read --topic test

4)配置消费分组权限

./bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:consumer --operation Read --group test-group

5)查看配置的权限

./bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=localhost:2181 --list

6)取消权限

./bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=localhost:2181 --remove --allow-principal User:producer --operation Write --topic test

测试
1)生产数据

	./bin/kafka-console-producer-jaas.sh --topic test --broker-list 192.168.1.20:9092 --producer.config config/producer-jaas.properties

2)消费数据

	./bin/kafka-console-consumer-jaas.sh --topic test --bootstrap-server 192.168.1.20:9092 --consumer.config config/consumer-jaas.properties

librdkafka1.3.0消费者

下载安装librdkafka1.3.0压缩包。
在0.9.0消费者基础上添加:

	pConf->set("security.protocol", “sasl_plaintext”, strErr);
	pConf->set("sasl.mechanism", “PLAIN”, strErr);
	pConf->set("sasl.username",“username”, strErr);
	pConf->set("sasl.password", “password”, strErr);

你可能感兴趣的:(kafka,kafka)