kafka-stream在Jeesuite框架中的使用

上一篇Jeesuite框架是如何消费Kafka消息的中提到Jeesuite中消费者启动会读取扫描标注了@ConsumerHandler的类,处理完会存放在Map中,现在看一下Kafka-stream是怎么在Jeesuite中应用.

在OldApiTopicConsumer.start()方法中
@Override
public void start() {
		//重置offset
		if(consumerContext.getOffsetLogHanlder() != null){	
			resetCorrectOffsets();
		}
		Map topicCountMap = new HashMap();
		for (String topicName : consumerContext.getMessageHandlers().keySet()) {
			int nThreads = 1;
			topicCountMap.put(topicName, nThreads);
			logger.info("topic[{}] assign fetch Threads {}",topicName,nThreads);
		}
		
		StringDecoder keyDecoder = new StringDecoder(new VerifiableProperties());
		MessageDecoder valueDecoder = new MessageDecoder(deserializer);

		Map>> consumerMap = this.connector.createMessageStreams(topicCountMap,
				keyDecoder, valueDecoder);

		for (String topicName : consumerContext.getMessageHandlers().keySet()) {
			final List> streams = consumerMap.get(topicName);

			for (final KafkaStream stream : streams) {
				MessageProcessor processer = new MessageProcessor(topicName, stream);
				this.fetchExecutor.execute(processer);
			}
		}
		//
		runing.set(true);
	}

 变量topicCountMap就是以topic为key,value为固定值1作为线程数,我把它改造一下就是

    @Override
    public void start() {
        ConsumerConnector connector = kafka.consumer.Consumer.createJavaConsumerConnector(new ConsumerConfig(context.getProperties()));
        Class deserializerClass = Class.forName(context.getProperties().getProperty("value.deserializer"));
        Deserializer deserializer = (Deserializer) deserializerClass.newInstance();
        StringDecoder keyDecoder = new StringDecoder(new VerifiableProperties());
        MessageDecoder valueDecoder = new MessageDecoder(deserializer);
        //1.组成HashMap();,key是主题,value是线程数,如果线程数是1,则List>大小也是1
        Map topicCountMap = new HashMap();
        Map messageHandlers = 上一篇中扫描的@ConsumerHandler
        for (String topicName : consumerContext.getMessageHandlers().keySet()) {
            int nThreads = 1;
            topicCountMap.put(topicName, nThreads);
        }
        //2.调用createMessageStreams()创建Stream
        Map>> consumerMap = connector.createMessageStreams(topicCountMap,
                keyDecoder, valueDecoder);
        //3.循环topic,取出List>,再执行处理消息任务.当线程数为1是,只要单层for循环即可
        for (String topicName : messageHandlers .keySet()) {
            final List> streams = consumerMap.get(topicName);
            for (final KafkaStream stream : streams) {
                MessageProcessor processer = new MessageProcessor(topicName, stream);
                //使用线程池执行线程任务
                this.fetchExecutor.execute(processer);
            }
        }
    } 
  

这个就是kafka-stream的使用了. 


 

你可能感兴趣的:(Kafka,kafka)