spring+springmvc+kafka分布式消息中间件集成方案

Honghu的消息服务平台已经抛弃了之前的ActiveMQ，改用高吞吐量比较大的Kafka分布式消息中间件方案：

kafka消息平台使用spring+kafka的集成方案，详情如下：
1. 使用最高版本2.1.0.RELEASE集成jar包：spring-integration-kafka

2. Zookeeper、Kafka分布式集群使用init.properties配置化方案。

kafka.servers=127.0.0.1:9092
kafka.topic=xxxooo

kafka.servers=127.0.0.1:9092  
kafka.topic=xxxooo

 
3. 使用消息生产者spring-context-producer配置化方案。

"1.0" encoding="UTF-8"?>
"http://www.springframework.org/schema/beans"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xmlns:context="http://www.springframework.org/schema/context"
     xsi:schemaLocation="http://www.springframework.org/schema/beans
         http://www.springframework.org/schema/beans/spring-beans.xsd
         http://www.springframework.org/schema/context
         http://www.springframework.org/schema/context/spring-context.xsd">



    "producerProperties" class="java.util.HashMap">


                "bootstrap.servers" value="localhost:9092" />
                "group.id" value="2" />
                "retries" value="10" />
                "batch.size" value="16384" />
                "linger.ms" value="1" />
                "buffer.memory" value="33554432" />
                "key.serializer"
                    value="org.apache.kafka.common.serialization.IntegerSerializer" />
                "value.serializer"
                    value="org.apache.kafka.common.serialization.StringSerializer" />





    "producerFactory"
        class="org.springframework.kafka.core.DefaultKafkaProducerFactory">

            "producerProperties" />




    "KafkaTemplate" class="org.springframework.kafka.core.KafkaTemplate">
        "producerFactory" />
        "autoFlush" value="true" />
        "defaultTopic" value="test" />

 
4. 使用消息消费者spring-context-producer配置化方案。

"1.0" encoding="UTF-8"?>
"http://www.springframework.org/schema/beans"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xmlns:context="http://www.springframework.org/schema/context"
     xsi:schemaLocation="http://www.springframework.org/schema/beans
         http://www.springframework.org/schema/beans/spring-beans.xsd
         http://www.springframework.org/schema/context
         http://www.springframework.org/schema/context/spring-context.xsd">



    "consumerProperties" class="java.util.HashMap">


                "bootstrap.servers" value="localhost:9092" />
                "group.id" value="0" />
                "enable.auto.commit" value="true" />
                "auto.commit.interval.ms" value="1000" />
                "session.timeout.ms" value="15000" />
                "key.deserializer"
                    value="org.apache.kafka.common.serialization.IntegerDeserializer" />
                "value.deserializer"
                    value="org.apache.kafka.common.serialization.StringDeserializer" />





    "consumerFactory"
        class="org.springframework.kafka.core.DefaultKafkaConsumerFactory">

            "consumerProperties" />




    "messageListernerConsumerService" class="com.sml.sz.kafka.KafKaConsumer" />


    "containerProperties"
        class="org.springframework.kafka.listener.config.ContainerProperties">
        "test" />
        "messageListener" ref="messageListernerConsumerService" />



    "messageListenerContainer"
        class="org.springframework.kafka.listener.KafkaMessageListenerContainer"
        init-method="doStart">
        "consumerFactory" />
        "containerProperties" />

 
5. 使用注解方式注入消息类型

@Autowired  

private KafkaTemplate kafkaTemplate;

6. 重写MessageListener 的getMessage方法获取消息（业务实现）

7. RestFul服务方式测试消息服务

@CrossOrigin(origins = "*", maxAge = 3600, methods = { RequestMethod.GET, RequestMethod.POST, RequestMethod.DELETE,
        RequestMethod.PUT })
@RestController
@RequestMapping(value = "/rest/kafka")
public class KafKaProducer {

    @RequestMapping(value = "/send", method = RequestMethod.GET)
    public JSONObject save() {
        System.out.println("+++++++++++++++++++++++++++++++");
        kafkaTemplate.sendDefault("HongHu KAFKA分布式消息服务测试");
        return null;
    }

    @Autowired
    private KafkaTemplate kafkaTemplate;



}

@CrossOrigin(origins = "*", maxAge = 3600, methods = { RequestMethod.GET, RequestMethod.POST, RequestMethod.DELETE,
		RequestMethod.PUT })
@RestController
@RequestMapping(value = "/rest/kafka")
public class KafKaProducer {
	
	@RequestMapping(value = "/send", method = RequestMethod.GET)
	public JSONObject save() {
		System.out.println("+++++++++++++++++++++++++++++++");
		kafkaTemplate.sendDefault("HongHu KAFKA分布式消息服务测试");  
		return null;
	}
	
	@Autowired  
    private KafkaTemplate kafkaTemplate;
	
  

}

 

@RestController
public class KafKaConsumer implements MessageListener {

    @Autowired
    private LogService logService;
    public void onMessage(ConsumerRecord records) {
        System.out.println("====================" + records);
        Object o = records.value();
        Log log = new Log();
        log.setIsNewRecord(true);
        log.setId(IdGen.uuid());
        log.setTitle(String.valueOf(o));
        logService.save(log);

    }

}

@RestController
public class KafKaConsumer implements MessageListener {

	@Autowired
	private LogService logService;
	public void onMessage(ConsumerRecord records) {
		System.out.println("====================" + records);
		Object o = records.value();
		Log log = new Log();
		log.setIsNewRecord(true);
		log.setId(IdGen.uuid());
		log.setTitle(String.valueOf(o));
		logService.save(log);

	}

}

 

愿意了解框架技术或者源码的朋友直接求求交流分享技术：3133806896


分布式的一些解决方案，有愿意了解的朋友可以找我们团队探讨


更多详细源码参考来源