Apache Storm流处理有序性探究

本文假设读者已经对Storm的基础结构有了全面理解,并知道Nimbus与supervisor在集群之中所扮演的角色。之所以要理解Storm集群的并行机制,是为了能够对数据流中数据地处理顺序有一个深入地理解,这样才能更有信心地使用工具。


首先是需要了解一些与Storm集群并行机制相关地概念:

工作进程(worker process,就是一个JVM进程,通过在supervisor服务器上执行jps命令可以看到一个或多个的worker)的隔离性

一个Storm集群可以运行多个Topologies,但是对于一个worker来说,它只为一个Topology工作,这在一定程度上提供了Topologies之间的隔离性。


执行器(executor)

在工作进程JVM的的内部,可以启动多个线程来一起工作,每个线程就是一个执行器(executor),一个执行器只能执行拓扑中的一个组件,要么是一个Spout,要么是一个Bolt。这里,我们看到一个工作进程内部,某些线程可以执行Spout任务,而另外一些线程可以执行Bolt任务。


任务(tasks)

任务是Storm集群中最细粒度的划分,每个任务要么是一个Spout任务,要么是一个Bolt任务,再结合上面对于执行器的说明,我们可以看到这些任务被分配到一些执行器中去执行。我们在定义拓扑的时候,会定义好每个Spout和每个Bolt任务的数量,一旦定义好之后,是不能在拓扑运行的过程中动态变更相应组件的任务数的。


要理解Storm的并行机制,只需要理解Storm数据流中元组的处理顺序即可。为了更与生产环境接近,使用Maven作为处理包依赖的处理工具,并通过其maven-shade-plugin插件,将需要的jar包依赖共同打包到最终的Storm任务jar包中,而不用对Storm的运行环境做环境变量做处理,这确实带来了非常大的便利。首先,需要通过配置POM文件明确各种包的依赖:


  4.0.0

  com.example
  istorm
  0.0.1-SNAPSHOT
  jar

  istorm
  http://maven.apache.org

  
    UTF-8
  

  
    
      junit
      junit
      3.8.1
      test
    
    
    
		org.apache.storm
		storm-kafka
		0.9.5
		compile
	
	
	
		org.apache.storm
		storm-core
		0.9.5
		provided
	

	
		redis.clients
		jedis
		2.7.3
	
	
	
		org.apache.kafka
		kafka_2.11
		0.8.2.1
		compile
		
	        
	            org.slf4j
	            slf4j-log4j12
	        
	        
	            log4j
	            log4j
	        
	    
	
  
  
  
	
		
			org.apache.maven.plugins
			maven-shade-plugin
			
				
					package
					
						shade
					
				
			
			
				withdepends-${artifactId}-${version}
			
		
	
    


在刚刚建立好Maven任务之后,需要下载所有的依赖包,这可能需要一些时间。有了这些依赖之后,下一步就要规划工程结构。为了更能够与生产环境接近,我们将Storm的数据源设定为Kafka,Storm和Kafka相关的依赖包已经在上面的POM文件中有所体现了,至于Kafka集群的搭建和使用,在此不做细致的说明。整个工程的结构是这样的:我们通过消费者向Kafka集群的一个TOPIC发送一系列的顺序的数字,1,2,3,4...N,而Storm(KafkaSpout)会读取这些数据,将其交付给SequenceBolt,它负责为序列数字添加上一些线程与任务相关的信息后,将其发送给PrinterBolt,这个PrinterBolt负责将收集到的数据发送给Kafka集群中的一个结果队列,我们能够从终端中读取这个这个队列中的结果,以观察最终的序列被处理的顺序。为了更能说明问题,我们的Topology能够通过参数来控制总的worker数量以及各个组件的tasks的数量,这在下面的代码示例中都会有所体现。


首先,是SequenceBolt的实现代码:

package com.example.istorm;

import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.Map;
import java.util.Random;

import org.apache.commons.lang.StringUtils;

import com.google.common.collect.ImmutableList;

import backtype.storm.task.TopologyContext;
import backtype.storm.topology.BasicOutputCollector;
import backtype.storm.topology.OutputFieldsDeclarer;
import backtype.storm.topology.base.BaseBasicBolt;
import backtype.storm.tuple.Fields;
import backtype.storm.tuple.Tuple;

public class SequenceBolt extends BaseBasicBolt {
	
	private static final long serialVersionUID = -5735813963113079832L;
	
	private int taskId;
	private Random rand;
	
	@Override
	public void prepare(Map stormConf, TopologyContext context) {
		this.taskId = context.getThisTaskId();
		this.rand = new Random();
	}
	
	public void execute(Tuple input, BasicOutputCollector collector) {
		String data = input.getString(0);
		String timestamp = new Date().toString();
		List dataList = new ArrayList();
		dataList.add("sequenceBoltThreadID:" + Thread.currentThread().getId() + "@" + timestamp + "@sequnceTaskID:" + this.taskId);
		dataList.add(data);
		// simulate the work latency
		try {
			Thread.sleep(this.rand.nextInt(47));
		} catch (InterruptedException e) {
			e.printStackTrace();
		}
		collector.emit(ImmutableList.of((Object) StringUtils.join(dataList, '\t')));
	}
	
	public void declareOutputFields(OutputFieldsDeclarer declarer) {
		declarer.declare(new Fields("sequence"));
	}
	
	public static void main(String[] args) throws InterruptedException{
		List dataList = new ArrayList();
		dataList.add("100");
		dataList.add("200");
		System.out.println(StringUtils.join(dataList, '\t'));
		System.out.println(ImmutableList.of((Object) StringUtils.join(dataList, '\t')));
		System.out.println("Sleep start");
		Random rand = new Random();
		Thread.sleep(rand.nextInt(1000));
		System.out.println("Sleep end");
	}
}

这个Bolt的逻辑非常简单,仅仅是在接收到序列数据之后,在其上加上了事件戳、线程信息、任务信息之后,直接提交这个数据。这么简单的逻辑,相信理解起来非常简单。下面是更简单的PrinterBolt,它仅仅负责将过来的数据发送到Kafka的结果队列之中:

package com.example.istorm;

import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.Map;

import org.apache.commons.lang.StringUtils;

import com.example.utils.KafkaUtils;

import backtype.storm.task.TopologyContext;
import backtype.storm.topology.BasicOutputCollector;
import backtype.storm.topology.OutputFieldsDeclarer;
import backtype.storm.topology.base.BaseBasicBolt;
import backtype.storm.tuple.Tuple;

public class PrinterBolt extends BaseBasicBolt {
	private static final long serialVersionUID = -6508964903665304388L;
	
	// Every printer bolt task will hold a kafkaUtils object that will be used to
	// post the result to the Kafka servers
	private KafkaUtils kafkaUtils = null;
	private int taskId;
	
	@Override
	public void prepare(Map stormConf, TopologyContext context) {
		this.kafkaUtils = new KafkaUtils("resulttopic"); // we will send result to this topic
		this.taskId = context.getThisTaskId();
    }
	
    @Override
    public void cleanup() {
    	if(this.kafkaUtils != null){
    		this.kafkaUtils.close();
    	}
    }    	

	public void execute(Tuple input, BasicOutputCollector collector) {
		String sequence = input.getStringByField("sequence");
		String timestamp = new Date().toString();
		
		List dataList = new ArrayList();
		dataList.add("printerBoltThreadID:" + Thread.currentThread().getId() + "@" + timestamp + "@printerTaskID:" + this.taskId);
		dataList.add(sequence);
		this.kafkaUtils.publishMessage(StringUtils.join(dataList, '\t'));
	}

	public void declareOutputFields(OutputFieldsDeclarer arg0) {
		// we don't emit anything
	}

}

在这个程序中的prepare方法中,创建了Kafka的工具类,用来向Kafka的某个TOPIC发送消息,这个工具类的代码也只有简单几行而已:

package com.example.utils;

import java.util.Properties;

import kafka.javaapi.producer.Producer;
import kafka.producer.KeyedMessage;
import kafka.producer.ProducerConfig;

public class KafkaUtils {

        private Producer producer;
        private String topic;

        public KafkaUtils(String topic){
                Properties props = new Properties();
                props.put("metadata.broker.list", "a00:9092,a01:9092,a02:9092");
                props.put("serializer.class", "kafka.serializer.StringEncoder");
                props.put("partitioner.class", "com.example.utils.SimplePartitioner");
                props.put("request.required_acks", "1");
                ProducerConfig config = new ProducerConfig(props);
                this.producer = new Producer(config);
                this.topic = topic;
        }

        public void publishMessage(String msg){
                KeyedMessage data = new KeyedMessage(this.topic, msg, msg);
                this.producer.send(data);
        }
        
        public void close(){
        	this.producer.close();
        }
        
        public static void main(String[] args){
            String topic = "datasrctopic";
            KafkaUtils sp = new KafkaUtils(topic);
            
            for(int i=0; i < 100; i++){
            	sp.publishMessage(Integer.toString(i));
            }
            sp.close();
        }
}

其中的main方法,实现了向Kafka中发送数据的功能,我们以后的测试数据就是通过执行这个方法获取的。为了更能说明问题,我实现了一个Kafka简单的分区函数,其实现也是非常简单的,其代码也是最简短的:

package com.example.utils;

import kafka.producer.Partitioner;
import kafka.utils.VerifiableProperties;

public class SimplePartitioner implements Partitioner {

        public SimplePartitioner(VerifiableProperties props){

        }

        public int partition(Object key, int a_numPartitions) {
                String partitionKey = (String) key;
                return partitionKey.hashCode() % a_numPartitions;
        }

}

这个代码,我们期望能够将消息分散到Kafka相应TOPIC中不同的分区中。最后,是将整个Topology初始化:


package com.example.istorm;

import backtype.storm.spout.SchemeAsMultiScheme;
import backtype.storm.topology.TopologyBuilder;

import java.util.ArrayList;
import java.util.List;

import backtype.storm.Config;
//import backtype.storm.LocalCluster;
import backtype.storm.StormSubmitter;
import backtype.storm.generated.AlreadyAliveException;
import backtype.storm.generated.InvalidTopologyException;
import storm.kafka.KafkaSpout;
import storm.kafka.SpoutConfig;
import storm.kafka.StringScheme;
import storm.kafka.ZkHosts;

public class KafkaTopology {
	public static void main(String[] args) throws AlreadyAliveException, InvalidTopologyException{
		// Zookeeper hosts for the Kafka cluster
		ZkHosts zkHosts = new ZkHosts("a00:2181,a01:2181,a02:2181");
		// Create the KafkaSpout configration
		// Second argument is the topic name
		// Third argument is the Zookeeper root for Kafka
		// Fourth argument is consumer group id
		SpoutConfig kafkaConfig = new SpoutConfig(zkHosts, "datasrctopic","/tmp", "id7");
		
		int workerNum = Integer.parseInt(args[0]);
		int kafkaSpoutNum = Integer.parseInt(args[1]);
		int kafkaSpoutTaskNum = Integer.parseInt(args[2]);
		int sequenceBoltNum = Integer.parseInt(args[3]);
		int sequenceBoltTaskNum = Integer.parseInt(args[4]);
		int printerBoltNum = Integer.parseInt(args[5]);
		int printerBoltTaskNum = Integer.parseInt(args[6]);
		
//		final List zkServerAddresses = new ArrayList();
//		zkServerAddresses.add("a00");
//		zkServerAddresses.add("a01");
//		zkServerAddresses.add("a02");
//		kafkaConfig.zkServers = zkServerAddresses;
//		kafkaConfig.zkPort = 2181;
		
		// Specify that the kafka messages are String
		kafkaConfig.scheme = new SchemeAsMultiScheme(new StringScheme());
		// We want to consume all the first messages in
		// the topic every time we run the topology to
		// help in debugging. In production, this property should be false
		kafkaConfig.forceFromStart = false;
		// Now we create the topology
		TopologyBuilder builder = new TopologyBuilder();
		// Set the kafka spout class
		builder.setSpout("KafkaSpout", new KafkaSpout(kafkaConfig), kafkaSpoutNum)
		.setNumTasks(kafkaSpoutTaskNum);
		// Configure the bolts
		builder.setBolt("SequenceBolt", new SequenceBolt(), sequenceBoltNum)
		.setNumTasks(sequenceBoltTaskNum)
		.globalGrouping("KafkaSpout");
		
		builder.setBolt("PrinterBolt", new PrinterBolt(), printerBoltNum)
		.setNumTasks(printerBoltTaskNum)
		.globalGrouping("SequenceBolt");
		
		Config conf = new Config();
		conf.setNumWorkers(workerNum);
		StormSubmitter.submitTopology("KafkaTopology", conf, builder.createTopology());
		
		/*
		// Create an instance of LocalCluster class
		// for executing in local mode.
		LocalCluster cluster = new LocalCluster();
		
		// Submit topology for execution
		cluster.submitTopology("KafkaTopology", conf, builder.createTopology());
		try{
			// Wait for some time before exiting
			Thread.sleep(100000);
		} catch(Exception exception){
			System.out.println("Thread interrupted exception: " + exception);
		}
		// Kill the KafkaTopology
		cluster.killTopology("KafkaTopology");
		// Shutdown the storm test cluster
		cluster.shutdown();
		*/
	}
}


这里需要注意的一点是,如果Storm是运行在LocalCluster的模式下,那么我们必须显式指定如下的代码,才能让Storm-Kafka记住上次的偏移量到Zookeeper服务中:

		final List zkServerAddresses = new ArrayList();
		zkServerAddresses.add("a00");
		zkServerAddresses.add("a01");
		zkServerAddresses.add("a02");
		kafkaConfig.zkServers = zkServerAddresses;
		kafkaConfig.zkPort = 2181;

当然,如果是在集群模式下运行,就不需要这几行代码了。在我们的Topology的配置中,我们使用了globalGrouping作为分发数据的方法,这种分发数据流的方法并不会将数据流分区,而是将整个数据流完整地分发到具有最小任务ID地BOLT中,有点儿类似于MapReduce中的Reduce阶段的数据流。

场景1:数据源的顺序确定。

对于数据源的顺序,我们经常会忽视。对于像Kafka这样的数据源,我们就不得不考虑其分区特性对于Storm数据源输入顺序的影响:Kafka在每个分区上,通过顺序偏移量的机制为所有消息提供了强的顺序保证,但是在不同的分区之间并没有这种顺序的保证。为了验证这个想法,我们在Kafka的数据源队列的TOPIC创建时,这样创建:

bin/kafka-topics.sh --create --zookeeper a00:2181,a01:2181,a02:2181 --replication-factor 1 --partitions 10 --topic datasrctopic

数据源TOPIC创建时,指定其有10个分区,我们期待Storm在读取这个TOPIC的数据时,总体上是无序的(虽然每个分区内的消息时有序的)。而最终的结果TOPIC,为了消除多分区对输出顺序的影响,我们仅仅指定其有1个分区:

bin/kafka-topics.sh --create --zookeeper a00:2181,a01:2181,a02:2181 --replication-factor 1 --partitions 1 --topic resulttopic

在有了如上的准备之后,我们就可以提交Topology到Storm集群上了:

bin/storm jar /tmp/withdepends-istorm-0.0.1-SNAPSHOT.jar com.example.istorm.KafkaTopology 1 1 1 1 1 1 1

为了消除其它影响,我们将worker数量以及各个组件的执行器数量和任务数量都设置为1,这样整体的顺序就取决于数据源的顺序了。这个时候,我们去执行一下KafkaUtils中的main函数,我们从结果队列中得到如下的输出:

printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	0
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	2
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	1
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	4
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	7
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	5
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	8
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	11
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	3
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	12
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	13
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	15
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	18
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	6
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	17
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	14
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	16
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	9
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	10
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	21
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	19
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	20
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	22
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	24
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	23
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	25
printerBoltThreadID:31@Mon Sep 14 15:31:01 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	28
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:01 CST 2015@sequnceTaskID:3	27
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	26
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	29
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	30
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	31
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	33
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	36
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	35
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	32
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	34
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	38
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	37
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	39
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	40
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	42
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	45
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	44
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	41
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	43
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	47
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	46
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	48
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	50
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	49
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	51
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	52
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	53
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	55
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	54
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	56
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	57
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	58
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	59
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	60
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	61
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	63
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	62
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	65
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	67
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	64
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	66
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	68
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	71
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	69
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	70
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	73
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	75
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	72
printerBoltThreadID:31@Mon Sep 14 15:31:02 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	74
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:02 CST 2015@sequnceTaskID:3	76
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	77
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	79
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	78
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	82
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	81
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	80
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	83
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	84
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	87
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	85
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	86
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	88
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	90
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	89
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	91
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	92
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	93
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	94
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	95
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	97
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	96
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	98
printerBoltThreadID:31@Mon Sep 14 15:31:03 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:31:03 CST 2015@sequnceTaskID:3	99

明显看到,整个输出时无序的。这个时候的无序完全是由于Kafka不能在多个分区上保持消息的有序性造成的,那么如果要让消息有序有两种方法,一种方法就是修改Kafka的分区函数,将其全部消息发向同一个分区;第二种方法,就是在创建TOPIC的时候,指定只有1个分区。当然,作为测试,我们采用第二种方法,在实际应用下,可能要利用第一种方法,采取某种Kafka的分区策略,比如按照用户的uid分区到Kafka的各个分区上,保证每个用户产生的消息是有序的。采用第二种方法就是简单地重建TOPIC:

bin/kafka-topics.sh --delete --zookeeper a00:2181,a01:2181,a02:2181 --topic datasrctopic
bin/kafka-topics.sh --create --zookeeper a00:2181,a01:2181,a02:2181 --replication-factor 1 --partitions 1 --topic datasrctopic

在做好如上地处理之后,我们再执行先前地程序,最终地输出就成为:

printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	0
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	1
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	2
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	3
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	4
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	5
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	6
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	7
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	8
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	9
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	10
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	11
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	12
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	13
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	14
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	15
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	16
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	17
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	18
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	19
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	20
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	21
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	22
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	23
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	24
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	25
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	26
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	27
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	28
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	29
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	30
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	31
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	32
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	33
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	34
printerBoltThreadID:31@Mon Sep 14 15:40:42 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	35
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:42 CST 2015@sequnceTaskID:3	36
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	37
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	38
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	39
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	40
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	41
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	42
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	43
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	44
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	45
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	46
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	47
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	48
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	49
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	50
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	51
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	52
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	53
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	54
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	55
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	56
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	57
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	58
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	59
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	60
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	61
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	62
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	63
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	64
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	65
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	66
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	67
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	68
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	69
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	70
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	71
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	72
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	73
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	74
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	75
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	76
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	77
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	78
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	79
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	80
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	81
printerBoltThreadID:31@Mon Sep 14 15:40:43 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	82
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:43 CST 2015@sequnceTaskID:3	83
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	84
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	85
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	86
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	87
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	88
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	89
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	90
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	91
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	92
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	93
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	94
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	95
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	96
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	97
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	98
printerBoltThreadID:31@Mon Sep 14 15:40:44 CST 2015@printerTaskID:2	sequenceBoltThreadID:33@Mon Sep 14 15:40:44 CST 2015@sequnceTaskID:3	99

正如,我们地预期,输出确实变成了顺序输出。这样就提醒我们,在需要顺序处理某些数据地时候,一定不能忽视数据源的顺序,具体到Kafka,就是采用合适的分区策略,利用单个分区的有序特性来达成自己的目的。


场景2:探究Global Grouping的特性。

Global Grouping不会对数据流做分区处理,而是将整个数据流完整发送到具有最小任务ID的BOLT任务上,也就是说,不论一个BOLT开几个任务,只有一个任务会获取数据流,也就是说,在我们的示例中,不论我们开启几个BOLT任务,我们得到也总是有序的输出。为测试,我们的想法,我们将以如下的方式来提交Topology到Storm集群中:

bin/storm jar /tmp/withdepends-istorm-0.0.1-SNAPSHOT.jar com.example.istorm.KafkaTopology 2 2 2 2 2 2 2

这样之后,我们向数据源TOPIC中发送数据,得到的输出仍旧是有序的(为了避免篇幅冗余,不再贴具体输出)。多个worker去读取数据的时候,也是要参照Zookeeper中存储的偏移量的,所以即使是多个worker去读Kafka队列(因为只有一个分区)中的数据,得到的数据也是有序的,并不会在数据读取时发生顺序上的变动。


场景3:探究Shuffle Grouping的特性。

Shuffle Grouping的分发方式是把数据流中的元组随机地在任务之间分发,既然这里提到了随机,那么就有可能出现失序地问题。但是,这里必须十分注意执行器和任务之间地关系。执行器就是一个独立地线程,而任务就是一个Bolt地实例对象,而一个执行器可以执行多个任务。如果每个Bolt不管有多少个任务,但是只有一个执行器来执行它们,那么理论上是不会失序的,那么实际上是怎样的呢?为了验证,我们理论上的想法,可以在启动Topology的时候,将每个Bolt的执行器都设置成1,而任务设置成多个,然后再做测试(不要忘了将Grouping改成Shuffle Grouping,并重新打包):

bin/storm jar /tmp/withdepends-istorm-0.0.1-SNAPSHOT.jar com.example.istorm.KafkaTopology 2 2 2 1 10 1 10

输出如下:

printerBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:13	0
printerBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:21	1
printerBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:15	2
printerBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:17	3
printerBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:14	4
printerBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:19	5
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:22	6
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:16	7
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:18	8
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:09 CST 2015@sequnceTaskID:20	9
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:13	10
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:21	11
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:19	12
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:18	13
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:17	14
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:16	15
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:20	16
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:15	17
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:22	18
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:14	19
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:17	20
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:22	21
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:13	22
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:15	23
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:19	24
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:20	25
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:21	26
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:14	27
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:16	28
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:18	29
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:16	30
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:13	31
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:22	32
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:18	33
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:14	34
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:21	35
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:20	36
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:15	37
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:19	38
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:17	39
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:14	40
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:16	41
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:13	42
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:20	43
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:17	44
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:18	45
printerBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:22	46
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:10 CST 2015@sequnceTaskID:19	47
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:21	48
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:15	49
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:17	50
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:21	51
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:16	52
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:22	53
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:13	54
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:15	55
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:20	56
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:19	57
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:18	58
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:14	59
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:22	60
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:16	61
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:19	62
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:18	63
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:17	64
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:13	65
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:15	66
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:20	67
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:21	68
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:14	69
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:17	70
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:13	71
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:20	72
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:18	73
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:19	74
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:21	75
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:14	76
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:15	77
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:16	78
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:22	79
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:14	80
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:21	81
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:13	82
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:22	83
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:16	84
printerBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:18	85
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:11 CST 2015@sequnceTaskID:20	86
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:15	87
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:19	88
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:17	89
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:17	90
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:20	91
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:19	92
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:15	93
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:16	94
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:18	95
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:21	96
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:22	97
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:14	98
printerBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:21:12 CST 2015@sequnceTaskID:13	99

可以看到,数据流中的元组被平均分配到了不同的任务中,但是输出依旧是有序的。那么继续我们的思路,如果要让输出失序,只要将其中一个BOLT的执行器设置成2个即可:

bin/storm jar /tmp/withdepends-istorm-0.0.1-SNAPSHOT.jar com.example.istorm.KafkaTopology 2 2 2 2 10 1 10

这里,我们将SequenceBolt的执行器设置成了2个,其对应的输出是:

printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:18	2
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:20	3
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:21	4
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:16	0
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:14	1
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:19	7
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:15	5
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:13	6
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:22	8
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:17	9
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:22	10
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:19	12
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:13	11
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:20	15
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:18	17
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:16	13
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:15	14
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:21	18
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:14	16
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:19	20
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:22	21
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:17	19
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:18	24
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:21	26
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:13	22
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:20	29
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:14	23
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:15	25
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:19	30
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:20	32
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:16	27
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:21	34
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:17	28
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:22	37
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:18	38
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:13	31
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:21	40
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:15	33
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:17	35
printerBoltThreadID:40@Mon Sep 14 16:24:37 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:22	42
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:16	36
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:37 CST 2015@sequnceTaskID:18	43
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:19	47
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:14	39
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:13	41
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:20	49
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:18	51
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:14	44
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:17	45
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:16	46
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:21	52
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:15	48
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:22	53
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:16	50
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:14	54
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:20	55
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:19	58
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:13	56
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:17	57
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:19	63
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:20	64
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:15	59
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:13	60
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:22	65
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:14	61
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:18	67
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:15	62
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:21	69
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:17	66
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:16	68
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:14	70
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:16	73
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:21	71
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:15	75
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:22	72
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:13	76
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:20	74
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:18	77
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:19	78
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:17	79
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:17	80
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:22	82
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:14	81
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:20	83
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:10	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:13	85
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:9	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:18	84
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:5	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:15	86
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:21	88
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:16	87
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:19	89
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:7	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:22	92
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:19	95
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:15	90
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:12	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:17	91
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:8	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:20	96
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:18	97
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:11	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:14	93
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:4	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:21	98
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:3	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:16	94
printerBoltThreadID:40@Mon Sep 14 16:24:38 CST 2015@printerTaskID:6	sequenceBoltThreadID:38@Mon Sep 14 16:24:38 CST 2015@sequnceTaskID:13	99

可以看到,在任务流中的元组被随机分配到不同的任务中的同时,其最终的总体顺序也被打乱了,这仅仅是因为我们为SequenceBolt分配了两个执行器。同样,如果我们只为PrinterBolt分配1个以上的执行器,也会造成输出的无序。


总结:希望通过这几个例子,能够有助于更深入理解Storm并行的各个组件,并在业务对流处理有序性有要求的时候能够采取正确的处理方案。类似Kafka这样的输入源,一定要考虑其自身特点,其分区内有序,分区外无序的特性,使得使用个性化的Kafka分区策略是必要的。而要保证Storm处理流程中的有序性,可以使用Global Grouping来完成。


与Kafka分区机制类似的是,Storm的Fields Grouping,比如你可以指定按照用户的uid字段进行Grouping,那么同一个用户的所有的数据都会进入到同一个任务中,这样的话你在一个任务中才能统计到某一个用户的完整信息。


当然,从另一个角度来说,要保证顺序就可能要做对应的性能牺牲,最好能够在业务设计的时候,尽量避免这种一致性的要求,但是如果不能绕过,那也是可以满足需求的,只不过可能要牺牲一些性能来妥协了。


你可能感兴趣的:(消息队列,分布式,数据,学习,BigData,Storm,Java)