Master-Worker模式

1.Master-Worker模式是常用的并行模式之一。它的核心思想是，系统由两类进程协作，Master进程和Worker进程。Master进程负责接收和分配任务，Worker进程负责处理子任务。当各个Worker进程将子任务处理完成后，将结果返回给Master进程，由Master进程做归纳和汇总。其处理过程如图所示：

                                                          图1-1Master-Worker模式工作示意图

2.Master-Worker模式的主要参与者如表 2-1

角色	作用
Worker	用于实际处理一个任务
Master	用于任务的分配和最终结果的合成
Main	启动系统，调度开启Master

3. Master-Worker的代码实现

import java.util.HashMap;
import java.util.Map;
import java.util.Queue;
import java.util.concurrent.ConcurrentLinkedQueue;

public class Master {

	// 任务队列
	protected Queue<Object> workQueue = new ConcurrentLinkedQueue<Object>();
	
	// worker进程队列
	protected Map<String, Thread> threadMap = new HashMap<String, Thread>();
	
	// 
	protected Map<String, Object> resultMap = new HashMap<String, Object>();
	
	// 是否所有的子任务都结束了
	public boolean isComplete(){
		for(Map.Entry<String, Thread> entry : threadMap.entrySet()){
			if(entry.getValue().getState() != Thread.State.TERMINATED){
				return false;
			}
		}
		return true;
	}
	
	public Master(Worker worker, int countWorker){
		worker.setWorkQueue(workQueue);
		worker.setResultMap(resultMap);
		
		for(int i = 0; i < countWorker; i++){
			threadMap.put(Integer.toString(i), new Thread(worker, Integer.toString(i)));
		}
	}
	
	
	public void submit(Object job){
		workQueue.add(job);
	}

	public Map<String, Object> getResultMap() {
		return resultMap;
	}
	
	
	public void execute(){
		for(Map.Entry<String, Thread> entry : threadMap.entrySet()){
			entry.getValue().start();
		}
	}
	
}

对应的Worker进程实现如下：

import java.util.Map;
import java.util.Queue;

public class Worker implements Runnable {
	
	protected Queue<Object> workQueue = null;
	
	protected Map<String, Object> resultMap = null;
	

	public void setWorkQueue(Queue<Object> workQueue) {
		this.workQueue = workQueue;
	}


	public void setResultMap(Map<String, Object> resultMap) {
		this.resultMap = resultMap;
	}

	
	public Object handle(Object input){
		return input;
	}


	@Override
	public void run() {
		// TODO Auto-generated method stub
		while(true){
			Object  input = workQueue.poll();
			if(input == null) break;
			
			Object re = handle(input);
			
			resultMap.put(Integer.toString(input.hashCode()), re);
		}
	}

}

以上两段代码展示了Master-Worker框架的全貌。应用程序通过重载Worker.handle()方法实现应用逻辑。

现在我们随便写一个Worker的实现。

public class PlusWorker extends Worker {

public class PlusWorker extends Worker {

	public Object handle(Object input){
		Integer i = (Integer)input;
		return i * i * i;
	}
	
}

使用Master-Worker框架计算的Main()函数实现如下：

import java.util.Map;
import java.util.Set;

public class testmain {

	public static void main(String[] args) {
		// TODO Auto-generated method stub
		
		Master m = new Master(new PlusWorker(), 5);
		for(int i = 0; i < 10; i++){
			m.submit(i);
		}
		
		m.execute();
		
		int ret = 0;
		
		Map<String, Object> resultMap =m.getResultMap();
		
		while(resultMap.size() > 0 || !m.isComplete()){
			
			Set<String> keys = resultMap.keySet();
			
			String key = null;
			for(String k : keys){
				key = k;
				break;
			}
			
			Integer i = null;
			
			if(key != null){
				i = (Integer)resultMap.get(key);
				
				if(i != null) ret += i;
				
				resultMap.remove(key);
			}
			
		}
		
		System.out.println("ret = " + ret);
		
	}

}