多线程分类处理数据

业务场景：消息队列消费端启动并发消费时，同一个单子的数据读写无法统一，比如一条线程判断缓存数为1，然后以这个结果做处理变为2，另一个线程提前已经操作变为2了。
解决方案: 手动编写线程，控制同一个单子始终在一条线程中，从而既保留多线程的处理效率，又保证了数据的准确。
实现思路：初始化线程时为每一个线程单独分配一个待处理队列Queue，建立一个线程管理的类，负责将数据提交到不同线程的Queue中，业务调用层只需要往管理类发数据即可，业务逻辑实现层实现Runnable并处理数据

测试代码

单元测试使用的String作为业务数据，实际工作中换成bean即可

public class Test {
    public static void main(String[] a){
        final ExecuteManager manager  = new ExecuteManager(10);
        manager.init();

        new Thread(new Runnable() {
            @Override
            public void run() {
                while(true){
                    int num = new Random().nextInt(100);
                    manager.sendMsg(num);
                    try {
                        TimeUnit.MILLISECONDS.sleep(10);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                }
            }
        }).start();


        synchronized (manager){
            try {
                manager.wait();
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
    }
}


class ExecuteManager {
    List> queues ;
    ExecutorService service;
    int threads;
    public ExecuteManager(int threads){
        this.threads  = threads;
    }

    public void init(){
        queues = new ArrayList>(threads);
        ExecutorService service = Executors.newFixedThreadPool(threads);
        for(int i = 0;i newQueue(){
        ArrayBlockingQueue queue = new ArrayBlockingQueue(100);
        this.queues.add(queue);
        return queue;
    }

    public void shutdown(){
        service.shutdown();
    }

    public void sendMsg(int num) {
        int index = num % 10;
        ArrayBlockingQueue queue = queues.get(index);
        try {
            queue.put(String.valueOf(num));
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

    }
}


class Worker implements  Runnable{
    private ArrayBlockingQueue queue;
    public Worker(ArrayBlockingQueue queue){
        this.queue = queue;
    }

    @Override
    public void run(){
        try{
            while(true){
                String s = queue.take();
                System.out.println(Thread.currentThread().getName()+"==========>"+s);
            }
        }catch (Exception e){
            e.printStackTrace();
        }

    }
}

我们将主单号取余，丢进不同线程的队列中，比如单号以2结尾的一定都是在第thread-3的线程中处理，避免了数据读写问题

实际应用（Springboot项目中）

1，参数配置，便于后期运维根据不同机器设置不同线程

#application.properties 线程
thread.threadsNum=4
thread.capacity=50

/**
 * 线程数配置
 */
@Component
@ConfigurationProperties(prefix = ThreadConstants.DS, ignoreUnknownFields = false)
public class ThreadConstants {
    // 对应配置文件里的配置键
    public final static String DS = "thread";

    private int threadsNum;

    private int capacity;

    //Alt+Ins get,set方法
}

2，Rabbitmq消费端监听类

@Component
public class WcForWsReceiver {
    @Autowired
    private ExecuteManager executeManager;

    @RabbitListener(bindings = @QueueBinding(
            value = @Queue(value = "old_scan_data",durable = "true"),
            exchange = @Exchange(value = "ws_exchange_old",durable = "true",type = "fanout")
    ))
    @RabbitHandler
    public void process(@Payload ScanData data) {
        executeManager.sendMsg(data);
    }
}

3，任务分发代理类

@Component
public class ExecuteManager {
    @Autowired
    private ThreadConstants threadConstants;

    private List> queues;

    public void init() {
        //固定长度的线程数，因为有规则控制数据进入指定线程
        int threadnum = threadConstants.getThreadsNum();
        queues = new ArrayList<>(threadnum);
        ExecutorService service = Executors.newFixedThreadPool(threadnum);
        for (int i = 0; i < threadnum; i++) {
            service.submit(new Worker(newQueue()));
        }
        System.out.println("线程池初始化");
    }

    public ArrayBlockingQueue newQueue() {
        //设置每个线程的队列大小
        ArrayBlockingQueue queue = new ArrayBlockingQueue<>(threadConstants.getCapacity());
        this.queues.add(queue);
        return queue;
    }

    public void sendMsg(ScanData scan) {
        //从队列实体中获取主单号
        int doc_id=Integer.parseInt(scan.getDOC_ID());
        int index =doc_id  % threadConstants.getThreadsNum();
        ArrayBlockingQueue queue = queues.get(index);
        try {
            //阻塞式的加入队列，避免数据丢失
            queue.put(scan);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

4，任务（伪）处理类

懒得动原来Service层的代码，想直接注入使用，发现注入失败，空指针异常，改用工具类强制获取Bean

import com.yd.express.data.ScanData;
import com.yd.express.util.SpringBeanUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.concurrent.ArrayBlockingQueue;
public class Worker implements  Runnable{
    private static Logger log = LoggerFactory.getLogger(Worker.class);
    private ArrayBlockingQueue queue;
    public Worker(ArrayBlockingQueue queue){
        this.queue = queue;
    }

    @Override
    public void run(){
        try{
            StockService stockservice= (StockService) SpringBeanUtils.getBean("StockService");
            while(true){
                ScanData data = queue.take();
                try {
                   System.out.println(Thread.currentThread().getName()+"==========>"+data.getSDOC_ID());
                    //实际业务处理
                    stockservice.doSomething(data);
                } catch (Exception e) {
                    log.error(data.toString(), e);
                }
            }
        }catch (Exception e){
            log.error( "线程异常", e);
        }
    }
}

@Service("StockService")
public interface StockService {
        void doSomething(ScanData data) throws Exception;
}

5，最后项目启动时初始化线程池，编码完毕，启动

@Component
public class WSBaseDataLoaderListener implements ApplicationListener{
    @Autowired
    ExecuteManager executeManager;
    @Override
    public void onApplicationEvent(ContextRefreshedEvent event) {
        executeManager.init();
    }
}