sql中in后面元素过多导致索引失效解决方案

1.1、定义线程池

public class ThreadPoolUtils {

    private static int corePoolSize = 50;
    private static int maximumPoolSize = 100;
    private static long keepAliveTime = 50L;
    private static int workQueueSize = 3000;
    private static ExecutorService executor;

    static {
        executor = new ThreadPoolExecutor(corePoolSize, maximumPoolSize, keepAliveTime, TimeUnit.SECONDS,
                new ArrayBlockingQueue<>(workQueueSize), (task, executor) -> log.info("ThreadPoolUtils process business  is full"));
    }

    public static void submit(Runnable task) {
        executor.submit(task);
    }

    //当前核心数两倍的线程池
    private static final int PROCESSORS = Runtime.getRuntime().availableProcessors() << 1;
    public static final ThreadPoolExecutor DOUBLE_PROCESSORS_THREAD_POOL = new ThreadPoolExecutor(
            PROCESSORS,
            PROCESSORS,
            20,
            TimeUnit.SECONDS,
            new ArrayBlockingQueue<>(1000),
            r -> {
                Thread thread = new Thread(r);
                thread.setName("PROCESSORS_DEFAULT_THREAD_POOL-" + r.getClass().getName());
                return thread;
            },
            new ThreadPoolExecutor.CallerRunsPolicy());

}

1.2、使用线程池分片查询数据库

List<String> inputList; // 查询条件,根据具体场景定义
List<String> res;
long start = System.currentTimeMillis();
int groupSize = 300;
// 分成 groupSize 个一组
// getData 是数据库查询的方法,inputList 为入参,返回结果为 List
List<List<String>> results = IntStream.range(0, inputList.size() / groupSize +1)
                    .mapToObj(i -> CompletableFuture.supplyAsync(() ->
                                    getData(inputList.subList(i * groupSize,Math.min((i + 1) * groupSize,inputList.size()) ))
                            , ThreadPoolUtils.DOUBLE_PROCESSORS_THREAD_POOL))
                    .map(CompletableFuture::join)
                    .collect(Collectors.toList());
// 处理结果达到预期格式
res = results.stream()
        .flatMap(List::stream)
        .collect(Collectors.toList());
log.info("线程池分片查询sql耗时:"+ (System.currentTimeMillis() - start) + "ms");

实际应用场景可根据根据 groupSize 自定义切片大小,寻找一个最优解。

你可能感兴趣的:(多线程,sql,数据库,java,mysql,spring,boot)