SpringBoot定时任务之分布式锁（1）
定时任务一般需要分布式锁进行任务同步，否则容易出现多个节点处理同一任务的情况。
本系列讲解使用并强化shedlock来实现分布式redis锁，主要需要完成的内容包括：
- 实现程序意外退出时，使用钩子函数删除redis分布式锁；
- 实现可重入锁；
- 实现锁的细分，例如，要批量处理商品，可以根据商品ID来设置锁，多个节点可以同时执行商品的处理任务，只是每个节点处理的商品不同；
- 实现锁续约，如果定时任务耗时较长，且时间动态变动，变动的范围又无法预估，在这种情况下，锁的超时时间是无法有效设置的，过早释放锁，会导致同一批任务又被其他节点再次处理，如果处理接口无法保证幂等性，则会导致数据的不一致，即使批处理接口能保证幂等性，也有可能导致两个节点同时处理一批任务而造成死锁等问题。
本次主要讲解环境的搭建，并解决第一个问题，即实现程序意外退出时，使用钩子函数删除redis分布式锁。
- pom依赖：
4.0.0
org.springframework.boot
spring-boot-starter-parent
2.2.5.RELEASE
com.springboot
shedlock
1.0.0
shedlock
distribute lock
1.8
org.springframework.boot
spring-boot-starter-data-redis
org.springframework.boot
spring-boot-starter-web
net.javacrumbs.shedlock
shedlock-spring
4.9.2
net.javacrumbs.shedlock
shedlock-provider-redis-spring
4.9.2
org.apache.commons
commons-lang3
3.4
pom
org.projectlombok
lombok
true
org.springframework.boot
spring-boot-starter-test
test
org.junit.vintage
junit-vintage-engine
org.springframework.boot
spring-boot-maven-plugin
- properties配置文件
shedlock.redis.host=127.0.0.1
shedlock.redis.port=6379
shedlock.redis.mode=STANDALONE
shedlock.redis.password=
shedlock.redis.timeout=100s- redis配置：三种模式的配置
package com.springboot.shedlock.config.redis;
import com.fasterxml.jackson.annotation.JsonAutoDetect;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.annotation.PropertyAccessor;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
import com.fasterxml.jackson.datatype.jsr310.deser.LocalDateDeserializer;
import com.fasterxml.jackson.datatype.jsr310.deser.LocalDateTimeDeserializer;
import com.fasterxml.jackson.datatype.jsr310.ser.LocalDateSerializer;
import com.fasterxml.jackson.datatype.jsr310.ser.LocalDateTimeSerializer;
import io.lettuce.core.ClientOptions;
import io.lettuce.core.TimeoutOptions;
import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.*;
import org.springframework.data.redis.connection.lettuce.LettuceClientConfiguration;
import org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;
import org.springframework.util.StringUtils;
import java.time.LocalDate;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.List;
@Configuration
@EnableConfigurationProperties(RedisProperties.class)
@ConditionalOnProperty("shedlock.redis.host")
@Slf4j
public class CommonRedisConfig {
@Bean
public RedisConnectionFactory redisConnectionFactory(RedisProperties properties) {
ClientOptions build = ClientOptions.builder()
.autoReconnect(true)
.timeoutOptions(TimeoutOptions.enabled(properties.getTimeout()))
.build();
LettuceClientConfiguration clientConfiguration = LettuceClientConfiguration.builder()
.clientName("my-redis")
.clientOptions(build)
.build();
RedisProperties.RedisMode mode = properties.getMode();
switch (mode) {
case CLUSTER:
return createCluster(properties, clientConfiguration);
case SENTINEL:
return createSentinel(properties, clientConfiguration);
case STANDALONE:
return createStandAlone(properties, clientConfiguration);
default:
throw new IllegalArgumentException("redisMode[STANDALONE|SENTINEL|CLUSTER]:" + mode);
}
}
private LettuceConnectionFactory createCluster(RedisProperties properties,
LettuceClientConfiguration clientConfiguration) {
RedisClusterConfiguration conf = new RedisClusterConfiguration();
String[] hosts = StringUtils.split(properties.getHost(), ",");
String[] ports = StringUtils.split(properties.getPort(), ",");
List nodes = new ArrayList<>(hosts.length);
for (int i = 0; i < hosts.length; i++) {
nodes.add(new RedisClusterNode(hosts[i], Integer.parseInt(ports[i])));
}
conf.setClusterNodes(nodes);
conf.setMaxRedirects(8);
if (!StringUtils.isEmpty(properties.getPassword())) {
conf.setPassword(properties.getPassword());
}
return new LettuceConnectionFactory(conf, clientConfiguration);
}
/**
* 注意配置第一个节点为Master
*
* @return LettuceConnectionFactory
*/
private LettuceConnectionFactory createSentinel(RedisProperties properties,
LettuceClientConfiguration clientConfiguration) {
RedisSentinelConfiguration configuration = new RedisSentinelConfiguration();
String[] hosts = StringUtils.split(properties.getHost(), ",");
String[] ports = StringUtils.split(properties.getPort(), ",");
for (int i = 0; i < hosts.length; i++) {
String host = hosts[i];
String port = ports[i];
RedisNode node = new RedisNode(host, Integer.parseInt(port));
configuration.addSentinel(node);
}
configuration.master(properties.getMasterName());
if (!StringUtils.isEmpty(properties.getPassword())) {
configuration.setPassword(properties.getPassword());
}
return new LettuceConnectionFactory(configuration, clientConfiguration);
}
private LettuceConnectionFactory createStandAlone(RedisProperties properties,
LettuceClientConfiguration clientConfiguration) {
RedisStandaloneConfiguration configuration = new RedisStandaloneConfiguration();
if (!StringUtils.isEmpty(properties.getPassword())) {
configuration.setPassword(properties.getPassword());
}
configuration.setHostName(properties.getHost());
configuration.setPort(Integer.parseInt(properties.getPort()));
return new LettuceConnectionFactory(configuration, clientConfiguration);
}
@Bean(name = "redisTemplate")
public RedisTemplate redisTemplate(RedisConnectionFactory factory) {
RedisTemplate template = new RedisTemplate();
template.setConnectionFactory(factory);
setSerializer(template);
template.afterPropertiesSet();
return template;
}
@Bean(name = "strRedisTemplate")
public RedisTemplate stringRedisTemplate(RedisConnectionFactory factory) {
RedisTemplate template = new RedisTemplate();
template.setConnectionFactory(factory);
setSerializer(template);
template.afterPropertiesSet();
return template;
}
private void setSerializer(RedisTemplate template) {
Jackson2JsonRedisSerializer package com.springboot.shedlock.config.redis;
import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
import java.time.Duration;
@Data
@ConfigurationProperties(prefix = "shedlock.redis")
public class RedisProperties {
public enum RedisMode {
SENTINEL, STANDALONE, CLUSTER
}
private String host;
private String port = "6379";
private String password;
private RedisMode mode = RedisMode.STANDALONE;
private Duration timeout;
private String masterName;
}
- 定时任务配置：使多个定时任务分线程同时启动，springboot默认的是只有一个线程，也就是一次只执行一个定时任务
package com.springboot.shedlock.config.scheduler;
import com.springboot.shedlock.scheduler.Job;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.TaskScheduler;
import org.springframework.scheduling.annotation.EnableScheduling;
import org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler;
import org.springframework.util.CollectionUtils;
import java.util.List;
@Configuration
@EnableScheduling
public class ScheduleConfig {
@Autowired
private List jobs;
@Bean
public TaskScheduler taskScheduler() {
ThreadPoolTaskScheduler taskScheduler = new ThreadPoolTaskScheduler();
taskScheduler.setPoolSize(getCount());
return taskScheduler;
}
public int getCount() {
int count;
if(CollectionUtils.isEmpty(jobs)){
count = 1;
} else if(jobs.size() > 10){
count = 10;
} else {
count = jobs.size();
}
return count;
}
}
- redis分布式锁配置：提供增强类，提供锁关闭功能：
package com.springboot.shedlock.config.lock;
import net.javacrumbs.shedlock.core.LockProvider;
import net.javacrumbs.shedlock.provider.redis.spring.RedisLockProvider;
import net.javacrumbs.shedlock.spring.annotation.EnableSchedulerLock;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.RedisConnectionFactory;
@Configuration
@EnableSchedulerLock(defaultLockAtMostFor = "120s")
public class LockConfig {
@Autowired
private RedisConnectionFactory redisConnectionFactory;
@Bean
public LockProvider lockProvider() {
// RedisLockProvider redisLockProvider = new RedisLockProvider(redisConnectionFactory);
RedisLockProviderEnhance redisLockProvider = new RedisLockProviderEnhance(redisConnectionFactory);
return redisLockProvider;
}
}package com.springboot.shedlock.config.lock;
import lombok.extern.slf4j.Slf4j;
import net.javacrumbs.shedlock.provider.redis.spring.RedisLockProvider;
import net.javacrumbs.shedlock.support.LockException;
import net.javacrumbs.shedlock.support.annotation.NonNull;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.StringRedisTemplate;
import java.util.Set;
@Slf4j
public class RedisLockProviderEnhance extends RedisLockProvider {
private final StringRedisTemplate redisTemplate;
public RedisLockProviderEnhance(@NonNull RedisConnectionFactory redisConn) {
super(redisConn);
redisTemplate = new StringRedisTemplate(redisConn);
}
public void unlockByKey(String key) {
try {
redisTemplate.delete(key);
} catch (Exception e) {
throw new LockException("Can not remove node", e);
}
}
public void unlockByPrefix(String prefix) {
try {
Set keys = redisTemplate.keys(prefix + "*");
log.error("删除的分布式锁包括：" + keys);
redisTemplate.delete(keys);
} catch (Exception e) {
throw new LockException("Can not remove node", e);
}
}
}
- 启动类：开启定时任务功能，调用钩子函数删除分布式锁
package com.springboot.shedlock;
import com.springboot.shedlock.config.lock.RedisLockProviderEnhance;
import net.javacrumbs.shedlock.core.LockProvider;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.scheduling.annotation.EnableScheduling;
@SpringBootApplication
@EnableScheduling
public class ShedlockApplication {
public static void main(String[] args) {
ConfigurableApplicationContext context = SpringApplication.run(ShedlockApplication.class, args);
RedisLockProviderEnhance lockProvider = context.getBean(RedisLockProviderEnhance.class);
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
lockProvider.unlockByPrefix("job-lock:default:");
}));
}
}
- 定时任务：所有定时任务继承同一个接口
package com.springboot.shedlock.scheduler;
public interface Job {
void exec();
}
package com.springboot.shedlock.scheduler;
import lombok.SneakyThrows;
import net.javacrumbs.shedlock.spring.annotation.SchedulerLock;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;
import java.util.concurrent.TimeUnit;
@Service
public class MyTestTaskScheduler implements Job {
@SneakyThrows
@Scheduled(cron = "0/5 * * * * ?")
@SchedulerLock(name = "myTask", lockAtMostFor="200000", lockAtLeastFor="200000")
@Override
public void exec() {
System.out.println("执行任务...");
TimeUnit.SECONDS.sleep(10000);
}
}
全量代码参考github：https://github.com/JohnZhaowen/shedlock.git