Talk is cheap, show me the code.
k | version |
---|---|
OS | windows 10 |
jdk | 1.8 |
maven | 3.6.0 |
spring boot | 2.3.4.RELEASE |
redis | 5.0.10 |
我在测试 redis stream 时,需要对比 消费消息后使用xdel
删除消息 和 消费消息后不使用xdel
删除消息。
这个对比实验的目的是为了研究,不使用xdel
去删除已消费的消息,是否会占用更多的 redis 内存。
实验方案是:向 redis stream 中推送 50000 条消息, 对比消费结束后的内存占用情况。
但是在消费数据过程中,大概消费了44000多条数据时,发生了异常:org.springframework.data.redis.RedisConnectionFailureException: Unable to connect to Redis; nested exception is java.util.concurrent.CompletionException: io.lettuce.core.RedisConnectionException: Unable to connect to localhost:6379
先一次性向 Redis stream 中生产了 50000 条数据,在消费数据过程中,大概在消费了44000多条数据时,发生了异常。
@RestController
@RequestMapping("/redis/stream")
public class RedisStreamController {
@Autowired
private RedisTemplate redisTemplate;
@GetMapping("/push-data")
public String pushData() {
for (int i = 0; i < 5_0000; i++) {
Map<String, String> map = new HashMap<>();
map.put("k", DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss:SSS").format(LocalDateTime.now()));
RecordId recordId = redisTemplate.opsForStream().add("stream_test1", map);
}
return "Success";
}
}
@Slf4j
@Component
public class Stream1Listener extends AbstractAutoRetryStreamListener<String, MapRecord<String, String, String>> {
@Override
public void doOnMessage(MapRecord<String, String, String> message) {
Map<String, String> msgMap = message.getValue();
// do something
double random = Math.random();
if (random > 0.5) {
log.warn("consume failure, consumer={} message={} random={}", getFullName(), message.getValue(), random);
throw new RuntimeException("Message processing failed");
}
log.info("consume success, consumer={} message={}", getFullName(), message.getValue());
}
@Override
public String getStream() {
return "stream_test1";
}
@Override
protected long maxRetries() {
return 0L;
}
}
Caused by: java.util.concurrent.CompletionException: io.lettuce.core.RedisConnectionException: Unable to connect to localhost:6379
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331)
at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346)
at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:632)
... 30 more
Caused by: io.lettuce.core.RedisConnectionException: Unable to connect to localhost:6379
at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:78)
at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:56)
at io.lettuce.core.RedisClient.lambda$transformAsyncConnectionException$20(RedisClient.java:767)
at io.lettuce.core.DefaultConnectionFuture.lambda$thenCompose$1(DefaultConnectionFuture.java:253)
... 22 more
Caused by: java.util.concurrent.CompletionException: io.netty.channel.AbstractChannel$AnnotatedSocketException: Address already in use: no further information: localhost/127.0.0.1:6379
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331)
at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346)
at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:632)
... 20 more
Caused by: io.netty.channel.AbstractChannel$AnnotatedSocketException: Address already in use: no further information: localhost/127.0.0.1:6379
Caused by: java.net.BindException: Address already in use: no further information
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:707)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
打开 Widnows 命令行窗口( Run as administor
)执行以下命令:
netsh int ipv4 set dynamicport tcp start=20000 num=90000
这个命令是Windows的Netsh工具命令,用于配置网络接口。具体来说,
netsh int ipv4 set dynamicport tcp start=20000 num=90000
这个命令是设置TCP协议的动态端口范围。其中,'start=20000’表示动态端口的起始值,而’num=90000’表示端口范围的大小,即从20000开始,共有90000个连续的端口可供动态分配。
CSDN - 并发测试异常[Address already in use: no futher infomation]
cnpython - java Spring Redis流消费者停止消费消息(地址已在使用中)
知乎 - 【Netty】网络连接数上限造成的问题