flume报错：Space for commit to queue couldn't be acquired. Sinks are likely not keeping up with sources

1.问题描述

自己采用flume监控log文件，结果报错

2018-11-26 19:32:04,687 (pool-5-thread-1) [ERROR - org.apache.flume.source.ExecSource$ExecRunnable.run(ExecSource.java:353)] Failed while running command: tail -F /opt/datas/streamingproject/access.log
org.apache.flume.ChannelException: Unable to put batch on required channel: org.apache.flume.channel.MemoryChannel{name: memory-channel}
        at org.apache.flume.channel.ChannelProcessor.executeChannelTransaction(ChannelProcessor.java:253)
        at org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:191)
        at org.apache.flume.source.ExecSource$ExecRunnable.flushEventBatch(ExecSource.java:382)
        at org.apache.flume.source.ExecSource$ExecRunnable.run(ExecSource.java:342)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flume.ChannelFullException: Space for commit to queue couldn't be acquired. Sinks are likely not keeping up with sources, or the buffer size is too tight
        at org.apache.flume.channel.MemoryChannel$MemoryTransaction.doCommit(MemoryChannel.java:130)
        at org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151)
        at org.apache.flume.channel.ChannelProcessor.executeChannelTransaction(ChannelProcessor.java:245)
        ... 8 more
2018-11-26 19:32:04,690 (timedFlushExecService19-0) [ERROR - org.apache.flume.source.ExecSource$ExecRunnable$1.run(ExecSource.java:328)] Exception occured when processing event batch
org.apache.flume.ChannelException: Unable to put batch on required channel: org.apache.flume.channel.MemoryChannel{name: memory-channel}
        at org.apache.flume.channel.ChannelProcessor.executeChannelTransaction(ChannelProcessor.java:253)
        at org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:191)
        at org.apache.flume.source.ExecSource$ExecRunnable.flushEventBatch(ExecSource.java:382)
        at org.apache.flume.source.ExecSource$ExecRunnable.access$100(ExecSource.java:255)
        at org.apache.flume.source.ExecSource$ExecRunnable$1.run(ExecSource.java:324)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flume.ChannelException: java.lang.InterruptedException
        at org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:154)
        at org.apache.flume.channel.ChannelProcessor.executeChannelTransaction(ChannelProcessor.java:245)
        ... 11 more
Caused by: java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
        at java.util.concurrent.Semaphore.tryAcquire(Semaphore.java:582)
        at org.apache.flume.channel.MemoryChannel$MemoryTransaction.doCommit(MemoryChannel.java:128)
        at org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151)
        ... 12 more
2018-11-26 19:32:04,690 (pool-5-thread-1) [INFO - org.apache.flume.source.ExecSource$ExecRunnable.run(ExecSource.java:376)] Command [tail -F /opt/datas/streamingproject/access.log] exited with 143

2.原因

参看Space for commit to queue couldn't be acquired. Sinks are likely not keeping up with sources, or the，是由于memory channel导致的。对于内存通道，当输入过多，而无法全部输出的时候，会导致内存通道崩溃，从而报错。

3.解决

将内存通道换为file通道。生产中多用file通道，安全稳定。但是缺点是慢。