tail: inotify resources exhausted tail: inotify cannot be used, reverting to polling
在使用tail 命令时发现以下异常信息：
- tail: inotify resources exhausted
tail: inotify cannot be used, reverting to polling
...并且您安装了CrashPlan [ * ]，那么您对inotify.max_user_watches的数量限制可能太低。我只提到CrashPlan，因为这似乎与Linux 上的CrashPlan相当常见。这可能是由于各种原因而发生的，以便找出造成这种情况的原因，请执行以下操作：
|
1
2
|
echo
1 >
/sys/kernel/debug/tracing/events/syscalls/sys_exit_inotify_add_watch/enable
echo
1 >
/sys/kernel/debug/tracing/tracing_enabled
|
这两个命令将使您能够“观看” inotify_add_watch事件。要实际观看它们，请在启用后等待几分钟，然后：
|
1
|
cat
/sys/kernel/debug/tracing/trace
|
你应该看到类似这样的输出：
|
1
2
3
4
五
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
三十
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
|
root@localhost:~# cat /sys/kernel/debug/tracing/trace | more
# tracer: nop
#
# TASK-PID CPU# TIMESTAMP FUNCTION
# | | | | |
java-13752 [010] 180569.026114: sys_inotify_add_watch -> 0x1
java-13752 [010] 180569.038573: sys_inotify_add_watch -> 0x2
java-13752 [010] 180569.039368: sys_inotify_add_watch -> 0x3
java-13752 [010] 180569.044214: sys_inotify_add_watch -> 0x4
java-13752 [010] 180569.051454: sys_inotify_add_watch -> 0x5
java-13752 [010] 180569.052107: sys_inotify_add_watch -> 0x6
java-13752 [010] 180569.059542: sys_inotify_add_watch -> 0x7
java-13752 [010] 180569.060265: sys_inotify_add_watch -> 0x8
java-13752 [010] 180569.060760: sys_inotify_add_watch -> 0x9
java-13752 [010] 180569.068002: sys_inotify_add_watch -> 0xa
java-13752 [010] 180569.068549: sys_inotify_add_watch -> 0xb
java-13752 [010] 180569.082694: sys_inotify_add_watch -> 0xc
java-13752 [010] 180569.089735: sys_inotify_add_watch -> 0xd
java-13752 [010] 180569.093624: sys_inotify_add_watch -> 0xe
java-13752 [010] 180569.094271: sys_inotify_add_watch -> 0xf
java-13752 [010] 180569.098156: sys_inotify_add_watch -> 0x10
java-13752 [010] 180569.098794: sys_inotify_add_watch -> 0x11
java-13752 [010] 180569.105731: sys_inotify_add_watch -> 0x12
java-13752 [010] 180569.109630: sys_inotify_add_watch -> 0x13
java-13752 [010] 180569.119702: sys_inotify_add_watch -> 0x14
java-13752 [010] 180569.123390: sys_inotify_add_watch -> 0x15
java-13752 [010] 180569.127319: sys_inotify_add_watch -> 0x16
java-13752 [010] 180569.127801: sys_inotify_add_watch -> 0x17
java-13752 [010] 180569.131432: sys_inotify_add_watch -> 0x18
java-13752 [010] 180569.135184: sys_inotify_add_watch -> 0x19
java-13752 [010] 180569.135616: sys_inotify_add_watch -> 0x1a
java-13752 [010] 180569.139202: sys_inotify_add_watch -> 0x1b
java-13752 [010] 180569.139622: sys_inotify_add_watch -> 0x1c
java-13752 [010] 180569.149321: sys_inotify_add_watch -> 0x1d
java-13752 [010] 180569.149717: sys_inotify_add_watch -> 0x1e
java-13752 [010] 180569.156260: sys_inotify_add_watch -> 0x1f
java-13752 [010] 180569.165739: sys_inotify_add_watch -> 0x20
java-13752 [010] 180569.169937: sys_inotify_add_watch -> 0x21
java-13752 [010] 180569.170296: sys_inotify_add_watch -> 0x22
java-13752 [010] 180569.177402: sys_inotify_add_watch -> 0x23
java-13752 [010] 180569.183846: sys_inotify_add_watch -> 0x24
java-13752 [010] 180569.187312: sys_inotify_add_watch -> 0x25
java-13752 [010] 180569.187802: sys_inotify_add_watch -> 0x26
java-13752 [010] 180569.191314: sys_inotify_add_watch -> 0x27
java-13752 [010] 180569.191781: sys_inotify_add_watch -> 0x28
java-13752 [010] 180569.198126: sys_inotify_add_watch -> 0x29
java-13752 [010] 180569.201667: sys_inotify_add_watch -> 0x2a
java-13752 [010] 180569.209703: sys_inotify_add_watch -> 0x2b
java-13752 [010] 180569.212063: sys_inotify_add_watch -> 0x2c
java-13752 [010] 180569.214432: sys_inotify_add_watch -> 0x2d
java-13752 [010] 180569.214729: sys_inotify_add_watch -> 0x2e
java-13752 [010] 180569.216971: sys_inotify_add_watch -> 0x2f
java-13752 [010] 180569.219159: sys_inotify_add_watch -> 0x30
java-13752 [010] 180569.219450: sys_inotify_add_watch -> 0x31
java-13752 [010] 180569.221780: sys_inotify_add_watch -> 0x32
java-13752 [010] 180569.222029: sys_inotify_add_watch -> 0x33
java-13752 [010] 180569.225990: sys_inotify_add_watch -> 0x34
java-13752 [010] 180569.228548: sys_inotify_add_watch -> 0x35
java-13752 [010] 180569.228797: sys_inotify_add_watch -> 0x36
java-13752 [010] 180569.232822: sys_inotify_add_watch -> 0x37
java-13752 [010] 180569.233054: sys_inotify_add_watch -> 0x38
java-13752 [010] 180569.237234: sys_inotify_add_watch -> 0x39
java-13752 [010] 180569.237551: sys_inotify_add_watch -> 0x3a
java-13752 [010] 180569.243332: sys_inotify_add_watch -> 0x3b
java-13752 [010] 180569.245901: sys_inotify_add_watch -> 0x3c
java-13752 [010] 180569.246179: sys_inotify_add_watch -> 0x3d
java-13752 [010] 180569.250486: sys_inotify_add_watch -> 0x3e
java-13752 [010] 180569.250802: sys_inotify_add_watch -> 0x3f
java-13752 [010] 180569.252945: sys_inotify_add_watch -> 0x40
java-13752 [010] 180569.253189: sys_inotify_add_watch -> 0x41
java-13752 [010] 180569.255402: sys_inotify_add_watch -> 0x42
java-13752 [010] 180569.255661: sys_inotify_add_watch -> 0x43
java-13752 [010] 180569.259566: sys_inotify_add_watch -> 0x44
java-13752 [010] 180569.261640: sys_inotify_add_watch -> 0x45
java-13752 [010] 180569.263669: sys_inotify_add_watch -> 0x46
java-13752 [010] 180569.265819: sys_inotify_add_watch -> 0x47
java-13752 [010] 180569.267893: sys_inotify_add_watch -> 0x48
java-13752 [010] 180569.269967: sys_inotify_add_watch -> 0x49
java-13752 [010] 180569.271976: sys_inotify_add_watch -> 0x4a
java-13752 [010] 180569.272240: sys_inotify_add_watch -> 0x4b
java-13752 [010] 180569.291990: sys_inotify_add_watch -> 0x4c
java-13752 [010] 180569.292369: sys_inotify_add_watch -> 0x4d
java-13752 [010] 180569.292726: sys_inotify_add_watch -> 0x4e
java-13752 [010] 180569.293091: sys_inotify_add_watch -> 0x4f
java-13752 [010] 180569.293420: sys_inotify_add_watch -> 0x50
java-13752 [010] 180569.293749: sys_inotify_add_watch -> 0x51
java-13752 [010] 180569.305760: sys_inotify_add_watch -> 0x52
java-13752 [010] 180569.306204: sys_inotify_add_watch -> 0x53
java-13752 [010] 180569.306665: sys_inotify_add_watch -> 0x54
java-13752 [010] 180569.307042: sys_inotify_add_watch -> 0x55
java-13752 [010] 180569.307385: sys_inotify_add_watch -> 0x56
java-13752 [010] 180569.307724: sys_inotify_add_watch -> 0x57
java-13752 [010] 180569.308032: sys_inotify_add_watch -> 0x58
java-13752 [010] 180569.321561: sys_inotify_add_watch -> 0x59
java-13752 [010] 180569.321968: sys_inotify_add_watch -> 0x5a
java-13752 [010] 180569.322274: sys_inotify_add_watch -> 0x5b
java-13752 [010] 180569.322552: sys_inotify_add_watch -> 0x5c
java-13752 [010] 180569.322830: sys_inotify_add_watch -> 0x5d
java-13752 [010] 180569.323106: sys_inotify_add_watch -> 0x5e
java-13752 [010] 180569.323378: sys_inotify_add_watch -> 0x5f
java-13752 [010] 180569.323635: sys_inotify_add_watch -> 0x60
java-13752 [010] 180569.337109: sys_inotify_add_watch -> 0x61
java-13752 [010] 180569.337452: sys_inotify_add_watch -> 0x62
java-13752 [010] 180569.337779: sys_inotify_add_watch -> 0x63
java-13752 [010] 180569.338094: sys_inotify_add_watch -> 0x64
java-13752 [010] 180569.338379: sys_inotify_add_watch -> 0x65
java-13752 [010] 180569.338660: sys_inotify_add_watch -> 0x66
--More--
|
注意任务和PID列：
|
1
2
|
root@localhost:~
# ps waux | grep java
root 13679 50.3 4.1 6393844 510320 pts
/1
SNl 11:58 1:18
/usr/local/crashplan/jre/bin/java
-Dfile.encoding=UTF-8 -Dapp=CrashPlanService -DappBaseName=CrashPlan -Xms20m -Xmx1024m -Djava.net.preferIPv4Stack=
true
-Dsun.net.inetaddr.ttl=300 -Dnetworkaddress.cache.ttl=300 -Dsun.net.inetaddr.negative.ttl=0 -Dnetworkaddress.cache.negative.ttl=0 -Dc42.native.md5.enabled=
false
-classpath
/usr/local/crashplan/lib/com
.backup42.desktop.jar:
/usr/local/crashplan/lang
com.backup42.service.CPService
|
PID并不总是与添加手表的过程相匹配，在上例中，CrashPlan可能会产生一个子过程（PID：13752，根据我们的跟踪）添加手表。
所以现在你知道为什么会发生这种情况，你应该这样做，首先，看看当前配置的限制是什么：
|
1
|
cat
/proc/sys/fs/inotify/max_user_watches
|
看来，Ubuntu服务器的默认限制是8192。要提高限制，请以root用户身份运行以下命令：
|
1
|
sysctl -w fs.inotify.max_user_watches=32768
|
或者，要使限制永久化，请编辑/etc/sysctl.conf并追加以下行：
|
1
|
fs.inotify.max_user_watches=32768
|
然后确保使用以下命令重新加载配置文件：
|
1
|
sysctl -p
|
限制32768可能有点高[ ** ]，所以你可能需要一个较低的取决于你的机器的可用资源（RAM，CPU等）。作为参考，我在具有12GB RAM或更多的生产服务器上使用此配置。因人而异。
把东西恢复到默认设置（默认为Ubuntu）：
|
1
2
|
echo
0 >
/sys/kernel/debug/tracing/events/syscalls/sys_exit_inotify_add_watch/enable
echo
1 >
/sys/kernel/debug/tracing/tracing_enabled
|
（在Ubuntu上，/ sys / kernel / debug / tracing / tracing_enabled的默认设置是“1”）
注释/延伸阅读：
- [*]这个问题不是CrashPlan特有的。任何大量使用inotify的环境迟早都会遇到这种情况，除非8192的默认值为低。
- [**]好的，这个设置并不是那么高 - 这取决于你的用例。例如，在下面的链接（来自CrashPlan的网站），建议使用限制1048576。
- 网络连接存储的实时备份：http : //support.code42.com/CrashPlan/Latest/Troubleshooting/Real-Time_Backup_For_Network-Attached_Drives#Related_Topics