2019-06-21 虚拟机进程经常触发OOM,被物理机杀死问题与解决方案

Jun 19 17:28:17 localhost kernel: Node 1 Normal: 3323*4kB (UEM) 2500*8kB (UEM) 876*16kB (UEM) 5*32kB (U) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 47468kB
Jun 19 17:28:17 localhost kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Jun 19 17:28:17 localhost kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Jun 19 17:28:17 localhost kernel: Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Jun 19 17:28:17 localhost kernel: Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Jun 19 17:28:17 localhost kernel: 480 total pagecache pages
Jun 19 17:28:17 localhost kernel: 26 pages in swap cache
Jun 19 17:28:17 localhost kernel: Swap cache stats: add 29067141, delete 29136938, find 10318233/18235303
Jun 19 17:28:17 localhost kernel: Free swap  = 32026832kB
Jun 19 17:28:17 localhost kernel: Total swap = 32972796kB
Jun 19 17:28:17 localhost kernel: 16752732 pages RAM
Jun 19 17:28:17 localhost kernel: 0 pages HighMem/MovableOnly
Jun 19 17:28:17 localhost kernel: 332320 pages reserved
Jun 19 17:28:17 localhost kernel: [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
Jun 19 17:28:17 localhost kernel: [ 4749]     0  4749    37475        0      75     1123             0 systemd-journal
Jun 19 17:28:17 localhost kernel: [ 4772]     0  4772    50268        0      31      728             0 lvmetad
Jun 19 17:28:17 localhost kernel: [ 4780]     0  4780    11787        1      25      686         -1000 systemd-udevd
Jun 19 17:28:17 localhost kernel: [ 9122]     0  9122    12256        0      24      123         -1000 auditd
Jun 19 17:28:17 localhost kernel: [ 9160]     0  9160    83533        0      83     3709             0 firewalld
Jun 19 17:28:17 localhost kernel: [ 9164]    81  9164     8200        0      17      146          -900 dbus-daemon
Jun 19 17:28:17 localhost kernel: [ 9172]     0  9172     6593        0      19       87             0 systemd-logind
Jun 19 17:28:17 localhost kernel: [ 9173]     0  9173     5413        0      14      111             0 irqbalance
Jun 19 17:28:17 localhost kernel: [ 9174]     0  9174    77540        0      79      228             0 rsyslogd
Jun 19 17:28:17 localhost kernel: [ 9177]     0  9177   141927        0      96     3203             0 tuned
Jun 19 17:28:17 localhost kernel: [ 9178]    70  9178     7036        0      20       84             0 avahi-daemon
Jun 19 17:28:17 localhost kernel: [ 9183]     0  9183    31048        0      19      164             0 crond
Jun 19 17:28:17 localhost kernel: [ 9185]    70  9185     7005        0      19       59             0 avahi-daemon
Jun 19 17:28:17 localhost kernel: [ 9396]     0  9396   116870        0      76      583             0 NetworkManager
Jun 19 17:28:17 localhost kernel: [ 9490]   999  9490   133097        0      59     2065             0 polkitd
Jun 19 17:28:17 localhost kernel: [ 9860]     0  9860    25466        0      50     3123             0 dhclient
Jun 19 17:28:17 localhost kernel: [ 9875]     0  9875    25466        0      50     3115             0 dhclient
Jun 19 17:28:17 localhost kernel: [ 9889]     0  9889    25466        0      50     3110             0 dhclient
Jun 19 17:28:17 localhost kernel: [ 9904]     0  9904    25466        0      51     3114             0 dhclient
Jun 19 17:28:17 localhost kernel: [10116]     0 10116    20078        0      42      232         -1000 sshd
Jun 19 17:28:17 localhost kernel: [10190]     0 10190     1633        0       8       29             0 iprupdate
Jun 19 17:28:17 localhost kernel: [10191]     0 10191     1633        0       9       30             0 iprinit
Jun 19 17:28:17 localhost kernel: [10285]     0 10285     9793        0       8       24             0 iprdump
Jun 19 17:28:17 localhost kernel: [10374]     0 10374    22265        0      42      284             0 master
Jun 19 17:28:17 localhost kernel: [10377]    89 10377    22308        0      43      272             0 qmgr
Jun 19 17:28:17 localhost kernel: [10574]     0 10574    27523        0      10       32             0 agetty
Jun 19 17:28:17 localhost kernel: [40277]     0 40277    71335        0      54      570             0 VBoxXPCOMIPCD
Jun 19 17:28:17 localhost kernel: [40283]     0 40283   226145        0      80     2477             0 VBoxSVC
Jun 19 17:28:17 localhost kernel: [46020]     0 46020  7965739  7409607   14684    43310             0 VBoxHeadless
Jun 19 17:28:17 localhost kernel: [47415]     0 47415  8035492  7479402   14821    43240             0 VBoxHeadless
Jun 19 17:28:17 localhost kernel: [49162]     0 49162  1471349  1018533    2385   103763             0 VBoxHeadless
Jun 19 17:28:17 localhost kernel: [50817]    89 50817    22291        0      45      259             0 pickup
Jun 19 17:28:17 localhost kernel: Out of memory: Kill process 47415 (VBoxHeadless) score 296 or sacrifice child
Jun 19 17:28:17 localhost kernel: Killed process 47415 (VBoxHeadless) total-vm:32141968kB, anon-rss:0kB, file-rss:29917608kB, shmem-rss:0kB
Jun 19 17:28:17 localhost kernel: VBoxDrvLinuxIOCtl: copy_to_user(0x7ffb37282c70,,0x18); uCmd=0xc0305698!
Jun 19 17:28:17 localhost kernel: VBoxDrvLinuxIOCtl: copy_to_user(0x7ffb37181c70,,0x18); uCmd=0xc0305698!
Jun 19 17:28:17 localhost kernel: VBoxDrvLinuxIOCtl: copy_to_user(0x7ffb36c7cc70,,0x18); uCmd=0xc0305698!
Jun 19 17:28:17 localhost kernel: VBoxDrvLinuxIOCtl: copy_to_user(0x7ffb37080c70,,0x18); uCmd=0xc0305698!
Jun 19 17:28:17 localhost kernel: VBoxDrvLinuxIOCtl: copy_to_user(0x7ffb36b7bc70,,0x18); uCmd=0xc0305698!
Jun 19 17:28:17 localhost kernel: VBoxDrvLinuxIOCtl: copy_to_user(0x7ffb36979c70,,0x18); uCmd=0xc0305698!
Jun 19 17:28:17 localhost kernel: VBoxDrvLinuxIOCtl: copy_to_user(0x7ffb36a7ac70,,0x18); uCmd=0xc0305698!
Jun 19 17:28:17 localhost kernel: VBoxDrvLinuxIOCtl: copy_to_user(0x7ffb36d7dc70,,0x18); uCmd=0xc0305698!
Jun 19 17:28:17 localhost kernel: VBoxDrvLinuxIOCtl: copy_to_user(0x7ffb045ebda0,,0x18); uCmd=0xc0305698!
Jun 19 17:28:17 localhost kernel: VBoxDrvLinuxIOCtl: copy_to_user(0x7ffb04811da0,,0x18); uCmd=0xc0305698!
Jun 19 17:28:17 localhost kernel: VBoxDrvLinuxIOCtl: copy_to_user(0x7ffb36e7ec90,,0x30); uCmd=0xc0305687!
Jun 19 17:28:17 localhost kernel: VBoxDrvLinuxIOCtl: copy_to_user(0x7ffb04373db0,,0x18); uCmd=0xc0305698!
Jun 19 17:28:17 localhost kernel: VBoxDrvLinuxIOCtl: copy_to_user(0x7ffb36777c70,,0x18); uCmd=0xc0305698!
Jun 19 17:28:17 localhost kernel: VBoxDrvLinuxIOCtl: copy_to_user(0x7ffb36878c70,,0x18); uCmd=0xc0305698!
Jun 19 17:28:17 localhost kernel: tuned invoked oom-killer: gfp_mask=0x200da, order=0, oom_score_adj=0
Jun 19 17:28:17 localhost kernel: tuned cpuset=/ mems_allowed=0-1
Jun 19 17:28:17 localhost kernel: CPU: 7 PID: 9287 Comm: tuned Kdump: loaded Tainted: G           OE  ------------   3.10.0-957.12.1.el7.x86_64 #1
Jun 19 17:28:17 localhost kernel: Hardware name: Dell Inc. PowerEdge R730/072T6D, BIOS 2.8.0 005/17/2018
 active_file:31 inactive_file:1121 isolated_file:0
 unevictable:18 dirty:7 writeback:19 unstable:0
 slab_reclaimable:125382 slab_unreclaimable:170275
 mapped:15283635 shmem:0 pagetables:33115 bounce:0
 free:55900 free_pcp:491 free_cma:0


方案一 禁掉oom,可能会导致系统挂起
方案二 分配给虚拟机更小的内存,否则虚拟机还是会逐渐占用更多的内存,然后让主机OOM(最优)
发现是虚拟机分配多少内存,就会预占物理机多少内存,而虚拟机分配的内存过多,导致虚拟机的今后才能被杀,重新配置虚拟机的内存之后,虚拟机运行正常
方案三 定期清理缓存和释放内存,crontab? 效果不好,还是挡不住virtualbox抢占内存 echo 3 > /proc/sys/vm/drop_caches

你可能感兴趣的:(2019-06-21 虚拟机进程经常触发OOM,被物理机杀死问题与解决方案)