在使用MPI时遇到了资源受限的报错
RLIMIT_MEMLOCK too small
这是由于shell默认的资源不足造成的
可以使用ulimite
命令解决
遇到RLIMIT_MEMLOCK too small
报错主要是由于默认设置的资源不足以完成指令,特别是在MPI运行多个线程时会出现这样的问题,根据提示来修改当前shell所需要的资源大小。首先可以利用ulimite -a
查看当前默认分配的资源:
>>> ulimit -a
>>>
ulimit -a
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 7220
max locked memory (kbytes, -l) 64 <<<<<这里就是报错的位置
max memory size (kbytes, -m) unlimited
open files (-n) 1000000
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 7220
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
我们看到报错的位置locked memory只有64k,需要放大这个资源限制:
# 直接设成不受限
ulimit -l unlimited
# 再查看结果
>>> ulimit -a
>>>
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 772787
max locked memory (kbytes, -l) unlimited <<<<<<已经修改成不受限
max memory size (kbytes, -m) unlimited
open files (-n) 1000000
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 772787
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
此外可以利用ulimit -h
查看更多使用请看
> ulimit --help
> >>
Syntax
ulimit [-abcdefHilmnpqrsStTuvx] [limit]
Key
-S Set a soft limit for the given resource.
-H Set a hard limit for the given resource.
-a All current limits are reported.
-b The maximum socket buffer size.
-c The maximum size of core files created.
-d The maximum size of a process's data segment.
-e The maximum scheduling priority ("nice")
-f The maximum size of files created by the shell(default option).
-i The maximum number of pending signals.
-l The maximum size that can be locked into memory.
-m The maximum resident set size.
-n The maximum number of open file descriptors.
-p The pipe buffer size.
-q The maximum number of bytes in POSIX message queues.
-r The maximum real-time scheduling priority.
-s The maximum stack size.
-t The maximum amount of cpu time in seconds.
-T The maximum number of threads.
-u The maximum number of processes available to a single user.
-v The maximum amount of virtual memory available to the process.
-x The maximum number of file locks.
# from:https://ss64.com/bash/ulimit.html
ref:ulimit, 详解1, 详解2,原始解决方法,修改core文件大小, shell详细资源管理,detail,其他资源管理方法
pic from pexels.com