WHAT IS NUMA（未完成）

Non Uniform Memory Access

first became available in commercial Linux distributions in 2004(SLES9 and RHEL 4)

local reclaim and memory migration

lowest possible NUMA distance is 10

off-node access that is slower than a local access.

14 is a common distance to a neighboring node

3. Linux and NUMA memory

3.1. Memory Management(传统UMA结构)

Central to all memory

management is the page allocator.

The page size is fixed in hardware at 4 KBytes for i386, x64 and many other architectures.(On Itanium the page size is usually configured to be 16k.)

There are two ways that pages mapped into user space are used:

The first type of pages is used for anonymous memory.

The page cache or buffers are pages that have an associated page on a secondary storage medium such as a disk.

Page cache pages can be removed if memory becomes tight because their content can be restored by reading the page from disk.

Most important is that the page cache contains the executable code for a process. A process may map additional files into its address space via the mmap() system call.

Both the executable code and the mapped files are directly addressable via virtual addresses from the user process.

The operating system may also maintain buffers in the page cache that are not mapped

into the address space of a process. This frequently occurs if a process manipulates files through system

calls like sys_write(), sys_read() that read and modify the contents of files.

The slab allocator retrieves individual pages or contiguous ranges of pages from the page allocator but then uses its own control structures to be able to hand out memory chunks of varying sizes as requested by the kernel or drivers.

Slab allocations are used to build up structures that maintain the current system state. This includes information about open files, recently used filenames and a variety of other state objects.

The device drivers utilize both the page allocator and the slab allocator to allocate memory to manage devices.

There are a couple of additional variations on page sized allocations for device drivers. First is the vmalloc subsystem.Vmalloc allows the allocation of larger chunks of memory that appear to be virtually contiguous within kernel context but the actual pages constituting this allocation may not be physically contiguous.Therefore vmalloc can generate a virtually contiguous memory for large chunks of memory.Accesses to memory obtained via the vmalloc allocator must use a page table to translate the virtual addresses to physical addresses and may be not as efficient as using a direct physical address as handed out from the page allocator.

Finally, the PCI subsystem itself may can be used by a device driver to request memory that is suitable for DMA transfers for a given device via dma_alloc_coherent(). The way of obtaining that type of memory varies with the type of underlying hardware

3.2. NUMA memory

On a NUMA system memory is managed separately for each node.

1 Unified Memory Architecture (UMA).

2 Non-Unified Memory Architecture (NUMA).