【infiniband】关于contig的支持

contig的英文翻译是：n.重叠群；叠连群；片段重叠群。

有个叫Contig的小工具，是一个单个文件碎片整理程序，其目的是使磁盘上的文件保持连续。

在读perfest源码时，看到is_contig_supported。

内核mm\page_alloc.c文件中有个__alloc_contig_pages函数，Linux内存管理：大页内存原理 - 知乎 (zhihu.com) 。

perfest相关代码：

ctx_init函数中：

	if (user_param->use_hugepages) {
		ctx->is_contig_supported = FAILURE;
	} else {
		ctx->is_contig_supported = check_for_contig_pages_support(ctx->context);
	}

create_single_mr函数中：

			#ifdef HAVE_VERBS_EXP
			exp_flags |= IBV_EXP_ACCESS_ALLOCATE_MR;


	#ifdef HAVE_VERBS_EXP
	if (ctx->is_contig_supported == SUCCESS || user_param->use_odp) {
		reg_mr_exp_in.pd = ctx->pd;
		reg_mr_exp_in.addr = ctx->buf[qp_index];
		reg_mr_exp_in.length = ctx->buff_size;
		reg_mr_exp_in.exp_access = exp_flags;
		reg_mr_exp_in.comp_mask = 0;
		ctx->mr[qp_index] = ibv_exp_reg_mr(®_mr_exp_in);
	}

Contiguous Pages

Contiguous Pages improves performance by allocating user memory regions over physical contiguous pages. It enables a user application to ask low level drivers to allocate contiguous memory for it as part of ibv_reg_mr.
Additional performance improvements can be reached by allocating Queue Pair (QP) and Completion Queue (CQ|) buffers to the Contiguous Pages.

"Contiguous Pages"是一种内存管理技术，它通过将用户内存区域分配到物理连续的内存页上来提高性能。这使得用户应用程序可以要求低级驱动程序为其分配连续的内存，作为ibv_reg_mr的一部分。通过将队列对（QP）和完成队列（CQ）缓冲区分配给Contiguous Pages，可以实现额外的性能改进。
To activate set the below environment variables with values of PREFER_CONTIG or CONTIG.

• For QP: MLX_QP_ALLOC_TYPE
• For CQ: MLX_CQ_ALLOC_TYPE

The following are all the possible values that can be allocated to the buffer:

Possible Value	Description
ANON	Use current pages ANON small ones.
HUGE	Force huge pages.
CONTIG	Force contiguous pages.
PREFER_CONTIG	Try contiguous fallback to ANON small pages. (Default)
PREFER_HUGE	Try huge fallback to ANON small pages.
ALL	Try huge fallback to contiguous if failed fallback to ANON small pages.

Note that values are NOT case sensitive.

Usage:

The application calls the ibv_exp_reg_mr API which turns on the IBV_EXP_ACCESS_ALLOCATE_MR bit and sets the input address to NULL. Upon success, the address field of the struct ibv_mr will hold the address to the allocated memory block. This block will be freed implicitly when the ibv_dereg_mr() is called.
The following are environment variables that can be used to control error cases/contiguity:

Paramters	Description
MLX_MR_ALLOC_TYPE	Configures the allocator type. ALL (Default) - uses all possible allocator and selects most efficient allocator ANON - enables the usage of anonymous pages and disables the allocator CONTIG - forces the usage of the contiguous pages allocator. If contiguous pages are not available the allocation fails
MLX_MR_MAX_LOG2_CONTIG_BSIZE	Sets the maximum contiguous block size order. Values: 12-23 Default: 23
MLX_MR_MIN_LOG2_CONTIG_BSIZE	Sets the minimum contiguous block size order. Values: 12-23 Default: 12

参考：

Linux内存管理：大页内存原理 - 知乎 (zhihu.com)

GitHub - mai-lang-chai/Sysinternals: 微软开发的系统管理小工具-简约教程-含渗透利器

Contig 文件通过 Defrag | Microsoft Learn

Optimized Memory Access - MLNX_EN v4.9-5.1.0.0 LTS - NVIDIA Networking Docs