最近工作需要了解RDMA的工作细节,这里参考别人的博客,简述RDMA的三种模式和对于不同的情景如何选择RDMA的模式:
https://www.rdmamojo.com/2013/06/01/which-queue-pair-type-to-use/
Reliable Connected (RC) QP
One RC QP is being connected (i.e. send and receive messages) to exactly one RC QP in a reliable way. It is guaranteed that messages are delivered from a requester to a responder at most once, in order and without corruption. The maximum supported message size is up to 2GB (this value may be lower, depends on the supported RDMA device attributes). RC QP supports Send operations (w/o immediate), RDMA Write operations (w/o immediate), RDMA Read operations and Atomic operations (it depends on the RDMA device support level in atomic operations).
If a message size is bigger than the path MTU, it is being fragmented in the side that sends the data and being reassembled in the receiver side.
Requester considers a message operation complete once there is an ack from the responder side that the message was read/written to its memory.
Responder considers a message operation complete once the message was read/written to its (local) memory.
Unreliable Connected (UC) QP
One UC QP is being connected (i.e. send and receive messages) to exactly one UC QP in an unreliable way. There isn’t any guaranteed that the messages will be received by the other side: corrupted or out of sequence packets are silently dropped. If a packet is being dropped, the whole message that it belongs to will be dropped. In this case, the responder won’t stop, but continues to receive incoming packets. There isn’t any guarantee about the packet ordering. The maximum supported message size is up to 2GB (this value may be lower, depends on the support RDMA device attributes). RC QP supports Send operations (w/o immediate) and RDMA Write operations (w/o immediate).
If a message size is bigger than the path MTU, it is being fragmented in the side that sends the data and being reassembled in the receiver side.
Requester considers a message operation complete once all of the message was sent to the fabric.
Responder considers a message operation complete once it received a complete message in correct sequence and it written the data to its (local) memory.
Unreliable Datagram (UD) QP
One QP can send and receive message to any other UD QP in either unicast (one to one) or multicast (one to many) way in an unreliable way. There isn’t any guaranteed that the messages will be received by the other side: corrupted or out of sequence packets are silently dropped. There isn’t any guarantee about the packet ordering. The maximum supported message size is the maximum path MTU. UD QP supports only Send operations.
Requester considers a message operation complete once the (one packet) message was sent to the fabric.
Responder considers a message operation complete once it received a complete message and it written the data to its (local) memory.
Summary
The following table describes the characteristics of each QP Transport Service Type: