SPDK

文章的具体内容将会一点的展开.

TODO:

SPDK综述, SPDK使用, SPDK代码概述

SPDK综述

SPDK使用

DPDK使用

1. 获取源代码

# git clone http://dpdk.org/git/dpdk

2. 进入目录

# cd dpdk

3. 启用pcap(需要安装libpcap-devel for centos, libpcap-dev for ubuntu)

# make config T=x86_64-native-linuxapp-gcc
# sed -ri 's,(PMD_PCAP=).*,\1y,' build/.config

4. 编译

# make

make 出现错误:

"error: implicit declaration of function ‘_mm_alignr_epi8’"

解决方法: cpu 不支持 sse3 指令(grep sse3 /proc/cpuinfo),

如果是虚拟机,主机支持,只需要虚拟机设置时CPU与主机一样;

如果主机不支持,那么只能更换主机cpu了.

SPDK 中 spdk_nvme_probe 流程:

spdk_nvme_probe -->
        nvme_pci_enumerate -->
+-
|               spdk_pci_enumerate -->
|                       enum_cb
|               <-- nvme_pci_enum_cb -->
|                       spdk_pci_device_get_class
|                       ctx->user_enum_cb
|       <-- nvme_enum_cb -->
|               spdk_nvme_ctrlr_opts_set_defaults
|               enum_ctx->probe_cb
| <-- probe_cb -->
|       spdk_pci_device_has_non_uio_driver
|               nvme_attach -->
|                       nvme_malloc
|                       nvme_ctrl_construct -->
|                               nvme_ctrlr_allocate_bars -->
|                                       nvme_pcicfg_map_bar -->
|                                               pci_device_map_range
|                                       nvme_ctrlr_map_cmb -->
|                                               nvme_pcicfg_get_bar_addr_len
|                                               nvme_pcicfg_map_bar_write_combine
|                               nvme_pcicfg_read32(spdk_pci_device_cfg_read32) -->
|                                       pci_device_cfg_read_u32
|                               nvme_pcicfg_write32(spdk_pci_device_cfg_write32) -->
|                                       pci_device_cfg_write_u32
|                               nvme_ctrlr_construct_admin_qpair -->
|                                       nvme_qpair_construct
+-
+-
|       nvme_ctrlr_process_init -->
|               nvme_mmio_write_4
|               nvme_ctrl_set_state
|       nvme_ctrlr_process_init -->
|               nvme_ctrlr_enable
|               nvme_ctrl_set_state
|       nvme_ctrlr_process_init -->
|               nvme_ctrlr_start -->
|                       nvme_qpair_reset
|                       nvme_qpair_enable
|                       nvme_ctrlr_identify -->
|                               nvme_ctrlr_cmd_identify_controller -->
|                                       nvme_allocate_request_contig -->
|                                               nvme_allocate_request -->
|                                                       nvme_alloc_request
|                                       nvme_ctrlr_submit_admin_request -->
|                                               nvme_qpair_submit_request -->
|                                                       nvme_qpair_check_enabled
|                               spdk_nvme_qpair_process_completions -->
|                                       nvme_qpair_check_enabled
|                                       nvme_qpair_complete_tracker
|                       <-- nvme_completion_poll_cb
|                                       spdk_mmio_write_4
|                       nvme_ctrlr_set_num_qpairs
|                       nvme_ctrlr_construct_io_qpairs
|                       nvme_ctrlr_construct_namespaces
|                       nvme_ctrlr_configure_aer
|                       nvme_ctrlr_set_supported_log_pages
|                       nvme_ctrlr_set_supported_features
|               nvme_ctrl_set_state
|       attach_cb -->
| <-- attach_cb
|       register_ns
+-

spdk_nvme_probe 主要分为两大部分(稍后有会详解)

SPDK代码概述

参考:

https://software.intel.com/en-us/articles/introduction-to-the-storage-performance-development-kit-spdk