在arm64上基于qemu的vhost user blk设备hotplug

在kata中，vhost user blk设备是hotplug进去的。本文记录一下如果在qemu中hotplug一个vhost user blk设备。

首先准备一个设备，以spdk设备为例。

build spdk

git clone https://github.com/spdk/spdk.git
cd spdk; ./configure; make -j `nproc`

制作vhost user blk

mkdir -p block/sockets
build/bin/spdk_tgt --logflag all -S block &
scripts/rpc.py bdev_malloc_create 64 4096 -b Malloc0
scripts/rpc.py vhost_create_blk_controller vhostblk0 Malloc0

在arm64上启动qemu

/usr/bin/qemu-system-aarch64 \
        -machine virt,usb=off,accel=kvm,gic-version=host\
        -cpu host,pmu=off \
        -m 2048M,slots=10,maxmem=128489M \
        -device pcie-root-port,port=0x8,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x1 \
        -device virtio-serial-pci,disable-modern=false,id=serial0 \
        -device virtconsole,chardev=charconsole0,id=console0 \
        -chardev socket,id=charconsole0,path=console.sock,server=on,wait=off \
        -device virtio-scsi-pci,id=scsi0,disable-modern=false \
        -object rng-random,id=rng0,filename=/dev/urandom \
        -device virtio-rng-pci,rng=rng0 \
        -rtc base=utc,driftfix=slew,clock=host \
        -global kvm-pit.lost_tick_policy=discard \
        -pflash /usr/share/kata-containers/kata-flash0.img \
        -pflash /usr/share/kata-containers/kata-flash1.img \
        -vga none -no-user-config -nodefaults -nographic --no-reboot  \
        -serial mon:stdio \
        -object memory-backend-file,id=dimm1,size=2048M,mem-path=/dev/hugepages,share=on,prealloc=on \
        -numa node,memdev=dimm1 \
        -monitor unix:qemu-monitor-socket,server,nowait \
        -drive if=none,file=focal-server-cloudimg-arm64-custom-20210929-0.qcow2,id=hd1 \
        -device virtio-blk-pci,drive=hd1 \
        -append "console=ttyAMA0 root=/dev/vda1 loglevel=8 acpi=on earlycon init=/bin/bash" \
        -smp 1,cores=1,threads=1,sockets=96,maxcpus=96 \
        -kernel Image \

启动vm后，在另一个shell中进入monitor

socat - unix:qemu-monitor-socket

hotplug blk 设备

chardev-add socket,id=char0,path=block/sockets/vhostblk0
#return OK
device_add vhost-user-blk-pci,id=blk0,chardev=char0,bus=pci.1
#return OK
#check qtree
info qtree

回到vm，看看是不是多了一块盘，lspci。

这里有个比较奇怪的issue，在x86上hotplug vhost user blk设备可以挂到pci-bridge上，但是在arm64的qemu/virt板上只能挂在pcie-root-port上。有待后续研究。。。