Linux端 USBIP测试

Linux端 USBIP测试

1. 环境准备

• 机器1：这台机器就是需要插上usb， IP为172.20.156.86， 安装的是centos7

• 机器2：IP是172.20.156.105， 安装的是centos7

注：
机器1插上USB， 所以机器1装的是host， 机器2装的是vhci， 这点需要搞清楚

2. 机器1安装USBIP-host

• centos7换源：

cp CentOS-Base.repo CentOS-Base.repo.bak
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo

• 下载centos源

rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
rpm -ivh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm

• 安装usbip

yum -y install kmod-usbip usbip-utils

• 插上USB

• 挂载USB

  步骤如下:
  fdisk -l

[root@localhost ~]# fdisk -l

Disk /dev/sda: 480.1 GB, 480103981056 bytes, 937703088 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: dos
Disk identifier: 0x000df1cc

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1            2048        6143        2048   83  Linux
/dev/sda2   *        6144     2103295     1048576   83  Linux
/dev/sda3         2103296   937701375   467799040   8e  Linux LVM

Disk /dev/mapper/centos-root: 85.9 GB, 85899345920 bytes, 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/centos-swap: 68.7 GB, 68719476736 bytes, 134217728 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/centos-var: 324.4 GB, 324404248576 bytes, 633602048 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/sdb: 31.5 GB, 31457280000 bytes, 61440000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x162a1965

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *          64    61438143    30719040    c  W95 FAT32 (LBA)

​ ls /dev

[root@localhost ~]# ls /dev | grep sdb
sdb
sdb1

​ mount

[root@localhost ~]# cd /mnt
[root@localhost mnt]# ls
[root@localhost mnt]# mkdir usb
[root@localhost mnt]# mount /dev/sdb1 /mnt/usb
[root@localhost mnt]# df -h
Filesystem               Size  Used Avail Use% Mounted on
devtmpfs                  63G     0   63G   0% /dev
tmpfs                     63G     0   63G   0% /dev/shm
tmpfs                     63G   35M   63G   1% /run
tmpfs                     63G     0   63G   0% /sys/fs/cgroup
/dev/mapper/centos-root   80G  3.7G   77G   5% /
/dev/sda2               1014M  193M  822M  19% /boot
/dev/mapper/centos-var   302G   12G  291G   4% /var
tmpfs                     13G     0   13G   0% /run/user/0
/dev/sdb1                 30G   20G  9.8G  67% /mnt/usb

• 安装服务器的mod

modprobe usbip-host

• 启动守护进程

usbipd -D

• 查看端口启动

[root@localhost yum.repos.d]# netstat -tunlp | grep 3240
tcp        0      0 0.0.0.0:3240            0.0.0.0:*               LISTEN      14217/usbipd        
tcp6       0      0 :::3240                 :::*                    LISTEN      14217/usbipd  

• 查看设备

[root@localhost yum.repos.d]# usbip list --local
 - busid 1-1.2.1 (046b:ff10)
   American Megatrends, Inc. : Virtual Keyboard and Mouse (046b:ff10)

 - busid 2-1.2 (ffff:5678)
   unknown vendor : unknown product (ffff:5678)

注：

​ busid 2-1.2是我之前插入服务器的usb设备

• bind设备

[root@localhost yum.repos.d]# usbip bind -b 2-1.2
usbip: info: bind device on busid 2-1.2: complete

3. 机器2安装USB IP

• 安装客户端的mod

[root@localhost /]# modprobe vhci-hcd

• 查看远程可连接的设备

[root@localhost /]# usbip list --remote=172.20.156.86
Exportable USB devices
======================
 - 172.20.156.86
      2-1.2: unknown vendor : unknown product (ffff:5678)
           : /sys/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2
           : (Defined at Interface level) (00/00/00)

• 连接远程服务器设备

usbip attach -r 172.20.156.86 -b 2-1.2

注：

​ 这一步可能会出现错误 —— tcp connect error，经检查， 需要关闭服务器的防火墙

​ 故解决方法为：

​ 在机器1 —— centos 执行如下命令：

[root@localhost yum.repos.d]# systemctl status firewalld.service
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2020-11-16 01:39:20 CST; 19h ago
Docs: man:firewalld(1)
Main PID: 1148 (firewalld)
CGroup: /system.slice/firewalld.service
     └─1148 /usr/bin/python2 -Es /usr/sbin/firewalld --nofork --nopid

Nov 16 01:39:20 localhost.localdomain systemd[1]: Starting firewalld - dynami....
Nov 16 01:39:20 localhost.localdomain systemd[1]: Started firewalld - dynamic....
Hint: Some lines were ellipsized, use -l to show in full.
[root@localhost yum.repos.d]# systemctl stop firewalld.service

​ 再次执行后就不会报错了

• 查看设备情况

Disk /dev/sdb: 29.3 GiB, 31457280000 bytes, 61440000 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x162a1965

Device     Boot Start      End  Sectors  Size Id Type
/dev/sdb1  *       64 61438143 61438080 29.3G  c W95 FAT32 (LBA)

• 查看设备挂载

[root@localhost /]# df -h
Filesystem               Size  Used Avail Use% Mounted on
devtmpfs                  63G     0   63G   0% /dev
tmpfs                     63G     0   63G   0% /dev/shm
tmpfs                     63G   11M   63G   1% /run
tmpfs                     63G     0   63G   0% /sys/fs/cgroup
/dev/mapper/centos-root  414G  1.8G  413G   1% /
/dev/sda2               1014M  150M  865M  15% /boot
tmpfs                     13G     0   13G   0% /run/user/0

​ 故， 需要手动挂载

• 手动挂载

[root@localhost /]# ls /dev | grep sdb
sdb
sdb1
[root@localhost /]# cd /mnt
[root@localhost mnt]# ls
[root@localhost mnt]# mkdir usb
[root@localhost mnt]# mount /dev/sdb1 /mnt/usb

[root@localhost mnt]# df -h
Filesystem               Size  Used Avail Use% Mounted on
devtmpfs                  63G     0   63G   0% /dev
tmpfs                     63G     0   63G   0% /dev/shm
tmpfs                     63G   11M   63G   1% /run
tmpfs                     63G     0   63G   0% /sys/fs/cgroup
/dev/mapper/centos-root  414G  1.8G  413G   1% /
/dev/sda2               1014M  150M  865M  15% /boot
tmpfs                     13G     0   13G   0% /run/user/0
/dev/sdb1                 30G   20G  9.8G  67% /mnt/usb
# 已经成功挂载

注：

​ 这一步可能会出现挂载失败的情况， 我们可以先detach后查看/dev目录， 我们可能会发现在没有sdb的时候竟然有sdb1， 删除/dev/sdb1就可以成功挂载

4. 机器2启动docker访问远程挂载设备

注：
如下是实现并行性访问USBIP的一种思路， 如果只想用USBIP实现远程挂载USB其实上面就已经完成了， 下面这些可以不用继续做了

• 下载docker

[root@localhost yum.repos.d]# yum update

[root@localhost yum.repos.d]# yum install -y yum-utils device-mapper-persistent-data lvm2

[root@localhost yum.repos.d]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo 
# 安装docker阿里源仓库

[root@localhost yum.repos.d]# yum install docker-ce-18.03.1.ce

• docker pull镜像

[root@localhost mnt]# docker pull centos
Using default tag: latest
latest: Pulling from library/centos
3c72a8ed6814: Pull complete 
Digest: sha256:76d24f3ba3317fa945743bb3746fbaf3a0b752f10b10376960de01da70685fbd
Status: Downloaded newer image for centos:latest

• docker启动容器

[root@localhost mnt]# docker run -it --name usbip-test --privileged=true --device=/dev/sdb1 --restart=always centos

• 挂载到容器

[root@0017d3f4bae9 /]# df -h
Filesystem               Size  Used Avail Use% Mounted on
overlay                  414G  2.1G  412G   1% /
tmpfs                     64M     0   64M   0% /dev
tmpfs                     63G     0   63G   0% /sys/fs/cgroup
/dev/mapper/centos-root  414G  2.1G  412G   1% /etc/hosts
shm                       64M     0   64M   0% /dev/shm
[root@0017d3f4bae9 /]# mkdir /mnt/usb  
[root@0017d3f4bae9 /]# mount /dev/sdb1 /mnt/usb
[root@0017d3f4bae9 /]# df -h
Filesystem               Size  Used Avail Use% Mounted on
overlay                  414G  2.1G  412G   1% /
tmpfs                     64M     0   64M   0% /dev
tmpfs                     63G     0   63G   0% /sys/fs/cgroup
/dev/mapper/centos-root  414G  2.1G  412G   1% /etc/hosts
shm                       64M     0   64M   0% /dev/shm
/dev/sdb1                 30G   20G  9.8G  67% /mnt/usb

• 访问之中数据

[root@0017d3f4bae9 usb]# ls
'??'  '???'   '?????'	  FOUND.000		      'linux???'
'??'  '????'  '?????'	 'System Volume Information'
'??'  '????'  '??offer'  'linux??'

注：

​ 说明可以通过docker访问远程挂载usb设备， 由于我的usb设备全部是中文文件夹，所以显示全是？

• 断开连接 —— 先退出容器

  先查看usb port：

[root@localhost mnt]# usbip port
Imported USB devices
====================
Port 00: <Port in Use> at High Speed(480Mbps)
       unknown vendor : unknown product (ffff:5678)
       3-1 -> usbip://172.20.156.86:3240/2-1.2
           -> remote bus/dev 002/003

​ 再断开连接：

[root@localhost mnt]# usbip detach -p 00

注：

​ 此处的00即port的00

• 结束