LXD教程入门实践 配置独立ip 挂载gpu显卡驱动 制作镜像

机器环境

• 以Ubuntu 16.04.6 LTS系统为例

安装lxd3.0

方式一、apt安装方式lxd

sudo apt install lxd

• ubuntu16系统使用apt方式安装的lxd是2.0
• ubuntu18系统使用apt方式安装的lxd是3.0
• 实际工作需要lxd3.0，故使用以下snap方式安装

方式二、snap安装方式lxd

sudo snap install lxd --channel=3.0/stable

安装zfs和bridge-utils

• zfs用于管理物理磁盘，支持lxd高级功能
• bridge-utils 用于搭建网桥

sudo apt-get install zfsutils-linux bridge-utils

---

配置网桥

• 一般教程，安装完lxd相关依赖，会进行sudo lxd init操作
• 建议在初始化前，配置好网桥，可实现独立ip
  • lxd3.0版本才有此功能，会用到bridge-utils依赖包

查看机器网络情况

root@ubuntu:~# ifconfig
enp2s0    Link encap:Ethernet  HWaddr 44:8a:5b:a4:62:8d  
          inet addr:192.168.1.152  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::468a:5bff:fea4:628d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:308676 errors:0 dropped:0 overruns:0 frame:0
          TX packets:23853 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:44473910 (44.4 MB)  TX bytes:1766587 (1.7 MB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:241 errors:0 dropped:0 overruns:0 frame:0
          TX packets:241 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:19272 (19.2 KB)  TX bytes:19272 (19.2 KB)

• 注意记住上图的enp2s0和lo

修改机器网络配置

编辑/etc/network/interfaces，参考以下

auto lo
iface lo inet loopback

auto br1
iface br1 inet dhcp
    bridge_ports enp2s0
iface enp2s0 inet dhcp

先关闭enp2s0，再打开br1

sudo ifdown enp2s0
sudo ifup br1

• 此时若是外网ssh连接的机器，可能会失去连接，建议在本机操作

将br1与容器相连

• 在容器已初始化的情况下，需执行命令进行br1网络与容器相连，参考文末链接文章
• 此文配置完网络后，还会初始化lxd容器，后续在sudo lxd init时可配置使br1与容器相连

---

LXD 初始化

sudo lxd init

ZFS设置大小要尽量大，注意网络相关选项，参考如下：

Do you want to configure a new storage pool (yes/no) [default=yes]? 
Name of the storage backend to use (dir or zfs) [default=zfs]: ^C
root@ubuntu:~# sudo lxd init
Do you want to configure a new storage pool (yes/no) [default=yes]? yes
Name of the storage backend to use (dir or zfs) [default=zfs]: zfs
Create a new ZFS pool (yes/no) [default=yes]? yes
Name of the new ZFS pool or dataset [default=lxd]: lxd
Would you like to use an existing block device (yes/no) [default=no]? no
Size in GB of the new loop device (1GB minimum) [default=43]: 43
Would you like LXD to be available over the network (yes/no) [default=no]? no
Do you want to configure the LXD bridge (yes/no) [default=yes]? yes

此时会跳到图形界面，根据提示选择不创建新的网桥，其中关键提示如下：

Would you like to configure LXD to use an existing bridge or host interface? (yes/no) [default=no]: yes
Name of the existing bridge or host interface: br1

执行完毕后，lxd安装成功，终端会有如下显示

LXD has been successfully configured.

---

容器操作

• lxd3.0安装完毕后，可安装不同操作系统的容器。
• 此时系统镜像来源于网络（此文会配置为清华源）
• 在安装了原生操作系统（此文以ubuntu16为例）的容器中，安装openssh-server，配置好公钥和ssh配置文件，再制作成本地镜像，后续可使用此镜像安装新容器，则新容器可直接ssh密钥登录

加速源

• 备份原image源，并添加清华源

  sudo lxc remote rename images images_bak 
  sudo lxc remote add images https://mirrors.tuna.tsinghua.edu.cn/lxc-images/ --protocol=simplestreams --public
  

• 列出远端images源的可用镜像

  sudo lxc image list images: 
  

下载镜像和安装容器

方式一、下载镜像到本地，再用本地镜像安装容器

• 将远端镜像下载到本地，暂不安装容器

  sudo lxc image copy images:ubuntu/16.04 local:
  

• 列出本地镜像

  sudo lxc image list
  

  root@ubunt:~# sudo lxc image list
  +-------+--------------+--------+--------------------------------------+--------+---------+-----------------------------+
  | ALIAS | FINGERPRINT  | PUBLIC |             DESCRIPTION              |  ARCH  |  SIZE   |         UPLOAD DATE         |
  +-------+--------------+--------+--------------------------------------+--------+---------+-----------------------------+
  |       | 92e395dbcf45 | no     | Ubuntu xenial amd64 (20191202_07:42) | x86_64 | 80.64MB | Dec 3, 2019 at 3:51am (UTC) |
  +-------+--------------+--------+--------------------------------------+--------+---------+-----------------------------+
  

• 根据本地镜像安装容器
  • 容器命名貌似不能带下划线

  sudo lxc launch local:92e395dbcf45 ubuntu16-clean
  

方式二、直接用远端镜像安装容器，此时远端镜像也会自动保存在本地

• 根据远端镜像安装容器

  sudo lxc launch images:ubuntu/18.04 ubuntu18-clean
  

• 列出本地镜像

  root@ubuntu:~# sudo lxc image list
  +-------+--------------+--------+--------------------------------------+--------+---------+-----------------------------+
  | ALIAS | FINGERPRINT  | PUBLIC |             DESCRIPTION              |  ARCH  |  SIZE   |         UPLOAD DATE         |
  +-------+--------------+--------+--------------------------------------+--------+---------+-----------------------------+
  |       | b9a35554db4d | no     | Ubuntu bionic amd64 (20191202_07:42) | x86_64 | 94.00MB | Dec 3, 2019 at 4:15am (UTC) |
  +-------+--------------+--------+--------------------------------------+--------+---------+-----------------------------+
  |       | 92e395dbcf45 | no     | Ubuntu xenial amd64 (20191202_07:42) | x86_64 | 80.64MB | Dec 3, 2019 at 3:51am (UTC) |
  +-------+--------------+--------+--------------------------------------+--------+---------+-----------------------------+
  

• 列出本地容器

  sudo lxc list
  

  root@ubuntu:~# sudo lxc list
  +----------------+---------+----------------------+------+------------+-----------+
  |      NAME      |  STATE  |         IPV4         | IPV6 |    TYPE    | SNAPSHOTS |
  +----------------+---------+----------------------+------+------------+-----------+
  | ubuntu16-clean | RUNNING | 192.168.1.57 (eth0)  |      | PERSISTENT | 0         |
  +----------------+---------+----------------------+------+------------+-----------+
  | ubuntu18-clean | RUNNING | 192.168.1.182 (eth0) |      | PERSISTENT | 0         |
  +----------------+---------+----------------------+------+------------+-----------+	
  

LXd 支持的镜像模板查看官方链接：https://uk.images.linuxcontainers.org/

进入容器

• bash登录容器

  sudo lxc exec ubuntu16-clean bash
  

• 查看网络，检查是否是独立ip
  • 可以看出网络是192.168开头，是独立内网ip

    root@ubuntu16-clean:~# ifconfig
    eth0      Link encap:Ethernet  HWaddr 00:16:3e:ee:e0:61  
              inet addr:192.168.1.57  Bcast:192.168.1.255  Mask:255.255.255.0
              inet6 addr: fe80::216:3eff:feee:e061/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:23344 errors:0 dropped:0 overruns:0 frame:0
              TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000 
              RX bytes:5027343 (5.0 MB)  TX bytes:1722 (1.7 KB)
    
    lo        Link encap:Local Loopback  
              inet addr:127.0.0.1  Mask:255.0.0.0
              inet6 addr: ::1/128 Scope:Host
              UP LOOPBACK RUNNING  MTU:65536  Metric:1
              RX packets:0 errors:0 dropped:0 overruns:0 frame:0
              TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1 
              RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
    

• 配置容器参数

  sudo lxc config edit gt-server
  

  architecture: x86_64
  config:
    volatile.base_image: f526bdf8ebf48f0a050cecea5fa9a16c5fb2120e23210708b33c4b9647e2a650
    volatile.eth0.hwaddr: 00:16:3e:a1:2e:eb
    volatile.idmap.base: "0"
    volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"$
    volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":10$
    volatile.last_state.power: RUNNING
  devices:
    root:
      path: /
      type: disk
  ephemeral: false
  profiles:
  - default
  stateful: false
  

• 配置默认容器参数（新容器的参数会继承default配置的参数，容器会优先使用自己的参数）

  sudo lxc profile edit default
  

• 后续ssh配置可参考另一篇博文
  Ubuntu16.04开启SSH服务远程登录 root登录 密钥登录

制作容器镜像

• ubuntu16-clean机器已执行的操作

  • 设置了root密码，安装了openssh-server，配置了/etc/ssh/sshd_config文件，将公钥写进了/root/.ssh/authorized_keys文件

• 将此时的ubuntu16-clean容器制作成新的镜像

  sudo lxc stop ubuntu16-clean
  sudo lxc publish ubuntu16-clean --alias my-image-ubuntu16  
  

  root@ubuntu:~# sudo lxc image list
  +-------------------+--------------+--------+--------------------------------------+--------+----------+-----------------------------+
  |       ALIAS       | FINGERPRINT  | PUBLIC |             DESCRIPTION              |  ARCH  |   SIZE   |         UPLOAD DATE         |
  +-------------------+--------------+--------+--------------------------------------+--------+----------+-----------------------------+
  | my-image-ubuntu16 | f526bdf8ebf4 | no     |                                      | x86_64 | 137.07MB | Dec 3, 2019 at 7:25am (UTC) |
  +-------------------+--------------+--------+--------------------------------------+--------+----------+-----------------------------+
  |                   | b9a35554db4d | no     | Ubuntu bionic amd64 (20191202_07:42) | x86_64 | 94.00MB  | Dec 3, 2019 at 4:15am (UTC) |
  +-------------------+--------------+--------+--------------------------------------+--------+----------+-----------------------------+
  |                   | 92e395dbcf45 | no     | Ubuntu xenial amd64 (20191202_07:42) | x86_64 | 80.64MB  | Dec 3, 2019 at 3:51am (UTC) |
  +-------------------+--------------+--------+--------------------------------------+--------+----------+-----------------------------+   
  

• 根据新镜像my-image-ubuntu16生成新的容器

  sudo lxc launch local:my-image-ubuntu16 gt-server
  

  root@ubuntu:~# sudo lxc list
  +----------------+---------+---------------------+------+------------+-----------+
  |      NAME      |  STATE  |        IPV4         | IPV6 |    TYPE    | SNAPSHOTS |
  +----------------+---------+---------------------+------+------------+-----------+
  | gt-server      | RUNNING | 192.168.1.57 (eth0) |      | PERSISTENT | 0         |
  +----------------+---------+---------------------+------+------------+-----------+
  | ubuntu16-clean | STOPPED |                     |      | PERSISTENT | 0         |
  +----------------+---------+---------------------+------+------------+-----------+
  | ubuntu18-clean | STOPPED |                     |      | PERSISTENT | 0         |
  +----------------+---------+---------------------+------+------------+-----------+
  

  此时gt-server机器已设置好root密码，可直接ssh密钥连接

容器常见操作

• 删除容器镜像

  sudo lxc image delete b9a35554db4d
  

  root@ubuntu:~# sudo lxc image delete b9a35554db4d
  root@ubuntu:~# sudo lxc image delete 92e395dbcf45
  root@ubuntu:~# sudo lxc image list
  +-------------------+--------------+--------+-------------+--------+----------+-----------------------------+
  |       ALIAS       | FINGERPRINT  | PUBLIC | DESCRIPTION |  ARCH  |   SIZE   |         UPLOAD DATE         |
  +-------------------+--------------+--------+-------------+--------+----------+-----------------------------+
  | my-image-ubuntu16 | f526bdf8ebf4 | no     |             | x86_64 | 137.07MB | Dec 3, 2019 at 7:25am (UTC) |
  +-------------------+--------------+--------+-------------+--------+----------+-----------------------------+
  

• 删除容器

  sudo lxc delete ubuntu16-clean
  

  root@ubuntu:~# sudo lxc delete ubuntu16-clean
  root@ubuntu:~# sudo lxc delete ubuntu18-clean
  root@ubuntu:~# sudo lxc list
  +-----------+---------+---------------------+------+------------+-----------+
  |   NAME    |  STATE  |        IPV4         | IPV6 |    TYPE    | SNAPSHOTS |
  +-----------+---------+---------------------+------+------------+-----------+
  | gt-server | RUNNING | 192.168.1.57 (eth0) |      | PERSISTENT | 0         |
  +-----------+---------+---------------------+------+------------+-----------+
  

---

GPU相关

• 为容器添加显卡

  #为容器添加所有GPU:
  sudo lxc config device add gt-server gpu gpu
  #添加指定GPU：
  sudo lxc config device add gt-server gpu0 gpu id=0
  

• 容器里安装显卡驱动

  方式一、apt安装

  sudo add-apt-repository ppa:graphics-drivers/ppa
  sudo apt-get update
  #可以查看到你可以安装的nvidia驱动版本
  sudo apt list | grep nvidia
  sudo apt-get install nvidia-430
  nvidia-smi
  

  方式二、nvidia官网下载runfile文件安装
  需要注意的是容器里面安装显卡驱动时需要加上后面的参数，安装时不需要安装到内核
  sudo sh ./NVIDIA-Linux-X86_64-[YOURVERSION].run --no-kernel-module

---

附录 常用的lxd命令

# 初始化配置
sudo lxd init

# 创建ubuntu容器样例
lxc launch images:ubuntu/xenial/amd64 first

# 显示可用镜像
lxc image list images: | less
lxc image list images: 'ubuntu'

# LXD/LXC 2.0使用清华镜像加速的方法:
创建一个remote链接，指向镜像站即可，或替换掉默认的images链接。
lxc remote add tuna-images https://mirrors.tuna.tsinghua.edu.cn/lxc-images/ --protocol=simplestreams --public

# 显示可用清华镜像
lxc image list tuna-images:

# 显示当前远程可用
lxc remote list

# 显示容器网络
lxc network show lxdbr0

# 显示容器
lxc list

# 运行容器bash
lxc exec first -- /bin/bash

# 直接运行容器命令
lxc exec first -- apt-get update

# 文件传输下载
lxc file pull first/etc/hosts .

# 文件传输上传
lxc file push hosts first/tmp/

# 停止容器
lxc stop first

# 删除容器
lxc delete first

# 显示容器信息
lxc info first

# 本地配置
lxc config edit 

# 显示容器详细信息
lxc config show first
lxc config show 

# 内存查看 free -m
lxc exec first -- free -m

# 设置内存最大值
lxc config set first limits.memory 2048MB

# 执行升级
lxc exec first -- apt-get update
lxc exec first -- apt-get dist-upgrade -y
lxc exec first -- apt-get autoremove --purge -y

# 暂停
lxc pause 

# 快照 Snapshots -stateful选项包含所有状态
lxc snapshot first clean

# 列出所有的快照
lxc info 
lxc snapshot  

# 快照恢复
lxc restore first clean

# 从快照生成容器镜像
lxc publish first/clean --alias clean-ubuntu

# 启动快照容器镜像
lxc launch clean-ubuntu second

# 删除容器镜像方式一
lxc delete --force second

# 删除容器镜像方式二
lxc delete -f second

# 删除一个快照
lxc delete /

# 从快照中创建一个新的容器
lxc copy / 

# 给快照重命名
lxc move / /

# lxc运行docker
lxc config set first security.nesting true
lxc restart first

# 本地镜像导出
lxc publish ${HOST}/auto-backup --alias ${BACKUP_NAME}
lxc image list
lxc image export ${BACKUP_NAME} ${BACKUP_DIR}/${BACKUP_NAME}

# 本地镜像删除
lxc image delete 

---

参考

• ubuntu16.04-LXD配置独立ip
• LXD 2.0 系列（五）：镜像管理
• 实验室GPU服务器的LXD虚拟化
• [LXD]虚拟容器LXD/LXC的常用命令集锦 持续更新~