Kylin v10基于cloud-init实现分区自动扩容(LVM分区模式)

1. 环境:

云平台:兼容OpenStack Queens的发行版
HOST OS:Kylin-Server-10-SP1-Release-Build20-20210518-arm64
虚拟机镜像ISO:Kylin-Server-10-SP1-Release-Build04-20200711-arm64.iso


2. 定制cloud-init镜像

cloud-init概念网上已有大量资料,这里不赘述。

2.1 安装OS

基于Kylin-Server-10-SP1-Release-Build04-20200711-arm64.iso,通过virt-manager安装操作系统,语言建议选择英文,软件选择根据自己情况选择,我这里选择Server with UKUI GUI,分区选择基于LVM的自定义分区,卷组设置为vklas,有别于物理机的klas。安装操作系统具体步骤参考网上其他资料或者Kylin手册

2.2 安装cloud-init及相关依赖包

配置源或者手动下载包https://update.cs2c.com.cn/NS/V10/V10SP1.1/os/adv/lic/base/aarch64/Packages/,一下通过yum安装软件包:

yum install cloud-init cloud-utils-growpart

2.3 配置启用密码登录

vim /etc/cloud/cloud.cfg

ssh_pwauth:   1

2.4 配置自动扩容

通过fdisk -l系统分区情况,如下所示:

Disk /dev/vda: 30 GiB, 32212254720 bytes, 62914560 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 4B4A0921-D392-4EAA-934B-8DAF77DFDB2F

Device       Start      End  Sectors  Size Type
/dev/vda1     2048   411647   409600  200M EFI System
/dev/vda2   411648  2508799  2097152    1G Linux filesystem
/dev/vda3  2508800 62914526 60405727 28.8G Linux LVM

通过blkid查看块设备详情,以识别一个块设备内容的类型(如文件系统、交换区)以及从内容的元数据(如卷标或 UUID 字段)中获取属性(如 tokens 和键值对),这里我们获取到我们需要的文件系统为XFS,如下所示:

/dev/vda1: SEC_TYPE="msdos" UUID="2783-50B3" TYPE="vfat" PARTLABEL="EFI System Partition" PARTUUID="70420fc9-f542-4a81-9732-4dd2d7ccdd74"
/dev/vda2: UUID="ec11ef2f-3137-44ca-b9ec-913ded527e24" TYPE="xfs" PARTUUID="0ac70b91-3e33-45c7-99f7-827a41cc4520"
/dev/vda3: UUID="mjo8Po-vWNy-cYSQ-xHC2-tYn9-Ho4A-ZnBqvo" TYPE="LVM2_member" PARTUUID="262a7de7-d914-4949-a3cf-381af4bc9322"
/dev/mapper/vklas-root: UUID="a53de3da-a8f1-420b-889e-2393696865d4" TYPE="xfs"
/dev/mapper/vklas-swap: UUID="7c98c77c-cef5-4188-a45f-66838a81baef" TYPE="swap"

根据以上信息,得知我们需要自动扩容vda3分区(growpart)、扩容LVM和xfs文件系统(runcmd)。
创建文件/etc/cloud/cloud.cfg.d/06_growpart.cfg
vim /etc/cloud/cloud.cfg.d/06_growpart.cfg

#cloud-config
growpart:
  mode: auto
  devices: [/dev/vda3]
  ignore_growroot_disabled: false
runcmd:
  - [pvresize,/dev/vda3]
  - [lvextend,-l,+100%FREE,/dev/mapper/vklas-root]
  - [xfs_growfs,/]

注意:文件系统为ext2、ext3、ext4,则采用通过resize2fs 进行调整

2.5 每次重启均调整

该需求涉及到云平台会有调整虚拟机的需求,也就是虚拟机磁盘进行调整后,文件系统也依然需要调整,故衍生出每次调整后自动扩容的了。不同的cloud-init模块有不同的执行频次:once-per-instance 、 always。可以在Modules文档中https://cloudinit.readthedocs.io/en/latest/topics/modules.html查看每个模块的Module frequency属性。例如,bootcmd在每次引导时运行,而runcmd只在第一次引导时运行。频次也可以在cloud.cfg配置文件模块列表中覆盖——cloud_init_modules, cloud_config_modules和cloud_final_modules。
故我们有bootcmd和runcmd两种方法实现该需求。

2.5.1 runcmd

通过修改/etc/cloud/cloud.cfg文件,覆盖模块频次
vim /etc/cloud/cloud.cfg

cloud_config_modules:
 - [runcmd,always]

2.5.2 bootcmd

修改/etc/cloud/cloud.cfg.d/06_growpart.cfg,将runcmd替换成bootcmd,全文如下:

#cloud-config
growpart:
  mode: auto
  devices: [/dev/vda3]
  ignore_growroot_disabled: false
bootcmd:
  - [pvresize,/dev/vda3]
  - [lvextend,-l,+100%FREE,/dev/mapper/vklas-root]
  - [xfs_growfs,/]

在云平台中调整后,还需再次重启才能生效

2.6 定制分区

略,可参考根分区扩容模式

2.7 清理网络信息

清理virt-manager启动虚拟机中所含有的现有网络信息,然后关闭虚拟机,镜像制作完毕。

2.8 上传至云平台

source /root/admin-openrc.sh
glance image-create --name Kylin-Server-10-SP1-Release-Build04-20200711-arm64-cloudinit --disk-format qcow2 --container-format bare --file Kylin-Server-10-SP1-Release-Build04-20200711-arm64-cloudinit.qcow2 --property hw_scsi_model=virtio-scsi --property hw_disk_bus=virtio --property os_distro=kylin --visibility public --progress

注意:由于上述分区采用的是virtio模式,故这个属性也必须设置为–property hw_disk_bus=virtio。

启动虚拟机,指定相应的磁盘大小,如500GB,进入虚拟机查看根分区扩容情况,df -hl查看如下所示:

Filesystem              Size  Used Avail Use% Mounted on
devtmpfs                7.5G     0  7.5G   0% /dev
tmpfs                   7.8G  192K  7.8G   1% /dev/shm
tmpfs                   7.8G   17M  7.7G   1% /run
tmpfs                   7.8G     0  7.8G   0% /sys/fs/cgroup
/dev/mapper/vklas-root  496G   13G  484G   3% /
tmpfs                   7.8G  256K  7.8G   1% /tmp
/dev/vda2              1014M  226M  789M  23% /boot
/dev/vda1               200M  5.8M  195M   3% /boot/efi
tmpfs                   1.6G  512K  1.6G   1% /run/user/0

虚拟机启动和调整参考各个openstack发行版本或商业版本,可能略有不同,这里不再赘述。


3. 问题记录:

1、growpart unexpected output in sfdisk --version [sfdisk,来自 util-linux 2.34]
自动扩容失败,通过查看/var/log/cloud-init.log,发现如下日志:

2022-06-06 02:53:53,571 - util.py[DEBUG]: Running command ['growpart', '--help'] with allowed return codes [0] (shell=False, capture=True)
2022-06-06 02:53:53,584 - util.py[DEBUG]: Reading from /sys/class/block/vda3/partition (quiet=False)
2022-06-06 02:53:53,584 - util.py[DEBUG]: Read 2 bytes from /sys/class/block/vda3/partition
2022-06-06 02:53:53,585 - util.py[DEBUG]: Reading from /sys/devices/pci0000:00/0000:00:01.4/0000:05:00.0/virtio3/block/vda/dev (quiet=False)
2022-06-06 02:53:53,585 - util.py[DEBUG]: Read 6 bytes from /sys/devices/pci0000:00/0000:00:01.4/0000:05:00.0/virtio3/block/vda/dev
2022-06-06 02:53:53,585 - util.py[DEBUG]: Running command ['growpart', '--dry-run', '/dev/vda', '3'] with allowed return codes [0] (shell=False, capture=True)
2022-06-06 02:53:53,600 - util.py[WARNING]: Failed growpart --dry-run for (/dev/vda, 3)
2022-06-06 02:53:53,600 - util.py[DEBUG]: Failed growpart --dry-run for (/dev/vda, 3)
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/cloudinit/config/cc_growpart.py", line 145, in resize
    util.subp(["growpart", '--dry-run', diskdev, partnum])
  File "/usr/lib/python3.7/site-packages/cloudinit/util.py", line 1847, in subp
    cmd=args)
cloudinit.util.ProcessExecutionError: Unexpected error while running command.
Command: ['growpart', '--dry-run', '/dev/vda', '3']
Exit code: 2
Reason: -
Stdout: -
Stderr: unexpected output in sfdisk --version [sfdisk,来自 util-linux 2.34]
2022-06-06 02:53:53,604 - util.py[DEBUG]: resize_devices took 0.021 seconds
2022-06-06 02:53:53,604 - cc_growpart.py[DEBUG]: '/dev/vda3' FAILED: failed to resize: disk=/dev/vda, ptnum=3: Unexpected error while running command.
Command: ['growpart', '--dry-run', '/dev/vda', '3']
Exit code: 2
Reason: -
Stdout: -
Stderr: unexpected output in sfdisk --version [sfdisk,来自 util-linux 2.34]

手动执行growpart /dev/vda 3依然报unexpected output in sfdisk --version [sfdisk,来自 util-linux 2.34],通过查阅资料发现为LANG的中文问题。通过执行export LANG="en_US.UTF-8"解决,需要cloud-init生效则需要修改/etc/locale.conf,如下内容:

LANG="en_US.UTF-8"

感谢phanx网友,原先的/etc/environment修改确实存在问题。
2、xfs_growfs: /dev/mapper/vklas-root is not a mounted XFS filesystem
线扩容系统盘报“xfs_growfs:is not a mounted XFS filesystem”错误,经查阅新旧版本的xfs_growfs命令使用问题。新版的xfs_growfs命令后面的参数需要为文件系统挂载点,而非文件系统本身。执行如下命令:

xfs_growfs /

4. 参考文献:

https://zhuanlan.zhihu.com/p/27664869
https://cloudinit.readthedocs.io/en/latest/topics/tutorial.html
https://github.com/canonical/cloud-init/pull/721
https://www.cnblogs.com/blog-lhong/p/11858396.html
https://forum.proxmox.com/threads/cloud-init-lvm-resize-not-working.68947/
https://cloud.tencent.com/developer/article/1501295
https://blog.csdn.net/qq_37242520/article/details/106421730
https://serverfault.com/questions/1090101/how-to-ensure-cloud-init-runs-exactly-once-and-once-only

你可能感兴趣的:(cloud-init,云计算,kylin,linux,运维)