openstack rocky AIO部署时遇到的问题及源码溯源

环境

ubuntu 18.04.2
python 2.7

问题一

按照官方教程一步步走
https://docs.openstack.org/openstack-ansible/latest/user/aio/quickstart.html

但是到 # scripts/bootstrap-aio.sh就出问题了
错误信息如下:
TASK [bootstrap-host : Fail if there is not enough space available in /] ******* fatal: [localhost]: FAILED! => {"changed": false, "msg": "Not enough space available in /.\nFound 0.0 GB, required 50 GB)\n"}

居然提示内存空间不足。

查看/可用空间,

文件系统            1K-块     已用       可用 已用% 挂载点
udev             32816228        0   32816228    0% /dev
tmpfs             6567676     2368    6565308    1% /run
/dev/nvme0n1p3  490140632 18205492  446967616    4% /
tmpfs            32838364    86376   32751988    1% /dev/shm
tmpfs                5120        4       5116    1% /run/lock
tmpfs            32838364        0   32838364    0% /sys/fs/cgroup
/dev/loop0          35456    35456          0  100% /snap/gtk-common-themes/818
/dev/loop1           4224     4224          0  100% /snap/gnome-calculator/352
/dev/loop2          14976    14976          0  100% /snap/gnome-logs/45
/dev/loop3           3840     3840          0  100% /snap/gnome-system-monitor/70
/dev/loop4          93184    93184          0  100% /snap/core/6350
/dev/loop6          15104    15104          0  100% /snap/gnome-characters/206
/dev/loop7           3840     3840          0  100% /snap/gnome-system-monitor/57
/dev/loop5           1024     1024          0  100% /snap/gnome-logs/57
/dev/loop8          36224    36224          0  100% /snap/gtk-common-themes/1198
/dev/loop9           2304     2304          0  100% /snap/gnome-calculator/260
/dev/loop10         91392    91392          0  100% /snap/core/6673
/dev/nvme0n1p2     594784     6248     588536    2% /boot/efi
/dev/loop12         55040    55040          0  100% /snap/core18/782
/dev/loop11        144128   144128          0  100% /snap/gnome-3-26-1604/82
/dev/loop13        144128   144128          0  100% /snap/gnome-3-26-1604/74
/dev/loop14         13312    13312          0  100% /snap/gnome-characters/139
tmpfs             6567672       16    6567656    1% /run/user/121
tmpfs             6567672       44    6567628    1% /run/user/1000
/dev/sda1       960568328    77888  911626492    1% /usr/local/openstack
/dev/sda2       479567536  3323764  451813336    1% /media/rise/7ab2a651-a610-4ccf-b1db-20030e67fe18
/dev/sda3       479567536    73764  455063336    1% /media/rise/c3ded1c0-8688-485f-97ff-96fcb8c79fb1
/dev/loop15        126976   126976          0  100% /snap/vscode/89
/dev/loop17    1073610752  1105952 1072504800    1% /var/lib/nova/instances
/dev/loop18    1073610752  1105952 1072504800    1% /srv/swift1.img
/dev/loop19    1073610752  1105952 1072504800    1% /srv/swift2.img
/dev/loop20    1073610752  1105952 1072504800    1% /srv/swift3.img
/dev/loop21     489684992    16672  487570432    1% /var/lib/machines
tmpfs             6567672        0    6567672    0% /run/user/0

完全没有问题啊!!!

经过半天的追查,在/opt/openstack-ansible/tests/roles/bootstrap-host/tasks/check-requirements.yml 文件中发现一个有趣的事实。

- name: Fail if there is not enough space available in /
  fail:
    msg: |
      Not enough space available in /.
      Found {{ root_gb_available }} GB, required {{ bootstrap_host_data_disk_min_size }} GB)
  when:
    - bootstrap_host_data_disk_device == None
    - (host_root_space_available_bytes | int) < (host_data_disk_min_size_bytes | int)
  tags:
    - check-disk-size

如上代码所示,直接原因是host_root_space_available_bytes为0,根据下面源码看出这个变量又依赖root_space_available

# Convert root_space_available to bytes.
- name: Set root disk facts
  set_fact:
    host_root_space_available_bytes: "{{ ( root_space_available.stdout | int) * 1024 | int }}"
  when:
    - bootstrap_host_data_disk_device == None
  tags:
    - check-disk-size

root_space_available又来自一个register,看下面源码:

- name: Identify the space available in /
  # NOTE(hwoarang): df does not work reliably on btrfs filesystems
  # https://btrfs.wiki.kernel.org/index.php/FAQ#How_much_free_space_do_I_have.3F
  # As such, use the btrfs tools to determine the real available size on the
  # disk
  shell: |
    if [[ $(df -T / | tail -n 1 | awk '{print $2}') == "btrfs" ]]; then
        btrfs fi usage --kbytes / | awk '/^.*Free / {print $3}'| sed 's/\..*//'
    else
        df -BK / | awk '!/^Filesystem/ {print $4}' | sed 's/K//' 
    fi
  when:
    - bootstrap_host_data_disk_device == None
  changed_when: false
  register: root_space_available
  tags:
    - check-disk-size

中间那个shell执行因为我的环境不是btrfs文件系统,所以进入else,重点是我环境是中文的,所以这个else的输出是

# df -BK / | awk '!/^Filesystem/ {print $4}' | sed 's/K//' 
可用
446940080

有没有搞错啊!!!居然还有中文和换行,所以解决方法也简单
所以要把这一行做个调整
df -BK / | awk '!/^Filesystem/ {print $4}' | sed 's/K//' | sed -n "2p"

OK!

问题二

TASK [lxc_hosts : Ensure that the LXC cache has been prepared] **************************************************************************
task path: /etc/ansible/roles/lxc_hosts/tasks/lxc_cache_preparation.yml:137
 The "physical_host" variable of "aio1" has been found to have a corresponding host entry in inventory.
 The "physical_host" variable of "aio1" terminates at "172.29.236.100" using the host variable "ansible_host".
FAILED - RETRYING: Ensure that the LXC cache has been prepared (120 retries left).
FAILED - RETRYING: Ensure that the LXC cache has been prepared (119 retries left).
FAILED - RETRYING: Ensure that the LXC cache has been prepared (118 retries left).
FAILED - RETRYING: Ensure that the LXC cache has been prepared (117 retries left).
FAILED - RETRYING: Ensure that the LXC cache has been prepared (116 retries left).
FAILED - RETRYING: Ensure that the LXC cache has been prepared (115 retries left).
FAILED - RETRYING: Ensure that the LXC cache has been prepared (114 retries left).
fatal: [aio1]: FAILED! => {"ansible_job_id": "616180643313.20921", "attempts": 8, "changed": true, "cmd": "chroot /var/lib/machines/ubuntu-bionic-amd64 /opt/cache-prep-commands.sh > /var/log/lxc-cache-prep-commands.log 2>&1", "delta": "0:01:09.560041", "end": "2019-04-04 10:55:54.090647", "finished": 1, "msg": "non-zero return code", "rc": 100, "start": "2019-04-04 10:54:44.530606", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}

这里截取部分日志,打印结果显示错误出现在/etc/ansible/roles/lxc_hosts/tasks/lxc_cache_preparation.yml:137,查看源码,如下:

- name: Ensure that the LXC cache has been prepared
  async_status:
    jid: "{{ _lxc_cache_prepare_commands.ansible_job_id }}"
  register: _lxc_cache_prepare_commands_result
  until: _lxc_cache_prepare_commands_result.finished
  delay: 10
  retries: "{{ lxc_cache_prep_timeout | int // 10 }}"

初步结论是job id超时出错,将delay设置为20,再次运行,解决了。

你可能感兴趣的:(openstack rocky AIO部署时遇到的问题及源码溯源)