kolla-ansible多节点部署openstack victoria

资产

控制+网络：虚拟机control1、control2、control3
ubuntu1804+4C+8G+100G+双网卡（因为不用三层网络，所以只用了一个网卡，但只有一个时部署会报错）
计算+存储：物理机compute1、compute2
ubuntu1804+双硬盘+双网卡
部署+终端管理：虚拟机kolla
ubuntu1804+ 2C+4G+100G+网卡

cat /etc/hosts

ip1 control1
ip2 control2
ip3 control3
ip4 compute1
ip5 compute2

整体配置

cat /root/.pip/pip.conf 
[global]
index-url = http://ip:8081/repository/tsinghua-pypi/simple/
trusted-host = ip

cat /etc/apt/sources.list
deb http://ip:8081/repository/tsinghua-apt/ bionic main restricted universe multiverse
deb http://ip:8081/repository/tsinghua-apt/ bionic-updates main restricted universe multiverse
deb http://ip:8081/repository/tsinghua-apt/ bionic-backports main restricted universe multiverse
deb http://ip:8081/repository/tsinghua-apt/ bionic-security main restricted universe multiverse

apt-get install docker.io python-pip3

计算+存储配置

pvcreate /dev/sdb
vgcreate cinder-volumes /dev/sdb

vi /etc/lvm/lvm.conf

devices {
     
filter = [ "a|sdb|", "r|.*|" ]
}

systemctl restart lvm2-lvmetad.service

#共享卷
vi /etc/systemd/system/docker.service.d/kolla.conf

[Service]
MountFlags=shared

systemctl daemon-reload
systemctl restart docker

部署+终端管理配置

pip3 install ansible==2.9
pip3 install kolla-ansible

ssh-keygen
ssh-copy-id -i /root/.ssh/id_rsa.pub control1
....

cp -r /usr/local/share/kolla-ansible/etc_examples/kolla /etc/
cp /usr/local/share/kolla-ansible/ansible/inventory/* /etc/kolla/

kolla-genpwd

vi /etc/kolla/passwords.yml
keystone_admin_password: 密码

grep -Ev "^$|#" /etc/kolla/globals.yml  
---
kolla_base_distro: "ubuntu"
kolla_install_type: "binary"
openstack_release: "victoria"
node_custom_config: "/etc/kolla/config"
kolla_internal_vip_address: "172.17.20.125"
network_interface: "eth0"
api_interface: "{
     { network_interface }}"
network_address_family: "ipv4"
api_address_family: "{
     { network_address_family }}"
storage_address_family: "{
     { network_address_family }}"
migration_address_family: "{
     { api_address_family }}"
dns_address_family: "{
     { network_address_family }}"
neutron_external_interface: "eth1"
neutron_plugin_agent: "openvswitch"
keepalived_virtual_router_id: "55"
enable_openstack_core: "yes"
enable_haproxy: "yes"
enable_cinder: "yes"
enable_cinder_backend_iscsi: "yes"
enable_cinder_backend_lvm: "yes"
enable_heat: "yes"
cinder_volume_group: "cinder-volumes"
nova_compute_virt_type: "kvm"

vi /etc/kolla/multinode


kolla-ansible -i /etc/kolla/multinode bootstrap-servers
kolla-ansible -i /etc/kolla/multinode prechecks
kolla-ansible -i /etc/kolla/multinode pull
kolla-ansible -i /etc/kolla/multinode deploy
kolla-ansible -i /etc/kolla/multinode post-deploy

apt-get install python3-openstackclient

踩坑

1.远程会话会致使免密登录失效
解决：
cd /root/.ssh | eval $(ssh-agent) && ssh-add id_rsa
2.拉取镜像时遇到些许失败，需要多拉几次kolla-ansible -i /etc/kolla/multinode pull
3.部署时遇到neutron container初始启动错误，找不到xxx文件/var/lib/kolla/config_files/config.json
解决：
/var/lib/kolla/config_files为容器内的目录，运行时通过config.json将宿主机相关配置文件的目录映射至容器内第一个目录内，再连接到另一个目录。
进入控制节点的/etc/kolla/neutron-server，发现是空目录，其他组件都有对应文件，百度、必应无果，果断从计算节点远程scp传输config.json  ml2_conf.ini neutron.conf openvswitch_agent.ini，解决
4.创建实例一直调度中
解决：
控制节点进入/var/log/kolla查日志，果断先查nova/nova-compute.log，发现
oslo_messaging.exceptions.MessageDeliveryFailure: Unable to connect to AMQP server on ip，果断重启rabbitmq的容器。
5.创建实例报错“Exceeded max scheduling attempts 3 for instance xxx. Last exception:”，依次查询日志/var/log/kolla/nova/nova-compute.log、/var/log/kolla/neutron/neutron-server.log发现
Instance failed network setup after 1 attempt(s): nova.exception.PortBindingFailed: 
Binding failed for port, please check neutron logs for more information
以及
Volume attachment could not be found with filter，考虑是前者端口绑定失败引起后者卷挂载失败
解决：
参考控制节点：
cat /etc/kolla/neutron-openvswitch-agent/openvswitch_agent.ini
修改计算节点：
vi /etc/kolla/neutron-openvswitch-agent/openvswitch_agent.ini
[OVS]
bridge_mappings = physnet1:br-ex #外部网络名称:外部网络网桥

apt -y install openvswitch-switch
service openvswitch-switch start

ovs-vsctl add-br br-ex
ovs-vsctl add-port br-ex eth1 #外部网桥 外部网络所连网卡名称

附

openstack二三层网络机制

qbr是security group
br-int、br-tun、br-ex 属于ovs或linux bridge部分
router、dhcp是网络命名空间

二层网络：直接流向计算节点
三层网络：流向计算节点后，再经由网络节点出入