vmware exsi 6.5虚拟平台
centos 7.6
openstack queens版本,三controller节点启用高可用性,三compute节点多挂硬盘复用为ceph的osd,monitoring节点启用ceilometer、aodh数据存至gnocchi,grafana展示。cinder、glance、gnocchi后端存储均为ceph。启用裸金属ironic。
kolla-ansible离线部署
机器名称 | IP地址 | 角色 | 备注 |
kolla-ansible-master | 10.0.49.130 | kolla主机 | |
controller01 | 10.0.49.131 | controller | 所有目标主机均为双网卡,一内一外,对外IP可以不配 |
controller02 | 10.0.49.132 | controller | |
controller03 | 10.0.49.133 | controller | |
compute01 | 10.0.49.134 | compute,storage | 挂第二个硬盘160G,为ceph的osd |
compute02 | 10.0.49.135 | compute,storage | 挂第二个硬盘160G,为ceph的osd |
compute03 | 10.0.49.136 | compute,storage | 挂第二个硬盘160G,为ceph的osd |
monitoring01 | 10.0.49.139 | monitoring | aodh,ceilometer,gnocchi,grafana |
network01 | 10.0.49.148 | network | |
10.0.49.149 | VirtualIPAddress |
systemctl stop firewalld
systemctl disable firewalld
vi /etc/selinux/config
SELINUX=disabled
10.0.49.130 kolla-ansible-master
10.0.49.131 controller01
10.0.49.132 controller02
10.0.49.133 controller03
10.0.49.134 compute01
10.0.49.135 compute02
10.0.49.136 compute03
10.0.49.148 network01
10.0.49.139 monitoring01
下载安装基本工具
yum install -y epel-release
yum install -y python-pip
pip install -U pip
yum install -y python-devel libffi-devel gcc openssl-devel libselinux-python git
yum install -y ansible
#修改pip源
mkdir ~/.pip
vim ~/.pip/pip.conf
[global]
index-url = https://pypi.tuna.tsinghua.edu.cn/simple
[install]
trusted-host = https://pypi.tuna.tsinghua.edu.cn
yum install -y yum-utils device-mapper-persistent-data lvm2
yum install -y python-devel libffi-devel gcc openssl-devel libselinux-python git openldap-devel
yum install -y ansible
#修改pip源
mkdir ~/.pip
vim ~/.pip/pip.conf
[global]
index-url = https://pypi.tuna.tsinghua.edu.cn/simple
[install]
trusted-host = https://pypi.tuna.tsinghua.edu.cn
在三个计算节点上运行
fdisk -l
...
Disk /dev/sdb: 171.8 GB, 171798691840 bytes, 335544320 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
...
/dev/sdb为第二个硬盘,打上ceph标签
parted /dev/sdb -s -- mklabel gpt mkpart KOLLA_CEPH_OSD_BOOTSTRAP 1 -1
有第三个、第四个盘时依次为/dev/sdc /dev/sdd等
编辑ceph配置文件
vim /etc/kolla/config/ceph.conf
[global]
osd pool default size = 3
osd pool default min size = 3
若部署失败,开始第二次部署前需删除分区并重打标签
fdisk /dev/sdb
依次按键为 d 2 d w
parted /dev/sdb -s -- mklabel gpt mkpart KOLLA_CEPH_OSD_BOOTSTRAP 1 -1
kolla-genpwd
# 生成的文件在/etc/kolla/passwords.yml,修改二行便于使用
vi /etc/kolla/passwords.yml
gnocchi_database_password: abcd@123 #gnocchi数据库密码,便于grafana使用
keystone_admin_password: abcd@123 #OpenStack登录密码
#检查
kolla-ansible -i ./multinode bootstrap-servers
PLAY RECAP *******************************************************************************************************************************************************************************************************************************************************************
compute01 : ok=39 changed=7 unreachable=0 failed=0 skipped=19 rescued=0 ignored=0
compute02 : ok=39 changed=7 unreachable=0 failed=0 skipped=19 rescued=0 ignored=0
compute03 : ok=39 changed=7 unreachable=0 failed=0 skipped=19 rescued=0 ignored=0
controller01 : ok=39 changed=7 unreachable=0 failed=0 skipped=19 rescued=0 ignored=0
controller02 : ok=39 changed=7 unreachable=0 failed=0 skipped=19 rescued=0 ignored=0
controller03 : ok=39 changed=7 unreachable=0 failed=0 skipped=19 rescued=0 ignored=0
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
monitoring01 : ok=39 changed=7 unreachable=0 failed=0 skipped=19 rescued=0 ignored=0
network01 : ok=39 changed=7 unreachable=0 failed=0 skipped=19 rescued=0 ignored=0
kolla-ansible -i ./multinode prechecks
PLAY RECAP *******************************************************************************************************************************************************************************************************************************************************************
compute01 : ok=25 changed=1 unreachable=0 failed=0 skipped=60 rescued=0 ignored=0
compute02 : ok=25 changed=1 unreachable=0 failed=0 skipped=60 rescued=0 ignored=0
compute03 : ok=25 changed=1 unreachable=0 failed=0 skipped=60 rescued=0 ignored=0
controller01 : ok=37 changed=1 unreachable=0 failed=0 skipped=181 rescued=0 ignored=0
controller02 : ok=36 changed=1 unreachable=0 failed=0 skipped=175 rescued=0 ignored=0
controller03 : ok=36 changed=1 unreachable=0 failed=0 skipped=175 rescued=0 ignored=0
localhost : ok=7 changed=1 unreachable=0 failed=0 skipped=12 rescued=0 ignored=0
monitoring01 : ok=20 changed=1 unreachable=0 failed=0 skipped=38 rescued=0 ignored=0
network01 : ok=25 changed=1 unreachable=0 failed=0 skipped=91 rescued=0 ignored=0
#开始部署
kolla-ansible -i ./multinode deploy
部署成功后可以用虚拟IP地址10.0.49.149登录
#摧毁集群
kolla-ansible destroy -i ./multinode --yes-i-really-really-mean-it
#grafana访问 10.0.49.139:3000
kolla-ansible -i ./multinode post-deploy
#使用文件
source /etc/kolla/admin-openrc.sh
#安装openstack部分客户端
pip install python-openstackclient python-glanceclient python-neutronclient
[root@kolla-ansible-master ~]# openstack network agent list
+--------------------------------------+----------------------+-----------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+----------------------+-----------+-------------------+-------+-------+---------------------------+
| 306ed9ef-3490-475e-b5b8-f736e337c7e0 | Open vSwitch agent | compute01 | None | :-) | UP | neutron-openvswitch-agent |
| a88e5126-6a39-4fed-8a50-d5d4f95d87d2 | Metadata agent | network01 | None | :-) | UP | neutron-metadata-agent |
| af71ed00-9d85-46ad-9a19-8eb0c11b7ace | Open vSwitch agent | network01 | None | :-) | UP | neutron-openvswitch-agent |
| d422f3f8-ac9a-44b2-9e88-e87e4027307f | L3 agent | network01 | nova | :-) | UP | neutron-l3-agent |
| dad9872f-4a93-4d0c-9bcd-7a20e0e71e3a | Open vSwitch agent | compute02 | None | :-) | UP | neutron-openvswitch-agent |
| dd0f0c52-3dde-4b28-bc1b-2f46d51892d6 | Loadbalancerv2 agent | network01 | None | :-) | UP | neutron-lbaasv2-agent |
| f0ec8c06-18a8-45f4-9ac0-4b6c19fdb306 | DHCP agent | network01 | nova | :-) | UP | neutron-dhcp-agent |
| f770b237-cdb1-4d39-9d3e-27352c772a2e | Open vSwitch agent | compute03 | None | :-) | UP | neutron-openvswitch-agent |
+--------------------------------------+----------------------+-----------+-------------------+-------+-------+---------------------------+
[root@kolla-ansible-master ~]# gnocchi resource-type list
+----------------------------+---------------------------------------------------------------------------------------+
| name | attributes |
+----------------------------+---------------------------------------------------------------------------------------+
| ceph_account | |
| generic | |
| host | - host_name: min_length: 0 , max_length: 255 , type: string , required: True |
| host_disk | - host_name: min_length: 0 , max_length: 255 , type: string , required: True |
| | - device_name: min_length: 0 , max_length: 255 , type: string , required: False |
| host_network_interface | - host_name: min_length: 0 , max_length: 255 , type: string , required: True |
| | - device_name: min_length: 0 , max_length: 255 , type: string , required: False |
| identity | |
| image | - container_format: min_length: 0 , max_length: 255 , type: string , required: True |
| | - disk_format: min_length: 0 , max_length: 255 , type: string , required: True |
| | - name: min_length: 0 , max_length: 255 , type: string , required: True |
| instance | - display_name: min_length: 0 , max_length: 255 , type: string , required: True |
| | - host: min_length: 0 , max_length: 255 , type: string , required: True |
| | - image_ref: min_length: 0 , max_length: 255 , type: string , required: False |
| | - flavor_name: min_length: 0 , max_length: 255 , type: string , required: True |
| | - server_group: min_length: 0 , max_length: 255 , type: string , required: False |
| | - flavor_id: min_length: 0 , max_length: 255 , type: string , required: True |
| instance_disk | - instance_id: required: True , type: uuid |
| | - name: min_length: 0 , max_length: 255 , type: string , required: True |
| instance_network_interface | - instance_id: required: True , type: uuid |
| | - name: min_length: 0 , max_length: 255 , type: string , required: True |
| ipmi | |
| manila_share | - status: min_length: 0 , max_length: 255 , type: string , required: True |
| | - host: min_length: 0 , max_length: 255 , type: string , required: True |
| | - protocol: min_length: 0 , max_length: 255 , type: string , required: False |
| | - name: min_length: 0 , max_length: 255 , type: string , required: False |
| | - availability_zone: min_length: 0 , max_length: 255 , type: string , required: False |
| network | |
| nova_compute | - host_name: min_length: 0 , max_length: 255 , type: string , required: True |
| port | - controller: min_length: 0 , max_length: 255 , type: string , required: True |
| stack | |
| swift_account | |
| switch | - controller: min_length: 0 , max_length: 255 , type: string , required: True |
| switch_port | - neutron_port_id: min_length: 0 , max_length: 255 , type: string , required: False |
| | - port_number_on_switch: max: 4294967295 , required: False , type: number , min: 0 |
| | - switch: min_length: 0 , max_length: 64 , type: string , required: True |
| | - controller: min_length: 0 , max_length: 255 , type: string , required: True |
| switch_table | - switch: min_length: 0 , max_length: 64 , type: string , required: True |
| | - controller: min_length: 0 , max_length: 255 , type: string , required: True |
| volume | - display_name: min_length: 0 , max_length: 255 , type: string , required: False |
| | - volume_type: min_length: 0 , max_length: 255 , type: string , required: False |
| volume_provider | |
| volume_provider_pool | - provider: min_length: 0 , max_length: 255 , type: string , required: True |
+----------------------------+---------------------------------------------------------------------------------------+
#修改初始文件
vim /usr/share/kolla-ansible/init-runonce
#配置公网浮动网段
# This EXT_NET_CIDR is your public network,that you want to connect to the internet via.
EXT_NET_CIDR='10.0.53.0/24'
EXT_NET_RANGE='start=10.0.53.150,end=10.0.53.199'
EXT_NET_GATEWAY='10.0.53.1'
#修改演示内网为实际所需
openstack network create --provider-network-type vxlan demo-net
openstack subnet create --subnet-range 192.168.1.0/24 --network demo-net \
--gateway 192.168.1.1 --dns-nameserver 60.191.134.196 demo-subnet
#初始化
/usr/share/kolla-ansible/init-runonce