网络是openstack最重要的资源之一,没有网络,虚拟机将被隔离。openstack的网络服务最主要的功能就是为虚拟机实例提供网络连接,最初由nova的一个单独模块compute实现,但是compute支持的网络服务有限,无法适应大规模、高密度和多项目的云计算,现已被专门的网络服务项目Neutron所取代。
Neutron为整个openstack环境提供软件定义网络支持,主要功能包括二层交换、三层路由、防火墙、、以及负载均衡等,Neturon在由其他openstack服务管理的网络接口设备之间提供网络连接即服务
实现虚拟化后,多个物理服务器可以被虚拟机所取代,部署在同一个物理服务器上。虚拟机由虚拟机管理器Hypervior实现,在linux中Hyervior通常采用KVM。在对服务器进行虚拟化的同时,也对网络进行虚拟化,Hypervisor为虚拟机创建一个或多个虚拟网卡,虚拟网卡等同于虚拟机的物理网卡卡。物理交换机在虚拟网络中被虚拟为虚拟交换机,虚拟机的虚拟网卡连接到虚拟交换机上,虚拟交换机再通过物理主机的物理网卡连接到外部网络,对于物理网络来说,虚拟化的主要工作是对网卡和交换设备的虚拟化。
开放虚拟交换机(Open vSwitch) 是与硬件交换机具备相同特性,可在不同虚拟平台之间移植,具有产品级质量的虚拟交换机,适合在生产环境中部署。 交换设备的虚拟化对虚拟网络来说至关重要。在传统的数据中心,管理员可以对物理交换机进行配置,控制服务器的网络接入,实现网络隔离、流量监控、Qos配置、流量优化等目标。在云环境中,采用Open vSwitch技术的虚拟交换机可使虚拟网络的管理、网络状态和流量的监控得以轻松实现。 Open Switch在云环境中的虚拟化平台上实现分布式虚拟交换机,可以将不同主机上的Open vSwitch交换机连接起来,形成一个大规模的虚拟网络
一个简化的典型的Neutron网络结构包括一个外部网络、一个内部网络和一个路由器。 外部网络负责连接OpenStack项目之外的网络环境,又称公共网络。与其他网络不同,它不仅仅是一个成拟网络,更重要的是,它表示OpenStack网络能被外部物理网络接入并访问。外部网络可能是企业的局域网(Intranet),也可能是互联网(Internet),这类网络并不是由Neutron直接管理。 内部网络完全由软件定义,又称私有网络。它是虚拟机实例所在的网络,能够直接连接到虚拟机。项目用户可以创建自己的内部网络。默认情况下,项目之间的内部网络是相互隔离的,不能共享。该网络由Neutron直接配置与管理。 路由器用于将内部网络与外部网络连接起来,因此,要使虚拟机访问外部网络,必须创建一个路由器。 Neutron需要实现的主要是内部网络和路由器。内部网络是对二层(L2)网络的抽象,模拟物理网络的二层局域网,对于项目来说,它是私有的。路由器则是对三层(L3)网络的抽象,模拟物理路由器,为用户提供路由、NAT等服务。
Neutron可以通过开发不同的插件和代理来支持不同的网络技术,这是一种相当开放的架构。不过随着所支持的网络提供者种类的增加,开发人员发现了两个突出的问题。一个问题是多种网络提供者无法共存。Core Plugin负责管理和维护Neutron二层虚拟网络的状态信息,一个Neutron网络只能由一个插件管理,而Core Plugin插件与相应的代理是 一 一 对应的。如果选择Linux Bridge插件,则只能选择Linux Bridge代理,必须在OpenStack的所有节点上使用Linux Bridge插件,则只能选择Linux Bridge代理,必须在OpenStackR的所有节点上使用Linux Bridge作为虚拟交换机。另一个问题是开发新的插件的工作量太大,而所有传统的Core Plugin之间存在大量反复代码 为解决这两个问题,从OpenStack的H版开始,Neutron实现了一个插件ML2,旨在取代所有的Core Plugin,允许在OpenStack网络中同时使用多种二层网络技术,不同的节点可以使用不同的网络实现机制。ML2能够与现有的代理无缝集成,以前使用的代理无须变更,只需将传统的Core Plugin替换成ML2,ML2使得对新的网络技术的支持更为简单,无须从头开发Core Plugin,只需要开发相应的机制驱动,大大减少编写和的代码。
Linux Bridge是成熟可靠的Neutron二层网络虚拟化技术,支持Local、Flat、VLAN和VXLAN这4种网络类型。
Linux Bridge可以将一台主机上的多个网卡桥接起来,充当一台交换机。它既可以桥接物理网卡,又可以是虚拟网卡。用于桥接虚拟机网卡(虚拟网卡)的是Tap接口,这是一个虚拟出来的网络设备,称为Tap设备,作为网桥的一个端口。Tap接口在逻辑上与物理接口具有相同的功能,可以接收和发送数据包。 如果选择Linux Bridge代理,在计算节点上数据包从虚拟机发送到物理网卡需要经过以下设备
与Linux Bridge相比,Open vSwitch (OvS)具有集中管控功能,而且性能更加优化,支持更多的功能,目前在OpenStack领域成为主流,它支持Local、Flat、VLAN、VXLAN、GRE和RGENEVE等所有网络类型。
[root@ct ~]# mysql -uroot -p
Enter password:
MariaDB [(none)]> CREATE DATABASE neutron;
Query OK, 1 row affected (0.001 sec)
MariaDB [(none)]> GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'localhost' IDENTIFIED BY 'NEUTRON_DBPASS';
Query OK, 0 rows affected (0.001 sec)
MariaDB [(none)]> GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'%' IDENTIFIED BY 'NEUTRON_DBPASS';
Query OK, 0 rows affected (0.001 sec)
MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.004 sec)
MariaDB [(none)]> exit;
Bye
[root@ct ~]# openstack user create --domain default --password NEUTRON_PASS neutron //创建neutron用户,用于keystone做认证
+---------------------+----------------------------------+
| Field | Value |
+---------------------+----------------------------------+
| domain_id | default |
| enabled | True |
| id | 6eb981651cb244709b8caa96dadf59da |
| name | neutron |
| options | {} |
| password_expires_at | None |
+---------------------+----------------------------------+
[root@ct ~]# openstack role add --project service --user neutron admin //讲neutron用户添加到service项目中拥有管理员权限
[root@ct ~]# openstack service create --name neutron --description "OpenStack Networking" network //创建network服务,服务类型为network
+-------------+----------------------------------+
| Field | Value |
+-------------+----------------------------------+
| description | OpenStack Networking |
| enabled | True |
| id | 870aa93aae414a668d55c9ebfbbfda98 |
| name | neutron |
| type | network |
+-------------+----------------------------------+
[root@ct ~]# openstack endpoint create --region RegionOne network public http://ct:9696
+--------------+----------------------------------+
| Field | Value |
+--------------+----------------------------------+
| enabled | True |
| id | 7bc6872314dc4de195a8ab0dd305ec09 |
| interface | public |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 870aa93aae414a668d55c9ebfbbfda98 |
| service_name | neutron |
| service_type | network |
| url | http://ct:9696 |
+--------------+----------------------------------+
[root@ct ~]# openstack endpoint create --region RegionOne network internal http://ct:9696
+--------------+----------------------------------+
| Field | Value |
+--------------+----------------------------------+
| enabled | True |
| id | 95a9691cad374b9eb3cc7f26bc6f0bb8 |
| interface | internal |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 870aa93aae414a668d55c9ebfbbfda98 |
| service_name | neutron |
| service_type | network |
| url | http://ct:9696 |
+--------------+----------------------------------+
[root@ct ~]# openstack endpoint create --region RegionOne network admin http://ct:9696
+--------------+----------------------------------+
| Field | Value |
+--------------+----------------------------------+
| enabled | True |
| id | 84a8eb4cd9b44027b775948f76c36fa1 |
| interface | admin |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 870aa93aae414a668d55c9ebfbbfda98 |
| service_name | neutron |
| service_type | network |
| url | http://ct:9696 |
+--------------+----------------------------------+
[root@ct ~]# yum -y install openstack-neutron openstack-neutron-ml2 openstack-neutron-linuxbridge ebtables conntrack-tools //ebtables包用来管理iptables规则的
[root@ct ~]# cp -a /etc/neutron/neutron.conf{,.bak}
[root@ct ~]# grep -Ev '^$|#' /etc/neutron/neutron.conf.bak > /etc/neutron/neutron.conf
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf database connection mysql+pymysql://neutron:NEUTRON_DBPASS@ct/neutron
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf DEFAULT core_plugin ml2
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf DEFAULT service_plugins router
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf DEFAULT allow_overlapping_ips true
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf DEFAULT transport_url rabbit://openstack:RABBIT_PASS@ct
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf DEFAULT auth_strategy keystone
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf DEFAULT notify_nova_on_port_status_changes true
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf DEFAULT notify_nova_on_port_data_changes true
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf keystone_authtoken www_authenticate_uri http://ct:5000
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_url http://ct:5000
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf keystone_authtoken memcached_servers ct:11211
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_type password
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf keystone_authtoken project_domain_name default
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf keystone_authtoken user_domain_name default
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf keystone_authtoken project_name service[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf keystone_authtoken username neutron
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf keystone_authtoken password NEUTRON_PASS
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf oslo_concurrency lock_path /var/lib/neutron/tmp
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf nova auth_url http://ct:5000
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf nova auth_type password
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf nova project_domain_name default
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf nova user_domain_name default
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf nova region_name RegionOne
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf nova project_name service
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf nova username nova
[root@ct ~]# openstack-config --set /etc/neutron/neutron.conf nova password NOVA_PASS
[root@ct ~]# cd /etc/neutron/
[root@ct neutron]# cat neutron.conf
[DEFAULT]
core_plugin = ml2 //启用二层网络插件
service_plugins = router //启用三层网络插件
allow_overlapping_ips = true
transport_url = rabbit://openstack:RABBIT_PASS@ct //配置rabbitmq连接
auth_strategy = keystone //认证的方式为keystone
notify_nova_on_port_status_changes = true //当网络接口发生改变时,通知计算节点
notify_nova_on_port_data_changes = true //当端口数据发生变化,通知计算节点
[cors]
[database] //配置数据库连接
connection = mysql+pymysql://neutron:NEUTRON_DBPASS@ct/neutron
[keystone_authtoken] //配置keystone认证信息
www_authenticate_uri = http://ct:5000
auth_url = http://ct:5000
memcached_servers = ct:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = neutron
password = NEUTRON_PASS
[oslo_concurrency] //配置锁路径
lock_path = /var/lib/neutron/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_middleware]
[oslo_policy]
[privsep]
[ssl]
[nova] // neutron需要给nova返回数据
auth_url = http://ct:5000 //到keystone认证nova
auth_type = password
project_domain_name = default
user_domain_name = default
region_name = RegionOne
project_name = service
username = nova //通过nova的用户和密码到keystone验证nov的token
password = NOVA_PASS
[root@ct neutron]# cp -a /etc/neutron/plugins/ml2/ml2_conf.ini{,.bak}
[root@ct neutron]# grep -Ev '^$|#' /etc/neutron/plugins/ml2/ml2_conf.ini.bak > /etc/neutron/plugins/ml2/ml2_conf.ini
[root@ct ml2]# openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2 type_drivers flat,vlan,vxlan
[root@ct ml2]# openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2 tenant_network_types
[root@ct ml2]# openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2 mechanism_drivers linuxbridge,l2population
[root@ct ml2]# openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2 extension_drivers port_security
[root@ct ml2]# openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2_type_flat flat_networks provider
[root@ct ml2]# openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2_type_vxlan vni_ranges 1:1000
[root@ct ml2]# openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini securitygroup enable_ipset true
[root@ct neutron]# cd plugins/ml2/
[root@ct ml2]# ls
linuxbridge_agent.ini ml2_conf.ini ml2_conf.ini.bak sriov_agent.ini
[root@ct ml2]# cat ml2_conf.ini
[DEFAULT]
[ml2]
type_drivers = flat,vlan,vxlan //配置类型驱动,单一扁平网络(桥接)和vlan;让二层网络支持桥接,支持基于vlan做子网划分
tenant_network_types = vxlan //租户网络类型
mechanism_drivers = linuxbridge,l2population /启用Linuxbridge和l2机制,(l2population机制是为了简化网络通信拓扑,减少网络广播)
extension_drivers = port_security //启用端口安全扩展驱动程序,基于iptables实现访问控制;但配置了扩展安全组会导致一些端口限制,造成一些服务无法启动
[ml2_type_flat]
flat_networks = provider //配置公共虚拟网络为flat网络
[ml2_type_vxlan]
vni_ranges = 1:1000 //为私有网络配置VXLAN网络识别的网络范围
[securitygroup]
enable_ipset = true //启用 ipset 增加安全组的方便性
[root@ct ml2]# cp -a /etc/neutron/plugins/ml2/linuxbridge_agent.ini{,.bak}
[root@ct ml2]# grep -Ev '^$|#' /etc/neutron/plugins/ml2/linuxbridge_agent.ini.bak > /etc/neutron/plugins/ml2/linuxbridge_agent.ini
[root@ct ml2]# ip a
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:d1:f3:45 brd ff:ff:ff:ff:ff:ff
inet 192.168.20.77/24 brd 192.168.20.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet6 fe80::ac6c:aa3e:2c80:1166/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:d1:f3:4f brd ff:ff:ff:ff:ff:ff
inet 192.168.30.10/24 brd 192.168.30.255 scope global noprefixroute ens37
valid_lft forever preferred_lft forever
inet6 fe80::f450:d74b:e5e5:eb0f/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@ct ml2]# openstack-config --set /etc/neutron/plugins/ml2/linuxbridge_agent.ini linux_bridge physical_interface_mappings provider:ens33 //指定控制节点net网卡
[root@ct ml2]# openstack-config --set /etan truen/plugins/ml2/linuxbridge_agent.ini vxlan enable_vxl
[root@ct ml2]# openstack-config --set /etc/neutron/plugins/ml2/linuxbridge_agent.ini vxlan local_ip 192.168.30.10 //指定控制节点内网ip
[root@ct ml2]# openstack-config --set /etc/neutron/plugins/ml2/linuxbridge_agent.ini vxlan l2_population true
[root@ct ml2]# openstack-config --set /etc/neutron/plugins/ml2/linuxbridge_agent.ini securitygroup enable_security_group true
[root@ct ml2]# openstack-config --set /etc/neutron/plugins/ml2/linuxbridge_agent.ini securitygroup firewall_driver neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
[root@ct ml2]# cat linuxbridge_agent.ini
[DEFAULT]
[linux_bridge]
physical_interface_mappings = provider:ens33 启用VXLAN覆盖网络,配置覆盖网络的物理网络接口的IP地址,启用layer-2 population
[vxlan]
enable_vxlan = true //允许用户创建自定义网络(3层网络)
local_ip = 192.168.30.10
l2_population = true
[securitygroup] //启用安全组并配置 Linux 桥接 iptables 防火墙驱动
enable_security_group = true
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
[root@ct ml2]# echo 'net.bridge.bridge-nf-call-iptables=1' >> /etc/sysctl.conf
[root@ct ml2]# echo 'net.bridge.bridge-nf-call-ip6tables=1' >> /etc/sysctl.conf
[root@ct ml2]# modprobe br_netfilter
[root@ct ml2]# sysctl -p
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
[root@ct ml2]# cp -a /etc/neutron/l3_agent.ini{,.bak}
[root@ct ml2]# grep -Ev '^$|#' /etc/neutron/l3_agent.ini.bak > /etc/neutron/l3_agent.ini
[root@ct ml2]# openstack-config --set /etc/neutron/l3_agent.ini DEFAULT interface_driver linuxbridge
[root@ct ml2]# cd ..
[root@ct plugins]# cd ..
[root@ct neutron]# cat l3_agent.ini
[DEFAULT]
interface_driver = linuxbridge
[root@ct neutron]# cp -a /etc/neutron/dhcp_agent.ini{,.bak}
[root@ct neutron]# grep -Ev '^$|#' /etc/neutron/dhcp_agent.ini.bak > /etc/neutron/dhcp_agent.ini
[root@ct neutron]# openstack-config --set /etc/neutron/dhcp_agent.ini DEFAULT interface_driver linuxbridge
[root@ct neutron]# openstack-config --set /etc/neutron/dhcp_agent.ini DEFAULT dhcp_driver neutron.agent.linux.dhcp.Dnsmasq
[root@ct neutron]# openstack-config --set /etc/neutron/dhcp_agent.ini DEFAULT enable_isolated_metadata true
[root@ct neutron]# cat dhcp_agent.ini
[DEFAULT]
interface_driver = linuxbridge //指定默认接口驱动为linux网桥
dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq //指定DHCP驱动
enable_isolated_metadata = true //开启iso元数据
[root@ct neutron]# cp -a /etc/neutron/metadata_agent.ini{,.bak}
[root@ct neutron]# grep -Ev '^$|#' /etc/neutron/metadata_agent.ini.bak > /etc/neutron/metadata_agent.ini
[root@ct neutron]# openstack-config --set /etc/neutron/metadata_agent.ini DEFAULT nova_metadata_host ct
[root@ct neutron]# openstack-config --set /etc/neutron/metadata_agent.ini DEFAULT metadata_proxy_shared_secret METADATA_SECRET
[root@ct neutron]# cat metadata_agent.ini
[DEFAULT]
nova_metadata_host = ct
metadata_proxy_shared_secret = METADATA_SECRET
[cache]
[root@ct neutron]# openstack-config --set /etc/nova/nova.conf neutron url http://ct:9696
[root@ct neutron]# openstack-config --set /etc/nova/nova.conf neutron auth_url http://ct:5000
[root@ct neutron]# openstack-config --set /etc/nova/nova.conf neutron auth_type password
[root@ct neutron]# openstack-config --set /etc/nova/nova.conf neutron project_domain_name default
[root@ct neutron]# openstack-config --set /etc/nova/nova.conf neutron user_domain_name default
[root@ct neutron]# openstack-config --set /etc/nova/nova.conf neutron region_name RegionOne
[root@ct neutron]# openstack-config --set /etc/nova/nova.conf neutron project_name service
[root@ct neutron]# openstack-config --set /etc/nova/nova.conf neutron username neutron
[root@ct neutron]# openstack-config --set /etc/nova/nova.conf neutron password NEUTRON_PASS
[root@ct neutron]# openstack-config --set /etc/nova/nova.conf neutron service_metadata_proxy true
[root@ct neutron]# openstack-config --set /etc/nova/nova.conf neutron metadata_proxy_shared_secret METADATA_SECRET
[root@ct neutron]# ln -s /etc/neutron/plugins/ml2/ml2_conf.ini /etc/neutron/plugin.ini
[root@ct ~]# su -s /bin/sh -c "neutron-db-manage --config-file /etc/neutron/neutron.conf \
> --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head" neutron
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
/usr/lib/python2.7/site-packages/pymysql/cursors.py:170: Warning: (1280, u"Name 'alembic_version_pkc' ignored for PRIMARY key.")
result = self._query(query)
正在对 neutron 运行 upgrade...
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade -> kilo
INFO [alembic.runtime.migration] Running upgrade kilo -> 354db87e3225
INFO [alembic.runtime.migration] Running upgrade 354db87e3225 -> 599c6a226151
INFO [alembic.runtime.migration] Running upgrade 599c6a226151 -> 52c5312f6baf
INFO [alembic.runtime.migration] Running upgrade 52c5312f6baf -> 313373c0ffee
INFO [alembic.runtime.migration] Running upgrade 313373c0ffee -> 8675309a5c4f
INFO [alembic.runtime.migration] Running upgrade 8675309a5c4f -> 45f955889773
INFO [alembic.runtime.migration] Running upgrade 45f955889773 -> 26c371498592
INFO [alembic.runtime.migration] Running upgrade 26c371498592 -> 1c844d1677f7
INFO [alembic.runtime.migration] Running upgrade 1c844d1677f7 -> 1b4c6e320f79
INFO [alembic.runtime.migration] Running upgrade 1b4c6e320f79 -> 48153cb5f051
INFO [alembic.runtime.migration] Running upgrade 48153cb5f051 -> 9859ac9c136
INFO [alembic.runtime.migration] Running upgrade 9859ac9c136 -> 34af2b5c5a59
INFO [alembic.runtime.migration] Running upgrade 34af2b5c5a59 -> 59cb5b6cf4d
INFO [alembic.runtime.migration] Running upgrade 59cb5b6cf4d -> 13cfb89f881a
INFO [alembic.runtime.migration] Running upgrade 13cfb89f881a -> 32e5974ada25
INFO [alembic.runtime.migration] Running upgrade 32e5974ada25 -> ec7fcfbf72ee
INFO [alembic.runtime.migration] Running upgrade ec7fcfbf72ee -> dce3ec7a25c9
INFO [alembic.runtime.migration] Running upgrade dce3ec7a25c9 -> c3a73f615e4
INFO [alembic.runtime.migration] Running upgrade c3a73f615e4 -> 659bf3d90664
INFO [alembic.runtime.migration] Running upgrade 659bf3d90664 -> 1df244e556f5
INFO [alembic.runtime.migration] Running upgrade 1df244e556f5 -> 19f26505c74f
INFO [alembic.runtime.migration] Running upgrade 19f26505c74f -> 15be73214821
INFO [alembic.runtime.migration] Running upgrade 15be73214821 -> b4caf27aae4
INFO [alembic.runtime.migration] Running upgrade b4caf27aae4 -> 15e43b934f81
INFO [alembic.runtime.migration] Running upgrade 15e43b934f81 -> 31ed664953e6
INFO [alembic.runtime.migration] Running upgrade 31ed664953e6 -> 2f9e956e7532
INFO [alembic.runtime.migration] Running upgrade 2f9e956e7532 -> 3894bccad37f
INFO [alembic.runtime.migration] Running upgrade 3894bccad37f -> 0e66c5227a8a
INFO [alembic.runtime.migration] Running upgrade 0e66c5227a8a -> 45f8dd33480b
INFO [alembic.runtime.migration] Running upgrade 45f8dd33480b -> 5abc0278ca73
INFO [alembic.runtime.migration] Running upgrade 5abc0278ca73 -> d3435b514502
INFO [alembic.runtime.migration] Running upgrade d3435b514502 -> 30107ab6a3ee
INFO [alembic.runtime.migration] Running upgrade 30107ab6a3ee -> c415aab1c048
INFO [alembic.runtime.migration] Running upgrade c415aab1c048 -> a963b38d82f4
INFO [alembic.runtime.migration] Running upgrade kilo -> 30018084ec99
INFO [alembic.runtime.migration] Running upgrade 30018084ec99 -> 4ffceebfada
INFO [alembic.runtime.migration] Running upgrade 4ffceebfada -> 5498d17be016
INFO [alembic.runtime.migration] Running upgrade 5498d17be016 -> 2a16083502f3
INFO [alembic.runtime.migration] Running upgrade 2a16083502f3 -> 2e5352a0ad4d
INFO [alembic.runtime.migration] Running upgrade 2e5352a0ad4d -> 11926bcfe72d
INFO [alembic.runtime.migration] Running upgrade 11926bcfe72d -> 4af11ca47297
INFO [alembic.runtime.migration] Running upgrade 4af11ca47297 -> 1b294093239c
INFO [alembic.runtime.migration] Running upgrade 1b294093239c -> 8a6d8bdae39
INFO [alembic.runtime.migration] Running upgrade 8a6d8bdae39 -> 2b4c2465d44b
INFO [alembic.runtime.migration] Running upgrade 2b4c2465d44b -> e3278ee65050
INFO [alembic.runtime.migration] Running upgrade e3278ee65050 -> c6c112992c9
INFO [alembic.runtime.migration] Running upgrade c6c112992c9 -> 5ffceebfada
INFO [alembic.runtime.migration] Running upgrade 5ffceebfada -> 4ffceebfcdc
INFO [alembic.runtime.migration] Running upgrade 4ffceebfcdc -> 7bbb25278f53
INFO [alembic.runtime.migration] Running upgrade 7bbb25278f53 -> 89ab9a816d70
INFO [alembic.runtime.migration] Running upgrade a963b38d82f4 -> 3d0e74aa7d37
INFO [alembic.runtime.migration] Running upgrade 3d0e74aa7d37 -> 030a959ceafa
INFO [alembic.runtime.migration] Running upgrade 030a959ceafa -> a5648cfeeadf
INFO [alembic.runtime.migration] Running upgrade a5648cfeeadf -> 0f5bef0f87d4
INFO [alembic.runtime.migration] Running upgrade 0f5bef0f87d4 -> 67daae611b6e
INFO [alembic.runtime.migration] Running upgrade 89ab9a816d70 -> c879c5e1ee90
INFO [alembic.runtime.migration] Running upgrade c879c5e1ee90 -> 8fd3918ef6f4
INFO [alembic.runtime.migration] Running upgrade 8fd3918ef6f4 -> 4bcd4df1f426
INFO [alembic.runtime.migration] Running upgrade 4bcd4df1f426 -> b67e765a3524
INFO [alembic.runtime.migration] Running upgrade 67daae611b6e -> 6b461a21bcfc
INFO [alembic.runtime.migration] Running upgrade 6b461a21bcfc -> 5cd92597d11d
INFO [alembic.runtime.migration] Running upgrade 5cd92597d11d -> 929c968efe70
INFO [alembic.runtime.migration] Running upgrade 929c968efe70 -> a9c43481023c
INFO [alembic.runtime.migration] Running upgrade a9c43481023c -> 804a3c76314c
INFO [alembic.runtime.migration] Running upgrade 804a3c76314c -> 2b42d90729da
INFO [alembic.runtime.migration] Running upgrade 2b42d90729da -> 62c781cb6192
INFO [alembic.runtime.migration] Running upgrade 62c781cb6192 -> c8c222d42aa9
INFO [alembic.runtime.migration] Running upgrade c8c222d42aa9 -> 349b6fd605a6
INFO [alembic.runtime.migration] Running upgrade 349b6fd605a6 -> 7d32f979895f
INFO [alembic.runtime.migration] Running upgrade 7d32f979895f -> 594422d373ee
INFO [alembic.runtime.migration] Running upgrade 594422d373ee -> 61663558142c
INFO [alembic.runtime.migration] Running upgrade 61663558142c -> 867d39095bf4, port forwarding
INFO [alembic.runtime.migration] Running upgrade 867d39095bf4 -> d72db3e25539, modify uniq port forwarding
INFO [alembic.runtime.migration] Running upgrade d72db3e25539 -> cada2437bf41
INFO [alembic.runtime.migration] Running upgrade cada2437bf41 -> 195176fb410d, router gateway IP QoS
INFO [alembic.runtime.migration] Running upgrade 195176fb410d -> fb0167bd9639
INFO [alembic.runtime.migration] Running upgrade fb0167bd9639 -> 0ff9e3881597
INFO [alembic.runtime.migration] Running upgrade 0ff9e3881597 -> 9bfad3f1e780
INFO [alembic.runtime.migration] Running upgrade 9bfad3f1e780 -> 63fd95af7dcd
INFO [alembic.runtime.migration] Running upgrade 63fd95af7dcd -> c613d0b82681
INFO [alembic.runtime.migration] Running upgrade b67e765a3524 -> a84ccf28f06a
INFO [alembic.runtime.migration] Running upgrade a84ccf28f06a -> 7d9d8eeec6ad
INFO [alembic.runtime.migration] Running upgrade 7d9d8eeec6ad -> a8b517cff8ab
INFO [alembic.runtime.migration] Running upgrade a8b517cff8ab -> 3b935b28e7a0
INFO [alembic.runtime.migration] Running upgrade 3b935b28e7a0 -> b12a3ef66e62
INFO [alembic.runtime.migration] Running upgrade b12a3ef66e62 -> 97c25b0d2353
INFO [alembic.runtime.migration] Running upgrade 97c25b0d2353 -> 2e0d7a8a1586
INFO [alembic.runtime.migration] Running upgrade 2e0d7a8a1586 -> 5c85685d616d
确定
[root@ct ~]# systemctl restart openstack-nova-api.service //重启nova-api服务
[root@ct ~]# systemctl enable neutron-server.service \
> neutron-linuxbridge-agent.service neutron-dhcp-agent.service \
> neutron-metadata-agent.service
Created symlink from /etc/systemd/system/multi-user.target.wants/neutron-server.service to /usr/lib/systemd/system/neutron-server.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/neutron-linuxbridge-agent.service to /usr/lib/systemd/system/neutron-linuxbridge-agent.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/neutron-dhcp-agent.service to /usr/lib/systemd/system/neutron-dhcp-agent.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/neutron-metadata-agent.service to /usr/lib/systemd/system/neutron-metadata-agent.service.
[root@ct ~]# systemctl start neutron-server.service \
> neutron-linuxbridge-agent.service neutron-dhcp-agent.service \
> neutron-metadata-agent.service
[root@ct ~]# netstat -anutp |grep 9696
tcp 0 0 0.0.0.0:9696 0.0.0.0:* LISTEN 96524/server.log
- 启动第三层服务
[root@ct ~]# systemctl enable neutron-l3-agent.service
Created symlink from /etc/systemd/system/multi-user.target.wants/neutron-l3-agent.service to /usr/lib/systemd/system/neutron-l3-agent.service.
[root@ct ~]# systemctl restart neutron-l3-agent.service
[root@ct1 ~]# yum -y install openstack-neutron-linuxbridge ebtables ipset conntrack-tools
[root@ct1 ~]# cp -a /etc/neutron/neutron.conf{,.bak}
[root@ct1 ~]# grep -Ev '^$|#' /etc/neutron/neutron.conf.bak > /etc/neutron/neutron.conf
[root@ct1 ~]# openstack-config --set /etc/neutron/neutron.conf DEFAULT transport_url rabbit://openstack:RABBIT_PASS@ct
[root@ct1 ~]# openstack-config --set /etc/neutron/neutron.conf DEFAULT auth_strategy keystone
[root@ct1 ~]# openstack-config --set /etc/neutron/neutron.conf keystone_authtoken www_authenticate_uri http://ct:5000
[root@ct1 ~]# openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_url http://ct:5000
[root@ct1 ~]# openstack-config --set /etc/neutron/neutron.conf keystone_authtoken memcached_servers ct:11211
[root@ct1 ~]# openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_type password
[root@ct1 ~]# openstack-config --set /etc/neutron/neutron.conf keystone_authtoken project_domain_name default
[root@ct1 ~]# openstack-config --set /etc/neutron/neutron.conf keystone_authtoken user_domain_name default
[root@ct1 ~]# openstack-config --set /etc/neutron/neutron.conf keystone_authtoken project_name service
[root@ct1 ~]# openstack-config --set /etc/neutron/neutron.conf keystone_authtoken username neutron
[root@ct1 ~]# openstack-config --set /etc/neutron/neutron.conf keystone_authtoken password NEUTRON_PASS
[root@ct1 ~]# openstack-config --set /etc/neutron/neutron.conf oslo_concurrency lock_path /var/lib/neutron/tmp
[root@ct1 ~]# cd /etc/neutron/
[root@ct1 neutron]# cat neutron.conf
[DEFAULT] //eutron的server端与agent端通讯也是通过rabbitmq进行通讯的
transport_url = rabbit://openstack:RABBIT_PASS@ct
auth_strategy = keystone //哦那个过keystone认证
[cors]
[database]
[keystone_authtoken] //指定keystone认证的信息
www_authenticate_uri = http://ct:5000
auth_url = http://ct:5000
memcached_servers = ct:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = neutron
password = NEUTRON_PASS
[oslo_concurrency] //配置锁路径(管理线程库)
lock_path = /var/lib/neutron/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_middleware]
[oslo_policy]
[privsep]
[ssl]
[root@ct1 neutron]# cp -a /etc/neutron/plugins/ml2/linuxbridge_agent.ini{,.bak}
[root@ct1 neutron]# grep -Ev '^$|#' /etc/neutron/plugins/ml2/linuxbridge_agent.ini.bak > /etc/neutron/plugins/ml2/linuxbridge_agent.ini
[root@ct1 ml2]# openstack-config --set /etc/neutron/plugins/ml2/linuxbridge_agent.ini linux_bridge physical_interface_mappings provider:ens33 //指定net网卡
[root@ct1 ml2]# openstack-config --set /elan truen/plugins/ml2/linuxbridge_agent.ini vxlan enable_vx
[root@ct1 ml2]# openstack-config --set /etc/neutron/plugins/ml2/linuxbridge_agent.ini vxlan local_ip 192.168.30.20 //指定ct1内网ip
[root@ct1 ml2]# openstack-config --set /etc/neutron/plugins/ml2/linuxbridge_agent.ini vxlan l2_population true
[root@ct1 ml2]# openstack-config --set /etc/neutron/plugins/ml2/linuxbridge_agent.ini securitygroup enable_security_group true
[root@ct1 ml2]# openstack-config --set /etc/neutron/plugins/ml2/linuxbridge_agent.ini securitygroup firewall_driver neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
[root@ct1 ml2]# cat linuxbridge_agent.ini
[DEFAULT]
[linux_bridge]
physical_interface_mappings = provider:ens33
[vxlan]
enable_vxlan = true
local_ip = 192.168.30.10
l2_population = true
[securitygroup]
enable_security_group = true //开启安全组
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver /指定安全组驱动文件
[root@ct1 ml2]# echo 'net.bridge.bridge-nf-call-iptables=1' >> /etc/sysctl.conf //允许虚拟机的数据通过物理机出去
[root@ct1 ml2]# echo 'net.bridge.bridge-nf-call-ip6tables=1' >> /etc/sysctl.conf
[root@ct1 ml2]# modprobe br_netfilter //用于向内核中加载模
[root@ct1 ml2]# sysctl -p
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
[root@ct1 ml2]# openstack-config --set /etc/nova/nova.conf neutron auth_url http://ct:5000
[root@ct1 ml2]# openstack-config --set /etc/nova/nova.conf neutron auth_type password
[root@ct1 ml2]# openstack-config --set /etc/nova/nova.conf neutron project_domain_name default
[root@ct1 ml2]# openstack-config --set /etc/nova/nova.conf neutron user_domain_name default
[root@ct1 ml2]# openstack-config --set /etc/nova/nova.conf neutron region_name RegionOne
[root@ct1 ml2]# openstack-config --set /etc/nova/nova.conf neutron project_name service
[root@ct1 ml2]# openstack-config --set /etc/nova/nova.conf neutron username neutron
[root@ct1 ml2]# openstack-config --set /etc/nova/nova.conf neutron password NEUTRON_PASS
[neutron]
auth_url = http://ct:5000
auth_type = password
project_domain_name = default
user_domain_name = default
region_name = RegionOne
project_name = service
username = neutron
password = NEUTRON_PASS
[root@ct1 ~]# systemctl restart openstack-nova-compute.service
[root@ct1 ~]# systemctl enable neutron-linuxbridge-agent.service //启动neutron网桥代理服务
[root@ct1 ~]# systemctl start neutron-linuxbridge-agent.service
[root@ct ~]# openstack extension list --network
+----------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Name | Alias | Description |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Subnet Pool Prefix Operations | subnetpool-prefix-ops | Provides support for adjusting the prefix list of subnet pools |
| Default Subnetpools | default-subnetpools | Provides ability to mark and use a subnetpool as the default. |
| Availability Zone | availability_zone | The availability zone extension. |
| Network Availability Zone | network_availability_zone | Availability zone support for network. |
| Subnet Onboard | subnet_onboard | Provides support for onboarding subnets into subnet pools |
| Auto Allocated Topology Services | auto-allocated-topology | Auto Allocated Topology Services. |
| Neutron L3 Configurable external gateway mode | ext-gw-mode | Extension of the router abstraction for specifying whether SNAT should occur on the external gateway |
| Port Binding | binding | Expose port bindings of a virtual port to external application |
| agent | agent | The agent management extension. |
| Subnet Allocation | subnet_allocation | Enables allocation of subnets from a subnet pool |
| DHCP Agent Scheduler | dhcp_agent_scheduler | Schedule networks among dhcp agents |
| Neutron external network | external-net | Adds external network attribute to network resource. |
| Empty String Filtering Extension | empty-string-filtering | Allow filtering by attributes with empty string value |
| Tag support for resources with standard attribute: subnet, trunk, network_segment_range, router, network, policy, subnetpool, port, security_group, floatingip | standard-attr-tag | Enables to set tag on resources with standard attribute. |
| Neutron Service Flavors | flavors | Flavor specification for Neutron advanced services. |
| Network MTU | net-mtu | Provides MTU attribute for a network resource. |
| Network IP Availability | network-ip-availability | Provides IP availability data for each network and subnet. |
| Quota management support | quotas | Expose functions for quotas management per tenant |
| If-Match constraints based on revision_number | revision-if-match | Extension indicating that If-Match based on revision_number is supported. |
| Prevent L3 router ports IP address change extension | l3-port-ip-change-not-allowed | Prevent change of IP address for some L3 router ports |
| Availability Zone Filter Extension | availability_zone_filter | Add filter parameters to AvailabilityZone resource |
| HA Router extension | l3-ha | Adds HA capability to routers. |
| Enforce Router's Admin State Down Before Update Extension | router-admin-state-down-before-update | Ensure that the admin state of a router is down (admin_state_up=False) before updating the distributed attribute |
| Filter parameters validation | filter-validation | Provides validation on filter parameters. |
| Multi Provider Network | multi-provider | Expose mapping of virtual networks to multiple physical networks |
| Quota details management support | quota_details | Expose functions for quotas usage statistics per project |
| Address scope | address-scope | Address scopes extension. |
| Neutron Extra Route | extraroute | Extra routes configuration for L3 router |
| Network MTU (writable) | net-mtu-writable | Provides a writable MTU attribute for a network resource. |
| Agent's Resource View Synced to Placement | agent-resources-synced | Stores success/failure of last sync to Placement |
| Subnet service types | subnet-service-types | Provides ability to set the subnet service_types field |
| Floating IP Pools Extension | floatingip-pools | Provides a floating IP pools API. |
| Neutron Port MAC address regenerate | port-mac-address-regenerate | Network port MAC address regenerate |
| Add security_group type to network RBAC | rbac-security-groups | Add security_group type to network RBAC |
| Provider Network | provider | Expose mapping of virtual networks to physical networks |
| Neutron Service Type Management | service-type | API for retrieving service providers for Neutron advanced services |
| Router Flavor Extension | l3-flavors | Flavor support for routers. |
| Port Security | port-security | Provides port security |
| Neutron Extra DHCP options | extra_dhcp_opt | Extra options configuration for DHCP. For example PXE boot options to DHCP clients can be specified (e.g. tftp-server, server-ip-address, bootfile-name) |
| Port filtering on security groups | port-security-groups-filtering | Provides security groups filtering when listing ports |
| Resource timestamps | standard-attr-timestamp | Adds created_at and updated_at fields to all Neutron resources that have Neutron standard attributes. |
| Resource revision numbers | standard-attr-revisions | This extension will display the revision number of neutron resources. |
| Pagination support | pagination | Extension that indicates that pagination is enabled. |
| Sorting support | sorting | Extension that indicates that sorting is enabled. |
| security-group | security-group | The security groups extension. |
| L3 Agent Scheduler | l3_agent_scheduler | Schedule routers among l3 agents |
| Floating IP Port Details Extension | fip-port-details | Add port_details attribute to Floating IP resource |
| Router Availability Zone | router_availability_zone | Availability zone support for router. |
| RBAC Policies | rbac-policies | Allows creation and modification of policies that control tenant access to resources. |
| Atomically add/remove extra routes | extraroute-atomic | Edit extra routes of a router on server side by atomically adding/removing extra routes |
| standard-attr-description | standard-attr-description | Extension to add descriptions to standard attributes |
| IP address substring filtering | ip-substring-filtering | Provides IP address substring filtering when listing ports |
| Neutron L3 Router | router | Router abstraction for basic L3 forwarding between L2 Neutron networks and access to external networks via a NAT gateway. |
| Allowed Address Pairs | allowed-address-pairs | Provides allowed address pairs |
| Port Bindings Extended | binding-extended | Expose port bindings of a virtual port to external application |
| project_id field enabled | project-id | Extension that indicates that project_id field is enabled. |
| Distributed Virtual Router | dvr | Enables configuration of Distributed Virtual Routers. |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
[root@ct ~]# openstack network agent list
+--------------------------------------+--------------------+------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+------+-------------------+-------+-------+---------------------------+
| 367bb2a8-25f4-47fe-af92-2bfcacc5be14 | L3 agent | ct | nova | :-) | UP | neutron-l3-agent |
| 502a580d-d1e7-42d4-a5c8-3638ca663142 | Metadata agent | ct | None | :-) | UP | neutron-metadata-agent |
| 60b71623-a0a5-452d-9153-e879eadb2cdb | Linux bridge agent | ct1 | None | :-) | UP | neutron-linuxbridge-agent |
| 8ea8dca1-4a37-488b-9c14-53b89422617d | DHCP agent | ct | nova | :-) | UP | neutron-dhcp-agent |
| c403b1ff-c9a1-48bb-ab46-9f4f9318312c | Linux bridge agent | ct2 | None | :-) | UP | neutron-linuxbridge-agent |
| ef668df2-0b26-4be4-a595-e7e300660709 | Linux bridge agent | ct | None | :-) | UP | neutron-linuxbridge-agent |
+--------------------------------------+--------------------+------+-------------------+-------+-------+---------------------------+
三个节点都需要配置neutron网络,
配置neutron组件的用户、认证、endpoint
设置提供者provider网络(这里是桥接模式)
1.配置二层网络
2.配置网桥(插件)
3.优化内核
4.配置网桥接口与外部对接
5.修改DHCP配置(修改配置文件、代理)
6.配置网桥与内部组件的配置(修改配置文件、代理)
设置neutron与nova对接的配置
其中C1、C2节点配置相同