A common problem for people who want to try Openstack without a full blown hardware setup is that they have just one network interface. Openstack identifies three distinct networks
Internal Network
This is where all your inter process communication happens. This is where your mysql-server/queue-server etc are listening and this is where your services exchange information among themselves. On a proper set up this network should be isolated and secured and the interface connected to this network should not be added to any bridge.
Data Network
This is where your Instances talk to each other and to their network’s l3 and dhcp services. This network again should be isolated and secured. There can be more than one data network. The data networks are mapped to a physical networks which will be available for neutron to use using config file parameters. It is the physical network that you denote as ‘provider:physical_network’ in the ‘neutron net-create‘ API call. You need not worry about choosing the physical_network for each network you create as neutron will choose it for you if you did not.
bridge_mappings
Present inside the ‘ovs’ section of ‘ml2_conf.ini’. You tell Neutron which physical networks are available for use through this parameter. You also have to tell which bridge to use in order to reach that physical network. Thus ‘bridge_mappings’ is a comma separated list of ‘physical_network:bridge_name’ pairs. You also have to make sure the bridges that you mapped to physical networks exists on the host.
flat_networks
Present under ‘ml2_type_flat’ section. Configured in case of flat networks. This is just a comma separated list of physical networks that are flat(no vlan involved)
network_vlan_ranges
Present under ‘ml2_type_vlan’ section. Configured in case of vlan networks. This is similar to flat_networks except that for each physical networks there is a start and an end vlan appended with a ‘:’ between them.
local_ip
Present under ‘ovs’ section’. In case you are using GRE mode this parameter will tell neutron which IP to bind and run GRE on. This in turn determines which interface and network should be used as data network. So it is a good idea to use an interface other than the one used for internal network.
Finally, unless you are using ‘GRE’ alone, you have to add one of the host’s network interface to every bridge specified so that all physical networks are now bridged to their corresponding data network. Using a little trick you can even map more than one physical network to a data network.
External Network
This network is used for two purposes.
To expose the services(nova-api, glance-api .,etc) to consumers outside of Openstack.
To allow your Instances to be accessible from outside of Openstack, through floating-ip.
It is a good idea to use two external networks for the above two purposes. That way you can restrict all ports other than those on which your exposed services are listening.
In Neutron and external network is one on which you have ‘router:external’ set to true. Only then can you create foating-ips on it. In all other ways all rules that apply to physical networks also apply here. Normally you would want to chose a flat physical network for creating external network. Otherwise you would have to ask your network administrator to set up vlan on the switch port connecting to the machine running your l3-agent and things start to get ugly.
The host interface connecting to the external network should not have any form of security. You should allow security groups to do that job.
Full blown OpenStack setup
Using the same Interface for all Networks
Finally we arrive at the purpose of this blog. This blog gives you plenty of information and reason why you should not do this but while you are experimenting all is fair.
Assumptions:
eth0 is the only available port
bridge_mappings=Physnet1:br-eth1,External_network:br-ex
network_vlan_ranges=Physnet1:100:200
flat_networks=External_network
ovs-vsctl add-br br-eth0
ovs-vsctl add-port br-eth0 eth0
ifconfig br-eth0 up
ip link set br-eth0 promisc on
ip link add proxy-br-eth1 type veth peer name eth1-br-proxy
ip link add proxy-br-ex type veth peer name ex-br-proxy
ovs-vsctl add-br br-eth1
ovs-vsctl add-br br-ex
ovs-vsctl add-port br-eth1 eth1-br-proxy
ovs-vsctl add-port br-ex ex-br-proxy
ovs-vsctl add-port br-eth0 proxy-br-eth1
ovs-vsctl add-port br-eth0 proxy-br-ex
ip link set eth1-br-proxy up promisc on
ip link set ex-br-proxy up promisc on
ip link set proxy-br-eth1 up promisc on
ip link set proxy-br-ex up promisc on
What we have done is added a new bridge br-eth0 and added eth0 to it.
Assign eth0’s ip address to br-eth0 and set the interface in promiscuous mode.
Then we create two veth pairs. In case you are not aware they are like virtual cables.
We connect br-eth1 and br-ex to br-eth0 using the veth pairs.
Then we enable promiscuous mode and bring up all the interfaces we use.
Single machine setup with 1 interface
Running Controller and Network on same host
Sometimes It is desired to have controller and Network node running on same machine and the machines have only two network interfaces each. The compute node requires only two interfaces as shown in the picture below. However in the network node we can combine the internal and external network by adding eth0 to br-ex and assigning br-ex with the ip address of eth0.
ovs-vsctl add-port br-ex eth0
ifconfig br-ex up
ip link set eth0 up promisc on
2 Machine with 2 interfaces each
If both your servers have only a single nic you may follow the below setup.
On network/controller node
#add all bridges
ovs-vsctl add-br br-int
ovs-vsctl add-br br-ex
ovs-vsctl add-br br-eth1
ovs-vsctl add-br br-proxy
#Create Veth pairs
ip link add proxy-br-eth1 type veth peer name eth1-br-proxy
ip link add proxy-br-ex type veth peer name ex-br-proxy
#Attach bridges using veth pair
ovs-vsctl add-port br-eth1 eth1-br-proxy
ovs-vsctl add-port br-ex ex-br-proxy
ovs-vsctl add-port br-proxy proxy-br-eth1
ovs-vsctl add-port br-proxy proxy-br-ex
#Assign eth0's ip address to br-proxy
ifconfig br-proxy up
#Bring up the interfaces
ip link set eth1-br-proxy up promisc on
ip link set ex-br-proxy up promisc on
ip link set proxy-br-eth1 up promisc on
ip link set proxy-br-ex up promisc on
On the Compute node
ovs-vsctl add-br br-eth1
ovs-vsctl add-port br-eth1 eth0
#Assign eth0's ip addres to br-eth1
ifconfig br-eth1 up
#Bring up the interfaces
ip link set eth0 up promisc on
The pictorial representation would be something like below
Dual machine setup with single nic each