Openstack安装Openstack（Juno版本）
国内现在做Openstack的培训也开始多起来，不过应该都是采用虚拟机的方式来培训，我个人感觉还是很土的，不能吃自己的狗食。建议日后大家参加Openstack的培训的时候，都问一句，是不是在Openstack下培训Openstack。
上次写过一篇Icehouse版本的。这次Juno版本，基本是一样的过程。不过调整一下顺序。用用户更容易操作。
这次我是使用刻通云平台进行，希望可以做到更加流畅。
（1）基本情况
默认是有一个基础网络。我们还是需要创建一个自己的网络，来满足openstack需求。
整理一下
| 角色 | 管理网络 | 虚拟机通讯网络 | 外部网络 |
| 控制节点 | eth0（10.0.0.11） | eth1（192.168.100.11） | |
| 网络节点 | eth0（10.0.0.21） | eth1（10.0.1.21） | eth2（192.168.100.21） |
| 计算节点 | eth0（10.0.0.31） | eth1（10.0.1.21） |
文档很清楚，
- 网络节点，需要3块网卡。
- 控制节点和网络节点，需要外部网络，就是需要所谓的公网的IP
- 计算节点是不需要公网IP
- 所有的虚拟机访问公网，都是需要经过网络节点。
- 192.168.100.0,就相当于公网的IP地址段
根据上图，我们组建我们自己的网络
- 创建router
- 创建管理网络，公网，虚拟机网络（记得按顺序创建）
- 公网连接router
- 申请公网IP
- 把IP绑定在router
（2）控制节点
1、网络搭建
创建一个ubuntu14.04的虚拟机，1core，2G内存，应该就够用了。网络，记得设置固定IP地址
为了登陆vnc，所以选择密码登陆
最后一步
虚拟机就创建完毕。
我们需要给控制节点添加一块网卡，连接到公网
看看拓扑图
vnc登陆
由于虚拟机获得两块网卡，默认网关就需要手工指定。
route add default gw 192.168.100.1这个时候,你就可以访问外网.
从远程访问虚拟机，可以通过端口映射，或者，这里直接在router上设置端口映射。这样你直接sshrouter的IP地址，就可以访问。
2、设置源
apt-get install ubuntu-cloud-keyring
echo "deb http://ubuntu-cloud.archive.canonical.com/ubuntu" \
"trusty-updates/juno main" > /etc/apt/sources.list.d/cloudarchive-juno.list更新
apt-get update && apt-get dist-upgrade3、NTP服务器
apt-get install -y ntp4、数据库
apt-get install mariadb-server python-mysqldb修改/etc/mysql/my.cnf
bind-address = 10.0.0.11
[mysqld]
default-storage-engine = innodb
innodb_file_per_table
collation-server = utf8_general_ci
init-connect = 'SET NAMES utf8'
character-set-server = utf8重启数据库
service mysql restart5、消息队列RabbitMQ
apt-get install -y rabbitmq-server6、keystone
安装
apt-get install -y keystone设置
创建keystone数据库，都是通过 mysql –u root –p 进入
CREATE DATABASE keystone;
GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'localhost' IDENTIFIED BY 'KEYSTONE_DBPASS';
GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'%' IDENTIFIED BY 'KEYSTONE_DBPASS';
exit;删除sqllite数据库
rm /var/lib/keystone/keystone.db编辑 /etc/keystone/keystone.conf
connection = mysql://keystone:[email protected]/keystone
[DEFAULT]
admin_token=ADMIN
log_dir=/var/log/keystone初始化keystone数据库
service keystone restart
keystone-manage db_sync设置环境变量
export OS_SERVICE_TOKEN=ADMIN
export OS_SERVICE_ENDPOINT=http://10.0.0.11:35357/v2.0创建管理员权力的用户
keystone user-create --name=admin --pass=admin_pass --email=admin@domain.com
keystone role-create --name=admin
keystone role-create --name=_member_
keystone tenant-create --name=admin --description="Admin Tenant"
keystone user-role-add --user=admin --tenant=admin --role=admin
keystone user-role-add --user=admin --role=_member_ --tenant=admin创建普通用户
keystone user-create --name=demo --pass=demo_pass --email=demo@domain.com
keystone tenant-create --name=demo --description="Demo Tenant"
keystone user-role-add --user=demo --role=_member_ --tenant=demo创建 service 租户
keystone tenant-create --name=service --description="Service Tenant"定义服务的API的endpoint
keystone service-create --name=keystone --type=identity --description="OpenStack Identity"创建endpoint
keystone endpoint-create \
--service-id=$(keystone service-list | awk '/ identity / {print $2}') \
--publicurl=http://192.168.100.11:5000/v2.0 \
--internalurl=http://10.0.0.11:5000/v2.0 \
--adminurl=http://10.0.0.11:35357/v2.0检测keystone
通过下面命令检查keystone的初始化是否正常
设置环境变量，创建creds 和 admin_creds 两个文件
cat <<EOF >>/root/creds
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=admin_pass
export OS_AUTH_URL="http://192.168.100.11:5000/v2.0/"
EOF
cat <<EOF >>/root/admin_creds
export OS_USERNAME=admin
export OS_PASSWORD=admin_pass
export OS_TENANT_NAME=admin
export OS_AUTH_URL=http://10.0.0.11:35357/v2.0
EOF检测
先退出ssh，让以前设置的环境变量失效。再登陆。
设置环境变量才能进行下面操作
source creds这样就可以
root@controller:~# keystone user-list
+----------------------------------+-------+---------+------------------+
| id | name | enabled | email |
+----------------------------------+-------+---------+------------------+
| 6f8bcafd62ec4e23ab2be28016829f91 | admin | True | admin@domain.com |
| 66713a75b7c14f73a1c5a015241f5826 | demo | True | demo@domain.com |
+----------------------------------+-------+---------+------------------+
root@controller:~# keystone role-list
+----------------------------------+----------+
| id | name |
+----------------------------------+----------+
| 9fe2ff9ee4384b1894a90878d3e92bab | _member_ |
| cd8dec7752d24a028f95657556f7573d | admin |
+----------------------------------+----------+
root@controller:~# keystone tenant-list
+----------------------------------+---------+---------+
| id | name | enabled |
+----------------------------------+---------+---------+
| efc81990ab4c433f94573e2e0fcf08c3 | admin | True |
| be10dc11d4034b389bef8bbcec657f6f | demo | True |
| cb45c886bc094f65940ba29d79eab8aa | service | True |
+----------------------------------+---------+---------+查看日志
日志在/var/log/keystone/ 下，先清空日志,看看日志是否还有错误信息.
echo "" > /var/log/keystone/keystone-all.log
echo "" > /var/log/keystone/keystone-manage.log
tail /var/log/keystone/*7、Glance
Openstack组件安装，都比较类似。
apt-get install -y glance python-glanceclient创建数据库 mysql –uroot –p
CREATE DATABASE glance;
GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'localhost' IDENTIFIED BY 'GLANCE_DBPASS';
GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'%' IDENTIFIED BY 'GLANCE_DBPASS';
exit;keystone创建glance用户和服务
keystone user-create --name=glance --pass=service_pass --email=glance@domain.com
keystone user-role-add --user=glance --tenant=service --role=admin设置endpoint
keystone service-create --name=glance --type=image --description="OpenStack Image Service"
keystone endpoint-create \
--service-id=$(keystone service-list | awk '/ image / {print $2}') \
--publicurl=http://192.168.100.11:9292 \
--internalurl=http://10.0.0.11:9292 \
--adminurl=http://10.0.0.11:9292编辑 /etc/glance/glance-api.conf
[database]
connection = mysql://glance:[email protected]/glance
[DEFAULT]
rpc_backend = rabbit
rabbit_host = 10.0.0.11
[keystone_authtoken]
auth_uri = http://10.0.0.11:5000
identity_uri = http://10.0.0.11:35357
admin_tenant_name = service
admin_user = glance
admin_password = service_pass
[paste_deploy]
flavor = keystone编辑 /etc/glance/glance-registry.conf
[database]
# The file name to use with SQLite (string value)
#sqlite_db = /var/lib/glance/glance.sqlite
connection = mysql://glance:[email protected]/glance
[keystone_authtoken]
auth_uri = http://10.0.0.11:5000
auth_host = 10.0.0.11
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = glance
admin_password = service_pass
[paste_deploy]
flavor = keystone重启服务
service glance-api restart; service glance-registry restart初始化glance数据库
glance-manage db_sync上传镜像
source creds
glance image-create --name "cirros-0.3.2-x86_64" --is-public true \
--container-format bare --disk-format qcow2 \
--location http://cdn.download.cirros-cloud.net/0.3.2/cirros-0.3.2-x86_64-disk.img查看镜像
# glance image-list
+--------------------------------------+---------------------+-------------+------------------+----------+--------+
| ID | Name | Disk Format | Container Format | Size | Status |
+--------------------------------------+---------------------+-------------+------------------+----------+--------+
| d7a6d71d-4222-44f4-82d0-49c14ba19676 | cirros-0.3.2-x86_64 | qcow2 | bare | 13167616 | active |
+--------------------------------------+---------------------+-------------+------------------+----------+--------+检测log
root@controller:~# tail /var/log/glance/*
==> /var/log/glance/api.log <==
-09-02 07:07:12.315 2946 WARNING glance.store.base [-] Failed to configure store correctly:
Store sheepdog could not be configured correctly. Reason:
Error in store configuration: [Errno 2] No such file or directory Disabling add method.
-09-02 07:07:12.316 2946 WARNING glance.store [-] Deprecated: glance.store.
sheepdog.Store not found in `known_store`.
Stores need to be explicitly enabled in the configuration file.你会发现log里有类似的所谓错误，这个不是问题。希望glance改进一下这个地方的log。不然让很多新手很郁闷。
8、Nova
安装软件
apt-get install -y nova-api nova-cert nova-conductor nova-consoleauth \
nova-novncproxy nova-scheduler python-novaclient创建nova 数据库 mysql –u root –p
CREATE DATABASE nova;
GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'localhost' IDENTIFIED BY 'NOVA_DBPASS';
GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'%' IDENTIFIED BY 'NOVA_DBPASS';
exit;配置keystone
keystone user-create --name=nova --pass=service_pass --email=nova@domain.com
keystone user-role-add --user=nova --tenant=service --role=admin设置endpoint
keystone service-create --name=nova --type=compute --description="OpenStack Compute"
keystone endpoint-create \
--service-id=$(keystone service-list | awk '/ compute / {print $2}') \
--publicurl=http://192.168.100.11:8774/v2/%\(tenant_id\)s \
--internalurl=http://10.0.0.11:8774/v2/%\(tenant_id\)s \
--adminurl=http://10.0.0.11:8774/v2/%\(tenant_id\)s编辑 /etc/nova/nova.conf
下面是我的nova.conf 文件的全部内容
[DEFAULT]
dhcpbridge_flagfile=/etc/nova/nova.conf
dhcpbridge=/usr/bin/nova-dhcpbridge
logdir=/var/log/nova
state_path=/var/lib/nova
lock_path=/var/lock/nova
force_dhcp_release=True
iscsi_helper=tgtadm
libvirt_use_virtio_for_bridges=True
connection_type=libvirt
root_helper=sudo nova-rootwrap /etc/nova/rootwrap.conf
verbose=True
ec2_private_dns_show_ip=True
api_paste_config=/etc/nova/api-paste.ini
volumes_path=/var/lib/nova/volumes
enabled_apis=ec2,osapi_compute,metadata
rpc_backend = rabbit
rabbit_host = 10.0.0.11
my_ip = 10.0.0.11
vncserver_listen = 10.0.0.11
vncserver_proxyclient_address = 10.0.0.11
auth_strategy = keystone
[keystone_authtoken]
auth_uri = http://10.0.0.11:5000
auth_host = 10.0.0.11
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = nova
admin_password = service_pass
[database]
connection = mysql://nova:[email protected]/nova删除sqlite数据库
rm /var/lib/nova/nova.sqlite初始化nova数据库
nova-manage db sync重启nova相关服务
service nova-api restart
service nova-cert restart
service nova-conductor restart
service nova-consoleauth restart
service nova-novncproxy restart
service nova-scheduler restart检查
# nova-manage service list
Binary Host Zone Status State Updated_At
nova-cert controller internal enabled 2014-08-26 14:13:08
nova-consoleauth controller internal enabled 2014-08-26 14:13:08
nova-conductor controller internal enabled 2014-08-26 14:13:08
nova-scheduler controller internal enabled 2014-08-26 14:13:089、Neutron
控制节点，也是需要安装Neutron server
apt-get install -y neutron-server neutron-plugin-ml2创建Neutron数据库 mysql –u root –p
CREATE DATABASE neutron;
GRANT ALL PRIVILEGES ON neutron.* TO neutron@'localhost' IDENTIFIED BY 'NEUTRON_DBPASS';
GRANT ALL PRIVILEGES ON neutron.* TO neutron@'%' IDENTIFIED BY 'NEUTRON_DBPASS';
exit;keystone创建neutron用户和角色
keystone user-create --name=neutron --pass=service_pass --email=neutron@domain.com
keystone user-role-add --user=neutron --tenant=service --role=admin注册服务和endpoint
keystone service-create --name=neutron --type=network --description="OpenStack Networking"
keystone endpoint-create \
--service-id=$(keystone service-list | awk '/ network / {print $2}') \
--publicurl=http://192.168.100.11:9696 \
--internalurl=http://10.0.0.11:9696 \
--adminurl=http://10.0.0.11:9696编辑 /etc/neutron/neutron.conf，关键的是nova_admin_tenant_id 需要你手工用命令获得，再填写
keystone tenant-list | awk '/ service / { print $2 }'#core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin
core_plugin = ml2
# service_plugins =
# Example: service_plugins = router,firewall,lbaas,aas,metering
service_plugins = router
# auth_strategy = keystone
auth_strategy = keystone
# allow_overlapping_ips = False
allow_overlapping_ips = True
rpc_backend = rabbit
rabbit_host = 10.0.0.11
notification_driver = neutron.openstack.common.notifier.rpc_notifier
# ======== neutron nova interactions ==========
# Send notification to nova when port status is active.
notify_nova_on_port_status_changes = True
# Send notifications to nova when port data (fixed_ips/floatingips) change
# so nova can update it's cache.
notify_nova_on_port_data_changes = True
# URL for connection to nova (Only supports one nova region currently).
nova_url = http://10.0.0.11:8774/v2
# Name of nova region to use. Useful if keystone manages more than one region
# nova_region_name =
# Username for connection to nova in admin context
nova_admin_username = nova
# The uuid of the admin nova tenant
nova_admin_tenant_id = cb45c886bc094f65940ba29d79eab8aa
# Password for connection to nova in admin context.
nova_admin_password = service_pass
# Authorization URL for connection to nova in admin context.
nova_admin_auth_url = http://10.0.0.11:35357/v2.0
[keystone_authtoken]
#auth_host = 127.0.0.1
#auth_port = 35357
#auth_protocol = http
#admin_tenant_name = %SERVICE_TENANT_NAME%
#admin_user = %SERVICE_USER%
#admin_password = %SERVICE_PASSWORD%
#signing_dir = $state_path/keystone-signing
auth_uri = http://10.0.0.11:5000
auth_host = 10.0.0.11
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = neutron
admin_password = service_pass
[database]
# This line MUST be changed to actually run the plugin.
# Example:
# connection = mysql://root:[email protected]:3306/neutron
# Replace 127.0.0.1 above with the IP address of the database used by the
# main neutron server. (Leave it as is if the database runs on this host.)
#connection = sqlite:////var/lib/neutron/neutron.sqlite
connection = mysql://neutron:[email protected]/neutron编辑 /etc/neutron/plugins/ml2/ml2_conf.ini
[ml2]
type_drivers = gre
tenant_network_types = gre
mechanism_drivers = openvswitch
[ml2_type_gre]
tunnel_id_ranges = 1:1000
[securitygroup]
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
enable_security_group = True编辑/etc/nova/nova.conf, 让nova支持neutron，在[DEFAULT] 添加
network_api_class=nova.network.neutronv2.api.API
neutron_url=http://10.0.0.11:9696
neutron_auth_strategy=keystone
neutron_admin_tenant_name=service
neutron_admin_username=neutron
neutron_admin_password=service_pass
neutron_admin_auth_url=http://10.0.0.11:35357/v2.0
libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver
linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver
firewall_driver=nova.virt.firewall.NoopFirewallDriver
security_group_api=neutron重启nova服务
service nova-api restart
service nova-scheduler restart
service nova-conductor restart这里面有一个bug，需要修复 http://www.tuicool.com/articles/vmaiiua
neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade juno重启neutron服务
service neutron-server restart查看log
root@controller:~# tail -f /var/log/neutron/*
-09-02 07:27:53.950 5373 WARNING neutron.api.extensions [-] Extension fwaas not supported by any of loaded plugins
-09-02 07:27:53.952 5373 WARNING neutron.api.extensions [-] Extension flavor not supported by any of loaded plugins
-09-02 07:27:53.962 5373 WARNING neutron.api.extensions [-] Extension lbaas_agent_scheduler not supported by any of loaded plugins
-09-02 07:27:53.967 5373 WARNING neutron.api.extensions [-] Extension lbaas not supported by any of loaded plugins
-09-02 07:27:53.969 5373 WARNING neutron.api.extensions [-] Extension metering not supported by any of loaded plugins
-09-02 07:27:53.973 5373 WARNING neutron.api.extensions [-] Extension port-security not supported by any of loaded plugins
-09-02 07:27:53.977 5373 WARNING neutron.api.extensions [-] Extension routed-service-insertion not supported by any of loaded plugins日志里显示找不到插件，这都是正常的。
10、Horizon
Dashboard的安装，倒是比较简单，不需要创建数据库。
apt-get install -y apache2 memcached libapache2-mod-wsgi openstack-dashboard编辑 /etc/openstack-dashboard/local_settings.py
#ALLOWED_HOSTS = ['horizon.example.com', ]
ALLOWED_HOSTS = ['localhost','192.168.100.11']
#OPENSTACK_HOST = "127.0.0.1"
OPENSTACK_HOST = "10.0.0.11"重启apache服务
service apache2 restart; service memcached restart这个时候，你可以通过 http://192.168.100.11/horizon
看到登录界面，应该是无法登录。
安装Openstack client端
在控制节点装上Openstack的client端，这样会方便很多，很多Neutron的操作，你都可以进行
apt-get -y install python-openstackclient（3）网络节点
看图理解的更好，这图来自redhat的官方文档。
网络节点需要3块网卡。经常有朋友问，1块网卡是否可以。其实1块网卡肯定也是可以的，不过不利于大家理解。不过大家都很难找到3块网卡的机器，所以在IaaS下来测试，就方便很多。
创建一个虚拟机，名字为：network, 删除网卡，并且添加3块网卡。ssh到虚拟机上，默认是无法访问外网的，原因也很简单，没有默认路由，手工添加默认路由就可以。
由于网络节点,比较特殊,我们需要把网卡的Ip设置成固定 /etc/netwrok/interface
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# Source interfaces
# Please check /etc/network/interfaces.d before changing this file
# as interfaces may have been defined in /etc/network/interfaces.d
# NOTE: the primary ethernet device is defined in
# /etc/network/interfaces.d/eth0
# See LP: #1262951
#source /etc/network/interfaces.d/*.cfg
# The management network interface
auto eth0
iface eth0 inet static
address 10.0.0.21
netmask 255.255.255.0
# VM traffic interface
auto eth1
iface eth1 inet static
address 10.0.1.21
netmask 255.255.255.0
# The public network interface
auto eth2
iface eth2 inet static
address 192.168.100.21
netmask 255.255.255.0
gateway 192.168.100.1
dns-nameservers 114.114.114.114设置完毕，重启虚拟机。
这个时候，你就可以访问外网，安装包。
apt-get update -y && apt-get upgrade -y && apt-get dist-upgrade同步时间
apt-get install -y ntp编辑 /etc/ntp.conf
server 10.0.0.11重启NTP服务
service ntp restart安装基础组件
apt-get install -y vlan bridge-utils编辑 /etc/sysctl.conf
net.ipv4.ip_forward=1
net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.default.rp_filter=0检测
sysctl -p安装Neutron组件
apt-get install -y neutron-plugin-ml2 neutron-plugin-openvswitch-agent \
dnsmasq neutron-l3-agent neutron-dhcp-agent编辑 /etc/neutron/neutron.conf , 这里修改的内容，比控制节点少很多。
#core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin
core_plugin = ml2
# service_plugins =
# Example: service_plugins = router,firewall,lbaas,aas,metering
service_plugins = router
# The strategy to be used for auth.
# Supported values are 'keystone'(default), 'noauth'.
auth_strategy = keystone
allow_overlapping_ips = True
rpc_backend = neutron.openstack.common.rpc.impl_kombu
rabbit_host = 10.0.0.11
[keystone_authtoken]
#auth_host = 127.0.0.1
#auth_port = 35357
#auth_protocol = http
#admin_tenant_name = %SERVICE_TENANT_NAME%
#admin_user = %SERVICE_USER%
#admin_password = %SERVICE_PASSWORD%
#signing_dir = $state_path/keystone-signing
auth_uri = http://10.0.0.11:5000
auth_host = 10.0.0.11
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = neutron
admin_password = service_pass编辑 /etc/neutron/l3_agent.ini
interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver
use_namespaces = True编辑 /etc/neutron/dhcp_agent.ini
interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver
dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq
use_namespaces = True编辑 /etc/neutron/metadata_agent.ini
auth_url = http://10.0.0.11:5000/v2.0
auth_region = regionOne
admin_tenant_name = service
admin_user = neutron
admin_password = service_pass
nova_metadata_ip = 10.0.0.11
metadata_proxy_shared_secret = helloOpenStack登录控制节点，修改 /etc/nova.conf 在[DEFAULT] 加入下面内容
service_neutron_metadata_proxy = true
neutron_metadata_proxy_shared_secret = helloOpenStack重启nova api服务
service nova-api restart编辑 /etc/neutron/plugins/ml2/ml2_conf.ini
[ml2]
type_drivers = gre
tenant_network_types = gre
mechanism_drivers = openvswitch
[ml2_type_gre]
tunnel_id_ranges = 1:1000
[ovs]
local_ip = 10.0.1.21
tunnel_type = gre
enable_tunneling = True
[securitygroup]
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
enable_security_group = True重启openvswitch
service openvswitch-switch restart创建br-ex
创建br-ex连接外网，这个不太好理解，看图
大概意思是：我们创建一个bridge br-ex，把br-ex绑定在eth2下，eth2是连接到公网的路由器上的。
ovs-vsctl add-br br-ex
ovs-vsctl add-port br-ex eth2下面内容是我操作的结果,大家慢慢理解.
编辑 /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# Source interfaces
# Please check /etc/network/interfaces.d before changing this file
# as interfaces may have been defined in /etc/network/interfaces.d
# NOTE: the primary ethernet device is defined in
# /etc/network/interfaces.d/eth0
# See LP: #1262951
#source /etc/network/interfaces.d/*.cfg
# The management network interface
auto eth0
iface eth0 inet static
address 10.0.0.21
netmask 255.255.255.0
# VM traffic interface
auto eth1
iface eth1 inet static
address 10.0.1.21
netmask 255.255.255.0
# The public network interface
# auto eth2
# iface eth2 inet static
# address 192.168.100.21
# netmask 255.255.255.0
# gateway 192.168.100.1
# dns-nameservers 114.114.114.114
auto eth2
iface eth2 inet manual
up ifconfig $IFACE 0.0.0.0 up
up ip link set $IFACE promisc on
down ip link set $IFACE promisc off
down ifconfig $IFACE down
auto br-ex
iface br-ex inet static
address 192.168.100.21
netmask 255.255.255.0
gateway 192.168.100.1
dns-nameservers 114.114.114.114重启虚拟机
替换br-ex和eth2的mac地址
由于网络的限制，目前192.168.100.21和192.168.100.11是无法通讯的，原因是因为出于安全的考虑，对网络访问的mac地址和ip地址做了绑定和限制。
通过ifconfig 查看网卡的mac地址，通过命令，把mac地址互换。
- br-ex mac 地址 c2:32:7d:cf:9d:4
- eth2 mac地址 fa:16:3e:80:5d:e6
ip link set eth2 addr c2:32:7d:cf:9d:43
ip link set br-ex addr fa:16:3e:80:5d:e6这个时候，外部网络的IP就可以互相访问。这些修改是临时性的，如果重启neutron服务，mac地址就会恢复。不过我们实验不需要重启服务。这里提供的是临时的方法，后面有彻底解决问题的办法。
设置环境变量
cat <<EOF >>/root/creds
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=admin_pass
export OS_AUTH_URL="http://192.168.100.11:5000/v2.0/"
EOF这样你就可以看到安装的agent
source creds
neutron agent-list# neutron agent-list
+--------------------------------------+--------------------+---------+-------+----------------+
| id | agent_type | host | alive | admin_state_up |
+--------------------------------------+--------------------+---------+-------+----------------+
| 3a80d2ea-bcf6-4835-b125-55144948024c | Open vSwitch agent | network | | True |
| 4219dd20-c4fd-4586-b2fc-c81bec0015d6 | L3 agent | network | | True |
| e956687f-a658-4226-a34f-368da61e9e44 | Metadata agent | network | | True |
| f3e841f8-b803-4134-9ba6-3152c3db5592 | DHCP agent | network | | True |
+--------------------------------------+--------------------+---------+-------+----------------+（4）计算节点
创建一个虚拟机，名字为：compute1, 删除网卡，并且添加2块网卡。ssh到虚拟机上.
计算节点默认是不需要接公网，不过由于我需要安装包，必须联网，所以你可以创建完虚拟机后，给虚拟机连接到外部网络，装完系统后，再断开就可以。
route add default gw 192.168.100.1这个时候，你就可以访问外网，安装包。
apt-get update -y && apt-get upgrade -y && apt-get dist-upgrade同步时间
apt-get install -y ntp编辑 /etc/ntp.conf
server 10.0.0.11重启NTP服务
service ntp restart安装kvm套件
apt-get install -y kvm libvirt-bin pm-utils安装计算节点组件
apt-get install -y nova-compute-kvm python-guestfs让内核只读
dpkg-statoverride --update --add root root 0644 /boot/vmlinuz-$(uname -r)创建脚本 /etc/kernel/postinst.d/statoverride
#!/bin/sh
version="$1"
# passing the kernel version is required
[ -z "${version}" ] && exit 0
dpkg-statoverride --update --add root root 0644 /boot/vmlinuz-${version}允许运行
chmod +x /etc/kernel/postinst.d/statoverride编辑 /etc/nova/nova.conf 文件，添加下面内容
[DEFAULT]
dhcpbridge_flagfile=/etc/nova/nova.conf
dhcpbridge=/usr/bin/nova-dhcpbridge
logdir=/var/log/nova
state_path=/var/lib/nova
lock_path=/var/lock/nova
force_dhcp_release=True
iscsi_helper=tgtadm
libvirt_use_virtio_for_bridges=True
connection_type=libvirt
root_helper=sudo nova-rootwrap /etc/nova/rootwrap.conf
verbose=True
ec2_private_dns_show_ip=True
api_paste_config=/etc/nova/api-paste.ini
volumes_path=/var/lib/nova/volumes
enabled_apis=ec2,osapi_compute,metadata
auth_strategy = keystone
rpc_backend = rabbit
rabbit_host = 10.0.0.11
my_ip = 10.0.0.31
vnc_enabled = True
vncserver_listen = 0.0.0.0
vncserver_proxyclient_address = 10.0.0.31
novncproxy_base_url = http://192.168.100.11:6080/vnc_auto.html
glance_host = 10.0.0.11
vif_plugging_is_fatal=false
vif_plugging_timeout=0
[database]
connection = mysql://nova:[email protected]/nova
[keystone_authtoken]
auth_uri = http://10.0.0.11:5000
auth_host = 10.0.0.11
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = nova
admin_password = service_pass
删除sqlite
rm /var/lib/nova/nova.sqlite重启compute服务
service nova-compute restart编辑 /etc/sysctl.conf
net.ipv4.ip_forward=1
net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.default.rp_filter=0马上生效
sysctl -p安装网络组件
apt-get install -y neutron-common neutron-plugin-ml2 neutron-plugin-openvswitch-agent编辑 /etc/neutron/neutron.conf
#core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin
core_plugin = ml2
# service_plugins =
# Example: service_plugins = router,firewall,lbaas,aas,metering
service_plugins = router
auth_strategy = keystone
allow_overlapping_ips = True
rpc_backend = neutron.openstack.common.rpc.impl_kombu
rabbit_host = 10.0.0.11
[keystone_authtoken]
#auth_host = 127.0.0.1
#auth_port = 35357
#auth_protocol = http
#admin_tenant_name = %SERVICE_TENANT_NAME%
#admin_user = %SERVICE_USER%
#admin_password = %SERVICE_PASSWORD%
#signing_dir = $state_path/keystone-signing
auth_uri = http://10.0.0.11:5000
auth_host = 10.0.0.11
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = neutron
admin_password = service_pass
编辑 /etc/neutron/plugins/ml2/ml2_conf.ini
[ml2]
type_drivers = gre
tenant_network_types = gre
mechanism_drivers = openvswitch
[ml2_type_gre]
tunnel_id_ranges = 1:1000
[ovs]
local_ip = 10.0.1.31
tunnel_type = gre
enable_tunneling = True
[securitygroup]
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
enable_security_group = True重启OVS
service openvswitch-switch restart再编辑 /etc/nova/nova.conf ，在[DEFAULT]里添加下面
network_api_class = nova.network.neutronv2.api.API
neutron_url = http://10.0.0.11:9696
neutron_auth_strategy = keystone
neutron_admin_tenant_name = service
neutron_admin_username = neutron
neutron_admin_password = service_pass
neutron_admin_auth_url = http://10.0.0.11:35357/v2.0
linuxnet_interface_driver = nova.network.linux_net.LinuxOVSInterfaceDriver
firewall_driver = nova.virt.firewall.NoopFirewallDriver
security_group_api = neutron编辑 /etc/nova/nova-compute.conf ,修改为使用qemu
[DEFAULT]
compute_driver=libvirt.LibvirtDriver
[libvirt]
virt_type=qemu重启相关服务
service nova-compute restart
service neutron-plugin-openvswitch-agent restart安装就全部完成。
登录控制节点
root@controller:~# source creds
root@controller:~# nova-manage service list
Binary Host Zone Status State Updated_At
nova-cert controller internal enabled 2014-09-02 10:31:03
nova-conductor controller internal enabled 2014-09-02 10:31:04
nova-scheduler controller internal enabled 2014-09-02 10:30:58
nova-consoleauth controller internal enabled 2014-09-02 10:31:00
nova-compute compute1 nova enabled 2014-09-02 10:30:57
root@controller:~#1、命令行创建虚拟机
在控制节点上，运行下面的命令就可以。镜像我上面已经上传。下面的操作，你完全可以在Dashboard里进行操作，这里命令行下，了解更加深入。
下面的操作，在控制节点完成。
创建外部网络
source creds
#Create the external network:
neutron net-create ext-net --shared --router:external=True
#Create the subnet for the external network:
neutron subnet-create ext-net --name ext-subnet \
--allocation-pool start=192.168.100.101,end=192.168.100.200 \
--disable-dhcp --gateway 192.168.100.1 192.168.100.0/24给租户创建内部网络
#Create the internal network:
neutron net-create int-net
#Create the subnet for the internal network:
neutron subnet-create int-net --name int-subnet \
--dns-nameserver 114.114.114.114 --gateway 172.16.1.1 172.16.1.0/24创建路由，并且连接到外部网络
#Create the router:
neutron router-create router1
#Attach the router to the internal subnet:
neutron router-interface-add router1 int-subnet
#Attach the router to the external network by setting it as the gateway:
neutron router-gateway-set router1 ext-net创建密钥
ssh-keygen添加公钥
nova keypair-add --pub-key ~/.ssh/id_rsa.pub key1设置安全组
# Permit ICMP (ping):
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
# Permit secure shell (SSH) access:
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0创建虚拟机
NET_ID=$(neutron net-list | awk '/ int-net / { print $2 }')
nova boot --flavor m1.tiny --image cirros-0.3.2-x86_64 --nic net-id=$NET_ID \
--security-group default --key-name key1 instance1查看虚拟机
nova list申请公网IP
neutron floatingip-create ext-net关联floating IP
nova floating-ip-associate instance1 192.168.100.102这个时候，你会发现你在控制节点上，根本是无法访问 router 192.168.100.101和floating ip 192.168.100.102。
访问虚拟机，你需要登录网络节点上，你可以用下面命令访问虚拟机
# ip netns
qdhcp-bf7f3043-d696-4735-9bc7-8c2e4d95c8d5
qrouter-7e8bbb53-1ea6-4763-a69c-a0c875b5224b第一个的虚拟机，第二个是路由器
# ip netns exec qdhcp-bf7f3043-d696-4735-9bc7-8c2e4d95c8d5 ifconfig
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:16 errors:0 dropped:0 overruns:0 frame:0
TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1216 (1.2 KB) TX bytes:1216 (1.2 KB)
tap1a85db16-da Link encap:Ethernet HWaddr fa:16:3e:ce:e0:e2
inet addr:172.16.1.3 Bcast:172.16.1.255 Mask:255.255.255.0
inet6 addr: fe80::f816:3eff:fece:e0e2/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:415 errors:0 dropped:0 overruns:0 frame:0
TX packets:105 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:64724 (64.7 KB) TX bytes:10228 (10.2 KB)2、访问公网
你可能发现一个很明显的问题，你在网络节点是可以ping 通虚拟机的floating IP，router的IP，不过你在控制节点是无法访问的。
如果希望比较完美，实现虚拟机可以ping通公网，那么需要我们多了解一下内容才行。可以发现全部的流量都是通过192.168.100.21这个端口出去，我们需要设置一下这个端口，运行所有的IP和mac地址通过。
登录网络节点，通过ping 192.168.100.101 和192.168.100.102 ，获得他们的mac地址。
# arp -a
? (10.0.0.11) at fa:16:3e:34:d0:7a [ether] on eth0
? (192.168.100.102) at fa:16:3e:0c:be:cd [ether] on br-ex
? (10.0.1.31) at fa:16:3e:eb:96:1c [ether] on eth1
? (192.168.100.101) at fa:16:3e:0c:be:cd [ether] on br-ex
? (192.168.100.1) at fa:16:3e:c2:a8:a8 [ether] on br-ex下面的操作，你可以在控制节点完成
通过curl获取token
使用token，修改192.168.100.21 port 的allow_address_pairs ，可以顺便把eth2和br-ex也修改，这样就不担心重启服务。
详细的操作，就参考这篇文档就可以。
http://www.chenshake.com/use-the-uos-api/
3、vnc访问
如果你登录Horizon，访问虚拟机，vnc可能无法访问，你需要登录uos，修改安全组规则。默认第一个虚拟机使用vnc的端口是6080。或者你全部打开端口。
参考资料
http://oddbit.com/rdo-hangout-multinode-packstack-slides/#/
https://github.com/ChaimaGhribi/OpenStack-Icehouse-Installation/blob/master/OpenStack-Icehouse-Installation.rst
参考文档 http://blog.oddbit.com/2014/05/23/open-vswitch-and-persistent-ma/
ovs-vsctl操作
root@network:~# ovs-vsctl show
05dd-bd0d-4af1-a331-c9394fbcb775
Bridge br-int
fail_mode: secure
Port br-int
Interface br-int
type: internal
ovs_version: "2.0.2"
root@network:~# ovs-vsctl add-br br-ex
root@network:~# ovs-vsctl show
05dd-bd0d-4af1-a331-c9394fbcb775
Bridge br-ex
Port br-ex
Interface br-ex
type: internal
Bridge br-int
fail_mode: secure
Port br-int
Interface br-int
type: internal
ovs_version: "2.0.2"
root@network:~# ovs-vsctl add-port br-ex eth2
root@network:~# ovs-vsctl show
05dd-bd0d-4af1-a331-c9394fbcb775
Bridge br-ex
Port "eth2"
Interface "eth2"
Port br-ex
Interface br-ex
type: internal
Bridge br-int
fail_mode: secure
Port br-int
Interface br-int
type: internal
ovs_version: "2.0.2"网络节点重启服务
service neutron-plugin-openvswitch-agent restart
service neutron-dhcp-agent restart
service neutron-l3-agent restart
service neutron-metadata-agent restart
service dnsmasq restart附录
建议采用的方式，我就简单，采用pptp
PPTP，默认拨号连接上去，你本地就无法上网了。你需要设置一下。
去掉这个勾就可以了。
这个时候，你拨号连接上去，访问虚拟机的时候，走通道，访问外网，还是走你以前的网络。
你还需要在你的增加一条路由才行，不过在win7，win8，增加路由，需要管理员权限。
按 Windows key + X, 再按 A
这时候，你就调出管理员权限的cmd
添加一条路由
route add 192.168.100.0 mask 255.255.255.0 10.100.100.1这时候，你就可以ping通那台控制节点的虚拟机。不容易吧。
其他参考：http://www.tuicool.com/articles/jIVR3eM
http://www.aboutyun.com/thread-9670-1-1.html