Apr
08
2013
2013年4月4日，已经正式发布了Grizzly版本，目前Ubuntu 12.04的Grizzly的deb包，也基本准备好了，这次Ubuntu的动作真的是很快。以前Folsom版本的时候，由于Quantum的复杂和很多问题，导致网络上有不少问题。Grizzly版本的Quantum，已经做了很大的改进。关于Quantum介绍，可以查看官方wiki
Qauntum 需要通过插件来实现网络管理，有各种的商业插件，目前开源的插件就2个，linux Bridge和Openvswith，这篇文档是针对Bridge，Bridge不支持GRE模式，只支持VLAN和Local两种模式，Local只能用于测试使用。如果是多节点，VLAN模式就需要交换机支持，对端口设置Trunk。
参考英文原文：https://github.com/mseknibilel/OpenStack-Grizzly-Install-Guide
Quantum linux-Bridge 插件介绍：https://wiki.openstack.org/wiki/Quantum-Linux-Bridge-Plugin
我会对原文进行一些调整
- 我的网络环境eth0连接公网，eth1是内网
- 组件需要的数据库，我统一创建
- 通过设置环境变量的办法，使得文章适应不同的场景（例如不同的IP和密码）
文档修改历史
2013年4月10凌晨：Grizzly正式版本的Ubuntu 12.04的包刚刚发布，使用最新的包，已经基本完成安装过程，目前只剩下Dashboard的使用。由于quantum的网络设置比较复杂，需要设置好网络，才能进一步测试。
2013年4月17日：修正了文档几个错误，nova.conf 里quantum的账号设置错误，还有就是quantum没设置 /etc/quantum/dhcp_agent.ini:不过目前还是没法创建虚拟机，非常郁闷。目前看到的情况是：Dashboard里还无法完全管理网络，尤其是bridge模式。
组件和网络拓扑图
Contents [hide]
准备环境
操作系统
安装Ubuntu 12.04.2, 最小化安装就可以。目前内核已经升级到3.5，这样应该会减少安装过程的麻烦。
源
默认Ubuntu 12.04的源是Essex版本，我们需要增加Grizzly的源。不过需要你手工添加。源的官方说明
apt-get update apt-get install ubuntu-cloud-keyring
添加Grizzly源
cat <<EOF >>/etc/apt/sources.list deb http://ubuntu-cloud.archive.canonical.com/ubuntu precise-proposed/grizzly main deb http://ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/grizzly main EOF
运行下面命令进行更新
apt-get update && apt-get -y dist-upgrade
Hostname
Hostname设置,其实不是必须的,不过养成一个习惯,也方便自己.
root@node08:~# cat /etc/hostname
node08
root@node08:~# cat /etc/hosts
127.0.0.1 localhost
127.0.1.1 node08.chenshake.com node08
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
root@node08:~# hostname
node08
root@node08:~# hostname -f
node08.chenshake.com
网络
# cat /etc/network/interfaces # This file describes network interfaces avaiulable on your system # and how to activate them. For more information, see interfaces(5). # Modified by convert_static.sh. auto lo iface lo inet loopback auto eth0 iface eth0 inet static address 10.1.199.8 hwaddress ether 00:e0:81:d8:42:f6 netmask 255.255.255.0 network 10.1.199.0 gateway 10.1.199.1 dns-search chenshake.com dns-nameservers 10.1.199.5 auto eth1 iface eth1 inet static address 10.10.10.8 netmask 255.255.255.0
重启网络
/etc/init.d/networking restart
IP转发
sed -i -r 's/^\s*#(net\.ipv4\.ip_forward=1.*)/\1/' /etc/sysctl.conf
echo 1 > /proc/sys/net/ipv4/ip_forward
检查修改结果
# sysctl -p net.ipv4.ip_forward = 1
查看当前机器路由
root@node08:~# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 10.1.199.1 0.0.0.0 UG 100 0 0 eth0
10.1.199.0 * 255.255.255.0 U 0 0 0 eth0
10.10.10.0 * 255.255.255.0 U 0 0 0 eth1
NTP服务
apt-get install -y ntp
环境变量
环境变量主要是为了使得文档更加灵活,参数可以设置
cat >/root/novarc <<EOF export OS_TENANT_NAME=admin export OS_USERNAME=admin export OS_PASSWORD=password export MYSQL_PASS=password export SERVICE_PASSWORD=password export RABBIT_PASSWORD=password export OS_AUTH_URL="http://localhost:5000/v2.0/" export SERVICE_ENDPOINT="http://localhost:35357/v2.0" export SERVICE_TOKEN=ADMIN export PUBLIC_IP="$(/sbin/ifconfig eth0 | awk '/inet addr/ {print $2}' | cut -f2 -d ":")" export LOCAL_IP="$(/sbin/ifconfig eth1 | awk '/inet addr/ {print $2}' | cut -f2 -d ":")" EOF
你可以根据你的需要调整上面password字段.它会自动获得你网卡的IP地址, 你可以根据你的情况调整。
查看novarc
cat /root/novarc export OS_TENANT_NAME=admin export OS_USERNAME=admin export OS_PASSWORD=password export MYSQL_PASS=password export SERVICE_PASSWORD=password export RABBIT_PASSWORD=password export OS_AUTH_URL="http://localhost:5000/v2.0/" export SERVICE_ENDPOINT="http://localhost:35357/v2.0" export SERVICE_TOKEN=ADMIN export PUBLIC_IP="10.1.199.8" export LOCAL_IP="10.10.10.8"
让环境变量生效
source novarc
echo "source novarc">>.bashrc
RabbitMQ和相关软件
apt-get -y install rabbitmq-server vlan bridge-utils curl
数据库
Openstack的组件都需要用到mysql
| 数据库 | 用户 | 密码 |
| mysql | root | password |
| keystone | keystone | password |
| nova | nova | password |
| glance | glance | password |
| cinder | cinder | password |
| quantum | quantum | password |
设置自动安装，无需输入密码
cat <<MYSQL_PRESEED | debconf-set-selections
mysql-server-5.5 mysql-server/root_password password $MYSQL_PASS
mysql-server-5.5 mysql-server/root_password_again password $MYSQL_PASS
mysql-server-5.5 mysql-server/start_on_boot boolean true
MYSQL_PRESEED
安装mysql
apt-get -y install mysql-server python-mysqldb
允许远程访问mysql
sed -i 's/127.0.0.1/0.0.0.0/g' /etc/mysql/my.cnf
重启服务
service mysql restart
创建数据库
mysql -uroot -p$MYSQL_PASS <<EOF CREATE DATABASE nova; GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'%' IDENTIFIED BY '$MYSQL_PASS'; CREATE DATABASE glance; GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'%' IDENTIFIED BY '$MYSQL_PASS'; CREATE DATABASE keystone; GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'%'IDENTIFIED BY '$MYSQL_PASS'; CREATE DATABASE cinder; GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'%'IDENTIFIED BY '$MYSQL_PASS'; CREATE DATABASE quantum; GRANT ALL PRIVILEGES ON quantum.* TO 'quantum'@'%'IDENTIFIED BY '$MYSQL_PASS'; FLUSH PRIVILEGES; EOF
Keystone
安装
apt-get install -y keystone
查看keystone运行状态
service keystone status
编辑 /etc/keystone/keystone.conf
需要注意的是admin_token 默认是ADMIN
[sql] # The SQLAlchemy connection string used to connect to the database #connection = sqlite:////var/lib/keystone/keystone.db connection = mysql://keystone:[email protected]:3306/keystone [signing] #token_format = PKI token_format = UUID
重启服务和初始化数据库
service keystone restart keystone-manage db_sync
使用脚本导入初始化数据
keystone.sh 和 endpoint.sh，脚本运行过程中，会出现 WARNING: Bypassing authentication using a token & endpoint (authentication credentials are being ignored). 提示，忽略就可以。
wget http://www.chenshake.com/wp-content/uploads/2013/04/keystone.sh_.txt
mv keystone.sh_.txt keystone.sh
bash keystone.sh
导入endpoint
wget http://www.chenshake.com/wp-content/uploads/2013/04/endpoint.sh_.txt
mv endpoint.sh_.txt endpoint.sh
bash endpoint.sh
验证keystone
keystone user-list keystone role-list keystone tenant-list keystone endpoint-list service keystone status
Troubleshooting Keystone
1. 查看 5000 和 35357 端口是否在监听
2. 查看 /var/log/keystone/keystone.log 报错信息
3. keystone.sh 脚本执行错误解决：(检查脚本内容变量设置)
如果你运行上面脚本出现问题，你可以删除数据库再导入数据，运行脚本出错，基本都是环境变量设置有误导致
mysql -uroot -p mysql> drop database keystone; mysql> create database keystone; mysql> quit;
记得需要同步一下数据库
keystone-manage db_sync
Glance
安装
apt-get -y install glance
配置
编辑 /etc/glance/glance-api.conf
#sql_connection = sqlite:////var/lib/glance/glance.sqlite sql_connection = mysql://glance:[email protected]/glance [keystone_authtoken] auth_host = 127.0.0.1 auth_port = 35357 auth_protocol = http #admin_tenant_name = %SERVICE_TENANT_NAME% #admin_user = %SERVICE_USER% #admin_password = %SERVICE_PASSWORD% admin_tenant_name = service admin_user = glance admin_password = password [paste_deploy] config_file = /etc/glance/glance-api-paste.ini flavor = keystone
重启服务
service glance-api restart
编辑 /etc/glance/glance-registry.conf
#sql_connection = sqlite:////var/lib/glance/glance.sqlite sql_connection = mysql://glance:[email protected]/glance [keystone_authtoken] auth_host = 127.0.0.1 auth_port = 35357 auth_protocol = http #admin_tenant_name = %SERVICE_TENANT_NAME% #admin_user = %SERVICE_USER% #admin_password = %SERVICE_PASSWORD% admin_tenant_name = service admin_user = glance admin_password = password [paste_deploy] config_file = /etc/glance/glance-registry-paste.ini flavor = keystone
重启服务
service glance-registry restart
同步数据库
glance-manage version_control 0 glance-manage db_sync
验证Glance
glance image-list
下面应该没任何输出，就表示正确。因为目前还没有上传image。
下载Image
我们下载CirrOS的image作为测试使用，只有10M。如果是ubuntu官方的image，220M，并且ubuntu官方的image，都是需要使用密钥登陆。
CirrOS
下载image
wget https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img
上传image
glance image-create --name=cirros-0.3.0-x86_64 --public --container-format=bare \ --disk-format=qcow2 < /root/cirros-0.3.0-x86_64-disk.img
Cirros，是可以使用用户名和密码登陆，也可以使用密钥登陆
user:cirros
password:cubswin:)
Ubuntu官方image
下载image
wget http://cloud-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64-disk1.img
上传image
glance image-create --name="Ubuntu 12.04 cloudimg amd64" --public --container-format=ovf \ --disk-format=qcow2 < /root/ubuntu-12.04-server-cloudimg-amd64-disk1.img
user：ubuntu
只能使用密钥登陆。
Quantum
安装
apt-get install -y quantum-server quantum-plugin-linuxbridge quantum-plugin-linuxbridge-agent \ dnsmasq quantum-dhcp-agent quantum-l3-agent
编辑 /etc/quantum/quantum.conf
#core_plugin = quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2 core_plugin = quantum.plugins.linuxbridge.lb_quantum_plugin.LinuxBridgePluginV2 [keystone_authtoken] auth_host = 127.0.0.1 auth_port = 35357 auth_protocol = http #admin_tenant_name = %SERVICE_TENANT_NAME% #admin_user = %SERVICE_USER% #admin_password = %SERVICE_PASSWORD% admin_tenant_name = service admin_user = quantum admin_password = password signing_dir = /var/lib/quantum/keystone-signing
配置bridge插件 /etc/quantum/plugins/linuxbridge/linuxbridge_conf.ini
# Default: tenant_network_type = local # Example: tenant_network_type = vlan tenant_network_type = vlan # Default: network_vlan_ranges = # Example: network_vlan_ranges = physnet1:1000:2999 network_vlan_ranges = physnet1:1000:2999 #sql_connection = sqlite:////var/lib/quantum/linuxbridge.sqlite sql_connection = mysql://quantum:[email protected]/quantum # Default: physical_interface_mappings = # Example: physical_interface_mappings = physnet1:eth1 physical_interface_mappings = physnet1:eth0
编辑 /etc/quantum/l3_agent.ini:
#interface_driver = quantum.agent.linux.interface.OVSInterfaceDriver interface_driver = quantum.agent.linux.interface.BridgeInterfaceDriver # use_namespaces = True use_namespaces = False
编辑 /etc/quantum/dhcp_agent.ini
interface_driver = quantum.agent.linux.interface.BridgeInterfaceDriver
use_namespaces = False
重启quantum所有服务
cd /etc/init.d/; for i in $( ls quantum-* ); do sudo service $i restart; done service dnsmasq restart
验证Quantum
quantum help
看到所有quantum相关的命令,目前quantum没有任何数据
quantum net-list
KVM
安装
apt-get install -y kvm libvirt-bin pm-utils
编辑 /etc/libvirt/qemu.conf ，添加下面内容
cgroup_device_acl = [
"/dev/null", "/dev/full", "/dev/zero",
"/dev/random", "/dev/urandom",
"/dev/ptmx", "/dev/kvm", "/dev/kqemu",
"/dev/rtc", "/dev/hpet","/dev/net/tun",
]
或者运行命令：这个地方用命令修改有点复杂，还没找到太好的办法。
cat <<EOF>>/etc/libvirt/qemu.conf
cgroup_device_acl = [
"/dev/null", "/dev/full", "/dev/zero",
"/dev/random", "/dev/urandom",
"/dev/ptmx", "/dev/kvm", "/dev/kqemu",
"/dev/rtc", "/dev/hpet","/dev/net/tun",
]
EOF
删除默认 virtual bridge
virsh net-destroy default virsh net-undefine default
允许迁移
编辑 /etc/libvirt/libvirtd.conf,
#auth_tcp = "sasl" auth_tcp = "none"
或者运行下面命令
sed -i '/#listen_tcp/s/#listen_tcp/listen_tcp/; /#auth_tcp/s/#auth_tcp/auth_tcp/; /auth_tcp/s/sasl/none/' /etc/libvirt/libvirtd.conf
编辑 /etc/init/libvirt-bin.conf
env libvirtd_opts="-d -l"
或者使用命令
sed -i '/env libvirtd_opts/s/-d/-d -l/' /etc/init/libvirt-bin.conf
编辑 /etc/default/libvirt-bin
libvirtd_opts="-d -l"
或者使用命令
sed -i '/libvirtd_opts/s/-d/-d -l/' /etc/default/libvirt-bin
重启服务
service libvirt-bin restart
Nova
安装
apt-get install -y nova-api nova-cert novnc nova-consoleauth nova-scheduler \ nova-novncproxy nova-doc nova-conductor nova-compute-kvm
编辑 /etc/nova/api-paste.ini
[filter:authtoken] paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory auth_host = 127.0.0.1 auth_port = 35357 auth_protocol = http #admin_tenant_name = %SERVICE_TENANT_NAME% #admin_user = %SERVICE_USER% #admin_password = %SERVICE_PASSWORD% admin_tenant_name = service admin_user = nova admin_password = password signing_dir = /tmp/keystone-signing-nova # Workaround for https://bugs.launchpad.net/nova/+bug/1154809 auth_version = v2.0
创建 /etc/nova/nova.conf
cat >/etc/nova/nova.conf <<EOF [DEFAULT] logdir=/var/log/nova state_path=/var/lib/nova lock_path=/run/lock/nova verbose=True api_paste_config=/etc/nova/api-paste.ini compute_scheduler_driver=nova.scheduler.simple.SimpleScheduler rabbit_host=$LOCAL_IP nova_url=http://$LOCAL_IP:8774/v1.1/ sql_connection=mysql://nova:$MYSQL_PASS@$LOCAL_IP/nova root_helper=sudo nova-rootwrap /etc/nova/rootwrap.conf # Auth use_deprecated_auth=false auth_strategy=keystone # Imaging service glance_api_servers=$LOCAL_IP:9292 image_service=nova.image.glance.GlanceImageService # Vnc configuration novnc_enabled=true novncproxy_base_url=http://$PUBLIC_IP:6080/vnc_auto.html novncproxy_port=6080 vncserver_proxyclient_address=$LOCAL_IP vncserver_listen=0.0.0.0 # Network settings network_api_class=nova.network.quantumv2.api.API quantum_url=http://$LOCAL_IP:9696 quantum_auth_strategy=keystone quantum_admin_tenant_name=service quantum_admin_username=quantum quantum_admin_password=$SERVICE_PASSWORD quantum_admin_auth_url=http://$LOCAL_IP:35357/v2.0 libvirt_vif_driver=nova.virt.libvirt.vif.QuantumLinuxBridgeVIFDriver linuxnet_interface_driver=nova.network.linux_net.LinuxBridgeInterfaceDriver firewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver # Compute # compute_driver=libvirt.LibvirtDriver # Cinder # volume_api_class=nova.volume.cinder.API osapi_volume_listen_port=5900 EOF
编辑 /etc/nova/nova-compute.conf:
[DEFAULT] libvirt_type=kvm compute_driver=libvirt.LibvirtDriver libvirt_vif_type=ethernet libvirt_vif_driver=nova.virt.libvirt.vif.QuantumLinuxBridgeVIFDriver
同步数据库
nova-manage db sync
重启nova相关服务
cd /etc/init.d/; for i in $( ls nova-* ); do sudo service $i restart; done
查看nova 服务
nova-manage service list
Cinder
安装
apt-get install -y cinder-api cinder-scheduler cinder-volume \ iscsitarget open-iscsi iscsitarget-dkms
配置iscsi服务
sed -i 's/false/true/g' /etc/default/iscsitarget
重启服务
service iscsitarget start service open-iscsi start
编辑 /etc/cinder/api-paste.ini
[filter:authtoken] paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory service_protocol = http service_host = 127.0.0.1 service_port = 5000 auth_host = 127.0.0.1 auth_port = 35357 auth_protocol = http #admin_tenant_name = %SERVICE_TENANT_NAME% #admin_user = %SERVICE_USER% #admin_password = %SERVICE_PASSWORD% admin_tenant_name = service admin_user = cinder admin_password = password
编辑 /etc/cinder/cinder.conf
[DEFAULT]
rootwrap_config=/etc/cinder/rootwrap.conf
sql_connection = mysql://cinder:[email protected]/cinder
api_paste_config = /etc/cinder/api-paste.ini
iscsi_helper=ietadm
volume_name_template = volume-%s
volume_group = cinder-volumes
verbose = True
auth_strategy = keystone
#osapi_volume_listen_port=5900
同步数据库
cinder-manage db sync
创建一个cinder volume的卷
如果使用文件模拟的方式,其实性能很差,基本是不可用。所以建议采用单独分区来测试
我的硬盘专门一个分区给volume使用
umount /dev/sda7 pvcreate /dev/sda7 vgcreate cinder-volumes /dev/sda7
去掉开机挂载
sed -i '/nova-volume/s/^/#/' /etc/fstab
重启服务
cd /etc/init.d/; for i in $( ls cinder-* ); do sudo service $i restart; done
查看cinder服务状态
cd /etc/init.d/; for i in $( ls cinder-* ); do sudo service $i status; done
Horizon
安装
apt-get install -y openstack-dashboard memcached
默认的ubuntu的theme一直都有问题，需要删掉。
dpkg --purge openstack-dashboard-ubuntu-theme
重启相关服务
service apache2 restart; service memcached restart
这个时候你就可以直接使用 http://10.1.199.8/horizon 访问
user:admin pass:password
如何使用Dashboard，后续补上。