openstack（kilo）安装部署文档（最小化安装，单机安装）

前言

之前在centos上安装了一个版本，是使用的一键安装，devstack来安装的，这个安装后很多细节都不太清楚，所以决定手动走一边安装过程。
我这里使用的是一台物理机进行的安装，IP 为10.1.82.161，安装的是最小集合，只安装了keystone,nova,glance和dashboard
这里我用的FEDORA21进行的安装，参考的官方文档：
http://docs.openstack.org/kilo/install-guide/install/yum/content/
听说用fuel可以快速安装，之后有时间也会看下

准备工作

yum install ntp
# systemctl enable ntpd.service
# systemctl start ntpd.service
# ntpq -c peers
# ntpq -c assoc

修改 /etc/hosts

10.1.82.161 controller

设置hostname

hostnamectl set-hostname controller

安装kilo的源

yum install http://rdo.fedorapeople.org/openstack-kilo/rdo-release-kilo.rpm

安装数据库mysql

# yum upgrade
# yum install mariadb mariadb-server MySQL-python

修改mysql的配置，（这里我不太确定是否加里一个新的cnf文件就会使用这个配置以及如何对应的,总之按照文档新建了配置文件并进行了配置）

vim /etc/my.cnf.d/mariadb_openstack.cnf
[mysqld]
bind-address = 10.0.0.11

[mysqld]
default-storage-engine = innodb
innodb_file_per_table
collation-server = utf8_general_ci
init-connect = 'SET NAMES utf8'
character-set-server = utf8

设置数据库开机启动和初始密码

# systemctl enable mariadb.service
# systemctl start mariadb.service

mysql_secure_installation

root 密码我设置了 qwer1234
安装rabbitmq并设置用户和权限

# yum install rabbitmq-server
# systemctl enable rabbitmq-server.service
# systemctl start rabbitmq-server.service

# rabbitmqctl add_user openstack RABBIT_PASS
# rabbitmqctl set_permissions openstack ".*" ".*" ".*"

keystone

安装

$ mysql -u root -p
CREATE DATABASE keystone;
GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'localhost' \
IDENTIFIED BY 'KEYSTONE_DBPASS';
GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'%' \
IDENTIFIED BY 'KEYSTONE_DBPASS';

安装软件包

yum install openstack-keystone httpd mod_wsgi python-openstackclient memcached python-memcached

# systemctl enable memcached.service
# systemctl start memcached.service

生成一个随机码

openssl rand -hex 10 

7f0ccd900a0e81f0a949
编辑/etc/keystone/keystone.conf ，注意以下几处的修改

[DEFAULT]
admin_token = 7f0ccd900a0e81f0a949

[database]
...
connection = mysql://keystone:KEYSTONE_DBPASS@controller/keystone
[memcache]
...
servers = localhost:11211
[token]
...
provider = keystone.token.providers.uuid.Provider
driver = keystone.token.persistence.backends.memcache.Token
[revoke]
...
driver = keystone.contrib.revoke.backends.sql.Revoke
[DEFAULT]
...
verbose = True

同步数据库信息

#su -s /bin/sh -c "keystone-manage db_sync" 
我的环境keystone 运行这个命令无反应，使用下一行
keystone-manage db_sync

修改 the /etc/httpd/conf/httpd.conf
ServerName controller

新建 /etc/httpd/conf.d/wsgi-keystone.conf

Listen 5000
Listen 35357


    WSGIDaemonProcess keystone-public processes=5 threads=1 user=keystone group=keystone display-name=%{GROUP}
    WSGIProcessGroup keystone-public
    WSGIScriptAlias / /var/www/cgi-bin/keystone/main
    WSGIApplicationGroup %{GLOBAL}
    WSGIPassAuthorization On
    LogLevel info
    ErrorLogFormat "%{cu}t %M"
    ErrorLog /var/log/httpd/keystone-error.log
    CustomLog /var/log/httpd/keystone-access.log combined



    WSGIDaemonProcess keystone-admin processes=5 threads=1 user=keystone group=keystone display-name=%{GROUP}
    WSGIProcessGroup keystone-admin
    WSGIScriptAlias / /var/www/cgi-bin/keystone/admin
    WSGIApplicationGroup %{GLOBAL}
    WSGIPassAuthorization On
    LogLevel info
    ErrorLogFormat "%{cu}t %M"
    ErrorLog /var/log/httpd/keystone-error.log
    CustomLog /var/log/httpd/keystone-access.log combined

设置

mkdir -p /var/www/cgi-bin/keystone
curl http://git.openstack.org/cgit/openstack/keystone/plain/httpd/keystone.py?h=stable/kilo \
| tee /var/www/cgi-bin/keystone/main /var/www/cgi-bin/keystone/admin

设置权限并设置开机启动

# chown -R keystone:keystone /var/www/cgi-bin/keystone
# chmod 755 /var/www/cgi-bin/keystone/*
# systemctl enable httpd.service
# systemctl start httpd.service

启动时候遇到错误

error:
(13)Permission denied: AH00072: make_sock: could not bind to address [::]:5000
6月 23 13:39:35 controller httpd[5137]: (13)Permission denied: AH00072: make_sock: could not bind to address 0.0.0.0:5000
6月 23 13:39:35 controller httpd[5137]: no listening sockets available, shutting down
6月 23 13:39:35 controller httpd[5137]: AH00015: Unable to open logs
6月 23 13:39:35 controller systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE
6月 23 13:39:35 controller systemd[1]: Failed to start The Apache HTTP Server.
6月 23 13:39:35 controller systemd[1]: Unit httpd.service entered failed state.
6月 23 13:39:35 controller systemd[1]: httpd.service failed.

设置setenforce 为0， 有效解决此问题
因此关闭selinux

设置keystone基础用户角色信息等

export OS_TOKEN=7f0ccd900a0e81f0a949
export OS_URL=http://controller:35357/v2.0

[root@controller ~]# openstack service create   --name keystone --description "OpenStack Identity" identity

这里又遇到了错误

ERROR: cliff.app 'super' object has no attribute 'load_commands'

尝试修复办法：
service firewalld stop 无效
去掉ServerName 无效
yum update 无效
加–debug 看信息

ERROR: openstackclient.shell Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/openstackclient/shell.py", line 176, in run
    return super(OpenStackShell, self).run(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 201, in run
    self.initialize_app(remainder)
  File "/usr/lib/python2.7/site-packages/openstackclient/shell.py", line 253, in initialize_app
    self.command_manager.add_command_group(cmd_group)
  File "/usr/lib/python2.7/site-packages/openstackclient/common/commandmanager.py", line 45, in add_command_group
    self.load_commands(group)
  File "/usr/lib/python2.7/site-packages/openstackclient/common/commandmanager.py", line 40, in load_commands
    return super(CommandManager, self).load_commands(namespace)
AttributeError: 'super' object has no attribute 'load_commands'

/usr/lib/python2.7/site-packages/openstackclient/common/commandmanager.py 此文件属于python-cliff，跟踪，发现确实无此方法，于是考虑升级它。
python-cliff-1.6.1-3.fc21.noarch 查到此包旧，升级为1.13 pip install cliff==1.13.0

还报错，chmod 777 /var/log/keystone/keystone.log
，重启HTTPD 。OK

openstack endpoint create \
  --publicurl http://controller:5000/v2.0 \
  --internalurl http://controller:5000/v2.0 \
  --adminurl http://controller:35357/v2.0 \
  --region RegionOne \
  identity



openstack project create --description "Admin Project" admin
openstack user create --password-prompt admin      
密码：admin
openstack role create admin
openstack role add --project admin --user admin admin
openstack project create --description "Service Project" service
openstack project create --description "Demo Project" demo
openstack user create --password-prompt demo
openstack role create user
openstack role add --project demo --user demo user

编辑 /usr/share/keystone/keystone-dist-paste.ini file and remove admin_token_auth from the [pipeline:public_api], [pipeline:admin_api], and [pipeline:api_v3] sections.

unset OS_TOKEN OS_URL

验证

openstack --os-auth-url http://controller:35357 \
  --os-project-name admin --os-username admin --os-auth-type password \
  token issue

vim admin-openrc.sh

export OS_PROJECT_DOMAIN_ID=default
export OS_USER_DOMAIN_ID=default
export OS_PROJECT_NAME=admin
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=ADMIN_PASS
export OS_AUTH_URL=http://controller:35357/v3

vim demo-openrc.sh

export OS_PROJECT_DOMAIN_ID=default
export OS_USER_DOMAIN_ID=default
export OS_PROJECT_NAME=demo
export OS_TENANT_NAME=demo
export OS_USERNAME=demo
export OS_PASSWORD=DEMO_PASS
export OS_AUTH_URL=http://controller:5000/v3

glance

安装

mysql -u root -p
CREATE DATABASE glance;
GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'localhost' \
  IDENTIFIED BY 'GLANCE_DBPASS';
GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'%' \
  IDENTIFIED BY 'GLANCE_DBPASS';

创建用户等信息

source admin-openrc.sh
openstack user create --password-prompt glance
openstack role add --project service --user glance admin
openstack service create --name glance \
  --description "OpenStack Image service" image
openstack endpoint create \
  --publicurl http://controller:9292 \
  --internalurl http://controller:9292 \
  --adminurl http://controller:9292 \
  --region RegionOne \
  image

安装软件

yum install openstack-glance python-glance python-glanceclient

配置

vim /etc/glance/glance-api.conf，注意以下几处的修改

[database]
...
connection = mysql://glance:GLANCE_DBPASS@controller/glance
[keystone_authtoken]
...
auth_uri = http://controller:5000
auth_url = http://controller:35357
auth_plugin = password
project_domain_id = default
user_domain_id = default
project_name = service
username = glance
password = glance

[paste_deploy]
...
flavor = keystone
[glance_store]
...
default_store = file
filesystem_store_datadir = /var/lib/glance/images/
[DEFAULT]
...
notification_driver = noop
[DEFAULT]
...
verbose = True

/etc/glance/glance-registry.conf 
[database]
...
connection = mysql://glance:GLANCE_DBPASS@controller/glance
[keystone_authtoken]
...
auth_uri = http://controller:5000
auth_url = http://controller:35357
auth_plugin = password
project_domain_id = default
user_domain_id = default
project_name = service
username = glance
password = glance

[paste_deploy]
...
flavor = keystone
[DEFAULT]
...
notification_driver = noop
[DEFAULT]
...
verbose = True

同步数据并启动服务

glance-manage db_sync
# systemctl enable openstack-glance-api.service openstack-glance-registry.service
# systemctl start openstack-glance-api.service openstack-glance-registry.service

openstack-glance-api.service 启动失败，开启DEBUG

6月 23 18:03:03 controller systemd[1]: Failed to start OpenStack Image Service (code-named Glance) API server.
Permission denied: '/var/log/glance/api.log

放开权限（其实我不太明白具体怎么设置，方便期间设置为了777，我也不明白为什么我装的时候老遇到日志文件没权限的问题）

chmod 777 /var/log/glance/api.log

修改

echo "export OS_IMAGE_API_VERSION=2" | tee -a admin-openrc.sh demo-openrc.sh

加一个映像

source admin-openrc.sh
mkdir /tmp/images
 wget -P /tmp/images http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img
glance image-create --name "cirros-0.3.4-x86_64" --file /tmp/images/cirros-0.3.4-x86_64-disk.img --disk-format qcow2 --container-format bare --visibility public --progress

遇到错误

不识别 --visibility public
添加错误 ，Error in store configuration. Adding images to store is disabled
 Error in store configuration. Adding images to store is disabled.
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data Traceback (most recent call last):
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 74, in upload
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     image.set_data(data, size)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 166, in set_data
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     self.base.set_data(data, size)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 429, in set_data
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     _send_notification(notify_error, 'image.upload', msg)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     six.reraise(self.type_, self.value, self.tb)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 378, in set_data
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     self.repo.set_data(data, size)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance/api/policy.py", line 196, in set_data
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     return self.image.set_data(*args, **kwargs)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance/quota/__init__.py", line 296, in set_data
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     self.image.set_data(data, size=size)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance/location.py", line 377, in set_data
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     context=self.context)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 364, in add_to_backend
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     return store_add_to_backend(image_id, data, size, store, context)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 339, in store_add_to_backend
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     context=context)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data   File "/usr/lib/python2.7/site-packages/glance_store/capabilities.py", line 224, in op_checker
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data     raise op_exec_map[op](**kwargs)
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data StoreAddDisabled: Configuration for store failed. Adding images to this store is disabled.
2015-06-24 09:23:40.390 2052 TRACE glance.api.v2.image_data

解决变法，关闭防火墙，SELINUX，放开/var/lib/glance/image权限777,重启服务，总之之后OK了，没具体测到到底是哪个导致的。看看结果

glance image-list

nova

安装

mysql -u root -p
CREATE DATABASE nova;
GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'localhost' \
  IDENTIFIED BY 'NOVA_DBPASS';
GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'%' \
  IDENTIFIED BY 'NOVA_DBPASS';

创建认证信息

source admin-openrc.sh
 openstack user create --password-prompt nova     
密码：nova
openstack role add --project service --user nova admin
openstack service create --name nova \
  --description "OpenStack Compute" compute
openstack endpoint create \
  --publicurl http://controller:8774/v2/%\(tenant_id\)s \
  --internalurl http://controller:8774/v2/%\(tenant_id\)s \
  --adminurl http://controller:8774/v2/%\(tenant_id\)s \
  --region RegionOne \
  compute

安装软件

yum install openstack-nova-api openstack-nova-cert openstack-nova-conductor \
  openstack-nova-console openstack-nova-novncproxy openstack-nova-scheduler \
  python-novaclient

配置

/etc/nova/nova.conf

[database]
...
connection = mysql://nova:NOVA_DBPASS@controller/nova
[DEFAULT]
...
rpc_backend = rabbit

[oslo_messaging_rabbit]
...
rabbit_host = controller
rabbit_userid = openstack
rabbit_password = RABBIT_PASS
[DEFAULT]
...
auth_strategy = keystone

[keystone_authtoken]
...
auth_uri = http://controller:5000
auth_url = http://controller:35357
auth_plugin = password
project_domain_id = default
user_domain_id = default
project_name = service
username = nova
password = nova
[DEFAULT]
...
my_ip = 10.1.82.161
[DEFAULT]
...
vncserver_listen = 127.0.0.1
vncserver_proxyclient_address = 127.0.0.1
[glance]
...
host = controller
[oslo_concurrency]
...
lock_path = /var/lib/nova/tmp
[DEFAULT]
...
verbose = True

同步数据并启动

nova-manage db sync
# systemctl enable openstack-nova-api.service openstack-nova-cert.service \
  openstack-nova-consoleauth.service openstack-nova-scheduler.service \
  openstack-nova-conductor.service openstack-nova-novncproxy.service
# systemctl start openstack-nova-api.service openstack-nova-cert.service \
  openstack-nova-consoleauth.service openstack-nova-scheduler.service \
  openstack-nova-conductor.service openstack-nova-novncproxy.service

安装nova-compute

我这里使用的是用一台机器进行安装的

yum install openstack-nova-compute sysfsutils
vim /etc/nova/nova.conf

进行配置
[DEFAULT]
…
rpc_backend = rabbit

[oslo_messaging_rabbit]
...
rabbit_host = controller
rabbit_userid = openstack
rabbit_password = RABBIT_PASS

[DEFAULT]
...
auth_strategy = keystone

[keystone_authtoken]
...
auth_uri = http://controller:5000
auth_url = http://controller:35357
auth_plugin = password
project_domain_id = default
user_domain_id = default
project_name = service
username = nova
password = NOVA_PASS

[DEFAULT]
...
my_ip = 10.1.82.166

[DEFAULT]
...
vnc_enabled = True
vncserver_listen = 0.0.0.0
vncserver_proxyclient_address = 127.0.0.1
novncproxy_base_url = http://controller:6080/vnc_auto.html

[glance]
...
host = controller

[oslo_concurrency]
...
lock_path = /var/lib/nova/tmp

[DEFAULT]
...
verbose = True

配置virt_type
执行egrep -c '(vmx|svm)' /proc/cpuinfo 若结果 0 ，应

[libvirt]
...
virt_type = qemu

否则，可用kvm
设置启动

# systemctl enable libvirtd.service openstack-nova-compute.service
# systemctl start libvirtd.service openstack-nova-compute.service

遇到错误

 libvirt version: 1.2.9.3, package: 2.fc21 (Fedora Project, 2015-06-06-15:23...t.org)
6月 24 11:52:45 controller libvirtd[13287]: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_vbox_network.so ...ssible
6月 24 11:52:45 controller libvirtd[13287]: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_vbox_storage.so ...ssible
6月 24 11:52:45 controller libvirtd[13287]: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_xen.so not accessible
6月 24 11:52:45 controller libvirtd[13287]: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_libxl.so not accessible
6月 24 11:52:45 controller libvirtd[13287]: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_uml.so not accessible
6月 24 11:52:45 controller libvirtd[13287]: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_vbox.so not accessible

感觉是KVM的包不全，所以安装

yum -y install kvm python-virtinst libvirt  bridge-utils virt-manager qemu-kvm-tools  virt-viewer  virt-v2v

之后又发现
rabbit_host 写localhost失败，换成controller，之后可运行

nova-network

这里使用nova-network配置网络，说实话，我对网络这一块搞的不是很明白，后面创建虚拟机的时候并没有创建网络和绑定。
服务端配置/etc/nova/nova.conf

[DEFAULT]
...
network_api_class = nova.network.api.API
security_group_api = nova

systemctl restart openstack-nova-api.service openstack-nova-scheduler.service \
  openstack-nova-conductor.service

computer node 计算节点配置，这里我用的是用一台机器

yum install openstack-nova-network openstack-nova-api

配置

[DEFAULT]
...
network_api_class = nova.network.api.API
security_group_api = nova
firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver
network_manager = nova.network.manager.FlatDHCPManager
network_size = 254
allow_same_net_traffic = False
multi_host = True
send_arp_for_ha = True
share_dhcp_address = True
force_dhcp_release = True
flat_network_bridge = br100
flat_interface = INTERFACE_NAME
public_interface = INTERFACE_NAME

INTERFACE_NAME改成你自己的网卡名称，启动

systemctl enable openstack-nova-network.service openstack-nova-metadata-api.service
# systemctl start openstack-nova-network.service openstack-nova-metadata-api.service

warning 同一个机器openstack-nova-metadata-api.service启动冲突，暂时未处理(此问题参考https://bugs.launchpad.net/nova/+bug/1237334)，下面创建一个网络

nova network-create demo-net --bridge br100 --multi-host T \
  --fixed-range-v4 10.1.82.161/22 --allowed-start 10.1.82.163 --allowed-end 10.1.82.165 --gateway 10.1.80.254 
nova net-list

dashboard

yum install openstack-dashboard httpd mod_wsgi memcached python-memcached             

vim /etc/openstack-dashboard/local_settings

OPENSTACK_HOST = "controller"
ALLOWED_HOSTS = '*'
CACHES = {
   'default': {
       'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
       'LOCATION': '127.0.0.1:11211',
   }
}
OPENSTACK_KEYSTONE_DEFAULT_ROLE = "user"
TIME_ZONE = "TIME_ZONE"

配置启动

setsebool -P httpd_can_network_connect on
chown -R apache:apache /usr/share/openstack-dashboard/static
systemctl enable httpd.service memcached.service
# systemctl start httpd.service memcached.service

错误：

The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-4c303042-6cb3-4fa4-93d1-1a2986940a1e)

尝试，创建网络，未解决
修复linuxnet_interface_driver=nova.network.linux_net.LinuxBridgeInterfaceDriver，未解决
后来发现auth_strategy=keystone 这一行配置在了网络段里，而没有在default段里，因此出现错误，修改之，又发现错误

 Not authorized for image 

查找，发现my_ip配置错误，改成正确的IP，重启

systemctl  restart openstack-nova-api.service openstack-nova-cert.service openstack-nova-compute.service openstack-nova-conductor.service 
openstack-nova-consoleauth.service openstack-nova-network.service openstack-nova-novncproxy.service openstack-nova-scheduler.service

系统可用，创建了WINDOWS虚拟机试了试，还可以，发现关闭实例时候会遇到一个问题，说是系统出现异常还是什么的请联系管理员，我还没具体去看是什么问题。下面会仔细研究下keystone的权限，以及调用流程等。

参考：

官网资料
nova-network工作原理：http://www.cnblogs.com/yuxc/p/3426463.html
http://lynnkong.iteye.com/blog/1699876

中文手册：http://docs.ocselected.org/openstack-manuals/kilo/