[openstack swift]SAIO
原文地址：http://swift.openstack.org/development_saio.html
openstack swift版本： 1.4.2-dev
SAIO - Swift All In One¶
安装开发用虚拟机¶
这篇文档介绍如何安装一系列虚拟机来模拟有4个结点的swift机群。* 获得 * Ubuntu 10.04 LTS (Lucid Lynx) * server 的镜像:
- Ubuntu Server ISO: http://releases.ubuntu.com/lucid/ubuntu-10.04.2-server-amd64.iso (717 MB)
- Ubuntu Live/Install: http://cdimage.ubuntu.com/releases/lucid/release/ubuntu-10.04.2-dvd-amd64.iso (4.2 GB)
- Ubuntu Mirrors: https://launchpad.net/ubuntu/+cdmirrors
- 由ubuntu镜像创建客户机(虚拟机)
其它linux版本的SAIO安装参照wiki:http://wiki.openstack.org/SAIOInstructions
安装依赖和核心代码¶
- 需要root权限运行以下:
- apt-get install python-software-properties
- add-apt-repository ppa:swift-core/ppa
- apt-get update
- apt-get install curl gcc bzr memcached python-configobjpython-coverage python-dev python-nose python-setuptools python-simplejsonpython-xattr sqlite3 xfsprogs python-webob python-eventletpython-greenlet python-pastedeploy python-netifaces
- 安装其它你所需要的工具，例如： screen, ssh, vim, etc.
- 接下来选择分区创建的方式，选其一即可 使用一个新的partition(逻辑卷)作为存储分区 或者 在已有的分区上建立存储分区.
使用一个新的partition(逻辑卷)作为存储分区¶
如果swift数据放在一个独立的分区上，那么按如下步骤操作。之前，请确认虚拟机挂载了新的硬件设备。
fdisk /dev/sdb (挂载新的逻辑卷，这里新设备为 /dev/sdb)
mkfs.xfs -i size=1024 /dev/sdb1
- 编辑 /etc/fstab 并在尾行插入
/dev/sdb1 /mnt/sdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0
mkdir /mnt/sdb1
mount /mnt/sdb1
mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
chown <your-user-name>:<your-group-name> /mnt/sdb1/*
mkdir /srv
for x in {1..4}; do ln -s /mnt/sdb1/$x /srv/$x; done
mkdir -p /etc/swift/object-server /etc/swift/container-server /etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 /var/run/swift
chown -R <your-user-name>:<your-group-name> /etc/swift /srv/[1-4]/ /var/run/swift – Make sure to include the trailing slash after /srv/[1-4]/
在 /etc/rc.local 尾行插入(注意在 exit 0 之前):
mkdir /var/run/swift chown <your-user-name>:<your-group-name> /var/run/swift然后，可以设置rsync守护进程了 设置rsync守护进程.
在已有的分区上建立存储分区¶
如果你只是想利用现有的逻辑分区，那么按照如下操作。
mkdir /srv
- dd if=/dev/zero of=/srv/swift-disk bs=1024 count=0 seek=1000000
(调整seek参数的值以分配你需要的分区大小，1000000为接近1G的空间)
mkfs.xfs -i size=1024 /srv/swift-disk
- 编辑，并在 /etc/fstab 尾行插入
/srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0
mkdir /mnt/sdb1
mount /mnt/sdb1
mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
chown <your-user-name>:<your-group-name> /mnt/sdb1/*
for x in {1..4}; do ln -s /mnt/sdb1/$x /srv/$x; done
mkdir -p /etc/swift/object-server /etc/swift/container-server /etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 /var/run/swift
chown -R <your-user-name>:<your-group-name> /etc/swift /srv/[1-4]/ /var/run/swift – Make sure to include the trailing slash after /srv/[1-4]/
编辑并在 /etc/rc.local 尾行插入(注意在 exit 0 之前):
mkdir /var/run/swift chown <your-user-name>:<your-group-name> /var/run/swift
设置rsync守护进程¶
创建配置文件 /etc/rsyncd.conf:
uid = <Your user name> gid = <Your group name> log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid address = 127.0.0.1 [account6012] max connections = 25 path = /srv/1/node/ read only = false lock file = /var/lock/account6012.lock [account6022] max connections = 25 path = /srv/2/node/ read only = false lock file = /var/lock/account6022.lock [account6032] max connections = 25 path = /srv/3/node/ read only = false lock file = /var/lock/account6032.lock [account6042] max connections = 25 path = /srv/4/node/ read only = false lock file = /var/lock/account6042.lock [container6011] max connections = 25 path = /srv/1/node/ read only = false lock file = /var/lock/container6011.lock [container6021] max connections = 25 path = /srv/2/node/ read only = false lock file = /var/lock/container6021.lock [container6031] max connections = 25 path = /srv/3/node/ read only = false lock file = /var/lock/container6031.lock [container6041] max connections = 25 path = /srv/4/node/ read only = false lock file = /var/lock/container6041.lock [object6010] max connections = 25 path = /srv/1/node/ read only = false lock file = /var/lock/object6010.lock [object6020] max connections = 25 path = /srv/2/node/ read only = false lock file = /var/lock/object6020.lock [object6030] max connections = 25 path = /srv/3/node/ read only = false lock file = /var/lock/object6030.lock [object6040] max connections = 25 path = /srv/4/node/ read only = false lock file = /var/lock/object6040.lock编辑 /etc/default/rsync ，更改相应配置属性
RSYNC_ENABLE=trueservice rsync restart
可选: 按照个人需要设置 rsyslog ，生成特有的日志文件¶
创建 /etc/rsyslog.d/10-swift.conf:
# 如果需要整合所有node的日志，请取消下行注释 #local1,local2,local3,local4,local5.* /var/log/swift/all.log # 如果需要proxy每小时的stats日志，请取消下行注释 #$template HourlyProxyLog,"/var/log/swift/hourly/%$YEAR%%$MONTH%%$DAY%%$HOUR%" #local1.*;local1.!notice ?HourlyProxyLog local1.*;local1.!notice /var/log/swift/proxy.log local1.notice /var/log/swift/proxy.error local1.* ~ local2.*;local2.!notice /var/log/swift/storage1.log local2.notice /var/log/swift/storage1.error local2.* ~ local3.*;local3.!notice /var/log/swift/storage2.log local3.notice /var/log/swift/storage2.error local3.* ~ local4.*;local4.!notice /var/log/swift/storage3.log local4.notice /var/log/swift/storage3.error local4.* ~ local5.*;local5.!notice /var/log/swift/storage4.log local5.notice /var/log/swift/storage4.error local5.* ~编辑 /etc/rsyslog.conf 并更改相应属性
$PrivDropToGroup adm
mkdir -p /var/log/swift/hourly
chown -R syslog.adm /var/log/swift
service rsyslog restart
获得swift源代码并配置测试环境¶
配置文件的示例如下，并含有注释
以下配置不需要root权限，通常使用的用户权限即可。bzr的配置不是必须的，无论你是否有launchpad的账号，都可以使用bzr branch 命令:
mkdir ~/bin
(可选) mkdir ~/.bazaar
(可选) 创建 ~/.bazaar/bazaar.conf 并在尾行插入
[DEFAULT] email = Your Name <your-email-address>(可选) 如果你需要使用lauchpad账户，获得源代码并做修改,则使用如下命令：bzr launchpad-login <launchpad_id>
创建swift代码仓库 bzr init-repo swift
bzr同步出最新的代码，例如:cd ~/swift; bzr branch lp:swift trunk
以开发的方式安装swift,例如 :cd ~/swift/trunk; sudo python setup.py develop
编辑 ~/.bashrc 并在尾行插入:
export SWIFT_TEST_CONFIG_FILE=/etc/swift/func_test.conf export PATH=${PATH}:~/bin. ~/.bashrc
配置各个结点(node)¶
配置文件的示例如下：
创建 /etc/swift/proxy-server.conf:
[DEFAULT] bind_port = 8080 user = <your-user-name> log_facility = LOG_LOCAL1 [pipeline:main] pipeline = healthcheck cache tempauth proxy-server [app:proxy-server] use = egg:swift#proxy allow_account_management = true [filter:tempauth] use = egg:swift#tempauth user_admin_admin = admin .admin .reseller_admin user_test_tester = testing .admin user_test2_tester2 = testing2 .admin user_test_tester3 = testing3 [filter:healthcheck] use = egg:swift#healthcheck [filter:cache] use = egg:swift#memcache创建 /etc/swift/swift.conf:
[swift-hash] # random unique string that can never change (DO NOT LOSE) swift_hash_path_suffix = changeme创建 /etc/swift/account-server/1.conf:
[DEFAULT] devices = /srv/1/node mount_check = false bind_port = 6012 user = <your-user-name> log_facility = LOG_LOCAL2 [pipeline:main] pipeline = account-server [app:account-server] use = egg:swift#account [account-replicator] vm_test_mode = yes [account-auditor] [account-reaper]创建 /etc/swift/account-server/2.conf:
[DEFAULT] devices = /srv/2/node mount_check = false bind_port = 6022 user = <your-user-name> log_facility = LOG_LOCAL3 [pipeline:main] pipeline = account-server [app:account-server] use = egg:swift#account [account-replicator] vm_test_mode = yes [account-auditor] [account-reaper]创建 /etc/swift/account-server/3.conf:
[DEFAULT] devices = /srv/3/node mount_check = false bind_port = 6032 user = <your-user-name> log_facility = LOG_LOCAL4 [pipeline:main] pipeline = account-server [app:account-server] use = egg:swift#account [account-replicator] vm_test_mode = yes [account-auditor] [account-reaper]创建 /etc/swift/account-server/4.conf:
[DEFAULT] devices = /srv/4/node mount_check = false bind_port = 6042 user = <your-user-name> log_facility = LOG_LOCAL5 [pipeline:main] pipeline = account-server [app:account-server] use = egg:swift#account [account-replicator] vm_test_mode = yes [account-auditor] [account-reaper]创建 /etc/swift/container-server/1.conf:
[DEFAULT] devices = /srv/1/node mount_check = false bind_port = 6011 user = <your-user-name> log_facility = LOG_LOCAL2 [pipeline:main] pipeline = container-server [app:container-server] use = egg:swift#container [container-replicator] vm_test_mode = yes [container-updater] [container-auditor]创建 /etc/swift/container-server/2.conf:
[DEFAULT] devices = /srv/2/node mount_check = false bind_port = 6021 user = <your-user-name> log_facility = LOG_LOCAL3 [pipeline:main] pipeline = container-server [app:container-server] use = egg:swift#container [container-replicator] vm_test_mode = yes [container-updater] [container-auditor]创建 /etc/swift/container-server/3.conf:
[DEFAULT] devices = /srv/3/node mount_check = false bind_port = 6031 user = <your-user-name> log_facility = LOG_LOCAL4 [pipeline:main] pipeline = container-server [app:container-server] use = egg:swift#container [container-replicator] vm_test_mode = yes [container-updater] [container-auditor]创建 /etc/swift/container-server/4.conf:
[DEFAULT] devices = /srv/4/node mount_check = false bind_port = 6041 user = <your-user-name> log_facility = LOG_LOCAL5 [pipeline:main] pipeline = container-server [app:container-server] use = egg:swift#container [container-replicator] vm_test_mode = yes [container-updater] [container-auditor]创建 /etc/swift/object-server/1.conf:
[DEFAULT] devices = /srv/1/node mount_check = false bind_port = 6010 user = <your-user-name> log_facility = LOG_LOCAL2 [pipeline:main] pipeline = object-server [app:object-server] use = egg:swift#object [object-replicator] vm_test_mode = yes [object-updater] [object-auditor]创建 /etc/swift/object-server/2.conf:
[DEFAULT] devices = /srv/2/node mount_check = false bind_port = 6020 user = <your-user-name> log_facility = LOG_LOCAL3 [pipeline:main] pipeline = object-server [app:object-server] use = egg:swift#object [object-replicator] vm_test_mode = yes [object-updater] [object-auditor]创建 /etc/swift/object-server/3.conf:
[DEFAULT] devices = /srv/3/node mount_check = false bind_port = 6030 user = <your-user-name> log_facility = LOG_LOCAL4 [pipeline:main] pipeline = object-server [app:object-server] use = egg:swift#object [object-replicator] vm_test_mode = yes [object-updater] [object-auditor]创建 /etc/swift/object-server/4.conf:
[DEFAULT] devices = /srv/4/node mount_check = false bind_port = 6040 user = <your-user-name> log_facility = LOG_LOCAL5 [pipeline:main] pipeline = object-server [app:object-server] use = egg:swift#object [object-replicator] vm_test_mode = yes [object-updater] [object-auditor]
配置脚本运行 Swift¶
创建 ~/bin/resetswift.
如果选择了“在已有的分区上建立存储分区”，那么如下脚本中，`/dev/sdb1` 需替换成 /srv/swift-disk.
如果没有选择“按照个人需要设置 rsyslog ，生成特有的日志文件”, 请注释掉 find /var/log/swift... 这一行:
#!/bin/bash swift-init all stop find /var/log/swift -type f -exec rm -f {} /; sudo umount /mnt/sdb1 sudo mkfs.xfs -f -i size=1024 /dev/sdb1 sudo mount /mnt/sdb1 sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4 sudo chown <your-user-name>:<your-group-name> /mnt/sdb1/* mkdir -p /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 sudo rm -f /var/log/debug /var/log/messages /var/log/rsyncd.log /var/log/syslog sudo service rsyslog restart sudo service memcached restart创建 ~/bin/remakerings:
#!/bin/bash cd /etc/swift rm -f *.builder *.ring.gz backups/*.builder backups/*.ring.gz swift-ring-builder object.builder create 18 3 1 swift-ring-builder object.builder add z1-127.0.0.1:6010/sdb1 1 swift-ring-builder object.builder add z2-127.0.0.1:6020/sdb2 1 swift-ring-builder object.builder add z3-127.0.0.1:6030/sdb3 1 swift-ring-builder object.builder add z4-127.0.0.1:6040/sdb4 1 swift-ring-builder object.builder rebalance swift-ring-builder container.builder create 18 3 1 swift-ring-builder container.builder add z1-127.0.0.1:6011/sdb1 1 swift-ring-builder container.builder add z2-127.0.0.1:6021/sdb2 1 swift-ring-builder container.builder add z3-127.0.0.1:6031/sdb3 1 swift-ring-builder container.builder add z4-127.0.0.1:6041/sdb4 1 swift-ring-builder container.builder rebalance swift-ring-builder account.builder create 18 3 1 swift-ring-builder account.builder add z1-127.0.0.1:6012/sdb1 1 swift-ring-builder account.builder add z2-127.0.0.1:6022/sdb2 1 swift-ring-builder account.builder add z3-127.0.0.1:6032/sdb3 1 swift-ring-builder account.builder add z4-127.0.0.1:6042/sdb4 1 swift-ring-builder account.builder rebalance创建 ~/bin/startmain:
#!/bin/bash swift-init main start创建 ~/bin/startrest:
#!/bin/bash swift-init rest startchmod +x ~/bin/*
remakerings
cd ~/swift/trunk; ./.unittests
startmain (警告 Unable to increase file descriptor limit. Running as non-root? 是正常的)
recreateaccounts
使用curl获得 X-Storage-Url 和 X-Auth-Token: curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0
检测一下是否可以 GET 到 account: curl -v -H 'X-Auth-Token: <上条命令的X-Auth-Token返回结果>' <上条命令的X-Storage-Url返回结果>
检测一下 swift 命令是否工作正常: swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat
cp ~/swift/trunk/test/functional/sample.conf /etc/swift/func_test.conf
cd ~/swift/trunk; ./.functests (注意: 功能测试会删除先前配置的任何account信息！)
cd ~/swift/trunk; ./.probetests (注意: probe 测试会重设你的swift环境，因为调用了 resetswift 。)
如果需要文档，请参照如下:
- On Ubuntu:
-
- sudo apt-get install python-sphinx 安装 Sphinx
- python setup.py build_sphinx 构建文档
- On MacOS:
-
- sudo easy_install -U sphinx 安装 Sphinx
- python setup.py build_sphinx 构建文档
调试相关¶
如果按照如上配置后出现了问题，swift仍然未按预期执行，或者测试失败、auth无法认证，之类总总，以下可能会帮助你发现和解决问题:
- 所有事情都记录在 /var/log/syslog, 因此出现问题首先应该查看这里，看看是否出现什么错误的提示(多数情况会是python的一些反馈）。
- 确保所有server 的服务都是开启的。基本上任何正常情况下 Proxy、 Account、 Container、Object server都应该是正常运行在后台的。
- 如果上述server没有开启，syslog中也没有任何错误被记录。那么最好手动启动一下服务，例如：swift-object-server /etc/swift/object-server/1.conf 将启动object server。如果在syslog中没有记录任何事情，那么在手动启动后应该会看到相应的反馈。