在一台机器上部署swift存储和swauth认证

在一台机器上部署swift存储和swauth认证

转载请说明出处：  http://blog.csdn.net/cywosp/article/details/7419082

1.  摘要说明

本文所涉及到的所有操作都是在Ubuntu Server 11.10 64位系统上通过验证。本文参考了Openstackswift的SAIO(Swift All In One)文档,具体信息如下：

Linux系统版本：Ubuntu Server11.10 64-bit oneiric

机器系统IP：   192.168.112.129

官方文档：     www.openstack.org

参考文档：      http://swift.openstack.org/development_saio.html

Swift版本：       1.4.8

Swauth版本：  1.0.4

2.  安装依赖包

sudo  apt-get --option Dpkg::Options::=--force-confold --assume-yes update

sudo  apt-get install pep8 pylint python-pip screen unzip wget psmisc git-core lsofopenssh-server vim-nox locate python-virtualenv python-unittest2 iputils-pingwget curl tcpdump euca2ools python-setuptools python-dev python-lxmlpython-pastescript python-pastedeploy python-paste sqlite3 python-pysqlite2python-sqlalchemy python-mysqldb python-webob python-greenlet python-routeslibldap2-dev libsasl2-dev python-bcrypt curl gcc python-configobjpython-coverage python-dev python-eventlet python-greenlet python-netifacespython-nose python-pastedeploy python-setuptools python-simplejson python-webobpython-xattr sqlite3 xfsprogs 

3.  创建swift用户

sudo useradd -mk /home/swift/ -s /bin/bash swift
sudo passwd swift    #为swift用户添加密码，在此我将其设为了swift
编辑/etc/sudoer文件，在文件末尾添加如下代码
swift  ALL=(ALL) NOPASSWD:ALL

4.  下载swift和swauth源码 (在swift用户下操作)

su swift      #切换到swift用户
sudo mkdir /home/swift/openstack       #创建一个目录来存放
cd /home/swift/openstack
git  clone https://github.com/openstack/swift.git #下载swift
cd  swift
git checkout 1.4.8       #使用1.4.8版本，在swift目录下你可以使用git tag命令查看有多少个版本
git clone https://github.com/gholt/swauth.git    #下载swauth
cd swauth
git checkout 1.0.4

5.  安装swift和swauth  (在swift用户下操作)

cd /home/swift/openstack/swift
sudo python setup.py install --record file.txt
#参数--record file.txt的作用是将所有文件被安装到的路径记录到#file.txt中，为了以后想卸载该软件的时候用，要想卸载该软件时，到#file.txt所在目录输入如下命令即可：sudo cat file.txt | xargs rm –rf

cd /home/swift/openstack/swauth
sudo python setup.py install --record file.txt 

6.  安装相关软件

sudo apt-get install memcached

7.  存储点的设置  (在swift用户下操作)

在这里有两种方式来设置存储点

1.  你的系统里有一个单独分区，使用此分区来做存储点，在这里假设系统中有/dev/sdb1（注：这里根据你自己系统的情况而定）这个分区未被使用，我们用它来做存储点。

sudo mkfs.xfs -i size=1024 /dev/sdb1   #以xfs方式格式化分区
sudo chmod a+w /etc/fstab
sudo echo “/dev/sdb1 /mnt/sdb1 xfs noatime,nodiratime nobarrier,logbufs=8 0 0”>> /etc/fstab        #系统启动时自动挂载
 
sudo mkdir /mnt/sdb1                #在/mnt下创建sdb1挂载目录
sudo mount /mnt/sdb1                #挂载
 
sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
sudo chown swift:swift /mnt/sdb1/*
sudo mkdir /srv
 
for x in {1..4}; do sudo ln -s /mnt/sdb1/$x /srv/$x; done #创建连接
 
sudo mkdir -p /etc/swift/object-server /etc/swift/container-server/etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3/srv/4/node/sdb4 /var/run/swift        #创建相关目录
 
sudo chown -R swift:swift /etc/swift /srv/[1-4]/ /var/run/swift
 
在/etc/rc.local的exit 0之前加入下列三行
mkdir /var/run/swift
chown swift:swift /var/run/swift
chmod a+w /var/run/swift

2.  如果系统中没有单独未使用的分区则可以创建一个回环设备来做存储点（在此我使用的就是此方法）

sudo mkdir /srv
sudo dd if=/dev/zero of=/srv/swift-disk bs=1024 count=0 seek=1000000   #这个命令是在/srv/下创建一个名为swift-disk的存
#储区，你可以改变seek的大小来改变swift-disk的大小
 
sudo mkfs.xfs -i size=1024 /srv/swift-disk
sudo chmod a+w /etc/fstab
sudo echo “/srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=80 0” >> /etc/fstab #系统启动
#时自动挂载
 
sudo mkdir /mnt/sdb1                #在/mnt下创建sdb1挂载目录
sudo mount /mnt/sdb1                #挂载
 
sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
sudo chown swift:swift /mnt/sdb1/*
 
for x in {1..4}; do sudo ln -s /mnt/sdb1/$x /srv/$x; done #创建连接
 
sudo mkdir -p /etc/swift/object-server /etc/swift/container-server/etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3/srv/4/node/sdb4 /var/run/swift        #创建相关目录
 
sudo chown -R swift:swift /etc/swift /srv/[1-4]/ /var/run/swift
 
在/etc/rc.local的exit 0之前加入下列三行
mkdir /var/run/swift
chown swift:swift /var/run/swift
chmod a+w /var/run/swift
 

8.  设置rsync

sudo touch /etc/rsyncd.conf    #在/etc/下创建rsyncd.conf
sudo chmod a+w /etc/rsyncd.conf
 
将以下配置信息写入/etc/rsyncd.conf中
uid =swift
gid =swift
logfile = /var/log/rsyncd.log
pidfile = /var/run/rsyncd.pid
address= 127.0.0.1
 
[account6012]
maxconnections = 25
path =/srv/1/node/
readonly = false
lockfile = /var/lock/account6012.lock
 
[account6022]
maxconnections = 25
path =/srv/2/node/
readonly = false
lockfile = /var/lock/account6022.lock
 
[account6032]
maxconnections = 25
path =/srv/3/node/
readonly = false
lockfile = /var/lock/account6032.lock
 
[account6042]
maxconnections = 25
path =/srv/4/node/
readonly = false
lockfile = /var/lock/account6042.lock
 
[container6011]
maxconnections = 25
path =/srv/1/node/
readonly = false
lockfile = /var/lock/container6011.lock
 
[container6021]
maxconnections = 25
path =/srv/2/node/
readonly = false
lockfile = /var/lock/container6021.lock
 
[container6031]
maxconnections = 25
path =/srv/3/node/
readonly = false
lockfile = /var/lock/container6031.lock
 
[container6041]
maxconnections = 25
path =/srv/4/node/
readonly = false
lockfile = /var/lock/container6041.lock
 
[object6010]
maxconnections = 25
path =/srv/1/node/
readonly = false
lockfile = /var/lock/object6010.lock
 
[object6020]
maxconnections = 25
path =/srv/2/node/
readonly = false
lockfile = /var/lock/object6020.lock
 
[object6030]
maxconnections = 25
path =/srv/3/node/
readonly = false
lockfile = /var/lock/object6030.lock
 
[object6040]
maxconnections = 25
path =/srv/4/node/
readonly = false
lockfile = /var/lock/object6040.lock
 
编辑/etc/default/rsync:
将RSYNC_ENABLE设置为true
 
重启服务：
sudo servicersync restart

9.  设置日志文件（可选）

sudo touch /etc/rsyslog.d/10-swift.conf
sudo chmod a+w /etc/rsyslog.d/10-swift.conf

在该文件中添加如下配置信息
#Uncomment the following to have a log containing all logs together
#local1,local2,local3,local4,local5.*   /var/log/swift/all.log
#Uncomment the following to have hourly proxy logs for stats processing
#$templateHourlyProxyLog,"/var/log/swift/hourly/%$YEAR%%$MONTH%%$DAY%%$HOUR%"
#local1.*;local1.!notice?HourlyProxyLog
 
local1.*;local1.!notice/var/log/swift/proxy.log
local1.notice           /var/log/swift/proxy.error
local1.*                ~
 
local2.*;local2.!notice/var/log/swift/storage1.log
local2.notice           /var/log/swift/storage1.error
local2.*                ~
 
local3.*;local3.!notice/var/log/swift/storage2.log
local3.notice           /var/log/swift/storage2.error
local3.*                ~
 
local4.*;local4.!notice/var/log/swift/storage3.log
local4.notice           /var/log/swift/storage3.error
local4.*                ~
 
local5.*;local5.!notice/var/log/swift/storage4.log
local5.notice           /var/log/swift/storage4.error
local5.*   

修改/etc/rsyslog.conf：
将$PrivDropToGroup  syslog改为$PrivDropToGroup adm
执行如下命令
sudo mkdir -p /var/log/swift/hourly
sudo chown -R syslog.adm /var/log/swift
sudo service rsyslog restart

10. 配置各个节点（以下文件不存在的话需要自己创建）

sudo chmod a+w -R /etc/swift/* 

1.  /etc/swift/proxy-server.conf
[DEFAULT]
bind_port= 8080
user =swift
log_facility= LOG_LOCAL1
 
[pipeline:main]
pipeline= healthcheck cache swauth proxy-server
 
[app:proxy-server]
use =egg:swift#proxy
allow_account_management= true
account_autocreate= true
 
[filter:swauth]
use =egg:swauth#swauth
#Highly recommended to change this.
super_admin_key= swauthkey
default_swift_cluster= local#http://127.0.0.1:8080/auth/v1.0
 
[filter:healthcheck]
use =egg:swift#healthcheck
 
[filter:cache]
use =egg:swift#memcache
 
2.  /etc/swift/swift.conf
[swift-hash]
#random unique string that can never change (DO NOT LOSE)
swift_hash_path_suffix= cynric   #cynric是可以随意更改的字符串
 
3.  /etc/swift/account-server/1.conf
[DEFAULT]
devices= /srv/1/node
mount_check= false
bind_port= 6012
user =swift
log_facility= LOG_LOCAL2
 
[pipeline:main]
pipeline= account-server
 
[app:account-server]
use =egg:swift#account
 
[account-replicator]
vm_test_mode= yes
 
[account-auditor]
 
[account-reaper]
4.  /etc/swift/account-server/2.conf
[DEFAULT]
devices= /srv/2/node
mount_check= false
bind_port= 6022
user =swift
log_facility= LOG_LOCAL3
 
[pipeline:main]
pipeline= account-server
 
[app:account-server]
use =egg:swift#account
 
[account-replicator]
vm_test_mode= yes
 
[account-auditor]
 
[account-reaper]
 
5.  /etc/swift/account-server/3.conf
[DEFAULT]
devices= /srv/3/node
mount_check= false
bind_port= 6032
user =swift
log_facility= LOG_LOCAL4
 
[pipeline:main]
pipeline= account-server
 
[app:account-server]
use =egg:swift#account
 
[account-replicator]
vm_test_mode= yes
 
[account-auditor]
 
[account-reaper]
 
6.  /etc/swift/account-server/4.conf
[DEFAULT]
devices= /srv/4/node
mount_check= false
bind_port= 6042
user =swift
log_facility= LOG_LOCAL5
 
[pipeline:main]
pipeline= account-server
 
[app:account-server]
use =egg:swift#account
 
[account-replicator]
vm_test_mode= yes
 
[account-auditor]
 
[account-reaper]
 
7.  /etc/swift/container-server/1.conf
[DEFAULT]
devices= /srv/1/node
mount_check= false
bind_port= 6011
user =swift
log_facility= LOG_LOCAL2
 
[pipeline:main]
pipeline= container-server
 
[app:container-server]
use =egg:swift#container
 
[container-replicator]
vm_test_mode= yes
 
[container-updater]
 
[container-auditor]
 
[container-sync]
 
8.  /etc/swift/container-server/2.conf
[DEFAULT]
devices= /srv/2/node
mount_check= false
bind_port= 6021
user =swift
log_facility= LOG_LOCAL3
 
[pipeline:main]
pipeline= container-server
 
[app:container-server]
use =egg:swift#container
 
[container-replicator]
vm_test_mode= yes
 
[container-updater]
 
[container-auditor]
 
[container-sync]
 
9.  /etc/swift/container-server/3.conf
[DEFAULT]
devices= /srv/3/node
mount_check= false
bind_port= 6031
user =swift
log_facility= LOG_LOCAL4
 
[pipeline:main]
pipeline= container-server
 
[app:container-server]
use =egg:swift#container
 
[container-replicator]
vm_test_mode= yes
 
[container-updater]
 
[container-auditor]
 
[container-sync]
 
10./etc/swift/container-server/4.conf
[DEFAULT]
devices= /srv/4/node
mount_check= false
bind_port= 6041
user =swift
log_facility= LOG_LOCAL5
 
[pipeline:main]
pipeline= container-server
 
[app:container-server]
use =egg:swift#container
 
[container-replicator]
vm_test_mode= yes
 
[container-updater]
 
[container-auditor]
 
[container-sync]
 
11./etc/swift/object-server/1.conf
[DEFAULT]
devices= /srv/1/node
mount_check= false
bind_port= 6010
user =swift
log_facility= LOG_LOCAL2
 
[pipeline:main]
pipeline= object-server
 
[app:object-server]
use =egg:swift#object
 
[object-replicator]
vm_test_mode= yes
 
[object-updater]
 
[object-auditor]
 
12./etc/swift/object-server/2.conf
[DEFAULT]
devices= /srv/2/node
mount_check= false
bind_port= 6020
user =swift
log_facility= LOG_LOCAL3
 
[pipeline:main]
pipeline= object-server
 
[app:object-server]
use =egg:swift#object
 
[object-replicator]
vm_test_mode= yes
 
[object-updater]
 
[object-auditor]
 
13./etc/swift/object-server/3.conf
[DEFAULT]
devices= /srv/3/node
mount_check= false
bind_port= 6030
user =swift
log_facility= LOG_LOCAL4
 
[pipeline:main]
pipeline= object-server
 
[app:object-server]
use =egg:swift#object
 
[object-replicator]
vm_test_mode= yes
 
[object-updater]
 
[object-auditor]
 
14./etc/swift/object-server/4.conf
[DEFAULT]
devices= /srv/4/node
mount_check= false
bind_port= 6040
user =swift
log_facility= LOG_LOCAL5
 
[pipeline:main]
pipeline= object-server
 
[app:object-server]
use =egg:swift#object
 
[object-replicator]
vm_test_mode= yes
 
[object-updater]
 
[object-auditor]

11. 创建swift运行脚本

sudo mkdir /home/swift/bin
sudo chown swift:swift -R /home/swift/
在/etc/profile文件末尾添加如下代码
export PATH=$PATH:/home/swift/bin

1.  创建/home/swift/bin/remakerings
#!/bin/bash
cd  /etc/swift
rm -f*.builder *.ring.gz backups/*.builder backups/*.ring.gz
 
swift-ring-builder object.builder create 18 3 1
swift-ring-builder container.builder create 18 3 1
swift-ring-builder account.builder create 18 3 1
 
swift-ring-builder object.builder add z1-127.0.0.1:6010/sdb1 1
swift-ring-builder object.builder add z2-127.0.0.1:6020/sdb2 1
swift-ring-builder object.builder add z3-127.0.0.1:6030/sdb3 1
swift-ring-builder object.builder add z4-127.0.0.1:6040/sdb4 1
 
swift-ring-builder container.builder add z1-127.0.0.1:6011/sdb1 1
swift-ring-builder container.builder add z2-127.0.0.1:6021/sdb2 1
swift-ring-builder container.builder add z3-127.0.0.1:6031/sdb3 1
swift-ring-builder container.builder add z4-127.0.0.1:6041/sdb4 1
 
swift-ring-builder account.builder add z1-127.0.0.1:6012/sdb1 1
swift-ring-builder account.builder add z2-127.0.0.1:6022/sdb2 1
swift-ring-builder account.builder add z3-127.0.0.1:6032/sdb3 1
swift-ring-builder account.builder add z4-127.0.0.1:6042/sdb4 1
 
swift-ring-builder object.builder rebalance
swift-ring-builder container.builder rebalance
swift-ring-builder account.builder rebalance
 
2.  创建/home/swift/bin/sfstart
#!/bin/bash
swift-init all start
sudo chown swift:swift -R /var/log/swift/
sudo chmod a+w /var/run/swift/*
 
3.  创建/home/swift/bin/sfstop
#!/bin/bash
Swift-init all stop
 
4.  创建/home/swift/bin/resetswift
#!/bin/bash
 
swift-init all stop
find /var/log/swift -type f -exec rm -f {} \;
sudo umount /mnt/sdb1
sudo mkfs.xfs -f -i size=1024 /srv/swift-disk
sudo chmod a+w /srv/swift-disk
sudo mount /mnt/sdb1
sudo rm -rf /srv/1 /srv/2 /srv/3 /srv/4
sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
sudo chown swift:swift /mnt/sdb1/*
for x in {1..4}; do sudo ln -s /mnt/sdb1/$x/srv/$x; done
sudo mkdir -p /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3/srv/4/node/sdb4
sudo chown swift:swift -R /srv/*
sudo rm -f /var/log/debug /var/log/messages /var/log/rsyncd.log/var/log/syslog
sudo chown -R swift:swift /mnt/sdb1/*
sudo service rsyslog restart
sudo service memcached restart
 
在/home/swift/bin/resetswift中，如果你使用的是单独分区来做存储点的，你需要把sudo mkfs.xfs -f -i size=1024 /srv/swift-disk和sudo chmod a+w /srv/swift-disk中的/srv/swift-disk改为该分区的设备名，例如本文第七步第一小点所说的那样，在这里我这里应该改为/dev/sdb1，具体情况根据你的系统而定
 
5.  使/home/swift/bin下的文件具有可执行权限
sudo  chmod a+x /home/swift/bin/*

12. 生成ring文件

source  /etc/profile  #下次启动系统是这条命令就不用执行了

remakerings     #执行该命令在/etc/swift中生成相应的文件

 

13. 启动swift服务

source  /etc/profile  #下次启动系统是这条命令就不用执行了

sfstart       #执行这个需要一段时间

#当启动的时候可能会报WARNING: Unable to increase filedescriptor limit.  Running as non-root?  这是正常情况

 

14. 测试swift服务

1.  初始化swauth

swauth-prep  -K swauthkey     #执行成功的话则不会有任何信息显示

#在这一步经常会出现Auth subsystem prep failed: 500 ServerError这种错误，发生错误的原因可能是你的/srv下的目录和/mnt/sdb1下的目录的拥有者不是属于swift用户。

2.  添加用户

swauth-add-user  -A  http://127.0.0.1:8080/auth/ -K swauthkey -a test tester testing   #在tenant test中添加tester用户并设置密码为testing

 

3.  检测swift运行状态

swift  -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat  -v       #运行成功则会返回类似如下的信息

 

StorageURL:http://127.0.0.1:8080/v1/AUTH_3f7fb037-57b6-4771-831a-8fd811bcc3c7

AuthToken: AUTH_tkf5b6840d9ce64d7298a800505a2ed857

Account:AUTH_3f7fb037-57b6-4771-831a-8fd811bcc3c7

Containers:0

Objects:0

Bytes:0

Accept-Ranges:bytes

X-Trans-Id:txdb39ed113c1c49299ebb364a4246a3a9 

注:红色字体部分是根据具体情况而定的，并不是每个人系统返回都是一样的，假如出现了错误你可以在/var/log/swift/目录下查询到相应的信息

15. 上传和下载文件

1.  获取X-Storage-Token和X-Storage-Url

curl  -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing'  http://127.0.0.1:8080/auth/v1.0        #执行完该条语句会返回相应的信息，例如（返回信息的一小部分）：

HTTP/1.1200 OK

X-Storage-Url:http://127.0.0.1:8080/v1/AUTH_3f7fb037-57b6-4771-831a-8fd811bcc3c7

X-Storage-Token:AUTH_tkf5b6840d9ce64d7298a800505a2ed857

X-Auth-Token:AUTH_tkf5b6840d9ce64d7298a800505a2ed857

swift  -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list       #查看test用户的container列表

3.  上传和列出文件

swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing upload cynric/etc/swift/proxy-server.conf #将/etc/swift/proxy-server.conf文件上传到cynric容器中

 

swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list cynric   #列出cynric容器中的文件

4.  下载文件

swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing download cynric   #将cynric容器下的所有文件下载到本地

注意：如果上传文件的时候，有嵌套目录，则上传到swift上后，也是以嵌套目录的形式存在，下载后，会在当前目录创建一样的嵌套目录。