openstack swift SAIO

阅读更多

openstack swift SAIO的安装  

2013-02-03 08:32:19|  分类： 云计算 |  标签：openstack  云计算  swift   |举报 |字号大中小 订阅

 

 

 

 

 

参考官方文档，及一些国内同行的博客
我的装系统是分了个lvm,/dev/sda7

注意：2.1~2.5必须以root身份进行操作
2.1、安装依赖包

apt-get install python-software-properties

add-apt-repository ppa:swift-core/ppa

apt-get update

apt-get install curl gcc bzr memcached python-configobj python-coverage python-dev python-nose python-setuptools python-simplejson python-xattr sqlite3 xfsprogs pythonwebob python-eventlet python-greenlet python-pastedeploy python-netifaces


2.2、创建swift用户

创建swift用户

sudo useradd -mk /home/swift -s /bin/bash swift

sudo password swift

 

编辑/etc/sudoers，添加以下一行

# User alias specification

swift   ALL=(ALL) NOPASSWD:ALL

Using a partition for storage

If you are going to use a separate partition for Swift data, be sure to add another device when creating the VM, and follow these instructions.

        mkfs.xfs -i size=1024 /dev/sda7

        Edit /etc/fstab and add

            /dev/sda7 /mnt/sda7 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0

        mkdir /mnt/sda7

        mount /mnt/sda7

        mkdir /mnt/sda7/1 /mnt/sda7/2 /mnt/sda7/3 /mnt/sda7/4

        chown swift:swift /mnt/sda7/*

        mkdir /srv

        for x in {1..4}; do ln -s /mnt/sda7/$x /srv/$x; done

        mkdir -p /etc/swift/object-server /etc/swift/container-server /etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 /var/run/swift

        chown -R swift:swift /etc/swift /srv/[1-4]/ /var/run/swift
    – Make sure to include the trailing slash after /srv/[1-4]/

        Add to /etc/rc.local (before the exit 0):

        mkdir -p /var/cache/swift /var/cache/swift2 /var/cache/swift3 /var/cache/swift4
        chown swift:swift /var/cache/swift*
        mkdir -p /var/run/swift
        chown swift:swift /var/run/swift


设置rsync

创建文件：/etc/rsyncd.conf:

uid = swift

gid = swift

log file = /var/log/rsyncd.log

pid file = /var/run/rsyncd.pid

address = 127.0.0.1

[account6012]

max connections = 25

path = /srv/1/node/

read only = false

lock file = /var/lock/account6012.lock

[account6022]

max connections = 25

path = /srv/2/node/

read only = false

lock file = /var/lock/account6022.lock

[account6032]

max connections = 25

path = /srv/3/node/

read only = false

lock file = /var/lock/account6032.lock

[account6042]

max connections = 25

path = /srv/4/node/

read only = false

lock file = /var/lock/account6042.lock

[container6011]

max connections = 25

path = /srv/1/node/

read only = false

lock file = /var/lock/container6011.lock

[container6021]

max connections = 25

path = /srv/2/node/

read only = false

lock file = /var/lock/container6021.lock

[container6031]

max connections = 25

path = /srv/3/node/

read only = false

lock file = /var/lock/container6031.lock

[container6041]

max connections = 25

path = /srv/4/node/

read only = false

lock file = /var/lock/container6041.lock

[object6010]

max connections = 25

path = /srv/1/node/

read only = false

lock file = /var/lock/object6010.lock

[object6020]

max connections = 25

path = /srv/2/node/

read only = false

lock file = /var/lock/object6020.lock

[object6030]

max connections = 25

path = /srv/3/node/

read only = false

lock file = /var/lock/object6030.lock

[object6040]

max connections = 25

path = /srv/4/node/

read only = false

lock file = /var/lock/object6040.lock

编辑/etc/default/rsync:

RSYNC_ENABLE设置为true

重启服务：

service rsync restart

2.5、日志设置（可选）

编辑/etc/rsyslog.d/10-swift.conf:

# Uncomment the following to have a log containing all logs together

#local1,local2,local3,local4,local5.* /var/log/swift/all.log

# Uncomment the following to have hourly proxy logs for stats processing

#$template HourlyProxyLog,"/var/log/swift/hourly/%$YEAR%%$MONTH%%$DAY%%

$HOUR%"

#local1.*;local1.!notice ?HourlyProxyLog

local1.*;local1.!notice /var/log/swift/proxy.log

local1.notice /var/log/swift/proxy.error

local1.* ~

local2.*;local2.!notice /var/log/swift/storage1.log

local2.notice /var/log/swift/storage1.error

local2.* ~

local3.*;local3.!notice /var/log/swift/storage2.log

local3.notice /var/log/swift/storage2.error

local3.* ~

local4.*;local4.!notice /var/log/swift/storage3.log

local4.notice /var/log/swift/storage3.error

local4.* ~

local5.*;local5.!notice /var/log/swift/storage4.log

local5.notice /var/log/swift/storage4.error

local5.* ~

 

修改/etc/rsyslog.conf：

$PrivDropToGroup adm

 

mkdir -p /var/log/swift/hourly

chown -R syslog.adm /var/log/swift

service rsyslog restart

 

注意：2.6~2.9必须以swift身份进行操作

2.6、获取代码和设置测试环境

su - swift

mkdir ~/bin

        Check out the swift repo with
git clone https://github.com/openstack/swift.git

        Build a development installation of swift, for example:
 cd ~/swift; sudo python setup.py develop

        Check out the python-swiftclient repo with
 cd ~
 git clone https://github.com/openstack/python-swiftclient.git

        Build a development installation of python-swiftclient, for example:
cd ~/python-swiftclient; sudo python setup.py develop

        Edit ~/.bashrc and add to the end:

        export SWIFT_TEST_CONFIG_FILE=/etc/swift/test.conf
        export PATH=${PATH}:~/bin

        . ~/.bashrc

编辑~/.bashrc，在最后加入：

export SWIFT_TEST_CONFIG_FILE=/etc/swift/func_test.conf

export PATH=${PATH}:~/bin

 
chmod a+x .bashrc
. ~/.bashrc

2.7、配置各个节点(以下这些文件不存在的话，就需要自己创建)

1）/etc/swift/proxy-server.conf

[DEFAULT]

bind_port = 8080

user = swift

log_facility = LOG_LOCAL1

[pipeline:main]

pipeline = healthcheck cache swauth proxy-server

[app:proxy-server]

use = egg:swift#proxy

allow_account_management = true

[filter:swauth]

use = egg:swift#swauth

# Highly recommended to change this.

super_admin_key = swauthkey

[filter:healthcheck]

use = egg:swift#healthcheck

[filter:cache]

use = egg:swift#memcache

 

2）/etc/swift/swift.conf

[swift-hash]

# random unique string that can never change (DO NOT LOSE)

swift_hash_path_suffix = 91d23be389715dab   //可以是任意的单字符串

 

3）/etc/swift/account-server/1.conf

[DEFAULT]

devices = /srv/1/node

mount_check = false

bind_port = 6012

user = swift

log_facility = LOG_LOCAL2

[pipeline:main]

pipeline = account-server

[app:account-server]

use = egg:swift#account

[account-replicator]

vm_test_mode = yes

[account-auditor]

[account-reaper]

 

4）/etc/swift/account-server/2.conf

[DEFAULT]

devices = /srv/2/node

mount_check = false

bind_port = 6022

user = swift

log_facility = LOG_LOCAL3

[pipeline:main]

pipeline = account-server

[app:account-server]

use = egg:swift#account

[account-replicator]

vm_test_mode = yes

[account-auditor]

[account-reaper]

 

5）/etc/swift/account-server/3.conf

[DEFAULT]

devices = /srv/3/node

mount_check = false

bind_port = 6032

user = swift

log_facility = LOG_LOCAL4

[pipeline:main]

pipeline = account-server

[app:account-server]

use = egg:swift#account

[account-replicator]

vm_test_mode = yes

[account-auditor]

[account-reaper]

6）/etc/swift/account-server/4.conf

[DEFAULT]

devices = /srv/4/node

mount_check = false

bind_port = 6042

user = swift

log_facility = LOG_LOCAL5

[pipeline:main]

pipeline = account-server

[app:account-server]

use = egg:swift#account

[account-replicator]

vm_test_mode = yes

[account-auditor]

[account-reaper]

 7）/etc/swift/container-server/1.conf

[DEFAULT]

devices = /srv/1/node

mount_check = false

bind_port = 6011

user = swift

log_facility = LOG_LOCAL2

[pipeline:main]

pipeline = container-server

[app:container-server]

use = egg:swift#container

[container-replicator]

vm_test_mode = yes

[container-updater]

[container-auditor]

[container-sync]

8）/etc/swift/container-server/2.conf

[DEFAULT]

devices = /srv/2/node

mount_check = false

bind_port = 6021

user = swift

log_facility = LOG_LOCAL3

[pipeline:main]

pipeline = container-server

[app:container-server]

use = egg:swift#container

[container-replicator]

vm_test_mode = yes

[container-updater]

[container-auditor]

[container-sync]

 9）/etc/swift/container-server/3.conf

[DEFAULT]

devices = /srv/3/node

mount_check = false

bind_port = 6031

user = swift

log_facility = LOG_LOCAL4

[pipeline:main]

pipeline = container-server

[app:container-server]

use = egg:swift#container

[container-replicator]

vm_test_mode = yes

[container-updater]

[container-auditor]

[container-sync]

10）/etc/swift/container-server/4.conf

[DEFAULT]

devices = /srv/4/node

mount_check = false

bind_port = 6041

user = swift

log_facility = LOG_LOCAL5

[pipeline:main]

pipeline = container-server

[app:container-server]

use = egg:swift#container

[container-replicator]

vm_test_mode = yes

[container-updater]

[container-auditor]

[container-sync]

11）/etc/swift/object-server/1.conf

[DEFAULT]

devices = /srv/1/node

mount_check = false

bind_port = 6010

user = swift

log_facility = LOG_LOCAL2

[pipeline:main]

pipeline = object-server

[app:object-server]

use = egg:swift#object

[object-replicator]

vm_test_mode = yes

[object-updater]

[object-auditor]

12）/etc/swift/object-server/2.conf

[DEFAULT]

devices = /srv/2/node

mount_check = false

bind_port = 6020

user = swift

log_facility = LOG_LOCAL3

[pipeline:main]

pipeline = object-server

[app:object-server]

use = egg:swift#object

[object-replicator]

vm_test_mode = yes

[object-updater]

[object-auditor]

 13）/etc/swift/object-server/3.conf

[DEFAULT]

devices = /srv/3/node

mount_check = false

bind_port = 6030

user = swift

log_facility = LOG_LOCAL4

[pipeline:main]

pipeline = object-server

[app:object-server]

use = egg:swift#object

[object-replicator]

vm_test_mode = yes

[object-updater]

[object-auditor]

 

14）/etc/swift/object-server/4.conf

[DEFAULT]

devices = /srv/4/node

mount_check = false

bind_port = 6040

user = swift

log_facility = LOG_LOCAL5

[pipeline:main]

pipeline = object-server

[app:object-server]

use = egg:swift#object

[object-replicator]

vm_test_mode = yes

[object-updater]

[object-auditor]

2.8、创建运行swift运行的脚本

1) ~/bin/resetswift

#!/bin/bash

swift-init all stop

sudo umount /mnt/sdb1

sudo mkfs.xfs -f -i size=1024 /srv/swift-disk

sudo mount /mnt/sdb1

sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4

sudo chown swift:swift /mnt/sdb1/*

mkdir -p /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4

sudo rm -f /var/log/debug /var/log/messages /var/log/rsyncd.log /var/log/syslog

sudo service rsyslog restart

sudo service memcached restart

 

2) ~/bin/remakerings

#!/bin/bash

cd /etc/swift

rm -f *.builder *.ring.gz backups/*.builder backups/*.ring.gz

swift-ring-builder object.builder create 18 3 1

swift-ring-builder object.builder add z1-127.0.0.1:6010/sdb1 1

swift-ring-builder object.builder add z2-127.0.0.1:6020/sdb2 1

swift-ring-builder object.builder add z3-127.0.0.1:6030/sdb3 1

swift-ring-builder object.builder add z4-127.0.0.1:6040/sdb4 1

swift-ring-builder object.builder rebalance

swift-ring-builder container.builder create 18 3 1

swift-ring-builder container.builder add z1-127.0.0.1:6011/sdb1 1

swift-ring-builder container.builder add z2-127.0.0.1:6021/sdb2 1

swift-ring-builder container.builder add z3-127.0.0.1:6031/sdb3 1

swift-ring-builder container.builder add z4-127.0.0.1:6041/sdb4 1

swift-ring-builder container.builder rebalance

swift-ring-builder account.builder create 18 3 1

swift-ring-builder account.builder add z1-127.0.0.1:6012/sdb1 1

swift-ring-builder account.builder add z2-127.0.0.1:6022/sdb2 1

swift-ring-builder account.builder add z3-127.0.0.1:6032/sdb3 1

swift-ring-builder account.builder add z4-127.0.0.1:6042/sdb4 1

swift-ring-builder account.builder rebalance

 3) ~/bin/startmain

#!/bin/bash

swift-init main start

 

4) ~/bin/startrest
        #!/bin/bash

swift-init rest start

2.9、脚本测试

1）添加~/bin/*的执行权限

# chmod +x ~/bin/*

2）重新产生rings

# remakerings

输出以下信息：

Device z1-127.0.0.1:6010/sdb1_"" with 1.0 weight got id 0

Device z2-127.0.0.1:6020/sdb2_"" with 1.0 weight got id 1

Device z3-127.0.0.1:6030/sdb3_"" with 1.0 weight got id 2

Device z4-127.0.0.1:6040/sdb4_"" with 1.0 weight got id 3

Reassigned 262144 (100.00%) partitions. Balance is now 0.00.

……

3）单元测试

# cd ~/swift; ./.unittests

输出以下信息说明单元测试通过：

……

TOTAL                                   7622   5983    78%  

----------------------------------------------------------------------

Ran 632 tests in 40.013s

 OK (SKIP=8)

 

4）启动startmain
cd ~/bin
#startmain

#####################################################################
       运行startmain可能会报错，此网页可以解决 ： https://answers.launchpad.net/swift/+question/162957
#####################################################################

 5）获取一个 X-Storage-Url 和 X-Auth-Token:

# curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0

 6）检查账户：

# curl -v -H 'X-Auth-Token: ' 

 7）检查swift工作:

# swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat

正确情况下，应该输出以下信息：

 Account: AUTH_test

 Containers: 0

 Objects: 0

 Bytes: 0

 Accept-Ranges: bytes

 8）# cp ~/swift/test/functional/sample.conf /etc/swift/func_test.conf

 9）功能测试

# cd ~/swift; ./.functests

正确情况下，输出类似于3）单元测试的结果。

 10）probe测试

# cd ~/swift; ./.probetests

该测试脚本会在每次测试前，调用resetswift脚本


三、上传/下载文件测试


3.1、获得test用户的Storage_Auth_Token和URL
       # curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0

得到以下回复信息：

* About to connect() to 127.0.0.1 port 8080 (#0)

*   Trying 127.0.0.1... connected

* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)

> GET /auth/v1.0 HTTP/1.1

> User-Agent: curl/7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15

> Host: 127.0.0.1:8080

> Accept: */*

> X-Storage-User: test:tester

> X-Storage-Pass: testing

< HTTP/1.1 200 OK

< X-Storage-Url: http://127.0.0.1:8080/v1/AUTH_test

< X-Storage-Token: AUTH_tk6474e5ee0cb04832b9d2a168e1a164d8

< X-Auth-Token: AUTH_tk6474e5ee0cb04832b9d2a168e1a164d8

< Content-Length: 0

< Date: Mon, 04 Jul 2011 01:36:57 GMT

* Connection #0 to host 127.0.0.1 left intact

Closing connection #0

3.2、创建container

创建一个名称为container_test的container（目录）

 # curl -X PUT -D -  -H "X-Auth_Token:AUTH_tk6474e5ee0cb04832b9d2a168e1a164d8" http://127.0.0.1:8080/v1/AUTH_test/container_test

//正确情况下，应该得到以下信息：

HTTP/1.1 201 Created

Content-Length: 18

Content-Type: text/html; charset=UTF-8

Date: Mon, 04 Jul 2011 01:39:38 GMT

 

查看test用户的container列表，发现只有一个目录：container_test：

# swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list

container_test

3.3、上传Object（文件）

# swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing upload container_test ./cvs/CVSROOT/config            //上传config文件到ljl1目录中

cvs/CVSROOT/config

# swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list container_test   //查看container_tese目录内的信息

bin/startmain    

cvs/CVSROOT/config

3.4、下载Object（文件）

swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing download  container_test

//将ljl1目录内的所有内容下载到目前所在目录

 
        注意：如果上传文件的时候，有嵌套目录，则上传到swift上后，也是以嵌套目录的形式存在，下载后，会在当前目录创建一样的嵌套目录。

四、遇到的问题

1、上传文件失败，查看log文件，

 File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 81, in set#012    self._set(name, value, 0, options | self.options)#012  File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 16, in _func#012    return func(first, *args)#012IOError: [Errno 95] Operation not supported

出错原因：查看/srv/下面的文件，正确的显示信息应该如下所示：可是我的四个目录显示是没有链接信息的，删除这四个目录，1，2，3，4，重新进行软连接，并修改目录的属主即可

swift@alen:/srv$ ls -l

lrwxrwxrwx 1 swift swift         11 2011-07-04 15:13 1 -> /mnt/sdb1/1

lrwxrwxrwx 1 swift swift         11 2011-07-04 15:13 2 -> /mnt/sdb1/2

lrwxrwxrwx 1 swift swift         11 2011-07-04 15:13 3 -> /mnt/sdb1/3

lrwxrwxrwx 1 swift swift         11 2011-07-04 15:13 4 -> /mnt/sdb1/4

-rw-r--r-- 1 swift swift 1024000000 2011-07-04 15:50 swift-disk