安装与配置Ceph RagosGW

安装与配置RagosGW

可以暂时不做这个，先熟悉ceph代码

如果配置这个的话，主要目的就是熟悉RADOSGW对象存储网关

 

操作之前先备份、做快照，因为看起来比较麻烦

在lxpnode1上创建radosgw

 

4.1 安装配置Apache

Apt-get 源：

1.      下载安装apache2

# apt-get install apache2

# apt-get install libapache2-mod-fastcgi

 

2.      配置/etc/apache2/apache2.conf

添加：

ServerName lxpnode1

3.      加载rewrite和fastcgi模块

# a2enmod rewrite

# a2enmod fastcgi

 

4.      启动apache服务

# service apache2 start

4.2 启用SSL

4.2.1安装

# apt-get install openssl ssl-cert

 

4.2.2 加载SSL模块

# a2enmod ssl

4.2.3 生成证书

# openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout/etc/apache2/ssl/apache.key -out /etc/apache2/ssl/apache.crt

全部回车

Generating a 2048 bit RSA private key

........................................................................................................................+++

...........................................+++

writing new private key to '/etc/apache2/ssl/apache.key'

-----

You are about to be asked to enter information that will beincorporated

into your certificate request.

What you are about to enter is what is called a Distinguished Nameor a DN.

There are quite a few fields but you can leave some blank

For some fields there will be a default value,

If you enter '.', the field will be left blank.

-----

Country Name (2 letter code) [AU]:

State or Province Name (full name) [Some-State]:

Locality Name (eg, city) []:

Organization Name (eg, company) [Internet Widgits Pty Ltd]:

Organizational Unit Name (eg, section) []:

Common Name (e.g. server FQDN or YOUR name) []:

Email Address []:

root@lxpnode1:/etc/apache2#

4.2.4 重启apache

# service apache2 restart

 

4.3 安装Ceph Object Gateway守护进程

使用海康的源，ubuntutrusty中的源都是0.80.x，而ubuntu vivid中的源版本为0.94.5，将源改为vivid

# apt-get install ceph

root@lxpnode1:/etc/apt# ceph -v

ceph version 0.94.5 (9764da52395923e0b32908d83a9f7304401fee43)

然后安装ceph-radosgw：

 

 

root@lxpnode1:/etc/apt# ceph -s

    cluster19c11869-2430-4b9a-8152-e60e36064826

     health HEALTH_OK

     monmap e1: 1 mons at{lxpnode1=192.168.11.6:6789/0}

            election epoch 1,quorum 0 lxpnode1

     osdmap e20: 2 osds: 2 up,2 in

      pgmap v1414: 124 pgs, 2pools, 148 bytes data, 4 objects

            3642 MB used,33380 MB / 39051 MB avail

                 124active+clean

目前仍是HEALTH_OK，没有影响

接下来安装radosgw：

# apt-get install radosgw

# apt-get install radosgw-agent

 

4.4配置Ceph Object Gateway

4.4.1 创建用户和keyring

（1）为网关服务器创建keyring

# ceph-authtool --create-keyring/etc/ceph/ceph.client.radosgw.keyring

# chmod +r ceph.client.radosgw.keyring

 

（2）为每一个网关实例生成一个名称和key. 在client.radosgw后面使用这个名称（下面实例的名称为gateway）：

# ceph-authtool/etc/ceph/ceph.client.radosgw.keyring -n client.radosgw.gateway --gen-key 

（3）key添加相应权限

# ceph-authtool -n client.radosgw.gateway--cap osd 'allow rwx' --cap mon 'allow rwx' /etc/ceph/ceph.client.radosgw.keyring

（4）将key添加到Ceph storage cluster:

# ceph -k/etc/ceph/ceph.client.admin.keyring auth add client.radosgw.gateway -i/etc/ceph/ceph.client.radosgw.keyring

（5）将key拷贝到其他网关服务器，这里只有一台，不必拷贝

#sudo scp /etc/ceph/ceph.client.radosgw.keyring  root@{hostname}:/etc/ceph/

 

注：1-4步在Ceph的admin节点执行，如果admin节点和网关服务器在一台机器上，第5步不需要执行。

 

4.4.2 创建池

#ceph osd pool create .rgw 32 32

#ceph osd pool create .rgw.root 32 32

#ceph osd pool create .rgw.control 32 32

#ceph osd pool create .rgw.gc 32 32

#ceph osd pool create .rgw.buckets 32 32

#ceph osd pool create .rgw.buckets.index 32 32

#ceph osd pool create .log 32 32

#ceph osd pool create .intent-log 32 32

#ceph osd pool create .usage 32 32

#ceph osd pool create .users 32 32

#ceph osd pool create .users.email 32 32

#ceph osd pool create .users.swift 32 32

#ceph osd pool create .users.uid 32 32

 

root@lxpnode1:/etc/ceph# rados lspools

rbd

rbd_pool

.rgw

.rgw.root

.rgw.control

.rgw.gc

.rgw.buckets

.rgw.buckets.index

.log

.intent-log

.usage

.users

.users.email

.users.swift

.users.uid

 

4.4.3 添加一个网关配置文件到Ceph

Apache版本：

 

root@lxpnode1:/etc/ceph# apache2 -v

Server version: Apache/2.4.7 (Ubuntu)

Apache2.2和早期的2.4版本，admin节点上的/etc/ceph/ceph.conf中添加：

[client.radosgw.gateway]

host = {hostname}

keyring = /etc/ceph/ceph.client.radosgw.keyring

rgw socket path = ""

log file =/var/log/radosgw/client.radosgw.gateway.log

rgw frontends = fastcgi socket_port=9000socket_host=0.0.0.0

rgw print continue = false

 

Apache 2.4.9 和后面版本:

[client.radosgw.gateway]

host = {hostname}

keyring =/etc/ceph/ceph.client.radosgw.keyring

rgw socket path =/var/run/ceph/ceph.radosgw.gateway.fastcgi.sock

log file =/var/log/radosgw/client.radosgw.gateway.log

rgw print continue = false

注：Apache 2.4.7用前一种方式radosgw服务起不来，用后一种是OK的，Apache2.4.7支持Unix Domain Socket。

 

4.4.4 分发Ceph配置文件

（1）更新/etc/ceph/ceph.conf到Ceph集群的根目录：

# ceph-deploy --overwrite-conf config pulllxpnode1

（2）将admin节点的ceph.conf拷贝到其他节点

# ceph-deploy  --overwrite-conf config push lxpnode2lxpnode3

 

4.4.5 拷贝ceph.client.admin.keyring 到网关服务器

我这里的网关服务器就是集群的mon，所以不必拷贝

 

4.4.6 创建CGI wrapper脚本

1.创建脚本:

#sudo vi /var/www/html/s3gw.fcgi

2.添加相关内容:

#!/bin/sh

exec /usr/bin/radosgw -c/etc/ceph/ceph.conf -n client.radosgw.gateway

3.授予脚本执行权限:

#sudo chmod +x /var/www/html/s3gw.fcgi

4.4.7 调整CGI wrapper脚本权限

#sudo chown www-data:www-data/var/www/html/s3gw.fcgi

Note: Ubuntu默认apache的用户是www-data:www-data

 

4.4.8 创建数据存放目录

#mkdir -p/var/lib/ceph/radosgw/ceph-radosgw.gateway

 

4.4.9 调整socket目录权限

# chown www-data:www-data /var/run/ceph

 

4.4.10 修改日志文件权限

root@lxpnode1:/etc/ceph# touch  /var/log/radosgw/client.radosgw.gateway.log

root@lxpnode1:/etc/ceph# chownwww-data:www-data /var/log/radosgw/client.radosgw.gateway.log

 

4.4.11 启动radosgw服务

# /etc/init.d/radosgw start

 

4.4.12 创建一个网关配置文件

1.Create thefile:

#sudovi /etc/apache2/sites-available/rgw.conf

2.add thefollowing contents to the file:

FastCgiExternalServer /var/www/html/s3gw.fcgi -socket /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock

<VirtualHost *:8080>

 ServerName {hostname}
 ServerAlias *.{hostname}
 ServerAdmin [email protected]
 DocumentRoot /var/www/html
 RewriteEngine On
 RewriteRule ^/(.*) /s3gw.fcgi?%{QUERY_STRING} [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]

 <IfModule mod_fastcgi.c>
 <Directory /var/www/html>
 Options +ExecCGI
 AllowOverride All
 SetHandler fastcgi-script
 Order allow,deny
 Allow from all
 AuthBasicAuthoritative Off
 </Directory>
 </IfModule>

 AllowEncodedSlashes On
 ErrorLog /var/log/apache2/error.log
 CustomLog /var/log/apache2/access.log combined
 ServerSignature Off
</VirtualHost>

如果部署的节点上已经有apache2服务使用80端口，需要配置apache多端口，rgw.conf中改为：

Listen 8080

3.使rgw.conf配置生效
For Debian/Ubuntu distributions, enable the site forrgw.conf.
root@lxpnode1:/etc/apache2#a2ensite rgw.conf

Enabling site rgw.

To activate the new configuration, you need to run:

  serviceapache2 reload

root@lxpnode1:/etc/apache2# a2dissite000-default.conf

Site 000-default disabled.

To activate the new configuration, you need to run:

  serviceapache2 reload

 

4.4.13 重启apache2

# service apache2 restart