CentOS6下安装基于Xen的Eucalyptus 2.0.3

Eucalyptus

  • 官网 http://open.eucalyptus.com/downloads
  • 手册 http://open.eucalyptus.com/wiki
  • 服务器角色
    • Cloud controller(CLC),控制调度云的资源,同时提供web界面以及兼容EC2的SOAP服务
    • Walrus,实现了基于桶的存储,同时提供兼容S3的SOAP服务以及REST接口
    • Cluster controller(CC),实现以集群为单位的调度
    • Storage controller(SC),基于块的存储控制器,EBS-style
    • Node controller(NC),虚拟节点,运行xen及虚拟机的机器

版本

  • CentOS 6 Minimal install
  • Xen 4.1.1 NC服务器上面需要完全安装,其余服务器需要几个编译好的rpm,参考 http://blog.csdn.net/phoenixie/article/details/6885313
  • Eucalyptus 2.0.3,下载offline源码包, http://eucalyptussoftware.com/downloads/releases/eucalyptus-2.0.3-src-offline.tar.gz , 依赖的库 http://eucalyptussoftware.com/downloads/releases/eucalyptus-2.0.3-src-deps.tar.gz
  • euca2ools 1.3.1, 使用fedora的,下载 http://eucalyptussoftware.com/downloads/releases/euca2ools-1.3.1-fedora-x86_64.tar.gz
  • dhcpd 3.0.5, CC上需要安装, 使用旧版本rhel5的, eucalyptus 2.0.3对稍微高版本的ISC dhcpd支持有问题, 下载http://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/dhcp-3.0.5-29.el5_7.1.src.rpm

准备工作

 1. 下载各个软件包
 2. 除了NC以外的服务器安装CentOS 6 Minimal方式
 3. NC服务器按照参考文档安装配置Xen环境,注意在Xen的内核配置里面添加一项  max_loop=256
 4. 配置好所有机器的NTP,使它们时钟保持同步

安装过程

 1. 创建 /etc/yum.repos.d/euca.repo 
[euca]
name=Eucalyptus
baseurl=http://www.eucalyptussoftware.com/downloads/repo/eucalyptus/2.0.3/yum/centos/x86_64
gpgcheck=0
 2. 通过yum安装必须的软件包 (所有机器上)
yum -y install perl-Convert-ASN1.noarch scsi-target-utils httpd java-1.6.0-openjdk-devel ant ant-nodeps libvirt-devel curl-devel httpd-devel apr-devel openssl-devel libxml2 libxml2-devel gnutls gnutls-devel libgcrypt-devel zlib-devel perl-Convert-ASN1 perl-Crypt-OpenSSL-RSA perl-Crypt-OpenSSL-Random chkfontpath fuse-libs swig gcc make patch compat-expat1 compat-db43 lzo2 vblade m2crypto lvm2 rsync ntp vconfig groff rpm-build
 3. 安装之前编译好的Xen的rpm包(给控制服务器,节点服务器已经装过xen的就不需要了)
rpm -Uvh xen-licenses-*.x86_64.rpm xen-devel-*.x86_64.rpm xen-libs-*.x86_64.rpm 
 4. 安装dhcpd
wget http://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/dhcp-3.0.5-29.el5_7.1.src.rpm
  
rpm -i dhcp-3.0.5-29.el5_7.1.src.rpm
  
cd ~/rpmbuild/SPECS/
rpmbuild -bb dhcp.spec  # 如果编译不过去,可能需要手动去掉-Werror编译选项
  
cd ~/rpmbuild/RPMS/x86_64/
rpm -Uvh dhcp-3.0.5-29.el6.1.x86_64.rpm # 如果已经装了dhcp,先手动删除掉原来的
 5. 安装Eucalyptus的依赖软件(eucalyptus-2.0.3-src-deps.tar.gz)
export EUCALYPTUS=/home/public/eucalyptus-2.0.3
export APACHE_INCLUDES=/usr/include/httpd/
export APR_INCLUDES=/usr/include/apr-1/
export AXIS2C_HOME=${EUCALYPTUS}/packages/axis2c-1.6.0

mkdir -p ${EUCALYPTUS}/packages/

tar xzf eucalyptus-2.0.3-src-deps.tar.gz
cd eucalyptus-src-deps

tar xzf axis2-1.4.tgz -C ${EUCALYPTUS}/packages/
cd ${EUCALYPTUS}/packages/
ln -sf axis2-1.4 axis2
cd -

tar xzf axis2c-src-1.6.0.tar.gz
cd axis2c-src-1.6.0
CFLAGS="-w" ./configure --prefix=${AXIS2C_HOME} -with-apache2=${APACHE_INCLUDES} --with-apr=${APR_INCLUDES} --enable-multi-thread=no
make
make install
cd -
rm -rf axis2c-src-1.6.0

export LD_LIBRARY_PATH=${AXIS2C_HOME}/lib
tar xzf rampartc-src-1.3.0-0euca2.tar.gz
cd rampartc-src-1.3.0
./configure --prefix=${AXIS2C_HOME} --enable-static=no --with-axis2=${AXIS2C_HOME}/include/axis2-1.6.0/
make
make install
rm -rf rampartc-src-1.3.0

cd ${EUCALYPTUS}/packages
ln -sf axis2c-1.6.0 axis2c
cd -
sed -i "s;<\!--phase name=\"Security\"/-->;<phase name=\"Security\"/>;g" ${AXIS2C_HOME}/axis2.xml

cd ..
rm -rf eucalyptus-src-deps
 6. 安装Eucalyptus (eucalyptus-2.0.3-src-offline.tar.gz)
export EUCALYPTUS=/home/public/eucalyptus-2.0.3
export AXIS2_HOME=${EUCALYPTUS}/packages/axis2
export AXIS2C_HOME=${EUCALYPTUS}/packages/axis2c
export JAVA_HOME="/usr/lib/jvm/java-openjdk/"
export JAVA="$JAVA_HOME/JRE/bin/java"

tar xzf eucalyptus-2.0.3-src-offline.tar.gz
cd eucalyptus-2.0.3-src-offline
./configure --with-axis2=${AXIS2_HOME} --with-axis2c=${AXIS2C_HOME} --enable-debug --prefix=${EUCALYPTUS}
make
make install
cd -
rm -rf eucalyptus-2.0.3-src-offline

cd /home/public/
ln -sf eucalyptus-2.0.3 eucalyptus

export EUCALYPTUS=/home/public/eucalyptus
ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cloud /etc/init.d/eucalyptus-cloud
ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cc /etc/init.d/eucalyptus-cc
ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-nc /etc/init.d/eucalyptus-nc

chkconfig eucalyptus-cloud off
chkconfig eucalyptus-cc off
chkconfig eucalyptus-nc off
 7. 安装工具(euca2ools-1.3.1-fedora-x86_64.tar.gz)
tar xzf euca2ools-1.3.1-fedora-x86_64.tar.gz
cd euca2ools-1.3.1-fedora-x86_64

rpm -Uvh euca2ools-1.3.1-1.x86_64.rpm

cd -
rm -rf euca2ools-1.3.1-fedora-x86_64

配置

 1. 创建一个专属用户 
adduser eucalyptus
 2. 初始化
export EUCALYPTUS=/home/public/eucalyptus
$EUCALYPTUS/usr/sbin/euca_conf \
 -d $EUCALYPTUS \
 --hypervisor xen \
 --instances /home/eucalyptus/instances \
 --user eucalyptus \
 --setup
# -d: Eucalyptus路径
# --hypervisor: 使用的hypervisor类型,可以是kvm或xen
# --instances: instance存储路径
# --user: 服务使用的用户
# --setup: 初始化

service eucalyptus-nc start
chkconfig eucalyptus-nc on
 3. 编辑 /home/public/eucalyptus/etc/eucalyptus/eucalyptus.conf,按自己的需求修改,其中 VNET_BRIDGE 是xen要使用的bridge名字,其余的VNET_MODE等请参考配置文件的说明,里面写的很详细。
 4. 复制eucalyptus安装目录到所有机器上,建立init.d的链接,添加service的服务,然后还需要运行一次初始化
 5. 启动NC
service eucalyptus-nc start # NC的Web service默认端口8775
chkconfig eucalyptus-nc on
 6. 启动CC
service eucalyptus-cc start # CC的Web service默认端口8774
chkconfig eucalyptus-cc on
 7. 启动相关服务,不同的服务可以部署在不同机器上面
export EUCALYPTUS=/home/public/eucalyptus
$EUCALYPTUS/usr/sbin/euca_conf -d $EUCALYPTUS --enable cloud
$EUCALYPTUS/usr/sbin/euca_conf -d $EUCALYPTUS --enable walrus
$EUCALYPTUS/usr/sbin/euca_conf -d $EUCALYPTUS --enable sc

service eucalyptus-cloud start # 三个服务都通过这个脚本启动,http端口8080,https端口8443,Web service端口8773
chkconfig eucalyptus-cloud on
 8. 注册服务
$EUCALYPTUS/usr/sbin/euca_conf --register-walrus $IP
$EUCALYPTUS/usr/sbin/euca_conf --register-cluster OurCloud $IP # 注册一个Cluster,名字叫OurCloud,默认端口8774
$EUCALYPTUS/usr/sbin/euca_conf --register-sc OurCloud $IP # 给Cluster添加一个SC
 9. 注册节点
$EUCALYPTUS/usr/sbin/euca_conf --register-nodes $IP
 10. 访问Cloud(CLC)所在IP的https://$IP:8443,用户名admin,密码admin,第一次登陆成功后会要求马上修改密码
 11. 在管理界面的导航栏找到Credentials,进入,点击按钮"Download Credentials",会下载一个zip文件
 12. 把zip文件传到安装了euca2ools的服务器上,建一个目录解压进去,以后每次使用euca2ools前需要执行一次下面的命令
source eucarc

 13. 安装完成


实例1: 使用官网提供的Ubuntu image

 1. 下载地址  http://open.eucalyptus.com/wiki/EucalyptusUserImageCreatorGuide 2. 添加 (euca-ubuntu-9.04-x86_64.tar.gz) 
tar xzf euca-ubuntu-9.04-x86_64.tar.gz
cd euca-ubuntu-9.04-x86_64

# 进入运行环境
source ~/.euca/eucarc

# 上传内核,得到一个eki-XXXXXXXX
euca-bundle-image -d ./tmp/ -i xen-kernel/vmlinuz-2.6.27.21-0.1-xen --kernel true
euca-upload-bundle -b ubuntu9-bucket -m ./tmp/vmlinuz-2.6.27.21-0.1-xen.manifest.xml
euca-register ubuntu9-bucket/vmlinuz-2.6.27.21-0.1-xen.manifest.xml

# 上传ramdisk,得到一个eri-XXXXXXXX
euca-bundle-image -d ./tmp/ -i xen-kernel/initrd-2.6.27.21-0.1-xen --ramdisk true
euca-upload-bundle -b ubuntu9-bucket -m ./tmp/initrd-2.6.27.21-0.1-xen.manifest.xml
euca-register ubuntu9-bucket/initrd-2.6.27.21-0.1-xen.manifest.xml

# 上传根分区,用上面两个命令得到的数字执行
euca-bundle-image -d ./tmp/ -i ubuntu.9-04.x86-64.img --kernel eki-XXXXXXXX --ramdisk eri-XXXXXXXX
euca-upload-bundle -b ubuntu9-bucket -m ./tmp/ubuntu.9-04.x86-64.img.manifest.xml
euca-register ubuntu9-bucket/ubuntu.9-04.x86-64.img.manifest.xml
 3. 创建一个登录虚拟机用的key
euca-add-keypair euca-key > ~/euca-key.private
chmod 0600 ~/euca-key.private
 4. 使用刚才上传虚拟机时候得到的emi值启动一个虚拟机实例
euca-run-instances -k euca-key -n 1 emi-XXXXXXXX
 5. 查看运行状态,没有dhcpd的话,这时拿不到IP
>euca-describe-instances

RESERVATION r-4F030A57 admin default
INSTANCE i-35C505C3 emi-XXXXXXXX 0.0.0.0 0.0.0.0 running euca-key 0 m1.small 2011-11-01T10:48:56.831Z OurCloud eki-XXXXXXXX eri-XXXXXXXX

实例2: 使用dhcpd分配IP

 0. CC服务器, IP eth3 192.168.11.3/24; NC服务器, IP eth3 192.168.11.4/24, Xen的bridge xenbr0; 实现从NC上面运行的虚拟机可以通过CC访问到外面的网络
 1. 配置CC的/home/public/eucalyptus/etc/eucalyptus/eucalyptus.conf
VNET_PUBINTERFACE="eth3"
VNET_PRIVINTERFACE="eth3"

VNET_DHCPDAEMON="/usr/sbin/dhcpd"
VNET_DHCPUSER="eucalyptus"

VNET_MODE="MANAGED-NOVLAN"
VNET_SUBNET="192.168.101.0"
VNET_NETMASK="255.255.255.0"
VNET_DNS="192.168.11.2"
VNET_ADDRSPERNET="16"   # 这个数字如果太大会无法创建虚拟机
VNET_PUBLICIPS="192.168.11.80-192.168.11.120"
VNET_CLOUDIP="192.168.11.3"
#VNET_MODE="SYSTEM"
 2. 配置NC的/home/public/eucalyptus/etc/eucalyptus/eucalyptus.conf
VNET_PUBINTERFACE="xenbr0"
VNET_PRIVINTERFACE="xenbr0"
VNET_BRIDGE="xenbr0"
VNET_MODE="MANAGED_NOVLAN"
#VNET_MODE="SYSTEM"
 3. 重新启动CC
service eucalyptus-cc cleanrestart
 4. 创建一个虚拟机看看 
>euca-run-instances -k euca-key -n 1 emi-XXXXXXXX 
RESERVATION r-502007DA admin admin-default
INSTANCE i-440907AE emi-XXXXXXXX 0.0.0.0 0.0.0.0 pending euca-key 2011-11-04T11:01:17.656Z eki-XXXXXXXX eri-XXXXXXXX

>euca-describe-instances 
RESERVATION r-502007DA admin default
INSTANCE i-440907AE emi-XXXXXXXX 192.168.11.101 192.168.101.131 running euca-key 0 m1.small 2011-11-04T11:01:17.656Z OurCloud eki-XXXXXXXX eri-XXXXXXXX

>ssh -i ~/euca-key.private 192.168.11.101
root@ubuntu:~#

>ip addr show eth0
UNKNOWN qlen 1000
    link/ether d0:0d:44:09:07:ae brd ff:ff:ff:ff:ff:ff
    inet 192.168.101.131/28 brd 192.168.101.143 scope global eth0
    inet6 fe80::d20d:44ff:fe09:7ae/64 scope link 
       valid_lft forever preferred_lft forever

>ping -c 1 192.168.11.2 
PING 192.168.11.2 (192.168.11.2) 56(84) bytes of data.
64 bytes from 192.168.11.2: icmp_seq=1 ttl=63 time=1.26 ms

--- 192.168.11.2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.264/1.264/1.264/0.000 ms

实例3: 自己建立CentOS6虚拟机

 1. 使用virtinst以minimal模式安装一个CentOS6虚拟机,硬盘弄一个分区就行了,其他分区都可省,在虚拟机里面安装配置好需要的软件,关掉虚拟机(安装参数可参考 http://blog.csdn.net/phoenixie/article/details/6885313)
 2. 提取虚拟机 / 分区,假设硬盘文件名为sda.img,2GB
> parted sda.img
GNU Parted 2.1
Using /home/vms/centos6/sda.img
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) U
Unit?  [compact]? b                                                       
(parted) p
Model:  (file)
Disk /home/vms/centos6/sda.img: 2147483648B
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start     End          Size         Type     File system  Flags
 1      1048576B  2147483647B  2146435072B  primary  ext3         boot

(parted) quit
> dd if=sda.img of=rootfs.img bs=1M skip=1 count=2047
2047+0 records in
2047+0 records out
2146435072 bytes (2.1 GB) copied, 37.3425 s, 57.5 MB/s
 3. 配置根分区文件
# 加载根分区
mkdir /mnt/rootfs
mount -o loop rootfs.img /mnt/rootfs/

# 复制出来kernel和initramfs
cp /mnt/rootfs/boot/vmlinuz-* ./kernel
cp /mnt/rootfs/boot/initramfs-* ./ramdisk

# 配置网卡
rm udev/rules.d/70-persistent-net.rules
vi /mnt/rootfs/etc/udev/rules.d/60-net.rules
ACTION=="add", SUBSYSTEM=="net", IMPORT{program}="/lib/udev/rename_device"
SUBSYSTEM=="net", RUN+="/etc/sysconfig/network-scripts/net.hotplug"

vi /mnt/rootfs/etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=dhcp
ONBOOT=yes
TYPE=Ethernet

# 关闭selinux
vi /mnt/rootfs/etc/sysconfig/selinux 
SELINUX=disabled

# 修改fstab,去掉所有的UUID方式
vi /mnt/rootfs/etc/fstab
/dev/xvda1         /             ext3     defaults,errors=remount-ro 0 0
/dev/xvda2         /mnt          ext3     defaults                   0 0
/dev/xvda3         swap          swap     defaults                   0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0

vi /mnt/rootfs/etc/mtab
/dev/sda1 / ext3 rw,errors=remount-ro 0 0
proc /proc proc rw 0 0
sysfs /sys sysfs rw 0 0

# 启动时自动下载可登录的ssh-rsa的public key
vi /mnt/rootfs/etc/rc.local 
# simple attempt to get the user ssh key using the meta-data service
mkdir -p /root/.ssh
echo >> /root/.ssh/authorized_keys
curl -m 10 -s http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key | grep 'ssh-rsa' >> /root/.ssh/authorized_keys
echo "AUTHORIZED_KEYS:"
echo "************************"
cat /root/.ssh/authorized_keys
echo "************************"

# 删除密码
vi /mnt/rootfs/etc/shadow
root:!:15280:0:99999:7:::

# 卸载
umount /mnt/rootfs
 4. 修改NC上面的 /home/public/eucalyptus/usr/share/eucalyptus/gen_libvirt_xml, 把所有的sda改为xvda, 这个修改后,目前官网提供的image将不能使用
 5. 最后, 上传
source ~/.euca/eucarc
# 上传内核,得到一个eki-KKKKKKKK
euca-bundle-image -d ./tmp/ -i ./kernel --kernel true
euca-upload-bundle -b centos6-bucket -m ./tmp/kernel.manifest.xml 
euca-register centos6-bucket/kernel.manifest.xml
# 上传ramdisk,得到一个eri-RRRRRRRR
euca-bundle-image -d ./tmp/ -i ./ramdisk --ramdisk true
euca-upload-bundle -b centos6-bucket -m ./tmp/ramdisk.manifest.xml
euca-register centos6-bucket/ramdisk.manifest.xml
# 上传根分区
euca-bundle-image -d ./tmp/ -i ./rootfs.img --kernel  eki-KKKKKKKK --ramdisk eri-RRRRRRRR
euca-upload-bundle -b centos6-bucket -m ./tmp/rootfs.img.manifest.xml 
euca-register centos6-bucket/rootfs.img.manifest.xml
 6. 现在可以试试了,需要注意的即将创建的虚拟机的磁盘,不能小于根分区文件的字节数,否则不能建立虚拟机;而且设定的虚拟机的内存,不能小于内核的最低需要;这两个参数可以在管理界面里面调整。

你可能感兴趣的:(虚拟机,centos,ubuntu,service,服务器,xen)