版本说明:
CloudFoundry:V2版本-v149+(本文后续章节中的Manifest适用于149+版本的CF Release)
OpenStack:Grizzly版本+nova-network
CloudFoundry部署交流QQ群:176302388
CloudFoundry目前官方的版本为V2版,release最新发布为149,不过CloudFoundry在GitHub上的仓库代码贡献非常频繁,一般只要提出有BUG,很快就会有修改并提交,所以,最好的部署方式是直接下载源码,创建dev版的release进行部署,当然,也可以直接使用官方发布的release进行打包部署,本文中使用149.1-dev进行打包部署,如果要使用官方release,指定release路径即可,如:bosh create release release/cf-149.yml。
注意:本文中的示例Manifest仅适用于149+版本的release。
默认情况下blobstore/0虚拟机中/var/vcap/store的存储空间比较小,而后边上传用于部署CloudFoundry的Stemcell包和Release包时,会因为存储空间不足导致错误,所以这里先增加/var/vcap/store的挂载空间,防止发生错误。
具体步骤参看文章:http://blog.csdn.net/tiger435/article/details/8808623#t8。
说明:本环境配置文件中的所有通过Stemcell创建的虚拟机的登陆账户密码为“root/c1oudc0w”或者“vcap/c1oudc0w”
注意:即使配置挂载空间后,以后重启blobstore的虚拟机的时候一定记得再次挂载,或者可以修改为开机挂载,本人就是忘记该步骤,导致重启虚拟机后,后边部署cf-service-release时出现莫名其妙找不到package的错误,折磨几个小时。
依次执行以下命令从GitHub获取并更新代码
root@bosh-cli:~# mkdir -p ~/src/cloudfoundry root@bosh-cli:~# cd ~/src/cloudfoundry root@bosh-cli:~/src/cloudfoundry# git clone -b release-candidate git://github.com/cloudfoundry/cf-release.git root@bosh-cli:~/src/cloudfoundry# cd cf-release root@bosh-cli:~/src/cloudfoundry/cf-release# ./update
下载并更新完毕之后,执行以下命令创建release包,这是一个相当漫长的过程,中间还会发生多次中断,只要重新执行bosh create release --force即可:
说明:bosh create release --force 是使用最新的CloudFoundry源码创建release-dev包,可能会存在一些奇怪的BUG或者错误、问题,好处就是使用这种方式构建release-dev包,下次代码更新后,再进行打包速度会比较快。若过程中出现问题,可以使用经过测试发布的release包,打包方式:bosh create release releases/cf-149.yml,会在releases目录下生成一个tgz的压缩包,之后执行bosh upload releases/cf-149.tgz上传release包即可。
root@bosh-cli:~/src/cloudfoundry/cf-release# bosh create release --force .... Building jobs ------------- Building health_manager_next... Final version: NOT FOUND Dev version: NOT FOUND Generating... Generated version 7.1-dev Building dashboard... Final version: Downloading `dashboard (10)'... Building ccdb_postgres... Final version: Downloading `ccdb_postgres (7)'... Building gorouter... Final version: Downloading `gorouter (2)'... Building dea_next... Final version: NOT FOUND Dev version: NOT FOUND Generating... Generated version 12.1-dev Building collector... Final version: NOT FOUND Dev version: NOT FOUND Generating... Generated version 6.1-dev Building uaa... Final version: NOT FOUND Dev version: NOT FOUND Generating... Generated version 27.1-dev Building syslog_aggregator... Final version: NOT FOUND Dev version: NOT FOUND Generating... Generated version 10.1-dev Building login... Final version: NOT FOUND Dev version: NOT FOUND Generating... Generated version 11.1-dev Building debian_nfs_server... Final version: FOUND LOCAL Building nats... Final version: NOT FOUND Dev version: NOT FOUND Generating... Generated version 10.1-dev Building postgres... Final version: Downloading `postgres (4)'... Building cloud_controller_ng... Final version: NOT FOUND Dev version: NOT FOUND Generating... Generated version 7.1-dev Building release ---------------- Generating manifest... ---------------------- Writing manifest... Release summary --------------- Packages +---------------------+----------+-------------+ | Name | Version | Notes | +---------------------+----------+-------------+ | git | 1 | | | nginx | 9 | | | ruby | 8 | | | health_manager_next | 18 | | | dashboard | 14 | | | libyaml | 1 | | | rootfs_lucid64 | 1 | | | insight_agent | 2 | | | ruby_next | 3 | | | gorouter | 4.1-dev | new version | | dea_next | 16.1-dev | new version | | collector | 8.1-dev | new version | | uaa | 24.1-dev | new version | | golang | 2 | | | warden | 26.1-dev | new version | | mysqlclient | 3 | | | syslog_aggregator | 4 | | | libpq | 5 | | | login | 13.1-dev | new version | | dea_jvm | 4 | | | buildpack_cache | 1.1-dev | new version | | debian_nfs_server | 3 | | | nats | 8 | | | postgres | 4 | | | sqlite | 3 | | | imagemagick | 2 | | | cloud_controller_ng | 13.1-dev | new version | | common | 5 | | +---------------------+----------+-------------+ Jobs +---------------------+----------+-------------+ | Name | Version | Notes | +---------------------+----------+-------------+ | health_manager_next | 7.1-dev | new version | | dashboard | 10 | | | ccdb_postgres | 7 | | | gorouter | 2 | | | dea_next | 12.1-dev | new version | | collector | 6.1-dev | new version | | uaa | 27.1-dev | new version | | syslog_aggregator | 10.1-dev | new version | | login | 11.1-dev | new version | | debian_nfs_server | 7 | | | nats | 10.1-dev | new version | | postgres | 4 | | | cloud_controller_ng | 7.1-dev | new version | +---------------------+----------+-------------+ Jobs affected by changes in this release +---------------------+----------+ | Name | Version | +---------------------+----------+ | health_manager_next | 7.1-dev | | dea_next | 12.1-dev | | collector | 6.1-dev | | uaa | 27.1-dev | | syslog_aggregator | 10.1-dev | | login | 11.1-dev | | nats | 10.1-dev | | cloud_controller_ng | 7.1-dev | | gorouter | 2 | +---------------------+----------+ Release version: 149.1-dev Release manifest: /root/src/cloudfoundry/cf-release/dev_releases/cf-release-149.1-dev.yml
root@bosh-cli:~/src/cloudfoundry/cf-release# bosh target https://10.68.19.132:25555 Target set to `bosh' root@bosh-cli:~/src/cloudfoundry/cf-release# bosh login Your username: admin Enter password: ***** Logged in as `admin' root@bosh-cli:~/src/cloudfoundry/cf-release# bosh create user Enter new username: venusource Enter new password: ********** Verify new password: ********** User `venusource' has been created root@bosh-cli:~/src/cloudfoundry/cf-release# bosh login Your username: venusource Enter password: ********** Logged in as `venusource'
上传Stemcell和部署Bosh的时候使用的是同一个Stemcell:
root@bosh-cli:~/src/cloudfoundry/cf-release# bosh upload stemcell ~/bosh-workspace/stemcells/bosh-stemcell-latest-openstack-kvm-ubuntu.tgz
Verifying stemcell...
File exists and readable OK
Using cached manifest...
Stemcell properties OK
Stemcell info
-------------
Name: bosh-stemcell
Version: 776
Checking if stemcell already exists...
No
Uploading stemcell...
latest-bosh-s: 100% |oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo| 251.3MB 11.0MB/s Time: 00:00:22
Director task 1
Update stemcell
extracting stemcell archive (00:00:12)
verifying stemcell manifest (00:00:00)
checking if this stemcell already exists (00:00:00)
uploading stemcell bosh-stemcell/776 to the cloud (00:02:05)
save stemcell bosh-stemcell/776 (4e433167-acf2-41b0-b691-934684a63e52) (00:00:00)
Done 5/5 00:02:17
Task 1 done
Started 2013-12-17 07:03:06 UTC
Finished 2013-12-17 07:05:23 UTC
Duration 00:02:17
Stemcell uploaded and created
上传Release到Bosh:
root@bosh-cli:~/src/cloudfoundry/cf-release# bosh upload release .... Release has been created cf-release/149.1-dev (00:00:00) Done 1/1 00:00:00 Task 2 done Started 2013-12-18 01:59:30 UTC Finished 2013-12-18 02:02:20 UTC Duration 00:02:50 Release uploaded
验证下Stemcell和Release是否正常:
root@bosh-cli:~# bosh releases +------------+------------+-------------+ | Name | Versions | Commit Hash | +------------+------------+-------------+ | cf-release | 149.1-dev* | 27784f08 | +------------+------------+-------------+ (*) Currently deployed (+) Uncommitted changes Releases total: 3 root@bosh-cli:~# bosh stemcells +---------------------------+---------+--------------------------------------+ | Name | Version | CID | +---------------------------+---------+--------------------------------------+ | bosh-openstack-kvm-ubuntu | 1478 | 316501e1-482d-472d-9fd9-b71a7b1223ee | +---------------------------+---------+--------------------------------------+ Stemcells total: 1
root@bosh-cli:~/src/cloudfoundry/cf-release# cd ~/bosh-workspace/deployments/ root@bosh-cli:~/bosh-workspace/deployments# vi cf-149.yml
下边给我本环境中配置后的Manifest.yml文件作为参考:
--- name: cf director_uuid: 785aa0a6-87c7-4452-a862-c118afc90b37 releases: - name: cf-release version: latest networks: - name: floating type: vip cloud_properties: {} - name: default type: dynamic cloud_properties: security_groups: - default update: canaries: 1 canary_watch_time: 30000-60000 update_watch_time: 30000-60000 max_in_flight: 4 compilation: workers: 6 network: default reuse_compilation_vms: true cloud_properties: instance_type: v1.medium resource_pools: - name: small network: default size: 4 stemcell: name: bosh-openstack-kvm-ubuntu version: latest cloud_properties: instance_type: v1.small - name: medium network: default size: 0 stemcell: name: bosh-openstack-kvm-ubuntu version: latest cloud_properties: instance_type: v1.medium jobs: - name: data release: cf-release template: - postgres - debian_nfs_server instances: 1 resource_pool: small persistent_disk: 4096 networks: - name: default default: - dns - gateway properties: db: databases - name: core release: cf-release template: - syslog_aggregator - nats - health_manager_next - uaa instances: 1 resource_pool: small networks: - name: default default: - dns - gateway - name: api release: cf-release template: - cloud_controller_ng - gorouter instances: 1 resource_pool: small networks: - name: default default: - dns - gateway - name: floating static_ips: - 10.68.19.133 properties: db: databases - name: dea release: cf-release template: - dea_next instances: 1 resource_pool: small networks: - name: default default: [dns, gateway] properties: cf: name: cf dns: vsc.com ip_addresses: ["10.68.19.133"] deployment_size: medium security_group: default persistent_disk: 4096 common_password: c1oudc0w dea_server_ram: 4096 domain: vsc.com system_domain: vsc.com.com system_domain_organization: system_domain app_domains: - vsc.com networks: apps: default management: default nats: address: 0.core.default.cf.bosh machines: - 0.core.default.cf.bosh port: 4222 user: nats password: c1oudc0w authorization_timeout: 5 router: address: 0.api.default.cf.bosh port: 8081 status: port: 8080 user: gorouter password: c1oudc0w dea: &dea memory_mb: 4096 disk_mb: 10240 directory_server_protocol: http dea_next: *dea syslog_aggregator: address: 0.core.default.cf.bosh port: 54321 nfs_server: address: 0.data.default.cf.bosh network: "*.cf.bosh" idmapd_domain: vsc.com debian_nfs_server: no_root_squash: true databases: &databases db_scheme: postgres address: 0.data.default.cf.bosh port: 5524 roles: - tag: admin name: ccadmin password: c1oudc0w - tag: admin name: uaaadmin password: c1oudc0w databases: - tag: cc name: ccdb citext: true - tag: uaa name: uaadb citext: true ccdb: &ccdb db_scheme: postgres address: 0.data.default.cf.bosh port: 5524 roles: - tag: admin name: ccadmin password: c1oudc0w databases: - tag: cc name: ccdb citext: true ccdb_ng: *ccdb uaadb: db_scheme: postgresql address: 0.data.default.cf.bosh port: 5524 roles: - tag: admin name: uaaadmin password: c1oudc0w databases: - tag: uaa name: uaadb citext: true cc_api_version: v2 cc: &cc logging_level: debug external_host: api srv_api_uri: http://api.vsc.com cc_partition: default db_encryption_key: c1oudc0w bootstrap_admin_email: [email protected] bulk_api_password: c1oudc0w uaa_resource_id: cloud_controller staging_upload_user: uploaduser staging_upload_password: c1oudc0w resource_pool: resource_directory_key: cc-resources # Local provider when using NFS fog_connection: provider: Local packages: app_package_directory_key: cc-packages droplets: droplet_directory_key: cc-droplets default_quota_definition: runaway ccng: *cc login: enabled: false uaa: url: http://uaa.vsc.com spring_profiles: postgresql no_ssl: true catalina_opts: -Xmx768m -XX:MaxPermSize=256m resource_id: account_manager jwt: signing_key: | -----BEGIN RSA PRIVATE KEY----- MIICXAIBAAKBgQDHFr+KICms+tuT1OXJwhCUmR2dKVy7psa8xzElSyzqx7oJyfJ1 JZyOzToj9T5SfTIq396agbHJWVfYphNahvZ/7uMXqHxf+ZH9BL1gk9Y6kCnbM5R6 0gfwjyW1/dQPjOzn9N394zd2FJoFHwdq9Qs0wBugspULZVNRxq7veq/fzwIDAQAB AoGBAJ8dRTQFhIllbHx4GLbpTQsWXJ6w4hZvskJKCLM/o8R4n+0W45pQ1xEiYKdA Z/DRcnjltylRImBD8XuLL8iYOQSZXNMb1h3g5/UGbUXLmCgQLOUUlnYt34QOQm+0 KvUqfMSFBbKMsYBAoQmNdTHBaz3dZa8ON9hh/f5TT8u0OWNRAkEA5opzsIXv+52J duc1VGyX3SwlxiE2dStW8wZqGiuLH142n6MKnkLU4ctNLiclw6BZePXFZYIK+AkE xQ+k16je5QJBAN0TIKMPWIbbHVr5rkdUqOyezlFFWYOwnMmw/BKa1d3zp54VP/P8 +5aQ2d4sMoKEOfdWH7UqMe3FszfYFvSu5KMCQFMYeFaaEEP7Jn8rGzfQ5HQd44ek lQJqmq6CE2BXbY/i34FuvPcKU70HEEygY6Y9d8J3o6zQ0K9SYNu+pcXt4lkCQA3h jJQQe5uEGJTExqed7jllQ0khFJzLMx0K6tj0NeeIzAaGCQz13oo2sCdeGRHO4aDh HH6Qlq/6UOV5wP8+GAcCQFgRCcB+hrje8hfEEefHcFpyKH+5g1Eu1k0mLrxK2zd+ 4SlotYRHgPCEubokb2S1zfZDWIXW3HmggnGgM949TlY= -----END RSA PRIVATE KEY----- verification_key: | -----BEGIN PUBLIC KEY----- MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDHFr+KICms+tuT1OXJwhCUmR2d KVy7psa8xzElSyzqx7oJyfJ1JZyOzToj9T5SfTIq396agbHJWVfYphNahvZ/7uMX qHxf+ZH9BL1gk9Y6kCnbM5R60gfwjyW1/dQPjOzn9N394zd2FJoFHwdq9Qs0wBug spULZVNRxq7veq/fzwIDAQAB -----END PUBLIC KEY----- cc: client_secret: c1oudc0w admin: client_secret: c1oudc0w batch: username: batchuser password: c1oudc0w client: autoapprove: - cf clients: cf: override: true authorized-grant-types: password,implicit,refresh_token authorities: uaa.none scope: cloud_controller.read,cloud_controller.write,openid,password.write,cloud_controller.admin,scim.read,scim.write access-token-validity: 7200 refresh-token-validity: 1209600 scim: users: - admin|c1oudc0w|scim.write,scim.read,openid,cloud_controller.admin - services|c1oudc0w|scim.write,scim.read,openid,cloud_controller.admin
修改cf.yml文件
director_uuid:修改为Bosh Director的UUID,可以通过bosh status查看;
releases的name属性,修改为之前创建CloudFoundry的Release包时输入的包名;
10.58.19.133:修改为可用的Floating IP;
vsc.com:域名,CF平台不能通过IP直接访问使用,必须配置域名,可以自己任意写个域名,然后配置本地DNS Server进行解析即可;
反馈:
root@bosh-cli:~/bosh-workspace/deployments# bosh deployment cf-149.yml
root@bosh-cli:~/bosh-workspace/deployments# bosh deploy Getting deployment properties from director... Unable to get properties list from director, trying without it... Compiling deployment manifest... Cannot get current deployment information from director, possibly a new deployment Please review all changes carefully Deploying `cf-149.yml' to `bosh' (type 'yes' to continue): yes Director task 10 Preparing deployment binding deployment (00:00:00) binding releases (00:00:00) binding existing deployment (00:00:00) binding resource pools (00:00:00) binding stemcells (00:00:00) binding templates (00:00:01) binding properties (00:00:00) binding unallocated VMs (00:00:00) binding instance networks (00:00:00) Done 9/9 00:00:01 Preparing package compilation Preparing DNS binding DNS (00:00:00) Done 1/1 00:00:00 Creating bound missing VMs small/0 (00:00:42) small/1 (00:00:57) small/2 (00:01:02) small/3 (00:01:10) Done 4/4 00:01:10 Binding instance VMs data/0 (00:00:01) dea/0 (00:00:01) core/0 (00:00:01) api/0 (00:00:01) Done 4/4 00:00:01 Preparing configuration binding configuration (00:00:02) Done 1/1 00:00:02 Updating job data data/0 (canary) (00:00:53) Done 1/1 00:00:53 Updating job core core/0 (canary) (00:01:08) Done 1/1 00:01:08 Updating job api api/0 (canary) (00:01:04) Done 1/1 00:01:04 Updating job dea dea/0 (canary) (00:03:20) Done 1/1 00:03:20 Task 10 done Started 2013-12-18 11:20:53 UTC Finished 2013-12-18 11:29:37 UTC Duration 00:08:44 Deployed `cf-149.yml' to `bosh'部署过程可能会因为OpenStack的VM虚拟机创建时间过长或PING虚拟机响应过长导致失败,只需要删除Deployments后再重新部署即可。
Deployment删除命令:
root@bosh-cli:~/bosh-workspace/deployments# bosh delete deployment cf部署成功之后可以使用命令查看虚拟机运行状态:
root@bosh-cli:~/bosh-workspace/deployments# bosh vms Deployment `cf' Director task 191 Task 191 done +-----------+---------+---------------+--------------------------+ | Job/index | State | Resource Pool | IPs | +-----------+---------+---------------+--------------------------+ | api/0 | running | small | 50.50.0.16, 10.68.19.133 | | core/0 | running | small | 50.50.0.17 | | data/0 | running | small | 50.50.0.14 | | dea/0 | running | small | 50.50.0.15 | +-----------+---------+---------------+--------------------------+ VMs total: 4
全部Job为Running状态则正常。
注意:请先配置DNS Server,对设置的虚拟域名(vsc.com)进行解析,参照文章:《配置DNS Server》和《配置OpenStack环境DNS》
6.1、安装CF命令行工具
gem install cf
6.2、切换CF指向为api.vsc.com
root@bosh-cli:~# cf target api.vsc.com Setting target to http://api.vsc.com... OK target: http://api.vsc.com organization: system_domain6.3、登陆到CF平台,本环境账号密码为admin/c1oudc0w
root@bosh-cli:~# cf login admin target: http://api.vsc.com Password> ******** Authenticating... OK There are no spaces. You may want to create one with create-space.6.4、创建组织
root@bosh-cli:~# cf create-org venusource Creating organization venusource... OK Switching to organization venusource... OK There are no spaces. You may want to create one with create-space.6.5、创建并切换空间
root@bosh-cli:~# cf create-space development Creating space development... OK Adding you as a manager... OK Adding you as a developer... OK Space created! Use `cf switch-space development` to target it. root@bosh-cli:~# cf switch-space development Switching to space development... OK target: http://api.vsc.com organization: venusource space: development6.6、下载测试Ruby工程
root@bosh-cli:~# git clone https://github.com/cloudfoundry-community/cf_demoapp_ruby_rack.git6.7、打包并发布测试工程
root@bosh-cli:~# cd cf_demoapp_ruby_rack/ root@bosh-cli:~/cf_demoapp_ruby_rack# rm manifest.yml root@bosh-cli:~/cf_demoapp_ruby_rack# bundle Resolving dependencies... Using rack (1.5.2) Using bundler (1.3.5) Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed. root@bosh-cli:~/cf_demoapp_ruby_rack# cf push Name> hello Instances> 1 1: 128M 2: 256M 3: 512M 4: 1G Memory Limit> 1 Creating hello... OK 1: hello 2: none Subdomain> hello 1: vsc.com 2: none Domain> mycloud.com Creating route hello.mycloud.com... OK Binding hello.mycloud.com to hello... OK Create services for application?> n Save configuration?> n Uploading hello... OK Starting hello... OK -----> Downloaded app package (4.0K) .... Checking hello... Staging in progress... 0/1 instances: 1 starting 0/1 instances: 1 starting 0/1 instances: 1 starting 1/1 instances: 1 running OK