Cloud Foundry使用dev_setup部署多节点之排错分析

多节点部署，说白了就和单节点一样，区别就是把一个东西拆开来放到不同节点上去安装而已。

（1）怎么部署多节点，可以先参看这个帖子：http://blog.csdn.net/cherry_sun/article/details/7711913

里面介绍了一个大体流程，对后面的的工作有很多好处。

（2）如果在安装过程中出现问题，可以参考这个博客，虽然问题可能不一样，但是思路基本是一样的：

http://blog.csdn.net/wearenoth/article/details/8035968

（3）安装完成后我又遇到一些比较诡异的问题，这些问题里面有一些我晓得大概为什么，但是更多都不明白是怎么回事，下面是我找到的一些答案：

注：我这个是按照官方给的教程弄的，https://github.com/cloudfoundry/oss-docs/tree/master/vcap/single_and_multi_node_deployments_with_dev_setup

本来部署就不是我工作的重点，但是不弄一次，很多东西又弄不明白，所以只能弄一次了。

注2：由于当时没有写笔记写的乱七八糟，所以看下文的时候会出现很多地方信息不太一致，但是方法还是基本差不多的。

问题0：到底有没有成功

问题描述：我在安装mysql的节点的时候，遇到下面这种情况，首先出现了我们最喜欢看到的Success的信息，但是在开启的时候出现了问题。所以问题就出来了，我到底是装成功了，还是没有装成功？

Deployment Info
***************
* Status: Success
* Config files: /home/vm/cloudfoundry/.deployments/mysql1/config
* Deployment name: mysql1
* Note:
  * If you want to run ruby/vmc please source the profile /home/vm/.cloudfoundry_deployment_profile
  * If you want to run cloudfoundry components by hand please source the profile /home/vm/.cloudfoundry_deployment_local
* Command to run cloudfoundry: /home/vm/cloudfoundry/vcap/dev_setup/bin/vcap_dev -n mysql1 start
vm@vm:~/cloudfoundry/vcap/dev_setup$ ./bin/vcap_dev start
Targeting deployment "mysql1" with cloudfoundry home "/home/vm/cloudfoundry"
Using cloudfoundry config from /home/vm/cloudfoundry/.deployments/mysql1/config
Executing /home/vm/cloudfoundry/.deployments/mysql1/deploy/rubies/ruby-1.9.2-p180/bin/ruby /home/vm/cloudfoundry/vcap/dev_setup/bin/vcap start mysql_node -c /home/vm/cloudfoundry/.deployments/mysql1/config -v /home/vm/cloudfoundry -l /home/vm/cloudfoundry/.deployments/mysql1/log
/home/vm/cloudfoundry/.deployments/mysql1/deploy/rubies/ruby-1.9.2-p180/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require': no such file to load -- nats/client (LoadError)
	from /home/vm/cloudfoundry/.deployments/mysql1/deploy/rubies/ruby-1.9.2-p180/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
	from /home/vm/cloudfoundry/vcap/dev_setup/bin/vcap:13:in `
'
vm@vm:~/cloudfoundry/vcap/dev_setup$ 

这个问题困扰了我很久，在网上也找到类似的问题，然后底下也给出了很多解决办法，我下面给出我找到的绝对比较有可能的几个回答，它们最后指向的矛头都是rvm的问题，而最终的问题都是和ruby版本相关。那答案真的是不是它呢？我们先看看我找到的答案是怎么说的吧：

网上的答案1：

Did you log out of the installation shell and create a new shell? This is needed I believe with the newer version of rvm..

https://github.com/cloudfoundry/vcap/blob/master/README.md

NOTE: The automated setup does not auto-start the system. Once you are done with the setup, exit your current shell, restart a new shell and continue the following steps

网上的答案2：

Correct Michael. The pull request in flight: https://github.com/cloudfoundry/vcap/pull/67

Does a few things:

 

#1 - it installs RVM correctly given some recent changes in the RVM installer. Major issue has to do with the new rvm blindly writing to .bash_profile which on a raw ubunto vm ends up disableing .bashrc. In addition, there was a bug in the original single-line automated installer that had the test for interactive shells backwards which disabled other parts of .bashrc, and finally there was a problem with the expansion of the $rvm_home variable... Net result is that RVM wasn't always working correctly after install

#2 - the automated setup process started vcap for you, but did it in a shell that was properly prepped for rvm. Once the install exits, you are left in the original shell, but that shell did not have rmv installed correctly and as a result is unable to run bin/vcap stop. Simplest solution is to restart the shell. I usually run this process by ssh into the vm so after the install, I just drop the ssh and reconnect in a new shell. this sets up rvm correctly and then cd cloudfoundry/vcap will prep your environment. the pull request no longer starts vcap for you so it feels more natural to run setup, log off and log back in. alternatively you could just source .bash_profile to get your rvm set correctly

#3 - The rake 0.9.0 change causes major issues. Its not a trivial thing to just gem uninstall it due to the fact that it's installed using rvm .default and .global gemsets. The pull request dances around this by removing rake from those global gemsets and then in its place gem install's 0.8.7 into each ruby environment. 

网上的答案3：

Hi Miguel,

  Please open up a new Terminal window/shell, which will source in the modified .bashrc and will

  setup rvm environment.   Then try doing a bin/vcap start.

  We need ruby1.9.2 in the environment, which is set by rvm.

  For example, when I traverse down to vcap directory, I see the following :

$ cd /cloudfoundry/vcap
Using /Users/svaiyapuri/.rvm/gems/ruby-1.9.2-p180
This is ensured by .rvmrc file in that directory, which invokes rvm to use 1.9.2, nats is installed under 1.9.2 gem tree

$ cat .rvmrc
rvm use 1.9.2-p180
 

  Best Regards,

  -senthil

当然这三个方法都没办法解决我的问题了，我搜了rvm是什么，然后费劲心思装上它，结果发现不会用。然后还想办法去修改ruby的版本，但是我后来发现以前部署单节点的时候ruby版本都没出现问题，所以慢慢的我就否定这三个答案了，可谓千辛万苦，但是最后的答案又是什么？我至今才开始学ruby on rails，所以它给的提示信息我只能猜着看了。

Executing /home/vm/cloudfoundry/.deployments/mysql1/deploy/rubies/ruby-1.9.2-p180/bin/ruby /home/vm/cloudfoundry/vcap/dev_setup/bin/vcap start mysql_node -c /home/vm/cloudfoundry/.deployments/mysql1/config -v /home/vm/cloudfoundry -l /home/vm/cloudfoundry/.deployments/mysql1/log
/home/vm/cloudfoundry/.deployments/mysql1/deploy/rubies/ruby-1.9.2-p180/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require': no such file to load -- nats/client (LoadError)

首先是：我们使用的ruby是

/home/vm/cloudfoundry/.deployments/mysql1/deploy/rubies/ruby-1.9.2-p180/bin/ruby

那说明肯定不是ruby版本问题了，所以上面的答案基本可以确定是错的了：然后看它的执行的是什么：

 /home/vm/cloudfoundry/vcap/dev_setup/bin/vcap 

我们可以看这个vcap是什么：

#!/usr/bin/env ruby
# Copyright (c) 2009-2011 VMware, Inc.
#
# Usage: bin/vcap_system [start|stop|restart|tail|status] [component ...]
#

require 'yaml'
require 'fileutils'
require 'optparse'

require 'rubygems'
require 'eventmachine'
require 'nats/client'

require File.expand_path(File.join("..", "lib", "vcap_components"), File.dirname(__FILE__))

ruby别的看不懂，这部分还是看的懂的，类似于C地下的包含头文件嘛，然后我想起来这篇博客：http://blog.csdn.net/cherry_sun/article/details/7711913

然后继续往下走：

mysql_node

这个就是启动mysql的意思了，然后继续往下，

 -c /home/vm/cloudfoundry/.deployments/mysql1/config -v /home/vm/cloudfoundry -l /home/vm/cloudfoundry/.deployments/mysql1/log

参数不难猜，但是如果想看的细致一些，我们可以到刚刚的vcap文件底下找答案：

$config_dir ||= ENV['CLOUD_FOUNDRY_CONFIG_PATH']
args = ARGV.dup
opts_parser = OptionParser.new do |opts|
  opts.on('--port PORT')                           { |port| $port = port.to_i }
  opts.on('--configdir CONFIGDIR', '-c CONFIGDIR') { |dir| $config_dir = File.expand_path(dir.to_s) }
  opts.on('--config CONFIGDIR')                    { |dir| $config_dir = File.expand_path(dir.to_s) }
  opts.on('--vcapdir VCAP_DIR', '-v VCAP_DIR')     { |dir| $vcap_home = File.expand_path(dir.to_s) }
  opts.on('--logdir LOG_DIR', '-l LOG_DIR')        { |dir| $log_dir = File.expand_path(dir.to_s) }
  opts.on('--no-color', '--nocolor', '--nc')       { $nocolor = true }
  opts.on('--noprompt', '-n')                      { $noprompt = true }
end

这样一看，基本就懂是什么意思了，-c是读取配置文件，这样我们就知道了mysql配置文件的位置；然后是-v是vcap的目录位置，这个我们都懂，不多说；然后是-l就是日志文件，那这样我们就知道mysql的日志文件在哪里了。虽然我们遇到的问题没有解决，但是我们从这里面已经知道了一些基本的东西，不久以后我们就会用到这些东西，这部分没看。

好了，分析到这我能知道的就是，我就是在执行vcap的时候没有找到这个“nats/client”这个库（？我不知道怎么说，在我眼里就是一个函数库）。

是不是感觉还是一头雾水，我当时就是这样的，就这样这个问题我卡了1天。

好了，我们好好回忆一下以前是不是有类似的问题，在上次单节点部署排错分析的时候我们遇到过一个问题，我们缺少一个头文件，结果编译就不成功。而ruby这种是不需要进行编译的，这两者之间有什么联系呢？我的感觉就是，说白了我没有装好nats这个库，然后现在问题就出来了，这个库应该装在什么地方？去哪里找？网上真的可以找的到么？一系列的问题让我们又陷入困境。

就在这时候，我突然灵光一闪（真的就是灵光一闪，有时候解决问题就是这样），我想起来vcap/services这个目录，我安装的东西在这个底下应该要能够找到。

解决方法：当然，老天保佑，果然是因为service这个目录下是空的导致我启动mysql失败了，因为贪图方便，我直接将整个service目录下的文件全部clone过来了。但是不管白猫黑猫，能抓老鼠就是好猫，这个问题总算解决了：

vm@vm:~/cloudfoundry/vcap/dev_setup/bin$ ./vcap_dev status
Targeting deployment "mysql1" with cloudfoundry home "/home/vm/cloudfoundry"
Using cloudfoundry config from /home/vm/cloudfoundry/.deployments/mysql1/config
Executing /home/vm/cloudfoundry/.deployments/mysql1/deploy/rubies/ruby-1.9.2-p180/bin/ruby /home/vm/cloudfoundry/vcap/dev_setup/bin/vcap status mysql_node -c /home/vm/cloudfoundry/.deployments/mysql1/config -v /home/vm/cloudfoundry -l /home/vm/cloudfoundry/.deployments/mysql1/log
mysql_node                    :	 RUNNING

问题1：VMC无法登陆

问题描述如下：大致情况就是，我可以注册，但是在登陆过程中出现了404错误。

root@arch3:~/cloudfoundry/vcap/dev_setup/bin# vmc target api.vcap.me
Successfully targeted to [http://api.vcap.me]
root@arch3:~/cloudfoundry/vcap/dev_setup/bin# vmc register
Email: [email protected]
Password: ***
Verify Password: ***
Creating New User: OK
Attempting login to [http://api.vcap.me]
Problem with login to 'http://api.vcap.me', 404: entity not found or
inaccessible, try again or register for an account.

解决方法：这个问题我找到的答案就是-将vmc重0.3.21版本降级到0.3.18版本，为什么会这样，我就不明白了。

问题2：部署的应用无法启动：

问题描述如下：这个是从网上copy下来的，当时情况是我执行vmc push hello，然后在Staging Application的时候出现了下面的这个错误。

root@ubuntu:/home/cloudfoundry# vmc start hello

Staging Application: .............Error (JSON 504):