ceph源码分析--ceph命令解析（终端敲命令之后发生的事）

在一次开发组内讨论中，那次是admin_socket的后台部分的课程分享。分享中同事提出了一个疑问。既然讲了admin_socket的的后台部分，那前端输入命令到底是怎么去调用的后台呢，或者前台的命令到底是怎么发送的呢？平时的ceph命令到底是怎么解析的呢？讨论会后带着这些疑问，看起了ceph代码。

在终端敲入了ceph stastus，终端就返回了集群的状态。那在这个过程中，到底是调用了什么。

主要涉及了
ceph-12.2.2\src\ceph.in
ceph-12.2.2\src\pybind\ceph_argparse.py
ceph-12.2.2\src\pybind\rados\rados.pyx

这里的pybind目录基本ceph所有的python脚本都在该目录下。

首先ceph命令都是由/usr/bin/ceph来执行，打开该文件发现内容与ceph.in相同。而在ceph.in中import了rados，

import rados
···
cluster_handle = run_in_thread(rados.Rados,name=name,
    clustername=clustername,conf_defaults=conf_defaults,
    conffile=conffile)
···

后续中有

json_command(cluster_handle, target=target,
             argdict=valid_dict,inbuf=inbuf)

该函数定义在ceph_argparse.py
且调用了

def send_command_retry(*args, **kwargs):
    while True:
        try:
            return send_command(*args, **kwargs)
        except Exception as e:
            if ('get_command_descriptions' in str(e) and
                'object in state configuring' in str(e)):
                continue
            else:
                raise

然后调用了

def send_command(cluster, target=('mon', ''), cmd=None,
                  inbuf=b'', timeout=0,verbose=False):
···
   if len(target) < 2 or target[1] == '': 
      ret, outbuf, outs = run_in_thread(cluster.mon_command,
                                  cmd, inbuf, timeout)
   else:
      ret, outbuf, outs = run_in_thread(cluster.mon_command,
                           cmd, inbuf, timeout, target[1])
···                  

调用了rados类中的mon_command()

cdef class Rados(object):
    """This class wraps librados functions"""
    # NOTE(sileht): attributes declared in .pyd

    def __init__(self, *args, **kwargs):
        PyEval_InitThreads()
           self.__setup(*args, **kwargs)
    def mon_command(self, cmd, inbuf, timeout=0, 
                    target=None):
    ···
    try:
        if target:
             with nogil:
                  ret =rados_mon_command_target(self.cluster,
                   _target,_cmd,
                   _cmdlen,_inbuf,
                   _inbuf_len,&_outbuf, &_outbuf_len,
                   &_outs, &_outs_len)
        else:
             with nogil:
                  ret = rados_mon_command(self.cluster,
                      _cmd, _cmdlen,
                      _inbuf, _inbuf_len,
                      &_outbuf, &_outbuf_len,&_outs,
                      &_outs_len)
    ···

根据rados.pyx头文件中

cdef extern from "rados/librados.h" nogil:
····
int rados_mon_command(rados_t cluster, const char **cmd,
                     size_t cmdlen,const char *inbuf, 
                     size_t inbuflen,char **outbuf, 
                     size_t *outbuflen,char **outs, 
                     size_t *outslen)
···

千辛万苦到了ceph的C++代码部分啦

这波骚操作令人窒息。笔者作为小菜鸡，第一次看通的时候也是长舒了一口气。

下面是常规操作。

extern "C" int rados_mon_command(rados_t cluster, 
                            const char **cmd,size_t cmdlen,
                            const char *inbuf, 
                            size_t inbuflen,char **outbuf,
                            size_t *outbuflen,char **outs,
                            size_t *outslen)
{
  tracepoint(librados, rados_mon_command_enter, cluster, cmdlen, inbuf, inbuflen);
  librados::RadosClient *client = (librados::RadosClient*)
                                cluster;
  bufferlist inbl;
  bufferlist outbl;
  string outstring;
  vector<string> cmdvec;

  for (size_t i = 0; i < cmdlen; i++) {
    tracepoint(librados, rados_mon_command_cmd, cmd[i]);
    cmdvec.push_back(cmd[i]);
  }

  inbl.append(inbuf, inbuflen);
  int ret = client->mon_command(cmdvec, inbl, &outbl, &outstring);

  do_out_buffer(outbl, outbuf, outbuflen);
  do_out_buffer(outstring, outs, outslen);
  tracepoint(librados, rados_mon_command_exit, ret, outbuf, 
              outbuflen, outs, outslen);
  return ret;
}

在最新版本中mgr负责了一些诸如ceph osd df命令的解析。mgr承担了一部分原本由monitor来完成的事情
在下面的函数里也进行了对命令的处理

void Monitor::handle_command(MonOpRequestRef op)