OpenStack Cinder服务启动过程中的资源加载和扩展源码解析之一

感谢朋友支持本博客，欢迎共同探讨交流，由于能力和时间有限，错误之处在所难免，欢迎指正！
如果转载，请保留作者信息。
博客地址：http://blog.csdn.net/gaoxingnengjisuan
邮箱地址：[email protected]

在OpenStack Cinder服务启动的过程中，对/cinder/api/contrib/和/cinder/api/v1/和/cinder/api/v2/下定义的资源功能进行了加载，那么具体是如何实现资源的加载，并且如何进行自定义资源功能的实现呢，这篇博客将会进行具体的分析。

在前面的博客cinder服务启动源码分析中，我们知道在服务类cinder.service.WSGIService的初始化过程中，有这样的源码实现：

    class WSGIService(object):  
        """ 
        Provides ability to launch API from a 'paste' configuration. 
        实现启动API的类； 
        """  
      
        def __init__(self, name, loader=None):  
            """ 
            Initialize, but do not start the WSGI server. 
            类的初始化方法，但是不启动服务； 
     
            :param name: The name of the WSGI server given to the loader. 
            给定的要加载的服务的名称； 
            :param loader: Loads the WSGI application using the given name. 
            :returns: None 
     
            """  
            # 给定的要加载的服务的名称；  
            # osapi_volume  
            self.name = name  
            # 根据给定的服务名称导入对应的管理类；  
            self.manager = self._get_manager()  
            self.loader = loader or wsgi.Loader()  
            self.app = self.loader.load_app(name)  
            # 获取主机host；  
            self.host = getattr(CONF, '%s_listen' % name, "0.0.0.0")  
            # 获取主机端口；  
            self.port = getattr(CONF, '%s_listen_port' % name, 0)  
            # Server：管理WSGI服务的类；  
            self.server = wsgi.Server(name,  
                                      self.app,  
                                      host=self.host,  
                                      port=self.port)  

对于语句：

self.loader = loader or wsgi.Loader()

self.app = self.loader.load_app(name)

之前我们没有进行深入的分析，实际上，这两条语句就实现了模块功能的扩展和加载过程。我们来看源码：

class Loader(object):
    """
    Used to load WSGI applications from paste configurations.   
    """

    def __init__(self, config_path=None):
        """Initialize the loader, and attempt to find the config.

        :param config_path: Full or relative path to the paste config.
        :returns: None

        """
        # 此处指向/etc/cinder/api-paste.ini
        config_path = config_path or CONF.api_paste_config
        # 获取config_path的全路径；
        self.config_path = utils.find_config(config_path)

def load_app(self, name):
        """
        Return the paste URLMap wrapped WSGI application.

        :param name: Name of the application to load.
        :returns: Paste URLMap object wrapping the requested application.
        :raises: `cinder.exception.PasteAppNotFound`
        加载osapi_volume程序；
        """
        try:
            return deploy.loadapp("config:%s" % self.config_path, name=name)
        except LookupError as err:
            LOG.error(err)
            raise exception.PasteAppNotFound(name=name, path=self.config_path)

我们可以看到config_path指向/etc/cinder/api-paste.ini，之前我们在博客Paste Deployment简介以及cinder-api-paste.ini的解析（1）和Paste Deployment简介以及cinder-api-paste.ini的解析（2）中，已经进行过解析， cinder通过api-paste.ini中的配置信息来进行指定WSGI Application的实现 。这里可以看到，在方法load_app中，调用了模块库deploy来实现对 api-paste.ini 中配置信息的解析和指定 WSGI Application 的实现。

而通过上述两篇博客我们也可以知道，通过对api-paste.ini中配置信息的解析，最终就调用了WSGI Applicationapiv1的实现，具体就是cinder.api.v1.router:APIRouter.factory，这个WSGI Application的具体功能就是实现模块功能的扩展和加载过程，所以这里我们就深入分析cinder.api.v1.router:APIRouter.factory的实现过程。

这里调用的方法是/cinder/api/openstack/__init__.py----class APIRouter(base_wsgi.Router)----def factory：

def factory(cls, global_config, **local_config):
    return cls()

输出示例：

cls = <class 'cinder.api.v1.router.APIRouter'>
global_config = {'__file__': '/etc/cinder/api-paste.ini', 'here': '/etc/cinder'}
local_config = {}

所以我们来看类/cinder/api/v1/router.py----class APIRouter的具体实现：

class APIRouter(cinder.api.openstack.APIRouter):
    """
    Routes requests on the OpenStack API to the appropriate controller
    and method.
    """
    ExtensionManager = extensions.ExtensionManager

    def _setup_routes(self, mapper, ext_mgr):
        ......

进而来看其父类/cinder/api/openstack/__init__.py----class APIRouter的初始化方法:

class APIRouter(base_wsgi.Router):
    def __init__(self, ext_mgr=None):
        if ext_mgr is None:
            if self.ExtensionManager:
                ext_mgr = self.ExtensionManager() (1)
            else:
                raise Exception(_("Must specify an ExtensionManager class"))
        
        mapper = ProjectMapper()        
        self.resources = {}
        self._setup_routes(mapper, ext_mgr) (2)
        self._setup_ext_routes(mapper, ext_mgr) （3）
        self._setup_extensions(ext_mgr) （4）
        super(APIRouter, self).__init__(mapper) （5）

在这个类的初始化方法中，就实现了cinder 模块功能的扩展和加载过程，这里我们就对其中比较重要的5条语句 进行详细的解析，来看看 cinder模块功能的扩展和加载过程 的具体实现过程。

(1).ext_mgr = self.ExtensionManager()

这里实现的具体语句是类/cinder/api/v1/router.py----class APIRouter中的语句：

ExtensionManager = extensions.ExtensionManager

具体来看类ExtensionManager的初始化方法：

class ExtensionManager(object):
    """
    Load extensions from the configured extension path.

    See cinder/tests/api/extensions/foxinsocks/extension.py for an
    example extension implementation.
    """

    def __init__(self):
        LOG.audit(_('Initializing extension manager.'))
        self.extensions = {}
        self._load_extensions()

继续来看方法_load_extensions的实现：

def _load_extensions(self):
        """
        Load extensions specified on the command line.
        """
        # extensions=['cinder.api.contrib.standard_extensions']
        extensions = list(self.cls_list)

        old_contrib_path = ('cinder.api.openstack.volume.contrib.'
                            'standard_extensions')
        new_contrib_path = 'cinder.api.contrib.standard_extensions'
        if old_contrib_path in extensions:
            extensions = [e.replace(old_contrib_path, new_contrib_path)
                          for e in extensions]

        for ext_factory in extensions:
            try:
                self.load_extension(ext_factory)
            except Exception as exc:
                LOG.warn(......)

这里最重要的语句就是：

for ext_factory in extensions:
    try:
        self.load_extension(ext_factory)
    except Exception as exc:
        LOG.warn(......)

输出示例：

extensions=['cinder.api.contrib.standard_extensions']

继续来看方法load_extension的实现源码：

def load_extension(self, ext_factory):
    """
    ext_factory=['cinder.api.contrib.standard_extensions']
    """

    # ext_factory = cinder.api.contrib.standard_extensions
    factory = importutils.import_class(ext_factory)
    factory(self)

这里ext_factory = cinder.api.contrib.standard_extensions，所以调用的是方法/cinder/api/contrib/__init__.py----def standard_extensions；

我们继续来看方法：

def standard_extensions(ext_mgr):
    extensions.load_standard_extensions(ext_mgr, LOG, __path__, __package__)

def load_standard_extensions(ext_mgr, logger, path, package, ext_list=None):
    """
    Registers all standard API extensions.

    ext_mgr = <cinder.api.extensions.ExtensionManager object at 0x1724190>
    logger = <cinder.openstack.common.log.ContextAdapter instance at 0x172c830>
    path = ['/usr/lib/python2.6/site-packages/cinder/api/contrib']
    package = cinder.api.contrib
    ext_list = None
    """

    # our_dir = path[0] = /usr/lib/python2.6/site-packages/cinder/api/contrib
    our_dir = path[0]
    for dirpath, dirnames, filenames in os.walk(our_dir):      
        # Compute the relative package name from the dirpath
        relpath = os.path.relpath(dirpath, our_dir)
        
        if relpath == '.':
            relpkg = ''
        else:
            relpkg = '.%s' % '.'.join(relpath.split(os.sep))

        # Now, consider each file in turn, only considering .py files
        for fname in filenames:
            
            root, ext = os.path.splitext(fname)

            # Skip __init__ and anything that's not .py
            if ext != '.py' or root == '__init__':
                continue

            # Try loading it
            classname = "%s%s" % (root[0].upper(), root[1:])
            
            classpath = ("%s%s.%s.%s" % (package, relpkg, root, classname))

            if ext_list is not None and classname not in ext_list:
                logger.debug("Skipping extension: %s" % classpath)
                continue

            try:
                ext_mgr.load_extension(classpath)
            except Exception as exc:
                logger.warn(_('Failed to load extension %(classpath)s: '
                              '%(exc)s'),
                            {'classpath': classpath, 'exc': exc})

        # Now, let's consider any subdirectories we may have...
        subdirs = []
        for dname in dirnames:            
            # Skip it if it does not have __init__.py
            if not os.path.exists(os.path.join(dirpath, dname, '__init__.py')):
                continue
 
            ext_name = ("%s%s.%s.extension" %
                        (package, relpkg, dname))
            try:
                ext = importutils.import_class(ext_name)
            except ImportError:
                # extension() doesn't exist on it, so we'll explore
                # the directory for ourselves
                subdirs.append(dname)
            else:
                try:
                    ext(ext_mgr)
                except Exception as exc:
                    logger.warn(_('Failed to load extension %(ext_name)s: %(exc)s), {'ext_name': ext_name, 'exc': exc})

        # Update the list of directories we'll explore...
        dirnames[:] = subdirs

我们来看这个方法的实现，首先来看两个变量的输出示例：

our_dir = path[0] = /usr/lib/python2.6/site-packages/cinder/api/contrib

filenames = [
'volume_tenant_attribute.pyc', 'volume_type_encryption.py', 'volume_host_attribute.pyc', 'snapshot_actions.py', 'types_extra_specs.pyo', 'volume_actions.pyc', 'getname.pyc', 'volume_type_encryption.pyo', 'snapshot_actions.pyo', 'quota_classes.pyc', 'getname.py', 'services.pyc', 'image_create.py', 'snapshot_actions.pyc', 'services.pyo', 'image_create.pyo', 'volume_encryption_metadata.py', 'qos_specs_manage.pyc', 'availability_zones.pyc', 'quota_classes.py', 'quotas.pyc', 'volume_transfer.pyo', 'volume_tenant_attribute.pyo', 'quota_classes.pyo', 'image_create.pyc', 'volume_actions.py~', 'scheduler_hints.py~', 'snapshot_actions.py~', 'hosts.py', 'volume_host_attribute.pyo', 'hosts.pyo', 'extended_snapshot_attributes.py~', 'volume_encryption_metadata.py~', 'quotas.pyo', 'types_extra_specs.py', 'volume_image_metadata.pyc', 'volume_type_encryption.pyc', 'volume_image_metadata.pyo', 'volume_mig_status_attribute.py', 'scheduler_hints.py', 'volume_mig_status_attribute.pyo', 'qos_specs_manage.py~', 'services.py', 'volume_host_attribute.py', 'volume_encryption_metadata.pyc', 'hosts.pyc', 'scheduler_hints.pyo', '__init__.py', 'volume_encryption_metadata.pyo', 'volume_mig_status_attribute.pyc', 'backups.py~', 'volume_host_attribute.py~', 'quota_classes.py~', 'extended_snapshot_attributes.pyc', 'availability_zones.pyo', 'quotas.py', 'volume_type_encryption.py~', 'volume_transfer.pyc', 'volume_mig_status_attribute.py~', 'scheduler_hints.pyc', 'backups.pyc', 'volume_tenant_attribute.py~', '__init__.pyc', 'hosts.py~', 'backups.pyo', 'types_extra_specs.pyc', 'qos_specs_manage.py', 'types_manage.py~', 'admin_actions.pyo', 'volume_tenant_attribute.py', 'volume_actions.pyo', 'types_extra_specs.py~', 'admin_actions.pyc', 'extended_snapshot_attributes.pyo', 'qos_specs_manage.pyo', 'volume_image_metadata.py', '__init__.py~', 'types_manage.pyo', 'availability_zones.py', 'backups.py', 'volume_actions.py', 'types_manage.py', 'types_manage.pyc', 'extended_snapshot_attributes.py', '__init__.pyo', 'services.py~', 'volume_image_metadata.py~', 'availability_zones.py~', 'quotas.py~', 'volume_transfer.py~', 'admin_actions.py', 'volume_transfer.py']

从而我们可以知道，这个方法中会遍历路径/usr/lib/python2.6/site-packages/cinder/api/contrib，调用方法ext_mgr.load_extension加载路径/usr/lib/python2.6/site-packages/cinder/api/contrib下每个以.py后缀结尾的文件对应的类，如：

cinder.api.contrib.volume_type_encryption.Volume_type_encryption
cinder.api.contrib.snapshot_actions.Snapshot_actions
cinder.api.contrib.getname.Getname
cinder.api.contrib.image_create.Image_create
cinder.api.contrib.volume_encryption_metadata.Volume_encryption_metadata
cinder.api.contrib.quota_classes.Quota_classes
cinder.api.contrib.hosts.Hosts
cinder.api.contrib.types_extra_specs.Types_extra_specs
cinder.api.contrib.volume_mig_status_attribute.Volume_mig_status_attribute
cinder.api.contrib.scheduler_hints.Scheduler_hints
cinder.api.contrib.services.Services
cinder.api.contrib.volume_host_attribute.Volume_host_attribute
cinder.api.contrib.quotas.Quotas
cinder.api.contrib.qos_specs_manage.Qos_specs_manage
cinder.api.contrib.volume_tenant_attribute.Volume_tenant_attribute
cinder.api.contrib.volume_image_metadata.Volume_image_metadata
cinder.api.contrib.availability_zones.Availability_zones
cinder.api.contrib.backups.Backups
cinder.api.contrib.volume_actions.Volume_actions
cinder.api.contrib.types_manage.Types_manage
cinder.api.contrib.extended_snapshot_attributes.Extended_snapshot_attributes
cinder.api.contrib.admin_actions.Admin_actions
cinder.api.contrib.volume_transfer.Volume_transfer

从源码中可以看到，这里再次调用了方法load_extension：

def load_extension(self, ext_factory):
    """
    Execute an extension factory.
    """
        
    LOG.debug(_("Loading extension %s"), ext_factory)

    # Load the factory
    factory = importutils.import_class(ext_factory)

    # Call it
    LOG.debug(_("Calling extension factory %s"), ext_factory)
    factory(self)

这个方法中factory(self)会调用上述各个类，进一步看源码，发现上述各个类中都没有实现def __init__方法，都要调用其父类中的def __init__方法；

以cinder.api.contrib.backups.Backups为例：

class Backups(extensions.ExtensionDescriptor):
    """Backups support."""
    name = 'Backups'
    alias = 'backups'
    namespace = 'http://docs.openstack.org/volume/ext/backups/api/v1'
    updated = '2012-12-12T00:00:00+00:00'

    def get_resources(self):
        resources = []
        res = extensions.ResourceExtension(
            Backups.alias, 
            BackupsController(),
            collection_actions={'detail': 'GET'},
            member_actions={'restore': 'POST'})
        resources.append(res)
        return resources

这里先会实现对变量name，alias，namespace，updated的赋值，然后会调用父类ExtensionDescriptor中的__init__方法。

class ExtensionDescriptor(object):
    """
    Base class that defines the contract for extensions.
    Note that you don't have to derive from this class to have a valid
    extension; it is purely a convenience.
    API扩展描述的基类；
    """

    name = None
    alias = None
    namespace = None
    updated = None

    def __init__(self, ext_mgr):
        """
        Register extension with the extension manager.
        """
        ext_mgr.register(self)

def register(self, ext):
    # Do nothing if the extension doesn't check out
    if not self._check_extension(ext):
        return

    alias = ext.alias
    LOG.audit(_('Loaded extension: %s'), alias)

    if alias in self.extensions:
        raise exception.Error("Found duplicate extension: %s" % alias)
        
    self.extensions[alias] = ext

以cinder.api.contrib.volume_image_metadata.Volume_image_metadata为例，这里就是把变量alias = os-vol-image-meta和类 cinder.api.contrib.volume_image_metadata.Volume_image_metadata 一一对应起来；

再以cinder.api.contrib.backups.Backups为例，这里就是把变量alias = backups和类cinder.api.contrib.backups.Backups一一对应起来；