openstack cinder snapshot快照源码分析

cinder snapshot快照源码分析

源码下载地址：
https://releases.openstack.org/index.html

---

1. 介绍

Cinder是OpenStack Block Storage服务，用于为Nova虚拟机，Ironic裸机主机，容器等提供卷。Cinder的一些目标是：

基于组件的体系结构：快速添加新行为

高度可用：扩展到非常严重的工作负载

容错：隔离的进程可以避免级联故障

---

2.命令行

命令行查看功能：

[root@cinder ~]# cinder -h | grep snap
    cgsnapshot-create   Creates a cgsnapshot.
    cgsnapshot-delete   Removes one or more cgsnapshots.
    cgsnapshot-list     Lists all cgsnapshots.
    cgsnapshot-show     Shows cgsnapshot details.
                        Creates a consistency group from a cgsnapshot or a
    snapshot-create     Creates a snapshot.
    snapshot-delete     Removes one or more snapshots.
    snapshot-list       Lists all snapshots.
    snapshot-manage     Manage an existing snapshot.
    snapshot-manageable-list
                        Lists all manageable snapshots.
    snapshot-metadata   Sets or deletes snapshot metadata.
    snapshot-metadata-show
                        Shows snapshot metadata.
    snapshot-metadata-update-all
                        Updates snapshot metadata.
    snapshot-rename     Renames a snapshot.
    snapshot-reset-state
                        Explicitly updates the snapshot state.
    snapshot-show       Shows snapshot details.
    snapshot-unmanage   Stop managing a snapshot.
[root@cinder ~]# 
[root@cinder ~]# 

查看命令详情帮助

[root@cinder ~]# cinder help snapshot-create
usage: cinder snapshot-create [--force []] [--name ]
                              [--description ]
                              [--metadata [ [ ...]]]
                              

Creates a snapshot.

Positional arguments:
                Name or ID of volume to snapshot.

Optional arguments:
  --force []
                        Allows or disallows snapshot of a volume when the
                        volume is attached to an instance. If set to True,
                        ignores the current status of the volume when
                        attempting to snapshot it rather than forcing it to be
                        available. Default=False.
  --name          Snapshot name. Default=None.
  --description 
                        Snapshot description. Default=None.
  --metadata [ [ ...]]
                        Snapshot metadata key and value pairs. Default=None.
[root@cinder ~]# 

---

3. 源码分析

从最新版本开始查找，目前为V3版本：

cinder snapshot快照功能的API入口在文件cinder/api/v3/snapshots.py

快照功能定义类：

class SnapshotsController(snapshots_v2.SnapshotsController):
    """The Snapshots API controller for the OpenStack API."""

    _view_builder_class = snapshot_views.ViewBuilder

实际使用v2版本， 源码文件在cinder/api/v2/snapshots.py

class SnapshotsController(wsgi.Controller):
    """The Snapshots API controller for the OpenStack API."""

    _view_builder_class = snapshot_views.ViewBuilder
    
    def __init__(self, ext_mgr=None):
        self.volume_api = volume.API()
        self.ext_mgr = ext_mgr
        super(SnapshotsController, self).__init__()

一. show 详情功能：

    def show(self, req, id):
        """Return data about the given snapshot."""
        context = req.environ['cinder.context']
    
        # Not found exception will be handled at the wsgi level
        snapshot = self.volume_api.get_snapshot(context, id)
        req.cache_db_snapshot(snapshot)
    
        return self._view_builder.detail(req, snapshot)

获取信息self.volume_api.get_snapshot(context, id)， 具体实现在API中，
定义在文件： cinder/volume/api.py

class API(base.Base):
    """API for interacting with the volume manager."""

		.....
		
	def get_snapshot(self, context, snapshot_id):
	    check_policy(context, 'get_snapshot')
	    snapshot = objects.Snapshot.get_by_id(context, snapshot_id)
	
	    # FIXME(jdg): The objects don't have the db name entries
	    # so build the resource tag manually for now.
	    LOG.info(_LI("Snapshot retrieved successfully."),
	             resource={'type': 'snapshot',
	                       'id': snapshot.id})
	    return snapshot

objects.Snapshot.get_by_id(context, snapshot_id)查询db，获取快照信息

可以看到该功能基本是直接获取db的操作， 比较简单

二. list， update与show差不多， 基本都是操作db信息， 这里不做分析

---

三. 创建create创建快照功能：

@wsgi.response(202)
    def create(self, req, body):
        """Creates a new snapshot."""
        kwargs = {}
        
#获取并校验参数信息        
        context = req.environ['cinder.context']

        self.assert_valid_body(body, 'snapshot')
    
        snapshot = body['snapshot']
        kwargs['metadata'] = snapshot.get('metadata', None)
    
        try:
            volume_id = snapshot['volume_id']
        except KeyError:
            msg = _("'volume_id' must be specified")
            raise exc.HTTPBadRequest(explanation=msg)


#获取并检查volume卷
        volume = self.volume_api.get(context, volume_id)
        force = snapshot.get('force', False)
        msg = _LI("Create snapshot from volume %s")
        LOG.info(msg, volume_id)
        self.validate_name_and_description(snapshot)

        # NOTE(thingee): v2 API allows name instead of display_name
        if 'name' in snapshot:
            snapshot['display_name'] = snapshot.pop('name')
    
        try:
            force = strutils.bool_from_string(force, strict=True)
        except ValueError as error:
            err_msg = encodeutils.exception_to_unicode(error)
            msg = _("Invalid value for 'force': '%s'") % err_msg
            raise exception.InvalidParameterValue(err=msg)
    
        if force:
            new_snapshot = self.volume_api.create_snapshot_force(
                context,
                volume,
                snapshot.get('display_name'),
                snapshot.get('description'),
                **kwargs)
        else:
            new_snapshot = self.volume_api.create_snapshot(
                context,
                volume,
                snapshot.get('display_name'),
                snapshot.get('description'),
                **kwargs)
        req.cache_db_snapshot(new_snapshot)
    
        return self._view_builder.detail(req, new_snapshot)

创建快照：

  new_snapshot = self.volume_api.create_snapshot(
      context,
      volume,
      snapshot.get('display_name'),
      snapshot.get('description'),
      **kwargs)

功能定义在API的create_snapshot ， 位置在： cinder/volume/api.py

    def create_snapshot(self, context,
                        volume, name, description,
                        metadata=None, cgsnapshot_id=None,
                        group_snapshot_id=None):
        result = self._create_snapshot(context, volume, name, description,
                                       False, metadata, cgsnapshot_id,
                                       group_snapshot_id)
        LOG.info(_LI("Snapshot create request issued successfully."),
                 resource=result)
        return result

​

其中实际调用_create_snapshot实现：

    def _create_snapshot(self, context,
                         volume, name, description,
                         force=False, metadata=None,
                         cgsnapshot_id=None,
                         group_snapshot_id=None):
# 创建db信息
        snapshot = self.create_snapshot_in_db(
            context, volume, name,
            description, force, metadata, cgsnapshot_id,
            True, group_snapshot_id)
        
        调用驱动创建快照
        self.volume_rpcapi.create_snapshot(context, volume, snapshot)
    
        return snapshot

self.volume_rpcapi.create_snapshot(context, volume, snapshot)进行远程rpc调用
最后实际的处理方法create_snapshot， 定义在nova/volume/manager.py中：

搜索create_snapshot

定义：

    def create_snapshot(self, context, volume_id, snapshot):
        """Creates and exports the snapshot."""
        context = context.elevated()
    
        self._notify_about_snapshot_usage(
            context, snapshot, "create.start")
    
        try:
            # NOTE(flaper87): Verify the driver is enabled
            # before going forward. The exception will be caught
            # and the snapshot status updated.
            utils.require_driver_initialized(self.driver)
    
            # Pass context so that drivers that want to use it, can,
            # but it is not a requirement for all drivers.
            snapshot.context = context

# 调用driver创建快照
            model_update = self.driver.create_snapshot(snapshot)


# 更新快照信息
            if model_update:
                snapshot.update(model_update)
                snapshot.save()
    
        except Exception:
            with excutils.save_and_reraise_exception():
                snapshot.status = fields.SnapshotStatus.ERROR
                snapshot.save()
    
        vol_ref = self.db.volume_get(context, snapshot.volume_id)
        if vol_ref.bootable:
            try:
                self.db.volume_glance_metadata_copy_to_snapshot(
                    context, snapshot.id, snapshot.volume_id)
            except exception.GlanceMetadataNotFound:
                # If volume is not created from image, No glance metadata
                # would be available for that volume in
                # volume glance metadata table
                pass
            except exception.CinderException as ex:
                LOG.exception(_LE("Failed updating snapshot"
                                  " metadata using the provided volumes"
                                  " %(volume_id)s metadata"),
                              {'volume_id': snapshot.volume_id},
                              resource=snapshot)
                snapshot.status = fields.SnapshotStatus.ERROR
                snapshot.save()
                raise exception.MetadataCopyFailure(reason=six.text_type(ex))


#更新snapshot信息

        snapshot.status = fields.SnapshotStatus.AVAILABLE
        snapshot.progress = '100%'
        snapshot.save()
    
        self._notify_about_snapshot_usage(context, snapshot, "create.end")
        LOG.info(_LI("Create snapshot completed successfully"),
                 resource=snapshot)
        return snapshot.id

其中最关键的步骤是调用driver信息创建snapshot：

model_update = self.driver.create_snapshot(snapshot)

在cinder/volume/drivers/ 下寻找对应的驱动文件

比如：
使用本地LVM， 则对应的驱动文件为cinder/volume/drivers/lvm.py

定义如下：

    def create_snapshot(self, snapshot):
        """Creates a snapshot."""

        self.vg.create_lv_snapshot(self._escape_snapshot(snapshot['name']),
                                   snapshot['volume_name'],
                                   self.configuration.lvm_type)

create_lv_snapshot实际使用的是操作系统的vg管理命令

定义如下：

 @utils.retry(putils.ProcessExecutionError)
 def create_lv_snapshot(self, name, source_lv_name, lv_type='default'):
     """Creates a snapshot of a logical volume.

     :param name: Name to assign to new snapshot
     :param source_lv_name: Name of Logical Volume to snapshot
     :param lv_type: Type of LV (default or thin)
    
     """
     source_lvref = self.get_volume(source_lv_name)
     if source_lvref is None:
         LOG.error(_LE("Trying to create snapshot by non-existent LV: %s"),
                   source_lv_name)
         raise exception.VolumeDeviceNotFound(device=source_lv_name)
     cmd = LVM.LVM_CMD_PREFIX + ['lvcreate', '--name', name, '--snapshot',
                                 '%s/%s' % (self.vg_name, source_lv_name)]
     if lv_type != 'thin':
         size = source_lvref['size']
         cmd.extend(['-L', '%sg' % (size)])
    
     try:
         self._execute(*cmd,
                       root_helper=self._root_helper,
                       run_as_root=True)
     except putils.ProcessExecutionError as err:
         LOG.exception(_LE('Error creating snapshot'))
         LOG.error(_LE('Cmd     :%s'), err.cmd)
         LOG.error(_LE('StdOut  :%s'), err.stdout)
         LOG.error(_LE('StdErr  :%s'), err.stderr)
         raise

_execute以root权限执行命令

cmd = LVM.LVM_CMD_PREFIX + ['lvcreate', '--name', name, '--snapshot',
                            '%s/%s' % (self.vg_name, source_lv_name)]

命令实际上使用的是lvcreate

lvcreate 创建, 可以直接在操作系统上查看功能

lvcreate -h

有关snapshot的功能：
  Create a COW snapshot LV of an origin LV.
  lvcreate -s|--snapshot -L|--size Size[m|UNIT] LV
        [ -l|--extents Number[PERCENT] ]
        [ -i|--stripes Number ]
        [ -I|--stripesize Size[k|UNIT] ]
        [ -c|--chunksize Size[k|UNIT] ]
        [    --type snapshot ]
        [ COMMON_OPTIONS ]
        [ PV ... ]

如果使用ceph存储， 则对接的驱动是RBD
对应驱动文件在cinder/volume/drivers/rbd.py

定义如下：

    def create_snapshot(self, snapshot):
        """Creates an rbd snapshot."""
        with RBDVolumeProxy(self, snapshot.volume_name) as volume:
            snap = utils.convert_str(snapshot.name)
            volume.create_snap(snap)
            volume.protect_snap(snap)

实际就是连接rbd， 调用对应的方法：
volume.create_snap(snap)
volume.protect_snap(snap)

对应调用create_snap 和protect_snap 方法

定义：

class RBDVolumeProxy(object):
    """Context manager for dealing with an existing rbd volume.

    This handles connecting to rados and opening an ioctx automatically, and
    otherwise acts like a librbd Image object.
    
    The underlying librados client and ioctx can be accessed as the attributes
    'client' and 'ioctx'.
    """
    def __init__(self, driver, name, pool=None, snapshot=None,
                 read_only=False):
        client, ioctx = driver._connect_to_rados(pool)
        if snapshot is not None:
            snapshot = utils.convert_str(snapshot)
    
        try:
            self.volume = driver.rbd.Image(ioctx,
                                           utils.convert_str(name),
                                           snapshot=snapshot,
                                           read_only=read_only)
        except driver.rbd.Error:
            LOG.exception(_LE("error opening rbd image %s"), name)
            driver._disconnect_from_rados(client, ioctx)
            raise
        self.driver = driver
        self.client = client
        self.ioctx = ioctx

---

四. 删除功能：

api入口： cinder/api/v2/snapshot.py

    def delete(self, req, id):
        """Delete a snapshot."""
        context = req.environ['cinder.context']
    
        LOG.info(_LI("Delete snapshot with id: %s"), id)
    
        # Not found exception will be handled at the wsgi level

# 获取快照信息， 并校验快照是否存在
        snapshot = self.volume_api.get_snapshot(context, id)

#删除快照
        self.volume_api.delete_snapshot(context, snapshot)

        return webob.Response(status_int=202)

API 在cinder/volume/api.py

    @wrap_check_policy
    def delete_snapshot(self, context, snapshot, force=False,
                        unmanage_only=False):
        # Build required conditions for conditional update
        expected = {'cgsnapshot_id': None,
                    'group_snapshot_id': None}
        # If not force deleting we have status conditions

# 如果不是进行强制删除， 则检察状态， 值允许available/error
        if not force:
            expected['status'] = (fields.SnapshotStatus.AVAILABLE,
                                  fields.SnapshotStatus.ERROR)
    
        result = snapshot.conditional_update(
            {'status': fields.SnapshotStatus.DELETING}, expected)
        if not result:
            status = utils.build_or_str(expected.get('status'),
                                        _('status must be %s and'))
            msg = (_('Snapshot %s must not be part of a group.') %
                   status)
            LOG.error(msg)
            raise exception.InvalidSnapshot(reason=msg)
    
        # Make RPC call to the right host
#获取volume卷信息
        volume = objects.Volume.get_by_id(context, snapshot.volume_id)
        
        
#volume.host  调用volume  host 主机的volume服务进行删除
        self.volume_rpcapi.delete_snapshot(context, snapshot, volume.host,
                                           unmanage_only=unmanage_only)
        LOG.info(_LI("Snapshot delete request issued successfully."),
                 resource=snapshot)

主机volume删除通过rpc进行调用

volume远程服务定义在cinder/volume/manager.py

   @coordination.synchronized('{snapshot.id}-{f_name}')
    def delete_snapshot(self, context, snapshot, unmanage_only=False):
        """Deletes and unexports snapshot."""
        context = context.elevated()
        snapshot._context = context
        project_id = snapshot.project_id

        self._notify_about_snapshot_usage(
            context, snapshot, "delete.start")
    
        try:
            # NOTE(flaper87): Verify the driver is enabled
            # before going forward. The exception will be caught
            # and the snapshot status updated.
            utils.require_driver_initialized(self.driver)
    
            # Pass context so that drivers that want to use it, can,
            # but it is not a requirement for all drivers.
            snapshot.context = context
            snapshot.save()
    
            if unmanage_only:
# unmanage 不进行管理， 进行信息剔除
                self.driver.unmanage_snapshot(snapshot)
            else:
# 删除快照信息
                self.driver.delete_snapshot(snapshot)
        except exception.SnapshotIsBusy:

#异常回滚，将snapshot的状态设置为available

            LOG.error(_LE("Delete snapshot failed, due to snapshot busy."),
                      resource=snapshot)
            snapshot.status = fields.SnapshotStatus.AVAILABLE
            snapshot.save()
            return
        except Exception:
            with excutils.save_and_reraise_exception():
                snapshot.status = fields.SnapshotStatus.ERROR_DELETING
                snapshot.save()
    
        # Get reservations
        try:

# 删除snapshot快照信息， 并调整quota信息

            if CONF.no_snapshot_gb_quota:
                reserve_opts = {'snapshots': -1}
            else:
                reserve_opts = {
                    'snapshots': -1,
                    'gigabytes': -snapshot.volume_size,
                }
            volume_ref = self.db.volume_get(context, snapshot.volume_id)
            QUOTAS.add_volume_type_opts(context,
                                        reserve_opts,
                                        volume_ref.get('volume_type_id'))
            reservations = QUOTAS.reserve(context,
                                          project_id=project_id,
                                          **reserve_opts)
        except Exception:
            reservations = None
            LOG.exception(_LE("Update snapshot usages failed."),
                          resource=snapshot)
        self.db.volume_glance_metadata_delete_by_snapshot(context, snapshot.id)
        snapshot.destroy()
        self._notify_about_snapshot_usage(context, snapshot, "delete.end")
    
        # Commit the reservations
        if reservations:
            QUOTAS.commit(context, reservations, project_id=project_id)
        LOG.info(_LI("Delete snapshot completed successfully"),
                 resource=snapshot)

self.driver.delete_snapshot(snapshot)调用驱动删除快照

驱动拥有多种， 驱动文件统一放在cinder/volume/dirvers/目录下

例如： 如果使用lvm 的vg管理功能则对应驱动文件为cinder/volume/dirvers/lvm.py

定义如下：

    def delete_snapshot(self, snapshot):
        """Deletes a snapshot."""
        
检察卷， 如果已经没有卷信息， 则直接返回true成功即可
        if self._volume_not_present(self._escape_snapshot(snapshot['name'])):
            # If the snapshot isn't present, then don't attempt to delete
            LOG.warning(_LW("snapshot: %s not found, "
                            "skipping delete operations"), snapshot['name'])
            LOG.info(_LI('Successfully deleted snapshot: %s'), snapshot['id'])
            return True

        # TODO(yamahata): zeroing out the whole snapshot triggers COW.
        # it's quite slow.
# 删除volume snapshot
        self._delete_volume(snapshot, is_snapshot=True)

_delete_volume 定义：

    def _delete_volume(self, volume, is_snapshot=False):
        """Deletes a logical volume."""
        if self.configuration.volume_clear != 'none' and \
                self.configuration.lvm_type != 'thin':
            self._clear_volume(volume, is_snapshot)
    
        name = volume['name']
        if is_snapshot:
            name = self._escape_snapshot(volume['name'])
        self.vg.delete(name)

vg delete 定义在cinder/brick/loacl_dev/lvm.py

    @utils.retry(putils.ProcessExecutionError)
    def delete(self, name):
        """Delete logical volume or snapshot.
    
        :param name: Name of LV to delete
    
        """
    
        def run_udevadm_settle():
            self._execute('udevadm', 'settle',
                          root_helper=self._root_helper, run_as_root=True,
                          check_exit_code=False)
    
        # LV removal seems to be a race with other writers or udev in
        # some cases (see LP #1270192), so we enable retry deactivation
        LVM_CONFIG = 'activation { retry_deactivation = 1} '
    
        try:
            self._execute(
                'lvremove',
                '--config', LVM_CONFIG,
                '-f',
                '%s/%s' % (self.vg_name, name),
                root_helper=self._root_helper, run_as_root=True)
        except putils.ProcessExecutionError as err:
            LOG.debug('Error reported running lvremove: CMD: %(command)s, '
                      'RESPONSE: %(response)s',
                      {'command': err.cmd, 'response': err.stderr})
    
            LOG.debug('Attempting udev settle and retry of lvremove...')
            run_udevadm_settle()
    
            # The previous failing lvremove -f might leave behind
            # suspended devices; when lvmetad is not available, any
            # further lvm command will block forever.
            # Therefore we need to skip suspended devices on retry.
            LVM_CONFIG += 'devices { ignore_suspended_devices = 1}'
    
            self._execute(
                'lvremove',
                '--config', LVM_CONFIG,
                '-f',
                '%s/%s' % (self.vg_name, name),
                root_helper=self._root_helper, run_as_root=True)
            LOG.debug('Successfully deleted volume: %s after '
                      'udev settle.', name)

lvremove 删除快照, 可以查看具体命令， 这里不做介绍

如果是ceph
则对应的驱动定义在cinder/volume/dirvers/rbd.py

delete_snapshot 删除snapshot定义

    def delete_snapshot(self, snapshot):
        """Deletes an rbd snapshot."""
        # NOTE(dosaboy): this was broken by commit cbe1d5f. Ensure names are
        #                utf-8 otherwise librbd will barf.
        volume_name = utils.convert_str(snapshot.volume_name)
        snap_name = utils.convert_str(snapshot.name)
    
        with RBDVolumeProxy(self, volume_name) as volume:
            try:
                volume.unprotect_snap(snap_name)
            except self.rbd.InvalidArgument:
                LOG.info(
                    _LI("InvalidArgument: Unable to unprotect snapshot %s."),
                    snap_name)
            except self.rbd.ImageNotFound:
                LOG.info(
                    _LI("ImageNotFound: Unable to unprotect snapshot %s."),
                    snap_name)
            except self.rbd.ImageBusy:
                children_list = self._get_children_info(volume, snap_name)
    
                if children_list:
                    for (pool, image) in children_list:
                        LOG.info(_LI('Image %(pool)s/%(image)s is dependent '
                                     'on the snapshot %(snap)s.'),
                                 {'pool': pool,
                                  'image': image,
                                  'snap': snap_name})
    
                raise exception.SnapshotIsBusy(snapshot_name=snap_name)
            try:
                volume.remove_snap(snap_name)
            except self.rbd.ImageNotFound:
                LOG.info(_LI("Snapshot %s does not exist in backend."),
                         snap_name)

RBDVolumeProxy 调用rbd

删除volume.remove_snap(snap_name)