“Cinder show 不存在的id,查询时间很长”分析

问题: 使用“cinder show”查询一个不存在的volumeid,使用时间很长。

[root@node1 ~]# time cinder show aksjdfkajsdkfjsakdf
ERROR: No volume with a name or ID of 'aksjdfkajsdkfjsakdf' exists.
real    0m35.374s
user    0m0.855s
sys     0m0.107s

分析:

1. cinderclient.v2.shell.do_show说明

指令“cinder show”的入口函数是cinderclient.v2.shell.do_show

@utils.arg('volume',
           metavar='',
           help='Name or ID of volume.')
def do_show(cs, args):
    """Shows volume details."""
    info = dict()
    volume = utils.find_volume(cs, args.volume)
    info.update(volume._info)

    if 'readonly' in info['metadata']:
        info['readonly'] = info['metadata']['readonly']

info.pop('links', None)
# 打印字典
    utils.print_dict(info,
                     formatters=['metadata', 'volume_image_metadata',
                                 'attachments'])

@utils.arg 是用于定义参数的标签。

指令参数根据必填、选填分为两种:

  • 1)@utils.arg('volume', metavar='', help='Name or ID of volume.')
    必填参数,使用时不用带参数名。例如”cinder show 123123”,这里的123123就是
  • 2)@utils.arg('--tenant', type=str, dest='tenant', nargs='?', metavar='', help='Display information from single tenant (Admin only).')
    选填参数,名字多了个破折号 --,指令使用时必须带参数名。例如”cinder list --tenant 123123”

根据参数值分类:

  • 1)@utils.arg('volume', metavar='', nargs='+', help='Name or ID of volume or volumes to delete.')
    如果设置nargs='+',代表值为数组型,以空格分隔,输入1个以上。如”cinder delete 111 2222 3333”,删除volume_id=111、volume_id=222、volume_id=333 的volume。
    此外还有nargs='*' 表示参数可设置零个或多个;nargs=' '+' 表示参数可设置一个或多个;nargs='?' 表示参数可设置零个或一个。
  • 2)@utils.arg('--metadata', type=str, nargs='*', metavar='', default=None, help='Snapshot metadata key and value pairs. Default=None.')
    设置metavar='',值为键值对形式。比如”cinder storage-create --metadata storage_protocol='fc' storage_ip='172.24.1.21' ”

@utils.arg定义的参数会传入def do_show(cs, args)的args,通过args.xxxx来调用。

2.utils.find_volume(cs, args.volume) 说明

cinderclient.utils.find_volume:
def find_volume(cs, volume):
    """Get a volume by name or ID."""
return find_resource(cs.volumes, volume)
def find_resource(manager, name_or_id):
    """Helper for the _find_* methods."""
    # first try to get entity as integer id
    try:
        if isinstance(name_or_id, int) or name_or_id.isdigit():
            return manager.get(int(name_or_id))
    except exceptions.NotFound:
        pass
    else:
        # now try to get entity as uuid
        try:
            uuid.UUID(name_or_id)
            return manager.get(name_or_id)
        except (ValueError, exceptions.NotFound):
            pass
    if sys.version_info <= (3, 0):
        name_or_id = encodeutils.safe_decode(name_or_id)
    try:
        try:
            resource = getattr(manager, 'resource_class', None)
            name_attr = resource.NAME_ATTR if resource else 'name'
            return manager.find(**{name_attr: name_or_id})
        except exceptions.NotFound:
            pass
        # finally try to find entity by human_id
        try:
            return manager.find(human_id=name_or_id)
        except exceptions.NotFound:
            msg = "No %s with a name or ID of '%s' exists." % \
                (manager.resource_class.__name__.lower(), name_or_id)
            raise exceptions.CommandError(msg)
    except exceptions.NoUniqueMatch:
        msg = ("Multiple %s matches found for '%s', use an ID to be more"
               " specific." % (manager.resource_class.__name__.lower(),
                               name_or_id))
        raise exceptions.CommandError(msg)

调用了cinderclient.utils.find_resource(manager, name_or_id)方法,find_resource的逻辑是:
① 如果id是int型,直接调用VolumeManager.get。如果没查到,走下一步。
② id转换成uuid,直接调用VolumeManager.get。如果没查到,走下一步。
③ 如果volumes类定义了NAME_ATTR属性,用{name_attr: name_or_id}作为条件调用VolumeManager.find查询。如果没查到,走下一步。
④ 用{human_id:name_or_id}作为条件调用VolumeManager.find查询。

3.VolumeManager.get说明

cinderclient.v2.volumes.VolumeManager#get
    def get(self, volume_id):
        """Get a volume.
        :param volume_id: The ID of the volume to get.
        :rtype: :class:`Volume`
        """
        return self._get("/volumes/%s" % volume_id, "volume")

self._get调用cinderclient.base.Manager#_get
    def _get(self, url, response_key=None):
        resp, body = self.api.client.get(url)
        if response_key:
            return self.resource_class(self, body[response_key], loaded=True,
                                       resp=resp)
        else:
            return self.resource_class(self, body, loaded=True, resp=resp)

self.resource_class在cinderclient.v2.volumes.VolumeManager定义:resource_class = Volume。所以返回值是body[response_key]转换出的cinderclient.v2.volumes.Volume(base.Resource)对象。

4.VolumeManager.find说明

VolumeManager.find调用cinderclient.base.ManagerWithFind#findall(self, **kwargs):

    def find(self, **kwargs):
        """
        Find a single item with attributes matching ``**kwargs``.
        This isn't very efficient for search options which require the
        Python side filtering(e.g. 'human_id')
        """
        matches = self.findall(**kwargs)
        num_matches = len(matches)
        if num_matches == 0:
            msg = "No %s matching %s." % (self.resource_class.__name__, kwargs)
            raise exceptions.NotFound(404, msg)
        elif num_matches > 1:
            raise exceptions.NoUniqueMatch
        else:
            matches[0].append_request_ids(matches.request_ids)
            return matches[0]
    def findall(self, **kwargs):
        """
        Find all items with attributes matching ``**kwargs``.
        This isn't very efficient for search options which require the
        Python side filtering(e.g. 'human_id')
        """
        # Want to search for all tenants here so that when attempting to delete
        # that a user like admin doesn't get a failure when trying to delete
        # another tenant's volume by name.
        search_opts = {'all_tenants': 1}
        # Pass 'name' or 'display_name' search_opts to server filtering to
        # increase search performance.
        if 'name' in kwargs:
            search_opts['name'] = kwargs['name']
        elif 'display_name' in kwargs:
            search_opts['display_name'] = kwargs['display_name']
        found = common_base.ListWithMeta([], None)
        searches = kwargs.items()
        listing = self.list(search_opts=search_opts)
        found.append_request_ids(listing.request_ids)
        # Not all resources attributes support filters on server side
        # (e.g. 'human_id' doesn't), so when doing findall some client
        # side filtering is still needed.
        for obj in listing:
            try:
                if all(getattr(obj, attr) == value
                       for (attr, value) in searches):
                    found.append(obj)
            except AttributeError:
                continue
        return found

findall的逻辑是:

① 检查kwargs字典key有无'name',有则加入查询条件search_opts。
② 检查kwargs字典key有无'display_name',有则加入查询条件search_opts。
③ self.list(search_opts=search_opts) 用查询条件查询VolumeManager.list。如果前两步都为否,则查询条件为空,那么list方法将查询所有卷信息返回。cinder的db api有配置单次查询最大条数是1000(配置项osapi_max_limit,默认1000),而我们的卷目前有8k多条,其中2528条可用的,所以得查三次才能查出所有卷记录。这占用时间很长。
④ 对第三步查出的卷列表做遍历,用kwargs字典的key和value检查,符合getattr(obj, attr) == value的记录添加进found数组返回。

5.human_id说明

上文介绍cinderclient.utils.find_volume方法里用{human_id:name_or_id}作为条件调用VolumeManager.find查询。human_id是什么?我们可以看到cinderclient.v3.volumes.Volume继承自cinderclient.apiclient.base.Resource,而Resource类里定义了HUMAN_ID = FalseNAME_ATTR = 'name',Resource类里还有一个human_id(self)属性方法。

cinderclient.apiclient.base.Resource

class Resource(RequestIdMixin):
    """Base class for OpenStack resources (tenant, user, etc.).
    This is pretty much just a bag for attributes.
    """
    HUMAN_ID = False
    NAME_ATTR = 'name'
    
    @property
    def human_id(self):
        """Human-readable ID which can be used for bash completion.
        """
        if self.NAME_ATTR in self.__dict__ and self.HUMAN_ID:
            return strutils.to_slug(getattr(self, self.NAME_ATTR))
        return None

根据human_id(self)属性方法的逻辑,我们可以这么理解:如果在cinderclient.v3.volumes.Volume类里有定义HUMAN_ID = True,且定义了类的名字属性NAME_ATTR ,那么volume.human_id=strutils.to_slug(getattr(self, self.NAME_ATTR))strutils.to_slug这个方法的作用是把参数转换成小写、移除non-word的字符(比如中文)、空格转连字符(-)、删除前后端空格。

那么现在我们基本明白human_id的作用了:

human_id是用于查询字符格式不标准的NAME_ATTR 的记录。db里可能存的NAME_ATTR包含了非不正常的字符,比如中文啊、多出的空格啊,cinderclient指令无法传递正确编码的参数到服务端查询。所以通过volume.human_id=strutils.to_slug(getattr(self, self.NAME_ATTR))方法设置human_id的值为NAME_ATTR(即name)转换格式后的值,这样对human_id来查询,就能查到了。比如下面的例子:
数据库里有个volume的display_name= miaomiao喵喵猫,


“Cinder show 不存在的id,查询时间很长”分析_第1张图片
image.png

但是在linux显示乱码:


“Cinder show 不存在的id,查询时间很长”分析_第2张图片
image.png

如果直接使用” cinder show miaomiao喵喵猫 ” 来查询,会提示错误 “ERROR: 'utf8' codec can't decode byte 0xdf in position 8: invalid continuation byte”,无法使用name来查询:


image.png

但只要我们在cinderclient.v3.volumes.Volume类里设置HUMAN_ID=True,strutils.to_slug(u'miaomiao喵喵猫') == 'miaomiao',即可使用`”cinder show miaomiao”查到结果。

“Cinder show 不存在的id,查询时间很长”分析_第3张图片
image.png

但是我们看cinderclient.v3.volumes.Volume没有复写HUMAN_ID,根据属性方法 human_id(self)的逻辑,如果self.HUMAN_ID==False,返回None作为volume的human_id属性值。所以,如果用{human_id:name_or_id}作为条件调用VolumeManager.find查询,getattr(obj, attr) == value是一定为False,find结果一定为空。

既然cinderclient.v3.volumes.Volume的human_id=None,为什么utils.find_resource(manager, name_or_id, **kwargs) 最后一步还要用human_id查询呢?用human_id查询是不是有什么特别的用途?
我用HUMAN_ID搜索cinderclient整个项目,发现只有cinderclient.apiclient.base.Resource里有用到,其它Resource的子类都没有复写。

“Cinder show 不存在的id,查询时间很长”分析_第4张图片
image.png

用human_id搜索cinderclient整个项目,也没用对human_id特别赋值的地方。


“Cinder show 不存在的id,查询时间很长”分析_第5张图片
image.png

这样看来,HUMAN_ID在cinderclient整个项目里的调用值都为False,human_id将始终未None,用{human_id:name_or_id}作为条件调用VolumeManager.find查询在当前的cinderclient里实在毫无意义。

解决方案:

可以考虑把{human_id:name_or_id}查询屏蔽。

你可能感兴趣的:(“Cinder show 不存在的id,查询时间很长”分析)