nova list命令的代码流程分析
1. 概要
这篇文章我们主要分析novalist命令的代码流程，其代码流程大致为：1.从keystone获得token。2. 根据获得的token去调用nova-api接口查询VM的列表。其中获得token之前需要查询keystone的版本信息，且所有的这些查询操作都是通过WSGI通信方式进行处理的。
2. 公共代码流程
nova命令的代码入口如下，
[root@jun ~]# cat /usr/bin/nova
#!/usr/bin/python2
# PBR Generated from u'console_scripts'
import sys
from novaclient.shell import main
if __name__ == "__main__":
sys.exit(main())
#/novaclient/shell.py
def main():
try:
argv = [encodeutils.safe_decode(a) for a in sys.argv[1:]]
OpenStackComputeShell().main(argv)
except Exception as e:
logger.debug(e, exc_info=1)
details = {'name': encodeutils.safe_encode(e.__class__.__name__),
'msg': encodeutils.safe_encode(six.text_type(e))}
print("ERROR (%(name)s): %(msg)s" % details,
file=sys.stderr)
sys.exit(1)
except KeyboardInterrupt:
print("... terminating nova client", file=sys.stderr)
sys.exit(130)
if __name__ == "__main__":
main()
最终调用OpenStackComputeShell类的main函数。如下，
#/novaclient/shell.py:OpenStackComputeShell
def main(self, argv):
# Parse args once to find version and debug settings
parser = self.get_base_parser()
(options, args) = parser.parse_known_args(argv)
self.setup_debugging(options.debug)
... ... ...
if must_auth:
if auth_plugin:
auth_plugin.parse_opts(args)
if not auth_plugin or not auth_plugin.opts:
if not os_username and not os_user_id:
raise exc.CommandError(
_("You must provide a username "
"or user id via --os-username, --os-user-id, "
"env[OS_USERNAME] or env[OS_USER_ID]"))
if not any([args.os_tenant_name, args.os_tenant_id,
args.os_project_id, args.os_project_name]):
raise exc.CommandError(_("You must provide a project name or"
" project id via --os-project-name,"
" --os-project-id, env[OS_PROJECT_ID]"
" or env[OS_PROJECT_NAME]. You may"
" use os-project and os-tenant"
" interchangeably."))
if not os_auth_url:
if os_auth_system and os_auth_system != 'keystone':
os_auth_url = auth_plugin.get_auth_url()
if not os_auth_url:
raise exc.CommandError(
_("You must provide an auth url "
"via either --os-auth-url or env[OS_AUTH_URL] "
"or specify an auth_system which defines a "
"default url with --os-auth-system "
"or env[OS_AUTH_SYSTEM]"))
project_id = args.os_project_id or args.os_tenant_id
project_name = args.os_project_name or args.os_tenant_name
if use_session:
# Not using Nova auth plugin, so use keystone
start_time = time.time()
keystone_session = ksession.Session.load_from_cli_options(args)
keystone_auth = self._get_keystone_auth(
keystone_session,
args.os_auth_url,
username=args.os_username,
user_id=args.os_user_id,
user_domain_id=args.os_user_domain_id,
user_domain_name=args.os_user_domain_name,
password=args.os_password,
auth_token=args.os_auth_token,
project_id=project_id,
project_name=project_name,
project_domain_id=args.os_project_domain_id,
project_domain_name=args.os_project_domain_name)
end_time = time.time()
self.times.append(
('%s %s' % ('auth_url', args.os_auth_url),
start_time, end_time))
if (options.os_compute_api_version and
options.os_compute_api_version != '1.0'):
if not any([args.os_tenant_id, args.os_tenant_name,
args.os_project_id, args.os_project_name]):
raise exc.CommandError(_("You must provide a project name or"
" project id via --os-project-name,"
" --os-project-id, env[OS_PROJECT_ID]"
" or env[OS_PROJECT_NAME]. You may"
" use os-project and os-tenant"
" interchangeably."))
if not os_auth_url:
raise exc.CommandError(
_("You must provide an auth url "
"via either --os-auth-url or env[OS_AUTH_URL]"))
self.cs = client.Client(
options.os_compute_api_version,
os_username, os_password, os_tenant_name,
tenant_id=os_tenant_id, user_id=os_user_id,
auth_url=os_auth_url, insecure=insecure,
region_name=os_region_name, endpoint_type=endpoint_type,
extensions=self.extensions, service_type=service_type,
service_name=service_name, auth_system=os_auth_system,
auth_plugin=auth_plugin, auth_token=auth_token,
volume_service_name=volume_service_name,
timings=args.timings, bypass_url=bypass_url,
os_cache=os_cache, http_log_debug=options.debug,
cacert=cacert, timeout=timeout,
session=keystone_session, auth=keystone_auth)
# Now check for the password/token of which pieces of the
# identifying keyring key can come from the underlying client
if must_auth:
helper = SecretsHelper(args, self.cs.client)
if (auth_plugin and auth_plugin.opts and
"os_password" not in auth_plugin.opts):
use_pw = False
else:
use_pw = True
tenant_id = helper.tenant_id
# Allow commandline to override cache
if not auth_token:
auth_token = helper.auth_token
if not management_url:
management_url = helper.management_url
if tenant_id and auth_token and management_url:
self.cs.client.tenant_id = tenant_id
self.cs.client.auth_token = auth_token
self.cs.client.management_url = management_url
self.cs.client.password_func = lambda: helper.password
elif use_pw:
# We're missing something, so auth with user/pass and save
# the result in our helper.
self.cs.client.password = helper.password
self.cs.client.keyring_saver = helper
try:
# This does a couple of bits which are useful even if we've
# got the token + service URL already. It exits fast in that case.
if not cliutils.isunauthenticated(args.func):
if not use_session:
# Only call authenticate() if Nova auth plugin is used.
# If keystone is used, authentication is handled as part
# of session.
self.cs.authenticate()
except exc.Unauthorized:
raise exc.CommandError(_("Invalid OpenStack Nova credentials."))
except exc.AuthorizationFailure:
raise exc.CommandError(_("Unable to authorize user"))
if options.os_compute_api_version == "3" and service_type != 'image':
# NOTE(cyeoh): create an image based client because the
# images api is no longer proxied by the V3 API and we
# sometimes need to be able to look up images information
# via glance when connected to the nova api.
image_service_type = 'image'
# NOTE(hdd): the password is needed again because creating a new
# Client without specifying bypass_url will force authentication.
# We can't reuse self.cs's bypass_url, because that's the URL for
# the nova service; we need to get glance's URL for this Client
if not os_password:
os_password = helper.password
self.cs.image_cs = client.Client(
options.os_compute_api_version, os_username,
os_password, os_tenant_name, tenant_id=os_tenant_id,
auth_url=os_auth_url, insecure=insecure,
region_name=os_region_name, endpoint_type=endpoint_type,
extensions=self.extensions, service_type=image_service_type,
service_name=service_name, auth_system=os_auth_system,
auth_plugin=auth_plugin,
volume_service_name=volume_service_name,
timings=args.timings, bypass_url=bypass_url,
os_cache=os_cache, http_log_debug=options.debug,
session=keystone_session, auth=keystone_auth,
cacert=cacert, timeout=timeout)
args.func(self.cs, args)
if args.timings:
self._dump_timings(self.times + self.cs.get_timings())
由于OpenStackComputeShell类的main函数的代码较多，所以我只截取了部分代码，我们在这里不重点分析该main函数的代码，我们只需知道最终我们所要分析的nova list的代码流程主要在args.func(self.cs,args)中执行的。其中构建self.cs(novaclient.v2.client.Client对象)时，传入的参数session为keystoneclient.session.Session对象，auth为keystoneclient.auth.identity.generic.password.Password对象。args.func在这里是do_list函数。下面是我环境上打印的args的信息。
args: Namespace(all_tenants=0,
bypass_url='',
debug=True,
deleted=False,
endpoint_type='publicURL',
fields=None,
flavor=None,
func=,
help=False,
host=None,
image=None,
insecure=False,
instance_name=None,
ip=None,
ip6=None,
minimal=False,
name=None,
os_auth_system='',
os_auth_token='',
os_auth_url='http://192.168.118.1:5000/v2.0/',
os_cacert=None,
os_cache=False,
os_cert=None,
os_compute_api_version='2',
os_domain_id=None,
os_domain_name=None,
os_key=None,
os_password='samsung',
os_project_domain_id=None,
os_project_domain_name=None,
os_project_id=None,
os_project_name=None,
os_region_name='RegionOne',
os_tenant_id='',
os_tenant_name='admin',
os_trust_id=None,
os_user_domain_id=None,
os_user_domain_name=None,
os_user_id=None,
os_username='admin',
reservation_id=None,
service_name='',
service_type=None,
sort=None,
status=None,
tenant=None,
timeout=600,
timings=False,
user=None,
volume_service_name=''
)
下面我们分析do_list函数，
#/novaclient/v2/shell.py
@cliutils.arg(
'--reservation-id',
dest='reservation_id',
metavar='',
default=None,
help=_('Only return servers that match reservation-id.'))
... ... ...
@cliutils.arg(
'--sort',
dest='sort',
metavar='[:]',
help=('Comma-separated list of sort keys and directions in the form'
' of [:]. The direction defaults to descending if'
' not specified.'))
def do_list(cs, args):
"""List active servers."""
imageid = None
flavorid = None
if args.image:
imageid = _find_image(cs, args.image).id
if args.flavor:
flavorid = _find_flavor(cs, args.flavor).id
# search by tenant or user only works with all_tenants
if args.tenant or args.user:
args.all_tenants = 1
search_opts = {
'all_tenants': args.all_tenants,
'reservation_id': args.reservation_id,
'ip': args.ip,
'ip6': args.ip6,
'name': args.name,
'image': imageid,
'flavor': flavorid,
'status': args.status,
'tenant_id': args.tenant,
'user_id': args.user,
'host': args.host,
'deleted': args.deleted,
'instance_name': args.instance_name}
filters = {'flavor': lambda f: f['id'],
'security_groups': utils._format_security_groups}
formatters = {}
field_titles = []
if args.fields:
for field in args.fields.split(','):
field_title, formatter = utils._make_field_formatter(field,
filters)
field_titles.append(field_title)
formatters[field_title] = formatter
id_col = 'ID'
detailed = not args.minimal
sort_keys = []
sort_dirs = []
if args.sort:
for sort in args.sort.split(','):
sort_key, _sep, sort_dir = sort.partition(':')
if not sort_dir:
sort_dir = 'desc'
elif sort_dir not in ('asc', 'desc'):
raise exceptions.CommandError(_(
'Unknown sort direction: %s') % sort_dir)
sort_keys.append(sort_key)
sort_dirs.append(sort_dir)
servers = cs.servers.list(detailed=detailed,
search_opts=search_opts,
sort_keys=sort_keys,
sort_dirs=sort_dirs)
convert = [('OS-EXT-SRV-ATTR:host', 'host'),
('OS-EXT-STS:task_state', 'task_state'),
('OS-EXT-SRV-ATTR:instance_name', 'instance_name'),
('OS-EXT-STS:power_state', 'power_state'),
('hostId', 'host_id')]
_translate_keys(servers, convert)
_translate_extended_states(servers)
if args.minimal:
columns = [
id_col,
'Name']
elif field_titles:
columns = [id_col] + field_titles
else:
columns = [
id_col,
'Name',
'Status',
'Task State',
'Power State',
'Networks'
]
# If getting the data for all tenants, print
# Tenant ID as well
if search_opts['all_tenants']:
columns.insert(2, 'Tenant ID')
formatters['Networks'] = utils._format_servers_list_networks
sortby_index = 1
if args.sort:
sortby_index = None
utils.print_list(servers, columns,
formatters, sortby_index=sortby_index)
这里获得VM的列表的语句为：
servers = cs.servers.list(detailed=detailed,
search_opts=search_opts,
sort_keys=sort_keys,
sort_dirs=sort_dirs)
所以我们重点分析该语句。
因为cs为novaclient.v2.client.Client对象，所以我们查看cs的构造函数，如下。
#/novaclient/v2/client.py:Client
class Client(object):
"""
Top-level object to access the OpenStack Compute API.
Create an instance with your creds::
>>> client = Client(USERNAME, PASSWORD, PROJECT_ID, AUTH_URL)
Or, alternatively, you can create a client instance using the
keystoneclient.session API::
>>> from keystoneclient.auth.identity import v2
>>> from keystoneclient import session
>>> from novaclient.client import Client
>>> auth = v2.Password(auth_url=AUTH_URL,
username=USERNAME,
password=PASSWORD,
tenant_name=PROJECT_ID)
>>> sess = session.Session(auth=auth)
>>> nova = client.Client(VERSION, session=sess)
Then call methods on its managers::
>>> client.servers.list()
...
>>> client.flavors.list()
...
It is also possible to use an instance as a context manager in which
case there will be a session kept alive for the duration of the with
statement::
>>> with Client(USERNAME, PASSWORD, PROJECT_ID, AUTH_URL) as client:
... client.servers.list()
... client.flavors.list()
...
It is also possible to have a permanent (process-long) connection pool,
by passing a connection_pool=True::
>>> client = Client(USERNAME, PASSWORD, PROJECT_ID,
... AUTH_URL, connection_pool=True)
"""
def __init__(self, username=None, api_key=None, project_id=None,
auth_url=None, insecure=False, timeout=None,
proxy_tenant_id=None, proxy_token=None, region_name=None,
endpoint_type='publicURL', extensions=None,
service_type='compute', service_name=None,
volume_service_name=None, timings=False, bypass_url=None,
os_cache=False, no_cache=True, http_log_debug=False,
auth_system='keystone', auth_plugin=None, auth_token=None,
cacert=None, tenant_id=None, user_id=None,
connection_pool=False, session=None, auth=None,
**kwargs):
"""
:param str username: Username
:param str api_key: API Key
:param str project_id: Project ID
:param str auth_url: Auth URL
:param bool insecure: Allow insecure
:param float timeout: API timeout, None or 0 disables
:param str proxy_tenant_id: Tenant ID
:param str proxy_token: Proxy Token
:param str region_name: Region Name
:param str endpoint_type: Endpoint Type
:param str extensions: Exensions
:param str service_type: Service Type
:param str service_name: Service Name
:param str volume_service_name: Volume Service Name
:param bool timings: Timings
:param str bypass_url: Bypass URL
:param bool os_cache: OS cache
:param bool no_cache: No cache
:param bool http_log_debug: Enable debugging for HTTP connections
:param str auth_system: Auth system
:param str auth_plugin: Auth plugin
:param str auth_token: Auth token
:param str cacert: cacert
:param str tenant_id: Tenant ID
:param str user_id: User ID
:param bool connection_pool: Use a connection pool
:param str session: Session
:param str auth: Auth
"""
# FIXME(comstud): Rename the api_key argument above when we
# know it's not being used as keyword argument
# NOTE(cyeoh): In the novaclient context (unlike Nova) the
# project_id is not the same as the tenant_id. Here project_id
# is a name (what the Nova API often refers to as a project or
# tenant name) and tenant_id is a UUID (what the Nova API
# often refers to as a project_id or tenant_id).
password = api_key
self.projectid = project_id
self.tenant_id = tenant_id
self.user_id = user_id
self.flavors = flavors.FlavorManager(self)
self.flavor_access = flavor_access.FlavorAccessManager(self)
self.images = images.ImageManager(self)
self.limits = limits.LimitsManager(self)
self.servers = servers.ServerManager(self)
self.versions = versions.VersionManager(self)
# extensions
self.agents = agents.AgentsManager(self)
self.dns_domains = floating_ip_dns.FloatingIPDNSDomainManager(self)
self.dns_entries = floating_ip_dns.FloatingIPDNSEntryManager(self)
self.cloudpipe = cloudpipe.CloudpipeManager(self)
self.certs = certs.CertificateManager(self)
self.floating_ips = floating_ips.FloatingIPManager(self)
self.floating_ip_pools = floating_ip_pools.FloatingIPPoolManager(self)
self.fping = fping.FpingManager(self)
self.volumes = volumes.VolumeManager(self)
self.volume_snapshots = volume_snapshots.SnapshotManager(self)
self.volume_types = volume_types.VolumeTypeManager(self)
self.keypairs = keypairs.KeypairManager(self)
self.networks = networks.NetworkManager(self)
self.quota_classes = quota_classes.QuotaClassSetManager(self)
self.quotas = quotas.QuotaSetManager(self)
self.security_groups = security_groups.SecurityGroupManager(self)
self.security_group_rules = \
security_group_rules.SecurityGroupRuleManager(self)
self.security_group_default_rules = \
security_group_default_rules.SecurityGroupDefaultRuleManager(self)
self.usage = usage.UsageManager(self)
self.virtual_interfaces = \
virtual_interfaces.VirtualInterfaceManager(self)
self.aggregates = aggregates.AggregateManager(self)
self.hosts = hosts.HostManager(self)
self.hypervisors = hypervisors.HypervisorManager(self)
self.hypervisor_stats = hypervisors.HypervisorStatsManager(self)
self.services = services.ServiceManager(self)
self.fixed_ips = fixed_ips.FixedIPsManager(self)
self.floating_ips_bulk = floating_ips_bulk.FloatingIPBulkManager(self)
self.os_cache = os_cache or not no_cache
self.availability_zones = \
availability_zones.AvailabilityZoneManager(self)
self.server_groups = server_groups.ServerGroupsManager(self)
# Add in any extensions...
if extensions:
for extension in extensions:
if extension.manager_class:
setattr(self, extension.name,
extension.manager_class(self))
self.client = client._construct_http_client(
username=username,
password=password,
user_id=user_id,
project_id=project_id,
tenant_id=tenant_id,
auth_url=auth_url,
auth_token=auth_token,
insecure=insecure,
timeout=timeout,
auth_system=auth_system,
auth_plugin=auth_plugin,
proxy_token=proxy_token,
proxy_tenant_id=proxy_tenant_id,
region_name=region_name,
endpoint_type=endpoint_type,
service_type=service_type,
service_name=service_name,
volume_service_name=volume_service_name,
timings=timings,
bypass_url=bypass_url,
os_cache=self.os_cache,
http_log_debug=http_log_debug,
cacert=cacert,
connection_pool=connection_pool,
session=session,
auth=auth,
**kwargs)
因为self.servers = servers.ServerManager(self),所以继续往下看。
#/novaclient/v2/servers.py:ServerManager
def list(self, detailed=True, search_opts=None, marker=None, limit=None,
sort_keys=None, sort_dirs=None):
"""
Get a list of servers.
:param detailed: Whether to return detailed server info (optional).
:param search_opts: Search options to filter out servers (optional).
:param marker: Begin returning servers that appear later in the server
list than that represented by this server id (optional).
:param limit: Maximum number of servers to return (optional).
:param sort_keys: List of sort keys
:param sort_dirs: List of sort directions
:rtype: list of :class:`Server`
"""
if search_opts is None:
search_opts = {}
qparams = {}
for opt, val in six.iteritems(search_opts):
if val:
qparams[opt] = val
if marker:
qparams['marker'] = marker
if limit:
qparams['limit'] = limit
# Transform the dict to a sequence of two-element tuples in fixed
# order, then the encoded string will be consistent in Python 2&3.
if qparams or sort_keys or sort_dirs:
# sort keys and directions are unique since the same parameter
# key is repeated for each associated value
# (ie, &sort_key=key1&sort_key=key2&sort_key=key3)
items = list(qparams.items())
if sort_keys:
items.extend(('sort_key', sort_key) for sort_key in sort_keys)
if sort_dirs:
items.extend(('sort_dir', sort_dir) for sort_dir in sort_dirs)
new_qparams = sorted(items, key=lambda x: x[0])
query_string = "?%s" % parse.urlencode(new_qparams)
else:
query_string = ""
detail = ""
if detailed:
detail = "/detail"
return self._list("/servers%s%s" % (detail, query_string), "servers")
最终调用的接口如下。
#/novaclient/base.py:Manager
class Manager(base.HookableMixin):
"""
Managers interact with a particular type of API (servers, flavors, images,
etc.) and provide CRUD operations for them.
"""
resource_class = None
cache_lock = threading.RLock()
def __init__(self, api):
self.api = api
def _list(self, url, response_key, obj_class=None, body=None):
if body:
_resp, body = self.api.client.post(url, body=body)
else:
_resp, body = self.api.client.get(url)
if obj_class is None:
obj_class = self.resource_class
data = body[response_key]
# NOTE(ja): keystone returns values as list as {'values': [ ... ]}
# unlike other services which just return the list...
if isinstance(data, dict):
try:
data = data['values']
except KeyError:
pass
with self.completion_cache('human_id', obj_class, mode="w"):
with self.completion_cache('uuid', obj_class, mode="w"):
return [obj_class(self, res, loaded=True)
for res in data if res]
这里传递进来的url=/servers/detail，且body未指定，所以body的值采用默认值None，因此执行_resp, body = self.api.client.get(url).
因为self.servers = servers.ServerManager(self)，所以这里self.api即为novaclient.v2.client.Client对象，那么novaclient.v2.client.Client对象中的client参数是什么呢？从上面所以我们查看cs的构造函数中可以看出，self.client=client._construct_http_client(参数)。那么_construct_http_client函数构造的client是什么呢，如下。
#/novaclient/client.py
def _construct_http_client(username=None, password=None, project_id=None,
auth_url=None, insecure=False, timeout=None,
proxy_tenant_id=None, proxy_token=None,
region_name=None, endpoint_type='publicURL',
extensions=None, service_type='compute',
service_name=None, volume_service_name=None,
timings=False, bypass_url=None, os_cache=False,
no_cache=True, http_log_debug=False,
auth_system='keystone', auth_plugin=None,
auth_token=None, cacert=None, tenant_id=None,
user_id=None, connection_pool=False, session=None,
auth=None, user_agent='python-novaclient',
interface=None, **kwargs):
if session:
return SessionClient(session=session,
auth=auth,
interface=interface or endpoint_type,
service_type=service_type,
region_name=region_name,
service_name=service_name,
user_agent=user_agent,
**kwargs)
else:
# FIXME(jamielennox): username and password are now optional. Need
# to test that they were provided in this mode.
return HTTPClient(username,
password,
user_id=user_id,
projectid=project_id,
tenant_id=tenant_id,
auth_url=auth_url,
auth_token=auth_token,
insecure=insecure,
timeout=timeout,
auth_system=auth_system,
auth_plugin=auth_plugin,
proxy_token=proxy_token,
proxy_tenant_id=proxy_tenant_id,
region_name=region_name,
endpoint_type=endpoint_type,
service_type=service_type,
service_name=service_name,
volume_service_name=volume_service_name,
timings=timings,
bypass_url=bypass_url,
os_cache=os_cache,
http_log_debug=http_log_debug,
cacert=cacert,
connection_pool=connection_pool)
因为传递进来的参数session有值，且为keystoneclient.session.Session对象，所以构建的self.client是一个SessionClient对象。且SessionClient与keystone有关。继续向下看。
#/novaclient/client.py:SessionClient
class SessionClient(adapter.LegacyJsonAdapter):
def __init__(self, *args, **kwargs):
self.times = []
super(SessionClient, self).__init__(*args, **kwargs)
def request(self, url, method, **kwargs):
# NOTE(jamielennox): The standard call raises errors from
# keystoneclient, where we need to raise the novaclient errors.
raise_exc = kwargs.pop('raise_exc', True)
start_time = time.time()
resp, body = super(SessionClient, self).request(url,
method,
raise_exc=False,
**kwargs)
end_time = time.time()
self.times.append(('%s %s' % (method, url),
start_time, end_time))
if raise_exc and resp.status_code >= 400:
raise exceptions.from_response(resp, body, url, method)
return resp, body
#/keystoneclient/adapter.py:LegacyJsonAdapter
class LegacyJsonAdapter(Adapter):
"""Make something that looks like an old HTTPClient.
A common case when using an adapter is that we want an interface similar to
the HTTPClients of old which returned the body as JSON as well.
You probably don't want this if you are starting from scratch.
"""
def request(self, *args, **kwargs):
headers = kwargs.setdefault('headers', {})
headers.setdefault('Accept', 'application/json')
try:
kwargs['json'] = kwargs.pop('body')
except KeyError:
pass
resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
body = None
if resp.text:
try:
body = jsonutils.loads(resp.text)
except ValueError:
pass
return resp, body
#/keystoneclient/adapter.py:Adapter
class Adapter(object):
"""An instance of a session with local variables.
A session is a global object that is shared around amongst many clients. It
therefore contains state that is relevant to everyone. There is a lot of
state such as the service type and region_name that are only relevant to a
particular client that is using the session. An adapter provides a wrapper
of client local data around the global session object.
:param session: The session object to wrap.
:type session: keystoneclient.session.Session
:param str service_type: The default service_type for URL discovery.
:param str service_name: The default service_name for URL discovery.
:param str interface: The default interface for URL discovery.
:param str region_name: The default region_name for URL discovery.
:param str endpoint_override: Always use this endpoint URL for requests
for this client.
:param tuple version: The version that this API targets.
:param auth: An auth plugin to use instead of the session one.
:type auth: keystoneclient.auth.base.BaseAuthPlugin
:param str user_agent: The User-Agent string to set.
:param int connect_retries: the maximum number of retries that should
be attempted for connection errors.
Default None - use session default which
is don't retry.
:param logger: A logging object to use for requests that pass through this
adapter.
:type logger: logging.Logger
"""
@utils.positional()
def __init__(self, session, service_type=None, service_name=None,
interface=None, region_name=None, endpoint_override=None,
version=None, auth=None, user_agent=None,
connect_retries=None, logger=None):
# NOTE(jamielennox): when adding new parameters to adapter please also
# add them to the adapter call in httpclient.HTTPClient.__init__
self.session = session
self.service_type = service_type
self.service_name = service_name
self.interface = interface
self.region_name = region_name
self.endpoint_override = endpoint_override
self.version = version
self.user_agent = user_agent
self.auth = auth
self.connect_retries = connect_retries
self.logger = logger
def get(self, url, **kwargs):
return self.request(url, 'GET', **kwargs)
因此_resp, body = self.api.client.get(url)将调用/keystoneclient/adapter.py:Adapter的get方法，而get方法调用的self.request函数，该函数先调用/novaclient/client.py:SessionClient的request函数，然后调用/keystoneclient/adapter.py:LegacyJsonAdapter的request函数，最终调用到/keystoneclient/adapter.py:Adapter的request函数。
#/keystoneclient/adapter.py:Adapter
def request(self, url, method, **kwargs):
endpoint_filter = kwargs.setdefault('endpoint_filter', {})
self._set_endpoint_filter_kwargs(endpoint_filter)
if self.endpoint_override:
kwargs.setdefault('endpoint_override', self.endpoint_override)
if self.auth:
kwargs.setdefault('auth', self.auth)
if self.user_agent:
kwargs.setdefault('user_agent', self.user_agent)
if self.connect_retries is not None:
kwargs.setdefault('connect_retries', self.connect_retries)
if self.logger:
kwargs.setdefault('logger', self.logger)
return self.session.request(url, method, **kwargs)
最终调用self.session.request(url,method, **kwargs)返回reps，其中self.session为keystoneclient.session.Session对象。
#/keystoneclient/session:Session
@utils.positional(enforcement=utils.positional.WARN)
def request(self, url, method, json=None, original_ip=None,
user_agent=None, redirect=None, authenticated=None,
endpoint_filter=None, auth=None, requests_auth=None,
raise_exc=True, allow_reauth=True, log=True,
endpoint_override=None, connect_retries=0, logger=_logger,
**kwargs):
"""Send an HTTP request with the specified characteristics.
Wrapper around `requests.Session.request` to handle tasks such as
setting headers, JSON encoding/decoding, and error handling.
Arguments that are not handled are passed through to the requests
library.
:param string url: Path or fully qualified URL of HTTP request. If only
a path is provided then endpoint_filter must also be
provided such that the base URL can be determined.
If a fully qualified URL is provided then
endpoint_filter will be ignored.
:param string method: The http method to use. (e.g. 'GET', 'POST')
:param string original_ip: Mark this request as forwarded for this ip.
(optional)
:param dict headers: Headers to be included in the request. (optional)
:param json: Some data to be represented as JSON. (optional)
:param string user_agent: A user_agent to use for the request. If
present will override one present in headers.
(optional)
:param int/bool redirect: the maximum number of redirections that
can be followed by a request. Either an
integer for a specific count or True/False
for forever/never. (optional)
:param int connect_retries: the maximum number of retries that should
be attempted for connection errors.
(optional, defaults to 0 - never retry).
:param bool authenticated: True if a token should be attached to this
request, False if not or None for attach if
an auth_plugin is available.
(optional, defaults to None)
:param dict endpoint_filter: Data to be provided to an auth plugin with
which it should be able to determine an
endpoint to use for this request. If not
provided then URL is expected to be a
fully qualified URL. (optional)
:param str endpoint_override: The URL to use instead of looking up the
endpoint in the auth plugin. This will be
ignored if a fully qualified URL is
provided but take priority over an
endpoint_filter. (optional)
:param auth: The auth plugin to use when authenticating this request.
This will override the plugin that is attached to the
session (if any). (optional)
:type auth: :py:class:`keystoneclient.auth.base.BaseAuthPlugin`
:param requests_auth: A requests library auth plugin that cannot be
passed via kwarg because the `auth` kwarg
collides with our own auth plugins. (optional)
:type requests_auth: :py:class:`requests.auth.AuthBase`
:param bool raise_exc: If True then raise an appropriate exception for
failed HTTP requests. If False then return the
request object. (optional, default True)
:param bool allow_reauth: Allow fetching a new token and retrying the
request on receiving a 401 Unauthorized
response. (optional, default True)
:param bool log: If True then log the request and response data to the
debug log. (optional, default True)
:param logger: The logger object to use to log request and responses.
If not provided the keystoneclient.session default
logger will be used.
:type logger: logging.Logger
:param kwargs: any other parameter that can be passed to
requests.Session.request (such as `headers`). Except:
'data' will be overwritten by the data in 'json' param.
'allow_redirects' is ignored as redirects are handled
by the session.
:raises keystoneclient.exceptions.ClientException: For connection
failure, or to indicate an error response code.
:returns: The response to the request.
"""
headers = kwargs.setdefault('headers', dict())
if authenticated is None:
authenticated = bool(auth or self.auth)
if authenticated:
auth_headers = self.get_auth_headers(auth)
if auth_headers is None:
msg = _('No valid authentication is available')
raise exceptions.AuthorizationFailure(msg)
headers.update(auth_headers)
if osprofiler_web:
headers.update(osprofiler_web.get_trace_id_headers())
# if we are passed a fully qualified URL and an endpoint_filter we
# should ignore the filter. This will make it easier for clients who
# want to overrule the default endpoint_filter data added to all client
# requests. We check fully qualified here by the presence of a host.
if not urllib.parse.urlparse(url).netloc:
base_url = None
if endpoint_override:
base_url = endpoint_override
elif endpoint_filter:
base_url = self.get_endpoint(auth, **endpoint_filter)
if not base_url:
raise exceptions.EndpointNotFound()
url = '%s/%s' % (base_url.rstrip('/'), url.lstrip('/'))
if self.cert:
kwargs.setdefault('cert', self.cert)
if self.timeout is not None:
kwargs.setdefault('timeout', self.timeout)
if user_agent:
headers['User-Agent'] = user_agent
elif self.user_agent:
user_agent = headers.setdefault('User-Agent', self.user_agent)
else:
user_agent = headers.setdefault('User-Agent', USER_AGENT)
if self.original_ip:
headers.setdefault('Forwarded',
'for=%s;by=%s' % (self.original_ip, user_agent))
if json is not None:
headers['Content-Type'] = 'application/json'
kwargs['data'] = jsonutils.dumps(json)
kwargs.setdefault('verify', self.verify)
if requests_auth:
kwargs['auth'] = requests_auth
if log:
self._http_log_request(url, method=method,
data=kwargs.get('data'),
headers=headers,
logger=logger)
# Force disable requests redirect handling. We will manage this below.
kwargs['allow_redirects'] = False
if redirect is None:
redirect = self.redirect
send = functools.partial(self._send_request,
url, method, redirect, log, logger,
connect_retries)
resp = send(**kwargs)
# handle getting a 401 Unauthorized response by invalidating the plugin
# and then retrying the request. This is only tried once.
if resp.status_code == 401 and authenticated and allow_reauth:
if self.invalidate(auth):
auth_headers = self.get_auth_headers(auth)
if auth_headers is not None:
headers.update(auth_headers)
resp = send(**kwargs)
if raise_exc and resp.status_code >= 400:
logger.debug('Request returned failure status: %s',
resp.status_code)
raise exceptions.from_response(resp, method, url)
return resp
这将是我们分析的重点。这里因为auth为keystoneclient.auth.identity.generic.password.Password对象，所以authenticated为True。因此执行如下的部分代码。
#/keystoneclient/session:Session.request
if authenticated:
auth_headers = self.get_auth_headers(auth)
if auth_headers is None:
msg = _('No valid authentication is available')
raise exceptions.AuthorizationFailure(msg)
headers.update(auth_headers)
在auth_headers =self.get_auth_headers(auth)代码中，则进行了获取token的操作。下一节具体分析。
3. 获取token
#/keystoneclient/session:Session
def get_auth_headers(self, auth=None, **kwargs):
"""Return auth headers as provided by the auth plugin.
:param auth: The auth plugin to use for token. Overrides the plugin
on the session. (optional)
:type auth: :py:class:`keystoneclient.auth.base.BaseAuthPlugin`
:raises keystoneclient.exceptions.AuthorizationFailure: if a new token
fetch fails.
:raises keystoneclient.exceptions.MissingAuthPlugin: if a plugin is not
available.
:returns: Authentication headers or None for failure.
:rtype: dict
"""
auth = self._auth_required(auth, 'fetch a token')
return auth.get_headers(self, **kwargs)
#/keystoneclient/session:Session
def _auth_required(self, auth, msg):
if not auth:
auth = self.auth
if not auth:
msg_fmt = _('An auth plugin is required to %s')
raise exceptions.MissingAuthPlugin(msg_fmt % msg)
return auth
从get_auth_headers函数的注释可以看出，该函数返回的正是keystone生成的一个token，形式为：{'X-Auth-Token':u'5eac0b85c71049328888f962ced04896'}
该token如何生成？我们继续向下分析。执行auth.get_headers(self, **kwargs)，且auth为keystoneclient.auth.identity.generic.password.Password对象。
#/keystoneclient/auth/identity/generic/password.py:Password
class Password(base.BaseGenericPlugin):
"""A common user/password authentication plugin.
:param string username: Username for authentication.
:param string user_id: User ID for authentication.
:param string password: Password for authentication.
:param string user_domain_id: User's domain ID for authentication.
:param string user_domain_name: User's domain name for authentication.
"""
@utils.positional()
def __init__(self, auth_url, username=None, user_id=None, password=None,
user_domain_id=None, user_domain_name=None, **kwargs):
super(Password, self).__init__(auth_url=auth_url, **kwargs)
self._username = username
self._user_id = user_id
self._password = password
self._user_domain_id = user_domain_id
self._user_domain_name = user_domain_name
#/keystoneclient/auth/identity/generic/base.py:BaseGenericPlugin
@six.add_metaclass(abc.ABCMeta)
class BaseGenericPlugin(base.BaseIdentityPlugin):
"""An identity plugin that is not version dependant.
Internally we will construct a version dependant plugin with the resolved
URL and then proxy all calls from the base plugin to the versioned one.
"""
def __init__(self, auth_url,
tenant_id=None,
tenant_name=None,
project_id=None,
project_name=None,
project_domain_id=None,
project_domain_name=None,
domain_id=None,
domain_name=None,
trust_id=None):
super(BaseGenericPlugin, self).__init__(auth_url=auth_url)
self._project_id = project_id or tenant_id
self._project_name = project_name or tenant_name
self._project_domain_id = project_domain_id
self._project_domain_name = project_domain_name
self._domain_id = domain_id
self._domain_name = domain_name
self._trust_id = trust_id
self._plugin = None
#/keystoneclient/auth/identity/base.py:BaseIdentityPlugin
@six.add_metaclass(abc.ABCMeta)
class BaseIdentityPlugin(base.BaseAuthPlugin):
# we count a token as valid if it is valid for at least this many seconds
MIN_TOKEN_LIFE_SECONDS = 1
def __init__(self,
auth_url=None,
username=None,
password=None,
token=None,
trust_id=None,
reauthenticate=True):
super(BaseIdentityPlugin, self).__init__()
self.auth_url = auth_url
self.auth_ref = None
self.reauthenticate = reauthenticate
self._endpoint_cache = {}
# NOTE(jamielennox): DEPRECATED. The following should not really be set
# here but handled by the individual auth plugin.
self.username = username
self.password = password
self.token = token
self.trust_id = trust_id
#/keystoneclient/auth/base.py:BaseAuthPlugin
class BaseAuthPlugin(object):
"""The basic structure of an authentication plugin."""
... ... ...
def get_headers(self, session, **kwargs):
"""Fetch authentication headers for message.
This is a more generalized replacement of the older get_token to allow
plugins to specify different or additional authentication headers to
the OpenStack standard 'X-Auth-Token' header.
How the authentication headers are obtained is up to the plugin. If the
headers are still valid they may be re-used, retrieved from cache or
the plugin may invoke an authentication request against a server.
The default implementation of get_headers calls the `get_token` method
to enable older style plugins to continue functioning unchanged.
Subclasses should feel free to completely override this function to
provide the headers that they want.
There are no required kwargs. They are passed directly to the auth
plugin and they are implementation specific.
Returning None will indicate that no token was able to be retrieved and
that authorization was a failure. Adding no authentication data can be
achieved by returning an empty dictionary.
:param session: The session object that the auth_plugin belongs to.
:type session: keystoneclient.session.Session
:returns: Headers that are set to authenticate a message or None for
failure. Note that when checking this value that the empty
dict is a valid, non-failure response.
:rtype: dict
"""
token = self.get_token(session)
if not token:
return None
return {IDENTITY_AUTH_HEADER_NAME: token}
auth.get_headers(self,**kwargs)最终调用到/keystoneclient/auth/base.py:BaseAuthPlugin的get_headers方法。在get_headers函数中，self.get_token方法获取token，然后构造一个字典返回回去，其中
| IDENTITY_AUTH_HEADER_NAME = 'X-Auth-Token' |
所以token的形式为：{'X-Auth-Token':u'5eac0b85c71049328888f962ced04896'}
token的获取方法如下：
#/keystoneclient/auth/identity/base.py:BaseIdentityPlugin
def get_token(self, session, **kwargs):
"""Return a valid auth token.
If a valid token is not present then a new one will be fetched.
:param session: A session object that can be used for communication.
:type session: keystoneclient.session.Session
:raises keystoneclient.exceptions.HttpError: An error from an invalid
HTTP response.
:return: A valid token.
:rtype: string
"""
return self.get_access(session).auth_token
#/keystoneclient/auth/identity/base.py:BaseIdentityPlugin
def _needs_reauthenticate(self):
"""Return if the existing token needs to be re-authenticated.
The token should be refreshed if it is about to expire.
:returns: True if the plugin should fetch a new token. False otherwise.
"""
if not self.auth_ref:
# authentication was never fetched.
return True
if not self.reauthenticate:
# don't re-authenticate if it has been disallowed.
return False
if self.auth_ref.will_expire_soon(self.MIN_TOKEN_LIFE_SECONDS):
# if it's about to expire we should re-authenticate now.
return True
# otherwise it's fine and use the existing one.
return False
#/keystoneclient/auth/identity/base.py:BaseIdentityPlugin
def get_access(self, session, **kwargs):
"""Fetch or return a current AccessInfo object.
If a valid AccessInfo is present then it is returned otherwise a new
one will be fetched.
:param session: A session object that can be used for communication.
:type session: keystoneclient.session.Session
:raises keystoneclient.exceptions.HttpError: An error from an invalid
HTTP response.
:returns: Valid AccessInfo
:rtype: :py:class:`keystoneclient.access.AccessInfo`
"""
if self._needs_reauthenticate():
self.auth_ref = self.get_auth_ref(session)
return self.auth_ref
这里_needs_reauthenticate函数根据3个条件来判断是否需要重新申请token。
1. self.auth_ref是否有值，没有值则需要重新申请token。注意，这里self.auth_ref的值一般是当申请token时，keystone一并返回的各种服务的endpoints，即譬如nova，neutron，cinder等服务的url。
2. 如果self.auth_ref没有值，则判断self. reauthenticate开启，这个值一般在创建BaseIdentityPlugin对象时进行设置，不过这里self. reauthenticate默认为True。
3. 如果self.auth_ref有值，且self. reauthenticate为True，则从self.auth_ref中读取token的expire时间，判断该token是否过期，如果过期，则_needs_reauthenticate函数返回True，即需要重新申请token。
如果3个条件判断完成后，发现self.auth_ref中的token并未过期，则采用目前的token。
self._needs_reauthenticate()返回的值为True，所以执行self.auth_ref= self.get_auth_ref(session)。
#/keystoneclient/auth/identity/generic/base.py:BaseGenericPlugin
def get_auth_ref(self, session, **kwargs):
if not self._plugin:
self._plugin = self._do_create_plugin(session)
return self._plugin.get_auth_ref(session, **kwargs)
因为self._plugin为None，所以调用_do_create_plugin方法。
#/keystoneclient/auth/identity/generic/base.py:BaseGenericPlugin
def _do_create_plugin(self, session):
plugin = None
try:
disc = self.get_discovery(session,
self.auth_url,
authenticated=False)
except (exceptions.DiscoveryFailure,
exceptions.HTTPError,
exceptions.ConnectionError):
LOG.warn(_LW('Discovering versions from the identity service '
'failed when creating the password plugin. '
'Attempting to determine version from URL.'))
url_parts = urlparse.urlparse(self.auth_url)
path = url_parts.path.lower()
if path.startswith('/v2.0') and not self._has_domain_scope:
plugin = self.create_plugin(session, (2, 0), self.auth_url)
elif path.startswith('/v3'):
plugin = self.create_plugin(session, (3, 0), self.auth_url)
else:
disc_data = disc.version_data()
for data in disc_data:
version = data['version']
if (_discover.version_match((2,), version) and
self._has_domain_scope):
# NOTE(jamielennox): if there are domain parameters there
# is no point even trying against v2 APIs.
continue
plugin = self.create_plugin(session,
version,
data['url'],
raw_status=data['raw_status'])
if plugin:
break
if plugin:
return plugin
# so there were no URLs that i could use for auth of any version.
msg = _('Could not determine a suitable URL for the plugin')
raise exceptions.DiscoveryFailure(msg)
我们看看get_discovery函数都做了什么操作？
#/keystoneclient/auth/identity/base.py:BaseIdentityPlugin
@utils.positional()
def get_discovery(self, session, url, authenticated=None):
"""Return the discovery object for a URL.
Check the session and the plugin cache to see if we have already
performed discovery on the URL and if so return it, otherwise create
a new discovery object, cache it and return it.
This function is expected to be used by subclasses and should not
be needed by users.
:param session: A session object to discover with.
:type session: keystoneclient.session.Session
:param str url: The url to lookup.
:param bool authenticated: Include a token in the discovery call.
(optional) Defaults to None (use a token
if a plugin is installed).
:raises keystoneclient.exceptions.DiscoveryFailure: if for some reason
the lookup fails.
:raises keystoneclient.exceptions.HttpError: An error from an invalid
HTTP response.
:returns: A discovery object with the results of looking up that URL.
"""
# NOTE(jamielennox): we want to cache endpoints on the session as well
# so that they maintain sharing between auth plugins. Create a cache on
# the session if it doesn't exist already.
try:
session_endpoint_cache = session._identity_endpoint_cache
except AttributeError:
session_endpoint_cache = session._identity_endpoint_cache = {}
# NOTE(jamielennox): There is a cache located on both the session
# object and the auth plugin object so that they can be shared and the
# cache is still usable
for cache in (self._endpoint_cache, session_endpoint_cache):
disc = cache.get(url)
if disc:
break
else:
disc = _discover.Discover(session, url,
authenticated=authenticated)
self._endpoint_cache[url] = disc
session_endpoint_cache[url] = disc
return disc
因为self._endpoint_cache和session._identity_endpoint_cache为空字典，所以执行
disc =_discover.Discover(session, url,
authenticated=authenticated)
#/keystoneclient/_discover.py:Discover
class Discover(object):
CURRENT_STATUSES = ('stable', 'current', 'supported')
DEPRECATED_STATUSES = ('deprecated',)
EXPERIMENTAL_STATUSES = ('experimental',)
@utils.positional()
def __init__(self, session, url, authenticated=None):
self._data = get_version_data(session, url,
authenticated=authenticated)
#/keystoneclient/_discover.py
@utils.positional()
def get_version_data(session, url, authenticated=None):
"""Retrieve raw version data from a url."""
headers = {'Accept': 'application/json'}
resp = session.get(url, headers=headers, authenticated=authenticated)
try:
body_resp = resp.json()
except ValueError:
pass
else:
# In the event of querying a root URL we will get back a list of
# available versions.
try:
return body_resp['versions']['values']
except (KeyError, TypeError):
pass
# Most servers don't have a 'values' element so accept a simple
# versions dict if available.
try:
return body_resp['versions']
except KeyError:
pass
# Otherwise if we query an endpoint like /v2.0 then we will get back
# just the one available version.
try:
return [body_resp['version']]
except KeyError:
pass
err_text = resp.text[:50] + '...' if len(resp.text) > 50 else resp.text
msg = _('Invalid Response - Bad version data returned: %s') % err_text
raise exceptions.DiscoveryFailure(msg)
get_version_data函数中的session.get函数回调到/keystoneclient/session:Session的get函数，最终又回到/keystoneclient/session:Session的request函数，根据刚才的分析我们知道，我们第一次进入request函数还未出去(由于authenticated的值为True，所以这里首先需要获取token)，这是在第一次进入request函数的基础上，再次进入该函数，相当于一层递归操作。这里authenticated=False，所以不会走第一次进入request函数的authenticated相关的操作，直接执行后面的操作。而执行这个get操作就是为了根据keystone返回的version数据来构造合适的版本的keystoneclient来进行token的申请。我的环境采用的version为2.0。
最终请求的debug信息类似如下信息：
| DEBUG (session:197) REQ: curl -g -i -X GET http://192.168.118.1:5000/v2.0/ -H "Accept: application/json" -H "User-Agent: python-keystoneclient" |
被keystone返回的debug信息类似如下信息(第一部分为header，第二部分为body)：
| DEBUG (session:226) RESP: [200] content-length: 339 vary: X-Auth-Token connection: keep-alive date: Mon, 14 Mar 2016 13:27:34 GMT content-type: application/json x-openstack-request-id: req-c3797498-4212-446c-b900-bd6066798c0c
RESP BODY: {"version": {"status": "stable", "updated": "2014-04-17T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json"}], "id": "v2.0", "links": [{"href": "http://192.168.118.1:5000/v2.0/", "rel": "self"}, {"href": "http://docs.openstack.org/", "type": "text/html", "rel": "describedby"}]}} |
其中对于最终如何获得这些信息，其实是通过基于WSGI架构设计的Restful API通信获得的，这部分的分析，我将在后面的文章中进行分析。
这样Discover类调用get_version_data函数根据url返回的version数据构造self._data成员变量。再次回到_do_create_plugin函数。
#/keystoneclient/auth/identity/generic/base.py:BaseGenericPlugin
def _do_create_plugin(self, session):
... ... ...
else:
disc_data = disc.version_data()
for data in disc_data:
version = data['version']
if (_discover.version_match((2,), version) and
self._has_domain_scope):
# NOTE(jamielennox): if there are domain parameters there
# is no point even trying against v2 APIs.
continue
plugin = self.create_plugin(session,
version,
data['url'],
raw_status=data['raw_status'])
if plugin:
break
if plugin:
return plugin
# so there were no URLs that i could use for auth of any version.
msg = _('Could not determine a suitable URL for the plugin')
raise exceptions.DiscoveryFailure(msg)
这里disc_data = disc.version_data()将归一化数据，即如下类似形式：
| [{'url': u'http://192.168.118.1:5000/v2.0/', 'version': (2, 0), 'raw_status': u'stable'}] |
#/keystoneclient/auth/identity/generic/password.py:Password
def create_plugin(self, session, version, url, raw_status=None):
if _discover.version_match((2,), version):
if self._user_domain_id or self._user_domain_name:
# If you specify any domain parameters it won't work so quit.
return None
return v2.Password(auth_url=url,
user_id=self._user_id,
username=self._username,
password=self._password,
**self._v2_params)
elif _discover.version_match((3,), version):
return v3.Password(auth_url=url,
user_id=self._user_id,
username=self._username,
user_domain_id=self._user_domain_id,
user_domain_name=self._user_domain_name,
password=self._password,
**self._v3_params)
#/keystoneclient/auth/identity/v2.py:Password
class Password(Auth):
"""A plugin for authenticating with a username and password.
A username or user_id must be provided.
:param string auth_url: Identity service endpoint for authorization.
:param string username: Username for authentication.
:param string password: Password for authentication.
:param string user_id: User ID for authentication.
:param string trust_id: Trust ID for trust scoping.
:param string tenant_id: Tenant ID for tenant scoping.
:param string tenant_name: Tenant name for tenant scoping.
:param bool reauthenticate: Allow fetching a new token if the current one
is going to expire. (optional) default True
:raises TypeError: if a user_id or username is not provided.
"""
@utils.positional(4)
def __init__(self, auth_url, username=_NOT_PASSED, password=None,
user_id=_NOT_PASSED, **kwargs):
super(Password, self).__init__(auth_url, **kwargs)
if username is _NOT_PASSED and user_id is _NOT_PASSED:
msg = 'You need to specify either a username or user_id'
raise TypeError(msg)
if username is _NOT_PASSED:
username = None
if user_id is _NOT_PASSED:
user_id = None
self.user_id = user_id
self.username = username
self.password = password
由于keystone采用的版本为2.0，所以create_plugin函数返回的为v2.Password对象。再次回到get_auth_ref函数。
#/keystoneclient/auth/identity/generic/base.py:BaseGenericPlugin
def get_auth_ref(self, session, **kwargs):
if not self._plugin:
self._plugin = self._do_create_plugin(session)
return self._plugin.get_auth_ref(session, **kwargs)
根据上面的分析self._plugin为v2.Password对象。执行v2.Password类的get_auth_ref函数，如下：
#/keystoneclient/auth/identity/v2.py:Auth
def get_auth_ref(self, session, **kwargs):
headers = {'Accept': 'application/json'}
url = self.auth_url.rstrip('/') + '/tokens'
params = {'auth': self.get_auth_data(headers)}
if self.tenant_id:
params['auth']['tenantId'] = self.tenant_id
elif self.tenant_name:
params['auth']['tenantName'] = self.tenant_name
if self.trust_id:
params['auth']['trust_id'] = self.trust_id
_logger.debug('Making authentication request to %s', url)
resp = session.post(url, json=params, headers=headers,
authenticated=False, log=False)
try:
resp_data = resp.json()['access']
except (KeyError, ValueError):
raise exceptions.InvalidResponse(response=resp)
return access.AccessInfoV2(**resp_data)
在执行 resp = session.post(url, json=params, headers=headers,
authenticated=False, log=False)
函数时，函数回调到/keystoneclient/session:Session的post函数，最终又回到/keystoneclient/session:Session的request函数。为了有利于我们debug调试，我们在这里可暂时将log=Flase修改为log=True。此时获得如下类似信息。
请求信息:
| DEBUG (session:197) REQ: curl -g -i -X POST http://192.168.118.1:5000/v2.0/tokens -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "admin"}}}' |
keystone回复的信息(第一部分为header，第二部分为body)：
| DEBUG (session:226) RESP: [200] content-length: 3381 vary: X-Auth-Token connection: keep-alive date: Mon, 14 Mar 2016 13:27:34 GMT content-type: application/json x-openstack-request-id: req-d6e2cf98-e522-45f3-b09a-67e31aa1db32
RESP BODY: {"access": {"token": {"issued_at": "2016-03-14T13:27:34.499050", "expires": "2016-03-14T14:27:34Z", "id": "5eac0b85c71049328888f962ced04896", "tenant": {"enabled": true, "description": "admin tenant", "name": "admin", "id": "09e04766c06d477098201683497d3878"}, "audit_ids": ["Fz3uPZygR_S7sRIINl6_Vg"]}, "serviceCatalog": " |
最终我们可以查看出keystone返回的token，即body中的”5eac0b85c71049328888f962ced04896”即为keystone返回的token。
最终返回到第一次调用/keystoneclient/session:Session的request函数的地方。
因此，最终auth_headers的值类似如下：
{'X-Auth-Token': u'5eac0b85c71049328888f962ced04896'}
此时token已经获取，下面需要执行的就是通过调用nova-api的函数来获取VM的列表。
4. 获取VM列表
我们回到第一次调用/keystoneclient/session:Session的request函数的地方。
#/keystoneclient/session:Session
@utils.positional(enforcement=utils.positional.WARN)
def request(self, url, method, json=None, original_ip=None,
user_agent=None, redirect=None, authenticated=None,
endpoint_filter=None, auth=None, requests_auth=None,
raise_exc=True, allow_reauth=True, log=True,
endpoint_override=None, connect_retries=0, logger=_logger,
**kwargs):
headers = kwargs.setdefault('headers', dict())
if authenticated is None:
authenticated = bool(auth or self.auth)
if authenticated:
auth_headers = self.get_auth_headers(auth)
if auth_headers is None:
msg = _('No valid authentication is available')
raise exceptions.AuthorizationFailure(msg)
headers.update(auth_headers)
if osprofiler_web:
headers.update(osprofiler_web.get_trace_id_headers())
# if we are passed a fully qualified URL and an endpoint_filter we
# should ignore the filter. This will make it easier for clients who
# want to overrule the default endpoint_filter data added to all client
# requests. We check fully qualified here by the presence of a host.
if not urllib.parse.urlparse(url).netloc:
base_url = None
if endpoint_override:
base_url = endpoint_override
elif endpoint_filter:
base_url = self.get_endpoint(auth, **endpoint_filter)
if not base_url:
raise exceptions.EndpointNotFound()
url = '%s/%s' % (base_url.rstrip('/'), url.lstrip('/'))
if self.cert:
kwargs.setdefault('cert', self.cert)
if self.timeout is not None:
kwargs.setdefault('timeout', self.timeout)
if user_agent:
headers['User-Agent'] = user_agent
elif self.user_agent:
user_agent = headers.setdefault('User-Agent', self.user_agent)
else:
user_agent = headers.setdefault('User-Agent', USER_AGENT)
if self.original_ip:
headers.setdefault('Forwarded',
'for=%s;by=%s' % (self.original_ip, user_agent))
if json is not None:
headers['Content-Type'] = 'application/json'
kwargs['data'] = jsonutils.dumps(json)
kwargs.setdefault('verify', self.verify)
if requests_auth:
kwargs['auth'] = requests_auth
if log:
self._http_log_request(url, method=method,
data=kwargs.get('data'),
headers=headers,
logger=logger)
# Force disable requests redirect handling. We will manage this below.
kwargs['allow_redirects'] = False
if redirect is None:
redirect = self.redirect
send = functools.partial(self._send_request,
url, method, redirect, log, logger,
connect_retries)
resp = send(**kwargs)
# handle getting a 401 Unauthorized response by invalidating the plugin
# and then retrying the request. This is only tried once.
if resp.status_code == 401 and authenticated and allow_reauth:
if self.invalidate(auth):
auth_headers = self.get_auth_headers(auth)
if auth_headers is not None:
headers.update(auth_headers)
resp = send(**kwargs)
if raise_exc and resp.status_code >= 400:
logger.debug('Request returned failure status: %s',
resp.status_code)
raise exceptions.from_response(resp, method, url)
return resp
我们知道最终auth_headers的值类似如下：
{'X-Auth-Token': u'5eac0b85c71049328888f962ced04896'}
在从keystone中获取相关的token信息的同时，keystone将各种服务的endpoints也返回给keystoneclient了，因此我们才能够知道nova服务的endpoint，即nova服务的url，然后才能用http向nova-api请求VM的列表。
#/keystoneclient/session:Session
@utils.positional(enforcement=utils.positional.WARN)
def request(self, url, method, json=None, original_ip=None,
user_agent=None, redirect=None, authenticated=None,
endpoint_filter=None, auth=None, requests_auth=None,
raise_exc=True, allow_reauth=True, log=True,
endpoint_override=None, connect_retries=0, logger=_logger,
**kwargs):
headers = kwargs.setdefault('headers', dict())
if authenticated is None:
authenticated = bool(auth or self.auth)
if authenticated:
auth_headers = self.get_auth_headers(auth)
if auth_headers is None:
msg = _('No valid authentication is available')
raise exceptions.AuthorizationFailure(msg)
headers.update(auth_headers)
if osprofiler_web:
headers.update(osprofiler_web.get_trace_id_headers())
# if we are passed a fully qualified URL and an endpoint_filter we
# should ignore the filter. This will make it easier for clients who
# want to overrule the default endpoint_filter data added to all client
# requests. We check fully qualified here by the presence of a host.
if not urllib.parse.urlparse(url).netloc:
base_url = None
if endpoint_override:
base_url = endpoint_override
elif endpoint_filter:
base_url = self.get_endpoint(auth, **endpoint_filter)
如上代码，在我的环境中，endpoint_filter的值如下：
| {'service_type': 'compute', 'interface': 'publicURL', 'region_name': 'RegionOne'} |
然后调用get_enpoint方法获取nova服务的endpoint。代码如下
#/keystoneclient/session:Session
def get_endpoint(self, auth=None, **kwargs):
"""Get an endpoint as provided by the auth plugin.
:param auth: The auth plugin to use for token. Overrides the plugin on
the session. (optional)
:type auth: :py:class:`keystoneclient.auth.base.BaseAuthPlugin`
:raises keystoneclient.exceptions.MissingAuthPlugin: if a plugin is not
available.
:returns: An endpoint if available or None.
:rtype: string
"""
auth = self._auth_required(auth, 'determine endpoint URL')
return auth.get_endpoint(self, **kwargs)
#/keystoneclient/auth/identity/base.py:BaseIdentityPlugin
def get_endpoint(self, session, service_type=None, interface=None,
region_name=None, service_name=None, version=None,
**kwargs):
"""Return a valid endpoint for a service.
If a valid token is not present then a new one will be fetched using
the session and kwargs.
:param session: A session object that can be used for communication.
:type session: keystoneclient.session.Session
:param string service_type: The type of service to lookup the endpoint
for. This plugin will return None (failure)
if service_type is not provided.
:param string interface: The exposure of the endpoint. Should be
`public`, `internal`, `admin`, or `auth`.
`auth` is special here to use the `auth_url`
rather than a URL extracted from the service
catalog. Defaults to `public`.
:param string region_name: The region the endpoint should exist in.
(optional)
:param string service_name: The name of the service in the catalog.
(optional)
:param tuple version: The minimum version number required for this
endpoint. (optional)
:raises keystoneclient.exceptions.HttpError: An error from an invalid
HTTP response.
:return: A valid endpoint URL or None if not available.
:rtype: string or None
"""
# NOTE(jamielennox): if you specifically ask for requests to be sent to
# the auth url then we can ignore the rest of the checks. Typically if
# you are asking for the auth endpoint it means that there is no
# catalog to query anyway.
if interface is base.AUTH_INTERFACE:
return self.auth_url
if not service_type:
LOG.warn(_LW('Plugin cannot return an endpoint without knowing '
'the service type that is required. Add service_type '
'to endpoint filtering data.'))
return None
if not interface:
interface = 'public'
service_catalog = self.get_access(session).service_catalog
url = service_catalog.url_for(service_type=service_type,
endpoint_type=interface,
region_name=region_name,
service_name=service_name)
if not version:
# NOTE(jamielennox): This may not be the best thing to default to
# but is here for backwards compatibility. It may be worth
# defaulting to the most recent version.
return url
# NOTE(jamielennox): For backwards compatibility people might have a
# versioned endpoint in their catalog even though they want to use
# other endpoint versions. So we support a list of client defined
# situations where we can strip the version component from a URL before
# doing discovery.
hacked_url = _discover.get_catalog_discover_hack(service_type, url)
try:
disc = self.get_discovery(session, hacked_url, authenticated=False)
except (exceptions.DiscoveryFailure,
exceptions.HTTPError,
exceptions.ConnectionError):
# NOTE(jamielennox): Again if we can't contact the server we fall
# back to just returning the URL from the catalog. This may not be
# the best default but we need it for now.
LOG.warn(_LW('Failed to contact the endpoint at %s for discovery. '
'Fallback to using that endpoint as the base url.'),
url)
else:
url = disc.url_for(version)
return url
get_endpoints函数执行service_catalog= self.get_access(session).service_catalog来获取相关服务的endpoint信息。
def get_access(self, session, **kwargs):
"""Fetch or return a current AccessInfo object.
If a valid AccessInfo is present then it is returned otherwise a new
one will be fetched.
:param session: A session object that can be used for communication.
:type session: keystoneclient.session.Session
:raises keystoneclient.exceptions.HttpError: An error from an invalid
HTTP response.
:returns: Valid AccessInfo
:rtype: :py:class:`keystoneclient.access.AccessInfo`
"""
if self._needs_reauthenticate():
self.auth_ref = self.get_auth_ref(session)
return self.auth_ref
其中_needs_reauthenticate函数的解释见上面的讲解，因为我们已经从keystone获取到token且token并未过期，所以_needs_reauthenticate函数返回False，即不需要重新获取token。因此直接返回self.auth_ref，而self.auth_ref则保存了各种服务的endpoints，因此我们能从self.auth_ref中获取nova服务的endpoint。本环境self.auth_ref的信息如下。
|
{u'token': {u'issued_at': u'2016-03-17T13:15:31.470321', u'expires': u'2016-03-17T14:15:31Z', u'id': u'8309ccdd9ed94d1ba54989b7764a35d4', u'tenant': {u'enabled': True, u'description': u'admin tenant', u'name': u'admin', u'id': u'09e04766c06d477098201683497d3878'}, u'audit_ids': [u'3w7Sg64nSqys-nMSYeH81A'] }, 'version': 'v2.0', u'serviceCatalog': [ {u'endpoints_links': [], u'endpoints': [{ u'adminURL': u'http://192.168.118.1:8774/v2/09e04766c06d477098201683497d3878', u'region': u'RegionOne', u'publicURL': u'http://192.168.118.1:8774/v2/09e04766c06d477098201683497d3878', u'internalURL': u'http://192.168.118.1:8774/v2/09e04766c06d477098201683497d3878', u'id': u'03966fa6606945b985d8ff3ba1912f00' }], u'type': u'compute', u'name': u'nova' },
{u'endpoints_links': [], u'endpoints': [{ u'adminURL': u'http://192.168.118.1:9696/', u'region': u'RegionOne', u'publicURL': u'http://192.168.118.1:9696/', u'internalURL': u'http://192.168.118.1:9696/', u'id': u'6ecc803364da42b7a250bf9fe2e71cc4' }], u'type': u'network', u'name': u'neutron' },
{u'endpoints_links': [], u'endpoints': [{ u'adminURL': u'http://192.168.118.1:8776/v2/09e04766c06d477098201683497d3878', u'region': u'RegionOne', u'publicURL': u'http://192.168.118.1:8776/v2/09e04766c06d477098201683497d3878', u'internalURL': u'http://192.168.118.1:8776/v2/09e04766c06d477098201683497d3878', u'id': u'1a22050d1c404a1fa257b7de8453f7b5' }], u'type': u'volumev2', u'name': u'cinderv2' },
{u'endpoints_links': [], u'endpoints': [{ u'adminURL': u'http://192.168.118.1:8774/v3', u'region': u'RegionOne', u'publicURL': u'http://192.168.118.1:8774/v3', u'internalURL': u'http://192.168.118.1:8774/v3', u'id': u'4095ec6b1d9c4e3ba87994a90a0d1ed5' }], u'type': u'computev3', u'name': u'novav3' },
{u'endpoints_links': [], u'endpoints': [{ u'adminURL': u'http://192.168.118.1:9292', u'region': u'RegionOne', u'publicURL': u'http://192.168.118.1:9292', u'internalURL': u'http://192.168.118.1:9292', u'id': u'7cd86837358f4c19bb551fe3c36fadf9' }], u'type': u'image', u'name': u'glance' },
{u'endpoints_links': [], u'endpoints': [{ u'adminURL': u'http://192.168.118.1:8777', u'region': u'RegionOne', u'publicURL': u'http://192.168.118.1:8777', u'internalURL': u'http://192.168.118.1:8777', u'id': u'3d9c418b79f641a291a885fe20ec6a84' }], u'type': u'metering', u'name': u'ceilometer' },
{u'endpoints_links': [], u'endpoints': [{ u'adminURL': u'http://192.168.118.1:8776/v1/09e04766c06d477098201683497d3878', u'region': u'RegionOne', u'publicURL': u'http://192.168.118.1:8776/v1/09e04766c06d477098201683497d3878', u'internalURL': u'http://192.168.118.1:8776/v1/09e04766c06d477098201683497d3878', u'id': u'65560ec5eed64581a72f4ac3d1fa519d' }], u'type': u'volume', u'name': u'cinder' },
{u'endpoints_links': [], u'endpoints': [{ u'adminURL': u'http://192.168.118.1:8773/services/Admin', u'region': u'RegionOne', u'publicURL': u'http://192.168.118.1:8773/services/Cloud', u'internalURL': u'http://192.168.118.1:8773/services/Cloud', u'id': u'1b6b0ae5906345d3a6ec0352c97e724d' }], u'type': u'ec2', u'name': u'nova_ec2' },
{u'endpoints_links': [], u'endpoints': [{ u'adminURL': u'http://192.168.118.1:35357/v2.0', u'region': u'RegionOne', u'publicURL': u'http://192.168.118.1:5000/v2.0', u'internalURL': u'http://192.168.118.1:5000/v2.0', u'id': u'11b8c840c71c4258a644bf93ddee8d0f' }], u'type': u'identity', u'name': u'keystone' } ],
u'user': {u'username': u'admin', u'roles_links': [], u'id': u'f59f17d8e9774eef8730b23ecdc86a4b', u'roles': [{u'name': u'admin'}], u'name': u'admin' },
u'metadata': {u'is_admin': 0, u'roles': [u'397eaf49b01549dab8be01804bec7972'] } } |
最终根据获取的nova的endpoint去构造http向nova-api请求VM的列表。信息如下：
通过debug信息得到的请求信息如下：
| DEBUG (session:197) REQ: curl -g -i -X GET http://192.168.118.1:8774/v2/09e04766c06d477098201683497d3878/servers/detail -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}dfc659e470d6b394760f1ce437535806bddbca61" |
注意上面的X-Auth-Token部分，即：
| -H "X-Auth-Token: {SHA1}dfc659e470d6b394760f1ce437535806bddbca61" |
其实这个X-Auth-Token中的value值即为token值，但是这次请求的token值为什么跟从keystone中返回的token值不一样呢？这是因为打印debug信息的时候，OpenStack为了安全性，对keystone返回token值做了一次hash处理，让打印出来的token值与实际获得的token值不一样，当然这只是 打印hash处理后的token值，实际代码中拿去请求VM列表的token值还是与从keystone返回回来的token值一致。
被nova-api返回的debug信息类似如下信息(第一部分为header，第二部分为body)：
| DEBUG (session:226) RESP: [200] date: Mon, 14 Mar 2016 13:27:34 GMT connection: keep-alive content-type: application/json content-length: 15 x-compute-request-id: req-d9689353-c84b-4aba-95b6-dacdb3224d70
RESP BODY: {"servers": []} |
由于我的环境上没有VM，所以body信息为空。
5. 总结
本文通过nova list命令分析了从novaclient->keystoneclient，再最终获取VM列表的过程。从debug信息可以看出，nova list命令主要执行了3条请求。
1. 从keystone中获取keystone的版本信息。请求信息如下：
| DEBUG (session:197) REQ: curl -g -i -X GET http://192.168.118.1:5000/v2.0/ -H "Accept: application/json" -H "User-Agent: python-keystoneclient" |
2. 根据从第一条请求返回的keystone版本信息，构建合适的plugin对象，本环境采用的是v2.Password plugin对象，然后调用该plugin的函数去从keystone中获取token值。请求信息如下：
| DEBUG (session:197) REQ: curl -g -i -X POST http://192.168.118.1:5000/v2.0/tokens -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "admin"}}}' |
3. 根据获取的token值从nova-api中请求VM的列表，请求信息如下：
| DEBUG (session:197) REQ: curl -g -i -X GET http://192.168.118.1:8774/v2/09e04766c06d477098201683497d3878/servers/detail -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}dfc659e470d6b394760f1ce437535806bddbca61" |
注意：虽然我们执行nova list命令是novaclient的命令，但是最终所有的代码流程都走到keystoneclient中去执行的，那是因为我们采用session的方式进行VM列表的获取，即当采用session的方式时，会创建一个SessionClient对象，然后最终所有的操作将在keystoneclient中完成，然后将返回回来的VM列表信息返回给novaclient，其他不是session的方式，则会创建一个HTTPClient对象，而对于该方式的调用，我也没研究过，有兴趣可以将/novaclient/shell.py: OpenStackComputeShell.main函数中的use_session变量值修改为False进行调试，看看这种不使用session方式的代码流程.