keystone WSGI流程
作为OpenStack两种主要的通信方式(RESTful API与消息总线)之一，理解RESTful API的设计思路和执行过程，有助于我们对OpenStack有更好的理解。RESTful只是设计风格而不是标准，Web服务中通常使用基于HTTP的符合RESTful风格的API。而WSGI(Web ServerGateway Interface)则是python语言中所定义的Web服务器和Web应用程序或框架之间的通用接口标准。
在OpenStack中随处可见基于WSGI的通信，如nova-api，keystone等等，其工作方式为：OpenStack的API服务进程接收到客户端的HTTP请求时，一个所谓的“路由”模块会将请求的URL装换成相应的资源，并路由到合适的操作函数上。
本文将介绍keystone的WSGI通信，并根据如何从keystone中获取token信息的举例方式进行介绍。
1. 创建WSGI server
keystone是通过/usr/bin/keyston-all脚本进行启动的。
#/usr/bin/keyston-all
import os
import sys
# If ../keystone/__init__.py exists, add ../ to Python search path, so that
# it will override what happens to be installed in /usr/(local/)lib/python...
possible_topdir = os.path.normpath(os.path.join(os.path.abspath(__file__),
os.pardir,
os.pardir))
if os.path.exists(os.path.join(possible_topdir,
'keystone',
'__init__.py')):
sys.path.insert(0, possible_topdir)
from keystone.server import eventlet as eventlet_server
if __name__ == '__main__':
eventlet_server.run(possible_topdir)
最终执行到/keystone/server/evenlet.py中的run函数。
#/keystone/server/eventlet.py
def run(possible_topdir):
dev_conf = os.path.join(possible_topdir,
'etc',
'keystone.conf')
config_files = None
if os.path.exists(dev_conf):
config_files = [dev_conf]
common.configure(
version=pbr.version.VersionInfo('keystone').version_string(),
config_files=config_files,
pre_setup_logging_fn=configure_threading)
paste_config = config.find_paste_config()
def create_servers():
admin_worker_count = _get_workers('admin_workers')
public_worker_count = _get_workers('public_workers')
servers = []
servers.append(create_server(paste_config,
'admin',
CONF.eventlet_server.admin_bind_host,
CONF.eventlet_server.admin_port,
admin_worker_count))
servers.append(create_server(paste_config,
'main',
CONF.eventlet_server.public_bind_host,
CONF.eventlet_server.public_port,
public_worker_count))
return servers
_unused, servers = common.setup_backends(
startup_application_fn=create_servers)
serve(*servers)
首先加载配置文件的配置选项，然后找到keystone的paste配置文件(paste_config= config.find_paste_config())，因为OpenStack使用paste的deploy组件来完成WSGI服务器和应用的构建，其中keystone的paste配置文件位于/usr/share/keystone目录下，文件名为keystone-dist-paste.ini，对于该配置文件的使用，我会在后续内容中进行介绍，更细致的使用可参考paste的deploy的官方文档(http://pythonpaste.org/deploy/)。
然后执行common.setup_backends方法加载backend driver和创建admin和main的WSGI的server。这里我们看看common.setup_backends如何操作的呢？
#/keystone/server/common.py
def setup_backends(load_extra_backends_fn=lambda: {},
startup_application_fn=lambda: None):
drivers = backends.load_backends()
drivers.update(load_extra_backends_fn())
res = startup_application_fn()
drivers.update(dependency.resolve_future_dependencies())
return drivers, res
#/keystone/backends.py
def load_backends():
# Configure and build the cache
cache.configure_cache_region(cache.REGION)
# Ensure that the identity driver is created before the assignment manager
# and that the assignment driver is created before the resource manager.
# The default resource driver depends on assignment, which in turn
# depends on identity - hence we need to ensure the chain is available.
_IDENTITY_API = identity.Manager()
_ASSIGNMENT_API = assignment.Manager()
DRIVERS = dict(
assignment_api=_ASSIGNMENT_API,
catalog_api=catalog.Manager(),
credential_api=credential.Manager(),
domain_config_api=resource.DomainConfigManager(),
endpoint_filter_api=endpoint_filter.Manager(),
endpoint_policy_api=endpoint_policy.Manager(),
federation_api=federation.Manager(),
id_generator_api=identity.generator.Manager(),
id_mapping_api=identity.MappingManager(),
identity_api=_IDENTITY_API,
oauth_api=oauth1.Manager(),
policy_api=policy.Manager(),
resource_api=resource.Manager(),
revoke_api=revoke.Manager(),
role_api=assignment.RoleManager(),
token_api=token.persistence.Manager(),
trust_api=trust.Manager(),
token_provider_api=token.provider.Manager())
auth.controllers.load_auth_methods()
return DRIVERS
参考这篇文章(http://bingotree.cn/?p=150)以及《OpenStack设计与实现》，我们知道，在”DRIVERS”字典中，每一个键值对都定义了一类keystone API实现，它们之间存在相互依赖的可能(所以在这里采用了依赖注入的设计模式)，如下：
#/keystone/identity/core.py:Manager
@dependency.provider('identity_api')
@dependency.requires('assignment_api', 'credential_api', 'id_mapping_api',
'resource_api', 'revoke_api')
class Manager(manager.Manager):
依赖注入模式就是在/keystone/common/dependency.py文件中实现的，对于上面的的作用为：如果一个class被加了@dependency.provider(‘xxx’),那么其就会生成一个实例，放置于_REGISTRY = {}全局变量中。名字叫做xxx。以后其他模块如果想要引用这个模块，那么只需要在class前面加上@dependency.requires(‘xxx’)或@dependency.optional(‘xxx’)就可以了。当加上了@dependency.requires后，这个模块就会有一个叫做xxx的属性，该属性的值就是对xxx的引用。
比方说下面这个最简单的例子：
@dependency.provider('XXX')
class XXX():
pass
@dependency.requires('XXX')
class YYY():
pass
#然后YYY的实例就有了XXX这个属性：
yyy = YYY()
print type(yyy.XXX)
当然啦，这里还有一个细节那就是在初始化identity这个Manager的时候，我们的@dependency.requires(‘assignment_api’, ‘credential_api’, ‘token_api’)是不满足要求的，因为这三个API我们还没通过dependency.provider注册。所以在源码中有这样的注释：”Objects must not rely on the existence of these attributes untilafter ‘resolve_future_dependencies’ has been called; they may not existbeforehand.”。这些require的东东需要等到resolve_future_dependencies被调用后才能被正常使用。
因此调用load_backends方法就加载了后续会使用的backend driver。然后执行res = startup_application_fn()代码创建WSGI的server。
#/keystone/server/eventlet.py
def create_servers():
admin_worker_count = _get_workers('admin_workers')
public_worker_count = _get_workers('public_workers')
servers = []
servers.append(create_server(paste_config,
'admin',
CONF.eventlet_server.admin_bind_host,
CONF.eventlet_server.admin_port,
admin_worker_count))
servers.append(create_server(paste_config,
'main',
CONF.eventlet_server.public_bind_host,
CONF.eventlet_server.public_port,
public_worker_count))
return servers
#/keystone/server/eventlet.py
def _get_workers(worker_type_config_opt):
# Get the value from config, if the config value is None (not set), return
# the number of cpus with a minimum of 2.
worker_count = CONF.eventlet_server.get(worker_type_config_opt)
if not worker_count:
worker_count = max(2, processutils.get_worker_count())
return worker_count
#/keystone/server/eventlet.py
def create_server(conf, name, host, port, workers):
app = keystone_service.loadapp('config:%s' % conf, name)
server = environment.Server(app, host=host, port=port,
keepalive=CONF.eventlet_server.tcp_keepalive,
keepidle=CONF.eventlet_server.tcp_keepidle)
if CONF.eventlet_server_ssl.enable:
server.set_ssl(CONF.eventlet_server_ssl.certfile,
CONF.eventlet_server_ssl.keyfile,
CONF.eventlet_server_ssl.ca_certs,
CONF.eventlet_server_ssl.cert_required)
return name, ServerWrapper(server, workers)
#/keystone/service.py
def loadapp(conf, name):
# NOTE(blk-u): Save the application being loaded in the controllers module.
# This is similar to how public_app_factory() and v3_app_factory()
# register the version with the controllers module.
controllers.latest_app = deploy.loadapp(conf, name=name)
return controllers.latest_app
其中create_servers方法首先获取欲创建的admin和main的WSGI server的进程个数(work count)，如果配置文件没有定义创建的进程个数，则OpenStack将通过计算计算机的cpu个数与2进行比较，选择最大的那个数作为创建的进程个数。我的环境没有在配置文件中进行设置创建进程的个数，且本环境的cpu个数为4，因此这里会创建8个WSGI server进程(4个admin WSGI server和4个main WSGI server)。如下：
| [root@jun ~]# ps -elf | grep keystone 5 S keystone 8882 4329 0 80 0 - 103806 poll_s 11:06 ? 00:00:01 keystone-admin -DFOREGROUND 5 S keystone 8883 4329 0 80 0 - 103806 poll_s 11:06 ? 00:00:01 keystone-main -DFOREGROUND 4 S keystone 50511 1 1 80 0 - 86478 poll_s 20:06 ? 00:00:09 /usr/bin/python /usr/bin/keystone-all 1 S keystone 50522 50511 0 80 0 - 114690 ep_pol 20:06 ? 00:00:01 /usr/bin/python /usr/bin/keystone-all 1 S keystone 50523 50511 0 80 0 - 114193 ep_pol 20:06 ? 00:00:00 /usr/bin/python /usr/bin/keystone-all 1 S keystone 50524 50511 0 80 0 - 114527 ep_pol 20:06 ? 00:00:00 /usr/bin/python /usr/bin/keystone-all 1 S keystone 50525 50511 0 80 0 - 114585 ep_pol 20:06 ? 00:00:00 /usr/bin/python /usr/bin/keystone-all 1 S keystone 50526 50511 0 80 0 - 86511 ep_pol 20:06 ? 00:00:00 /usr/bin/python /usr/bin/keystone-all 1 S keystone 50527 50511 0 80 0 - 86511 ep_pol 20:06 ? 00:00:00 /usr/bin/python /usr/bin/keystone-all 1 S keystone 50528 50511 0 80 0 - 86511 ep_pol 20:06 ? 00:00:00 /usr/bin/python /usr/bin/keystone-all 1 S keystone 50529 50511 0 80 0 - 86511 ep_pol 20:06 ? 00:00:00 /usr/bin/python /usr/bin/keystone-all 0 S root 51358 7967 0 80 0 - 28161 pipe_w 20:17 pts/2 00:00:00 grep --color=auto keystone |
其中进程号为50511的进程为创建WSGI server的父进程，进程号为50522~50529的进程为8个WSGI server进程。
在得到欲创建的WSGI server的进程数后，则去执行create_server函数，该函数执行app = keystone_service.loadapp('config:%s' % conf, name)代码来从paste配置文件中生成一个WSGI应用。这里只是加载WSGI应用，并没有执行生成的WSGI应用，只有在keystone服务进程收到keystoneclient的HTTP请求时，才会触发使用WSGI应用，对于如何使用该WSGI应用，我们会在下一小节进行详细分析。
在加载WSGI应用之后，创建WSGI server，且如果配置文件中使能了SSL，则需对创建的WSGI server进行SSL 设置。
#/keystone/common/environment/eventlet_server.py:Server
class Server(object):
"""Server class to manage multiple WSGI sockets and applications."""
def __init__(self, application, host=None, port=None, keepalive=False,
keepidle=None):
self.application = application
self.host = host or '0.0.0.0'
self.port = port or 0
# Pool for a green thread in which wsgi server will be running
self.pool = eventlet.GreenPool(POOL_SIZE)
self.socket_info = {}
self.greenthread = None
self.do_ssl = False
self.cert_required = False
self.keepalive = keepalive
self.keepidle = keepidle
self.socket = None其中create_server只是创建创建了一个Server对象，并没有监听相应的端口(而创建这个Server对象的进程就是WSGI server的父进程，即上面举例的50511号进程)。
最后create_server将创建的server和workers使用ServerWrapper类进行封装，并将WSGI server的name(即admin和main)和封装后的WSGI server进行返回。
最后setup_backends函数执行drivers.update(dependency.resolve_future_dependencies())后返回。回到/keystone/server/eventlet.py的run函数，最终run函数执行serve方法，该方法会创建dmin和main的WSGI server。
#/keystone/server/eventlet.py
def serve(*servers):
logging.warning(_('Running keystone via eventlet is deprecated as of Kilo '
'in favor of running in a WSGI server (e.g. mod_wsgi). '
'Support for keystone under eventlet will be removed in '
'the "M"-Release.'))
if max([server[1].workers for server in servers]) > 1:
launcher = service.ProcessLauncher()
else:
launcher = service.ServiceLauncher()
for name, server in servers:
try:
server.launch_with(launcher)
except socket.error:
logging.exception(_('Failed to start the %(name)s server') % {
'name': name})
raise
# notify calling process we are ready to serve
systemd.notify_once()
for name, server in servers:
launcher.wait()
因为admin和main的WSGI server的works都为4，所以执行lanuncher = service.ProcessLauncher()，即创建一个ProcessLauncher对象。
#/keystone/openstack/common/service.py:ProcessLauncher
class ProcessLauncher(object):
_signal_handlers_set = set()
@classmethod
def _handle_class_signals(cls, *args, **kwargs):
for handler in cls._signal_handlers_set:
handler(*args, **kwargs)
def __init__(self, wait_interval=0.01):
"""Constructor.
:param wait_interval: The interval to sleep for between checks
of child process exit.
"""
self.children = {}
self.sigcaught = None
self.running = True
self.wait_interval = wait_interval
rfd, self.writepipe = os.pipe()
self.readpipe = eventlet.greenio.GreenPipe(rfd, 'r')
self.handle_signal()
然后在对admin和main的WSGI server分别执行server.launch_with(launcher)语句。
#/keystone/server/eventlet.py:ServerWrapper
class ServerWrapper(object):
"""Wraps a Server with some launching info & capabilities."""
def __init__(self, server, workers):
self.server = server
self.workers = workers
def launch_with(self, launcher):
self.server.listen()
if self.workers > 1:
# Use multi-process launcher
launcher.launch_service(self.server, self.workers)
else:
# Use single process launcher
launcher.launch_service(self.server)
#/keystone/common/environment/eventlet_server.py:Server
def listen(self, key=None, backlog=128):
"""Create and start listening on socket.
Call before forking worker processes.
Raises Exception if this has already been called.
"""
# TODO(dims): eventlet's green dns/socket module does not actually
# support IPv6 in getaddrinfo(). We need to get around this in the
# future or monitor upstream for a fix.
# Please refer below link
# (https://bitbucket.org/eventlet/eventlet/
# src/e0f578180d7d82d2ed3d8a96d520103503c524ec/eventlet/support/
# greendns.py?at=0.12#cl-163)
info = socket.getaddrinfo(self.host,
self.port,
socket.AF_UNSPEC,
socket.SOCK_STREAM)[0]
try:
self.socket = eventlet.listen(info[-1], family=info[0],
backlog=backlog)
except EnvironmentError:
LOG.error(_LE("Could not bind to %(host)s:%(port)s"),
{'host': self.host, 'port': self.port})
raise
LOG.info(_LI('Starting %(arg0)s on %(host)s:%(port)s'),
{'arg0': sys.argv[0],
'host': self.host,
'port': self.port})
这里执行launch_with函数时，首先使用WSGI server的父进程对相应的端口进行监听。
| [root@jun ~]# netstat -tnulp | grep 50511 tcp 0 0 0.0.0.0:5000 0.0.0.0:* LISTEN 50511/python tcp 0 0 0.0.0.0:35357 0.0.0.0:* LISTEN 50511/python |
如上所示，WSGI server的父进程(50511号进程)开启两个socket去分别监听本环境的5000和35357号端口，其中5000号端口是为main的WSGI server提供的，35357号端口为admin的WSGI server提供的。即WSGI server的父进程接收到5000号端口的HTTP请求时，则将把该请求转发给为main开启的WSGI server去处理，而WSGI server的父进程接收到35357号端口的HTTP请求时，则将把该请求转发给为admin开启的WSGI server去处理。
在开启WSGI server的父进程的socket监听后，才会正式创建admin和main的WSGI server。即执行launcher.launch_service(self.server,self.workers)语句。
#/keystone/openstack/common/service.py:ProcessLauncher
def launch_service(self, service, workers=1):
wrap = ServiceWrapper(service, workers)
LOG.info(_LI('Starting %d workers'), wrap.workers)
while self.running and len(wrap.children) < wrap.workers:
self._start_child(wrap)
#/keystone/openstack/common/service.py:ProcessLauncher
def _start_child(self, wrap):
if len(wrap.forktimes) > wrap.workers:
# Limit ourselves to one process a second (over the period of
# number of workers * 1 second). This will allow workers to
# start up quickly but ensure we don't fork off children that
# die instantly too quickly.
if time.time() - wrap.forktimes[0] < wrap.workers:
LOG.info(_LI('Forking too fast, sleeping'))
time.sleep(1)
wrap.forktimes.pop(0)
wrap.forktimes.append(time.time())
pid = os.fork()
if pid == 0:
launcher = self._child_process(wrap.service)
while True:
self._child_process_handle_signal()
status, signo = self._child_wait_for_exit_or_signal(launcher)
if not _is_sighup_and_daemon(signo):
break
launcher.restart()
os._exit(status)
LOG.info(_LI('Started child %d'), pid)
wrap.children.add(pid)
self.children[pid] = wrap
return pid
其中在创建的子进程中执行(pid== 0)中执行launcher =self._child_process(wrap.service)创建一个Launcher对象，并加载/keystone/openstack/common/service.py:Services中的run_service函数。那么它如何加载run_service函数的呢？如下
#/keystone/openstack/common/service.py:ProcessLauncher
def _child_process(self, service):
self._child_process_handle_signal()
# Reopen the eventlet hub to make sure we don't share an epoll
# fd with parent and/or siblings, which would be bad
eventlet.hubs.use_hub()
# Close write to ensure only parent has it open
os.close(self.writepipe)
# Create greenthread to watch for parent to close pipe
eventlet.spawn_n(self._pipe_watcher)
# Reseed random number generator
random.seed()
launcher = Launcher()
launcher.launch_service(service)
return launcher
#/keystone/openstack/common/service.py:Launcher
class Launcher(object):
"""Launch one or more services and wait for them to complete."""
def __init__(self):
"""Initialize the service launcher.
:returns: None
"""
self.services = Services()
self.backdoor_port = eventlet_backdoor.initialize_if_enabled()
def launch_service(self, service):
"""Load and start the given service.
:param service: The service you would like to start.
:returns: None
"""
service.backdoor_port = self.backdoor_port
self.services.add(service)
#/keystone/openstack/common/service.py:Services
class Services(object):
def __init__(self):
self.services = []
self.tg = threadgroup.ThreadGroup()
self.done = event.Event()
def add(self, service):
self.services.append(service)
self.tg.add_thread(self.run_service, service, self.done)
从上面的代码可以看出，加载run_service函数是在/keystone/openstack/common/service.py:Launcher中的launch_service函数中进行加载。当然，这里只是简单的加载run_service函数，并不会立即执行该函数。run_service函数的执行是在/keystone/openstack/common/service.py:ProcessLauncher中_start_child中的这条语句执行的：status,signo = self._child_wait_for_exit_or_signal(launcher)。如下
#/keystone/openstack/common/service.py:ProcessLauncher
def _child_wait_for_exit_or_signal(self, launcher):
status = 0
signo = 0
# NOTE(johannes): All exceptions are caught to ensure this
# doesn't fallback into the loop spawning children. It would
# be bad for a child to spawn more children.
try:
launcher.wait()
except SignalExit as exc:
signame = _signo_to_signame(exc.signo)
LOG.info(_LI('Child caught %s, exiting'), signame)
status = exc.code
signo = exc.signo
except SystemExit as exc:
status = exc.code
except BaseException:
LOG.exception(_LE('Unhandled exception'))
status = 2
finally:
launcher.stop()
return status, signo
当执行launcher.wait()语句时，将去执行run_service函数。那么run_service函数做了哪些操作呢？
#/keystone/openstack/common/service.py:Services
@staticmethod
def run_service(service, done):
"""Service start wrapper.
:param service: service to run
:param done: event to wait on until a shutdown is triggered
:returns: None
"""
service.start()
done.wait()
这里run_service执行service.start则会去真正的创建WSGI server，这里service即为/keystone/common/environment/eventlet_server.py中的Server对象。即run_service执行Server对象的start函数。
#/keystone/common/environment/eventlet_server.py:Server
def start(self, key=None, backlog=128):
"""Run a WSGI server with the given application."""
if self.socket is None:
self.listen(key=key, backlog=backlog)
dup_socket = self.socket.dup()
if key:
self.socket_info[key] = self.socket.getsockname()
# SSL is enabled
if self.do_ssl:
if self.cert_required:
cert_reqs = ssl.CERT_REQUIRED
else:
cert_reqs = ssl.CERT_NONE
dup_socket = eventlet.wrap_ssl(dup_socket, certfile=self.certfile,
keyfile=self.keyfile,
server_side=True,
cert_reqs=cert_reqs,
ca_certs=self.ca_certs)
# Optionally enable keepalive on the wsgi socket.
if self.keepalive:
dup_socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
if self.keepidle is not None:
dup_socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPIDLE,
self.keepidle)
self.greenthread = self.pool.spawn(self._run,
self.application,
dup_socket)
#/keystone/common/environment/eventlet_server.py:Server
def _run(self, application, socket):
"""Start a WSGI server with a new green thread pool."""
logger = log.getLogger('eventlet.wsgi.server')
socket_timeout = CONF.eventlet_server.client_socket_timeout or None
try:
eventlet.wsgi.server(
socket, application, log=EventletFilteringLogger(logger),
debug=False, keepalive=CONF.eventlet_server.wsgi_keep_alive,
socket_timeout=socket_timeout)
except greenlet.GreenletExit:
# Wait until all servers have completed running
pass
except Exception:
LOG.exception(_LE('Server error'))
raise
最终在所创建的每个子进程中创建一个协程来开启一个WSGI server，即在_run函数中创建和开启了一个WSGIserver。
那么在开启了WSGI server后，当WSGI server的父进程收到有HTTP请求，则将其请求发送到合适的WSGIserver上进行处理，而WSGI server如何处理HTTP请求，我们将在下一节进行分析。
2. 分析keystoneclient到keystone的HTTP请求过程
这里我们利用上一篇文章《novalist命令的代码流程》中的获取token的HTTP请求作为例子进行讲解。即如下的HTTP请求。
| DEBUG (session:197) REQ: curl -g -i -X POST http://192.168.118.1:5000/v2.0/tokens -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "admin"}}}' |
目前OpenStack使用paste的deploy组件完成WSGI服务器和应用的构建，且deploy的工作是基于.ini结尾的配置文件，这里基于keystone的配置文件为keystone-dist-paste.ini。其内容如下。
| # Keystone PasteDeploy configuration file.
[filter:debug] paste.filter_factory = keystone.common.wsgi:Debug.factory
[filter:request_id] paste.filter_factory = oslo_middleware:RequestId.factory
[filter:build_auth_context] paste.filter_factory = keystone.middleware:AuthContextMiddleware.factory
[filter:token_auth] paste.filter_factory = keystone.middleware:TokenAuthMiddleware.factory
[filter:admin_token_auth] paste.filter_factory = keystone.middleware:AdminTokenAuthMiddleware.factory
[filter:json_body] paste.filter_factory = keystone.middleware:JsonBodyMiddleware.factory
[filter:user_crud_extension] paste.filter_factory = keystone.contrib.user_crud:CrudExtension.factory
[filter:crud_extension] paste.filter_factory = keystone.contrib.admin_crud:CrudExtension.factory
[filter:ec2_extension] paste.filter_factory = keystone.contrib.ec2:Ec2Extension.factory
[filter:ec2_extension_v3] paste.filter_factory = keystone.contrib.ec2:Ec2ExtensionV3.factory
[filter:federation_extension] paste.filter_factory = keystone.contrib.federation.routers:FederationExtension.factory
[filter:oauth1_extension] paste.filter_factory = keystone.contrib.oauth1.routers:OAuth1Extension.factory
[filter:s3_extension] paste.filter_factory = keystone.contrib.s3:S3Extension.factory
[filter:endpoint_filter_extension] paste.filter_factory = keystone.contrib.endpoint_filter.routers:EndpointFilterExtension.factory
[filter:endpoint_policy_extension] paste.filter_factory = keystone.contrib.endpoint_policy.routers:EndpointPolicyExtension.factory
[filter:simple_cert_extension] paste.filter_factory = keystone.contrib.simple_cert:SimpleCertExtension.factory
[filter:revoke_extension] paste.filter_factory = keystone.contrib.revoke.routers:RevokeExtension.factory
[filter:url_normalize] paste.filter_factory = keystone.middleware:NormalizingFilter.factory
[filter:sizelimit] paste.filter_factory = oslo_middleware.sizelimit:RequestBodySizeLimiter.factory
[app:public_service] paste.app_factory = keystone.service:public_app_factory
[app:service_v3] paste.app_factory = keystone.service:v3_app_factory
[app:admin_service] paste.app_factory = keystone.service:admin_app_factory
[pipeline:public_api] # The last item in this pipeline must be public_service or an equivalent # application. It cannot be a filter. pipeline = sizelimit url_normalize request_id build_auth_context token_auth admin_token_auth json_body ec2_extension user_crud_extension public_service
[pipeline:admin_api] # The last item in this pipeline must be admin_service or an equivalent # application. It cannot be a filter. pipeline = sizelimit url_normalize request_id build_auth_context token_auth admin_token_auth json_body ec2_extension s3_extension crud_extension admin_service
[pipeline:api_v3] # The last item in this pipeline must be service_v3 or an equivalent # application. It cannot be a filter. pipeline = sizelimit url_normalize request_id build_auth_context token_auth admin_token_auth json_body ec2_extension_v3 s3_extension simple_cert_extension revoke_extension oauth1_extension endpoint_filter_extension endpoint_policy_extension service_v3
[app:public_version_service] paste.app_factory = keystone.service:public_version_app_factory
[app:admin_version_service] paste.app_factory = keystone.service:admin_version_app_factory
[pipeline:public_version_api] pipeline = sizelimit url_normalize public_version_service
[pipeline:admin_version_api] pipeline = sizelimit url_normalize admin_version_service
[composite:main] use = egg:Paste#urlmap /v2.0 = public_api /v3 = api_v3 / = public_version_api
[composite:admin] use = egg:Paste#urlmap /v2.0 = admin_api /v3 = api_v3 / = admin_version_api |
参考书籍《OpenStack设计与实现》知道，paste配置文件分为多个section，每个section以type：name的格式命名。
(1) type = composite
这个类型的section会把URL请求分发到对应的Application，use表明具体的分发方式，比如”egg:Paste#urlmap”表示使用Paste包中的urlmap模块，这个section里的其他形式如”key = value”的行是使用urlmap进行分发时的参数。
(2) type = app
一个app就是一个具体的WSGI Application。
(3) type = filter-app
接收一个请求后，会首先调用filter-app中的use所指定的app进行过滤，如果这个请求没有被过滤，就会转发到next所指定的app进行下一步的处理。
(4) type = filter
与filter-app类型的区别只是没有next。
(5) type = pipeline
pipeline由一系列filter组成。这个filter链条的末尾是一个app。pipeline类型主要是对filter-app进行了简化，否则，如果多个filter，就需要多个filter-app，然后使用next进行连接。OpenStack的paste的deploy的配置文件主要采用的pipeline的方式。
参看这篇文章http://blog.csdn.net/ztejiagn/article/details/8722765，我们知道pipeline的使用方式。例如：
| [pipeline:test1] pipeline = filter1 filter2 filter3 app
[filter:filter1] paste.filter_factory = xxx
[filter:filter2] paste.filter_factory = yyy
[filter:filter3] paste.filter_factory = zzz |
假设在ini文件中, 某条pipeline的顺序是filter1, filter2, filter3，app, 那么，最终运行的app_real是这样组织的： app_real =filter1(filter2(filter3(app)))
在app真正被调用的过程中，filter1.__call__(environ,start_response)被首先调用，若某种检查未通过，filter1做出反应；否则交给filter2.__call__(environ,start_response)进一步处理，若某种检查未通过，filter2做出反应，中断链条，否则交给filter3.__call__(environ,start_response)处理，若filter3的某种检查都通过了，最后交给app.__call__(environ,start_response)进行处理。
下面我们具体分析keystone如何处理keystoneclient的HTTP请求。
| DEBUG (session:197) REQ: curl -g -i -X POST http://192.168.118.1:5000/v2.0/tokens -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "admin"}}}' |
这里，我们的基本url为http://192.168.118.1:5000，而5000端口是为main的WSGI server开启的监听端口，所以当WSGI server的父进程监听到5000端口有HTTP请求，则它将会把HTTP请求转发给main的WSGI server进行处理，而main的WSGI server则根据keystone-dist-paste.ini配置文件内容对其HTTP请求作相应处理。
1. composite的处理
| [composite:main] use = egg:Paste#urlmap /v2.0 = public_api /v3 = api_v3 / = public_version_api |
因为url为http://192.168.118.1:5000/v2.0/tokens，因为基本url的后面接的信息为/v2.0，所以将到public_api的section作相应操作。
2. pipeline的处理
| [pipeline:public_api] # The last item in this pipeline must be public_service or an equivalent # application. It cannot be a filter. pipeline = sizelimit url_normalize request_id build_auth_context token_auth admin_token_auth json_body ec2_extension user_crud_extension public_service |
2.1 sizelimite filter处理
| [filter:sizelimit] paste.filter_factory = oslo_middleware.sizelimit:RequestBodySizeLimiter.factory |
#/oslo_middleware/sizelimit.py:RequestBodySizeLimiter
class RequestBodySizeLimiter(base.Middleware):
"""Limit the size of incoming requests."""
@webob.dec.wsgify
def __call__(self, req):
max_size = CONF.oslo_middleware.max_request_body_size
if (req.content_length is not None and
req.content_length > max_size):
msg = _("Request is too large.")
raise webob.exc.HTTPRequestEntityTooLarge(explanation=msg)
if req.content_length is None and req.is_body_readable:
limiter = LimitingReader(req.body_file, max_size)
req.body_file = limiter
return self.application
#/oslo_middleware/base.py:Middleware
"""Base class(es) for WSGI Middleware."""
import webob.dec
class Middleware(object):
"""Base WSGI middleware wrapper.
These classes require an application to be initialized that will be called
next. By default the middleware will simply call its wrapped app, or you
can override __call__ to customize its behavior.
"""
@classmethod
def factory(cls, global_conf, **local_conf):
"""Factory method for paste.deploy."""
return cls
def __init__(self, application):
self.application = application
def process_request(self, req):
"""Called on each request.
If this returns None, the next application down the stack will be
executed. If it returns a response then that response will be returned
and execution will stop here.
"""
return None
def process_response(self, response):
"""Do whatever you'd like to the response."""
return response
@webob.dec.wsgify
def __call__(self, req):
response = self.process_request(req)
if response:
return response
response = req.get_response(self.application)
return self.process_response(response)
这里，在paste的deploy的配置文件中，sizelimite filter的paste.filter_factory的value值为factory函数名，在执行paste的deploy.loadapp调用时，则执行了该factory函数，而该函数返回自身类，从而创建了一个自身类。所以当HTTP请求到来时，以函数的形式调用对象时，将会执行该对象的__call__方法。
因此对于本例中的HTTP请求到来时，首先用sizelimite filter去过来HTTP请求，即执行RequestBodySizeLimiter类的__call__方法。分析该方法可知，sizelimite filter主要判断request的内容是否超过配置文件keystone.conf设置的最大的request大小。keystone.conf配置文件默认的max_request_body_size为114688。
此时经过sizelimiterfilter过滤后的request的environ成员变量(字典)的信息为：
| {'SCRIPT_NAME': '/v2.0', 'webob.adhoc_attrs': {'response': |
2.2 url_normalize filter处理
| [filter:url_normalize] paste.filter_factory = keystone.middleware:NormalizingFilter.factory |
#/keystone/common/wsgi.py:Middleware
class Middleware(Application):
"""Base WSGI middleware.
These classes require an application to be
initialized that will be called next. By default the middleware will
simply call its wrapped app, or you can override __call__ to customize its
behavior.
"""
@classmethod
def factory(cls, global_config, **local_config):
"""Used for paste app factories in paste.deploy config files.
Any local configuration (that is, values under the [filter:APPNAME]
section of the paste config) will be passed into the `__init__` method
as kwargs.
A hypothetical configuration would look like:
[filter:analytics]
redis_host = 127.0.0.1
paste.filter_factory = keystone.analytics:Analytics.factory
which would result in a call to the `Analytics` class as
import keystone.analytics
keystone.analytics.Analytics(app, redis_host='127.0.0.1')
You could of course re-implement the `factory` method in subclasses,
but using the kwarg passing it shouldn't be necessary.
"""
def _factory(app):
conf = global_config.copy()
conf.update(local_config)
return cls(app, **local_config)
return _factory
@webob.dec.wsgify()
def __call__(self, request):
try:
response = self.process_request(request)
if response:
return response
response = request.get_response(self.application)
return self.process_response(request, response)
except exception.Error as e:
LOG.warning(six.text_type(e))
return render_exception(e, request=request,
user_locale=best_match_language(request))
except TypeError as e:
LOG.exception(six.text_type(e))
return render_exception(exception.ValidationError(e),
request=request,
user_locale=best_match_language(request))
except Exception as e:
LOG.exception(six.text_type(e))
return render_exception(exception.UnexpectedError(exception=e),
request=request,
user_locale=best_match_language(request))
#/keystone/middleware/core.py:NormalizingFilter
class NormalizingFilter(wsgi.Middleware):
"""Middleware filter to handle URL normalization."""
def process_request(self, request):
"""Normalizes URLs."""
# Removes a trailing slash from the given path, if any.
if (len(request.environ['PATH_INFO']) > 1 and
request.environ['PATH_INFO'][-1] == '/'):
request.environ['PATH_INFO'] = request.environ['PATH_INFO'][:-1]
# Rewrites path to root if no path is given.
elif not request.environ['PATH_INFO']:
request.environ['PATH_INFO'] = '/'
url_normalize filter调用父类Middleware的__call__方法，在__call__方法中，调用自身的process_request函数，该函数主要处理sizelimiter filter传递下来的request的environ成员变量(字典)中的PATH_INFO的value值，由于本文例子sizelimiterfilter过滤后request的environ字典中的PATH_INFO的值为：’/tokens’，所以这里并未做任何处理。
此时打印出来的request的environ成员变量的值与sizelimiter filter过滤后的相同，即为：
| {'SCRIPT_NAME': '/v2.0', 'webob.adhoc_attrs': {'response': |
2.3 request_id filter处理
| [filter:request_id] paste.filter_factory = oslo_middleware:RequestId.factory |
#/oslo_middleware/request_id.py:RequestId
ENV_REQUEST_ID = 'openstack.request_id'
class RequestId(base.Middleware):
"""Middleware that ensures request ID.
It ensures to assign request ID for each API request and set it to
request environment. The request ID is also added to API response.
"""
@webob.dec.wsgify
def __call__(self, req):
req_id = context.generate_request_id()
req.environ[ENV_REQUEST_ID] = req_id
response = req.get_response(self.application)
if HTTP_RESP_HEADER_REQUEST_ID not in response.headers:
response.headers.add(HTTP_RESP_HEADER_REQUEST_ID, req_id)
return response
该filter相当于在request的environ成员变量(字典)中的增加key为openstack.request_id，value值req_id(由context的generate_request_id函数产生的由req-开头的16进制的一串数字)的字典。
最终生成的request的environ成员变量的信息为：
| {'SCRIPT_NAME': '/v2.0', 'webob.adhoc_attrs': {'response': |
2.4 build_auth_context token_auth admin_token_authjson_body filter处理
| [filter:build_auth_context] paste.filter_factory = keystone.middleware:AuthContextMiddleware.factory
[filter:token_auth] paste.filter_factory = keystone.middleware:TokenAuthMiddleware.factory
[filter:admin_token_auth] paste.filter_factory = keystone.middleware:AdminTokenAuthMiddleware.factory
[filter:json_body] paste.filter_factory = keystone.middleware:JsonBodyMiddleware.factory |
这4个filter其实也跟前面的filter一样，没做任何实质性的工作，其代码流程跟url_normalize filter处理流程类似，我们就不在这里讲解了。
2.5 ec2_extension filter处理
| [filter:ec2_extension] paste.filter_factory = keystone.contrib.ec2:Ec2Extension.factory |
#/keystone/contrib/ec2/routers.py:Ec2Extension
class Ec2Extension(wsgi.ExtensionRouter):
def add_routes(self, mapper):
ec2_controller = controllers.Ec2Controller()
# validation
mapper.connect(
'/ec2tokens',
controller=ec2_controller,
action='authenticate',
conditions=dict(method=['POST']))
# crud
mapper.connect(
'/users/{user_id}/credentials/OS-EC2',
controller=ec2_controller,
action='create_credential',
conditions=dict(method=['POST']))
mapper.connect(
'/users/{user_id}/credentials/OS-EC2',
controller=ec2_controller,
action='get_credentials',
conditions=dict(method=['GET']))
mapper.connect(
'/users/{user_id}/credentials/OS-EC2/{credential_id}',
controller=ec2_controller,
action='get_credential',
conditions=dict(method=['GET']))
mapper.connect(
'/users/{user_id}/credentials/OS-EC2/{credential_id}',
controller=ec2_controller,
action='delete_credential',
conditions=dict(method=['DELETE']))
#/keystone/common/wsgi.py:ExtensionRouter
class ExtensionRouter(Router):
"""A router that allows extensions to supplement or overwrite routes.
Expects to be subclassed.
"""
def __init__(self, application, mapper=None):
if mapper is None:
mapper = routes.Mapper()
self.application = application
self.add_routes(mapper)
mapper.connect('{path_info:.*}', controller=self.application)
super(ExtensionRouter, self).__init__(mapper)
def add_routes(self, mapper):
pass
@classmethod
def factory(cls, global_config, **local_config):
"""Used for paste app factories in paste.deploy config files.
Any local configuration (that is, values under the [filter:APPNAME]
section of the paste config) will be passed into the `__init__` method
as kwargs.
A hypothetical configuration would look like:
[filter:analytics]
redis_host = 127.0.0.1
paste.filter_factory = keystone.analytics:Analytics.factory
which would result in a call to the `Analytics` class as
import keystone.analytics
keystone.analytics.Analytics(app, redis_host='127.0.0.1')
You could of course re-implement the `factory` method in subclasses,
but using the kwarg passing it shouldn't be necessary.
"""
def _factory(app):
conf = global_config.copy()
conf.update(local_config)
return cls(app, **local_config)
return _factory
这里引入了python的routes模块，具体使用方法参考官网链接http://routes.readthedocs.org/en/latest/index.html。当HTTP请求到来时，route将根据请求的url信息转换成相应的资源，并路由到合适的函数上。对于ec2_extension filter的重点就在add_routes函数，这里提供了url到相应资源的转换且能route到合适的函数上，如我们提供HTTP请求的url为http://192.168.118.1:5000/v2.0/ec2tokens，且请求类型为POST，通过-d再加上其他一下验证参数，这样将会去执行 Ec2Controller类中的authenticate函数，执行完成后将得到的ec2类型的token值返回给请求者。当然，我们举例的url为http://192.168.118.1:5000/v2.0/tokens，则不会路由到ec2_extension filter上的资源。
2.6 user_crud_extension filter处理
| [filter:user_crud_extension] paste.filter_factory = keystone.contrib.user_crud:CrudExtension.factory |
#/keystone/contrib/user_crud/core.py:CrudExtension
class CrudExtension(wsgi.ExtensionRouter):
"""Provides a subset of CRUD operations for internal data types."""
def add_routes(self, mapper):
user_controller = UserController()
mapper.connect('/OS-KSCRUD/users/{user_id}',
controller=user_controller,
action='set_user_password',
conditions=dict(method=['PATCH']))
user_crud_extension filter处理与ec2_extension filter处理类似，且它们有相同的父类，这里user_crud_extension filter只是增加了一个route。
其实大部分route都在public_service filter上。
2.7 public_service app处理
| [app:public_service] paste.app_factory = keystone.service:public_app_factory |
#/keystone/service.py
@fail_gracefully
def public_app_factory(global_conf, **local_conf):
controllers.register_version('v2.0')
return wsgi.ComposingRouter(routes.Mapper(),
[assignment.routers.Public(),
token.routers.Router(),
routers.VersionV2('public'),
routers.Extension(False)])
#/keystone/common/wsgi.py:ComposingRouter
class ComposingRouter(Router):
def __init__(self, mapper=None, routers=None):
if mapper is None:
mapper = routes.Mapper()
if routers is None:
routers = []
for router in routers:
router.add_routes(mapper)
super(ComposingRouter, self).__init__(mapper)
从上面可以public_serviceapp增加了4个route，即assignment.routers.Public()，token.routers.Router(),routers.VersionV2('public')和routers.Extension(False)]，那么我们看看这4个route都增加了哪些资源呢？
#/keystone/assignment/routers.py:Public
class Public(wsgi.ComposableRouter):
def add_routes(self, mapper):
tenant_controller = controllers.TenantAssignment()
mapper.connect('/tenants',
controller=tenant_controller,
action='get_projects_for_token',
conditions=dict(method=['GET']))
#/keystone/token/routers.py:Router
class Router(wsgi.ComposableRouter):
def add_routes(self, mapper):
token_controller = controllers.Auth()
mapper.connect('/tokens',
controller=token_controller,
action='authenticate',
conditions=dict(method=['POST']))
mapper.connect('/tokens/revoked',
controller=token_controller,
action='revocation_list',
conditions=dict(method=['GET']))
mapper.connect('/tokens/{token_id}',
controller=token_controller,
action='validate_token',
conditions=dict(method=['GET']))
# NOTE(morganfainberg): For policy enforcement reasons, the
# ``validate_token_head`` method is still used for HEAD requests.
# The controller method makes the same call as the validate_token
# call and lets wsgi.render_response remove the body data.
mapper.connect('/tokens/{token_id}',
controller=token_controller,
action='validate_token_head',
conditions=dict(method=['HEAD']))
mapper.connect('/tokens/{token_id}',
controller=token_controller,
action='delete_token',
conditions=dict(method=['DELETE']))
mapper.connect('/tokens/{token_id}/endpoints',
controller=token_controller,
action='endpoints',
conditions=dict(method=['GET']))
# Certificates used to verify auth tokens
mapper.connect('/certificates/ca',
controller=token_controller,
action='ca_cert',
conditions=dict(method=['GET']))
mapper.connect('/certificates/signing',
controller=token_controller,
action='signing_cert',
conditions=dict(method=['GET']))
#/keystone/routers.py:VersionV2
class VersionV2(wsgi.ComposableRouter):
def __init__(self, description):
self.description = description
def add_routes(self, mapper):
version_controller = controllers.Version(self.description)
mapper.connect('/',
controller=version_controller,
action='get_version_v2')
#/keystone/routers.py:Extension
class Extension(wsgi.ComposableRouter):
def __init__(self, is_admin=True):
if is_admin:
self.controller = controllers.AdminExtensions()
else:
self.controller = controllers.PublicExtensions()
def add_routes(self, mapper):
extensions_controller = self.controller
mapper.connect('/extensions',
controller=extensions_controller,
action='get_extensions_info',
conditions=dict(method=['GET']))
mapper.connect('/extensions/{extension_alias}',
controller=extensions_controller,
action='get_extension_info',
conditions=dict(method=['GET']))
这些就是public_serviceapp增加的所有route资源，你也可以按照上面的mapper.connect的方式添加你自己的所需要的资源，然后利用HTTP请求route到你的函数上去执行的相应的操作。
因为我们本文中举的例子为：
| DEBUG (session:197) REQ: curl -g -i -X POST http://192.168.118.1:5000/v2.0/tokens -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "admin"}}}' |
所以执行的是/keystone/token/routers.py:Router的第一个route资源。即
mapper.connect('/tokens',
controller=token_controller,
action='authenticate',
conditions=dict(method=['POST']))
因为token_controller为/keystone/token/controllers.py:Auth对象，所以此时执行该类中的authenticate函数。如下
#/keystone/token/controllers.py:Auth
@controller.v2_deprecated
def authenticate(self, context, auth=None):
"""Authenticate credentials and return a token.
Accept auth as a dict that looks like::
{
"auth":{
"passwordCredentials":{
"username":"test_user",
"password":"mypass"
},
"tenantName":"customer-x"
}
}
In this case, tenant is optional, if not provided the token will be
considered "unscoped" and can later be used to get a scoped token.
Alternatively, this call accepts auth with only a token and tenant
that will return a token that is scoped to that tenant.
"""
if auth is None:
raise exception.ValidationError(attribute='auth',
target='request body')
if "token" in auth:
# Try to authenticate using a token
auth_info = self._authenticate_token(
context, auth)
else:
# Try external authentication
try:
auth_info = self._authenticate_external(
context, auth)
except ExternalAuthNotApplicable:
# Try local authentication
auth_info = self._authenticate_local(
context, auth)
user_ref, tenant_ref, metadata_ref, expiry, bind, audit_id = auth_info
# Validate that the auth info is valid and nothing is disabled
try:
self.identity_api.assert_user_enabled(
user_id=user_ref['id'], user=user_ref)
if tenant_ref:
self.resource_api.assert_project_enabled(
project_id=tenant_ref['id'], project=tenant_ref)
except AssertionError as e:
six.reraise(exception.Unauthorized, exception.Unauthorized(e),
sys.exc_info()[2])
# NOTE(morganfainberg): Make sure the data is in correct form since it
# might be consumed external to Keystone and this is a v2.0 controller.
# The user_ref is encoded into the auth_token_data which is returned as
# part of the token data. The token provider doesn't care about the
# format.
user_ref = self.v3_to_v2_user(user_ref)
if tenant_ref:
tenant_ref = self.v3_to_v2_project(tenant_ref)
auth_token_data = self._get_auth_token_data(user_ref,
tenant_ref,
metadata_ref,
expiry,
audit_id)
if tenant_ref:
catalog_ref = self.catalog_api.get_catalog(
user_ref['id'], tenant_ref['id'])
else:
catalog_ref = {}
auth_token_data['id'] = 'placeholder'
if bind:
auth_token_data['bind'] = bind
roles_ref = []
for role_id in metadata_ref.get('roles', []):
role_ref = self.role_api.get_role(role_id)
roles_ref.append(dict(name=role_ref['name']))
(token_id, token_data) = self.token_provider_api.issue_v2_token(
auth_token_data, roles_ref=roles_ref, catalog_ref=catalog_ref)
# NOTE(wanghong): We consume a trust use only when we are using trusts
# and have successfully issued a token.
if CONF.trust.enabled and 'trust_id' in auth:
self.trust_api.consume_use(auth['trust_id'])
return token_data
2.7.1 验证HTTP请求
这里我们举例的HTTP请求传递给authenticate函数的auth变量的值为：
| {u'tenantName': u'admin', u'passwordCredentials': {u'username': u'admin', u'password': u'admin'}} |
所以，HTTP请求将在_authenticate_local函数中进行验证。这里是通过传递用户名(或用户id)和密码的方式进行验证。
另外，对于keystone的token的HTTP请求还有另外一种形式，即auth变量的值为如下形式：
| "auth": {"tenantName": "admin","token": {"id": "9b390c37959f44b5ad5e8b2aa403267a "}} |
此时完整的HTTP请求为：
| curl -g -i -X POST http://192.168.118.1:5000/v2.0/tokens -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -d '{"auth": {"tenantName": "admin","token": {"id": "9b390c37959f44b5ad5e8b2aa403267a"}}}' |
即不通过用户名(或用户id)和密码的进行验证，而是通过未过期的token进行验证，这种形式的HTTP请求将在_authenticate_token函数中进行验证。
这里我们主要分析如何通过用户名和密码对HTTP请求进行验证。
#/keystone/token/controllers.py:Auth
def _authenticate_local(self, context, auth):
"""Try to authenticate against the identity backend.
Returns auth_token_data, (user_ref, tenant_ref, metadata_ref)
"""
if 'passwordCredentials' not in auth:
raise exception.ValidationError(
attribute='passwordCredentials', target='auth')
if "password" not in auth['passwordCredentials']:
raise exception.ValidationError(
attribute='password', target='passwordCredentials')
password = auth['passwordCredentials']['password']
if password and len(password) > CONF.identity.max_password_length:
raise exception.ValidationSizeError(
attribute='password', size=CONF.identity.max_password_length)
if (not auth['passwordCredentials'].get("userId") and
not auth['passwordCredentials'].get("username")):
raise exception.ValidationError(
attribute='username or userId',
target='passwordCredentials')
user_id = auth['passwordCredentials'].get('userId')
if user_id and len(user_id) > CONF.max_param_size:
raise exception.ValidationSizeError(attribute='userId',
size=CONF.max_param_size)
username = auth['passwordCredentials'].get('username', '')
if username:
if len(username) > CONF.max_param_size:
raise exception.ValidationSizeError(attribute='username',
size=CONF.max_param_size)
try:
user_ref = self.identity_api.get_user_by_name(
username, CONF.identity.default_domain_id)
user_id = user_ref['id']
except exception.UserNotFound as e:
raise exception.Unauthorized(e)
try:
user_ref = self.identity_api.authenticate(
context,
user_id=user_id,
password=password)
except AssertionError as e:
raise exception.Unauthorized(e.args[0])
metadata_ref = {}
tenant_id = self._get_project_id_from_auth(auth)
tenant_ref, metadata_ref['roles'] = self._get_project_roles_and_ref(
user_id, tenant_id)
expiry = provider.default_expire_time()
bind = None
audit_id = None
return (user_ref, tenant_ref, metadata_ref, expiry, bind, audit_id)
_authenticate_local函数前面都是对用户名(或用户id)和密码的长度进行判断。然后执行self.identity_api.get_user_by_name方法，该方法主要目的是获取用户的相关信息，然后从中获取用户的ID(这是在HTTP请求采用用户名和密码进行验证的时候调用的接口)。
在获取用户的ID后，使用用户ID和密码对HTTP请求进行验证。执行如下代码：
user_ref = self.identity_api.authenticate(
context,
user_id=user_id,
password=password)
其中self.identity_api是我们在/keystone/backends.py的load_backends函数中定义的。即self.identity_api为/keystone/identity/core.py中的Manager对象。
#/keystone/identity/core.py:Manager
@notifications.emit_event('authenticate')
@domains_configured
@exception_translated('assertion')
def authenticate(self, context, user_id, password):
domain_id, driver, entity_id = (
self._get_domain_driver_and_entity_id(user_id))
ref = driver.authenticate(entity_id, password)
return self._set_domain_id_and_mapping(
ref, domain_id, driver, mapping.EntityType.USER)
#/keystone/identity/core.py:Manager
def _get_domain_driver_and_entity_id(self, public_id):
"""Look up details using the public ID.
:param public_id: the ID provided in the call
:returns: domain_id, which can be None to indicate that the driver
in question supports multiple domains
driver selected based on this domain
entity_id which will is understood by the driver.
Use the mapping table to look up the domain, driver and local entity
that is represented by the provided public ID. Handle the situations
were we do not use the mapping (e.g. single driver that understands
UUIDs etc.)
"""
conf = CONF.identity
# First, since we don't know anything about the entity yet, we must
# assume it needs mapping, so long as we are using domain specific
# drivers.
if conf.domain_specific_drivers_enabled:
local_id_ref = self.id_mapping_api.get_id_mapping(public_id)
if local_id_ref:
return (
local_id_ref['domain_id'],
self._select_identity_driver(local_id_ref['domain_id']),
local_id_ref['local_id'])
# So either we are using multiple drivers but the public ID is invalid
# (and hence was not found in the mapping table), or the public ID is
# being handled by the default driver. Either way, the only place left
# to look is in that standard driver. However, we don't yet know if
# this driver also needs mapping (e.g. LDAP in non backward
# compatibility mode).
driver = self.driver
if driver.generates_uuids():
if driver.is_domain_aware:
# No mapping required, and the driver can handle the domain
# information itself. The classic case of this is the
# current SQL driver.
return (None, driver, public_id)
else:
# Although we don't have any drivers of this type, i.e. that
# understand UUIDs but not domains, conceptually you could.
return (conf.default_domain_id, driver, public_id)
# So the only place left to find the ID is in the default driver which
# we now know doesn't generate UUIDs
if not CONF.identity_mapping.backward_compatible_ids:
# We are not running in backward compatibility mode, so we
# must use a mapping.
local_id_ref = self.id_mapping_api.get_id_mapping(public_id)
if local_id_ref:
return (
local_id_ref['domain_id'],
driver,
local_id_ref['local_id'])
else:
raise exception.PublicIDNotFound(id=public_id)
# If we reach here, this means that the default driver
# requires no mapping - but also doesn't understand domains
# (e.g. the classic single LDAP driver situation). Hence we pass
# back the public_ID unmodified and use the default domain (to
# keep backwards compatibility with existing installations).
#
# It is still possible that the public ID is just invalid in
# which case we leave this to the caller to check.
return (conf.default_domain_id, driver, public_id)
这里我们重点关注从_get_domain_driver_and_entity_id函数返回的driver是什么？从上面的代码可以看出driver= self.driver，那么我们看看Manger类如何定义driver的？
#/keystone/identity/core.py:Manager
@dependency.provider('identity_api')
@dependency.requires('assignment_api', 'credential_api', 'id_mapping_api',
'resource_api', 'revoke_api')
class Manager(manager.Manager):
"""Default pivot point for the Identity backend.
See :mod:`keystone.common.manager.Manager` for more details on how this
dynamically calls the backend.
This class also handles the support of domain specific backends, by using
the DomainConfigs class. The setup call for DomainConfigs is called
from with the @domains_configured wrapper in a lazy loading fashion
to get around the fact that we can't satisfy the assignment api it needs
from within our __init__() function since the assignment driver is not
itself yet initialized.
Each of the identity calls are pre-processed here to choose, based on
domain, which of the drivers should be called. The non-domain-specific
driver is still in place, and is used if there is no specific driver for
the domain in question (or we are not using multiple domain drivers).
Starting with Juno, in order to be able to obtain the domain from
just an ID being presented as part of an API call, a public ID to domain
and local ID mapping is maintained. This mapping also allows for the local
ID of drivers that do not provide simple UUIDs (such as LDAP) to be
referenced via a public facing ID. The mapping itself is automatically
generated as entities are accessed via the driver.
This mapping is only used when:
- the entity is being handled by anything other than the default driver, or
- the entity is being handled by the default LDAP driver and backward
compatible IDs are not required.
This means that in the standard case of a single SQL backend or the default
settings of a single LDAP backend (since backward compatible IDs is set to
True by default), no mapping is used. An alternative approach would be to
always use the mapping table, but in the cases where we don't need it to
make the public and local IDs the same. It is felt that not using the
mapping by default is a more prudent way to introduce this functionality.
"""
_USER = 'user'
_GROUP = 'group'
def __init__(self):
super(Manager, self).__init__(CONF.identity.driver)
self.domain_configs = DomainConfigs()
self.event_callbacks = {
notifications.ACTIONS.deleted: {
'domain': [self._domain_deleted],
},
}
#/keystone/common/manager.py:Manager
class Manager(object):
"""Base class for intermediary request layer.
The Manager layer exists to support additional logic that applies to all
or some of the methods exposed by a service that are not specific to the
HTTP interface.
It also provides a stable entry point to dynamic backends.
An example of a probable use case is logging all the calls.
"""
def __init__(self, driver_name):
self.driver = importutils.import_object(driver_name)
def __getattr__(self, name):
"""Forward calls to the underlying driver."""
f = getattr(self.driver, name)
setattr(self, name, f)
return f
从上述的代码可以看出，/keystone/identity/core.py中的Manager类中driver是在其父类/keystone/common/manager.py:Manager中进行定义的，且该driver是在keystone中的配置文件中进行读取。默认配置如下：
cfg.StrOpt('driver',
default=('keystone.identity.backends'
'.sql.Identity'),
help='Identity backend driver.'),
即/keystone/identity/core.py中的Manager类中driver的值是/keystone/identity/backends/sql.py中的Identity对象。
此外，我们/keystone/common/manager.py:Manager中也可以发现，该类相当于是一个代理类(从__getattr__函数可以看出)：当子类调用既不在子类自身中也不在父类中的函数时，此时将会去调用定义的driver中的函数。
我们发现driver的值是/keystone/identity/backends/sql.py中的Identity对象。所以/keystone/identity/core.py中Manager类中的authenticate函数此代码(ref= driver.authenticate(entity_id, password))将调用/keystone/identity/backends/sql.py中的Identity对象中的authenticate函数。
#/keystone/identity/backends/sql.py:Identity
# Identity interface
def authenticate(self, user_id, password):
session = sql.get_session()
user_ref = None
try:
user_ref = self._get_user(session, user_id)
except exception.UserNotFound:
raise AssertionError(_('Invalid user / password'))
if not self._check_password(password, user_ref):
raise AssertionError(_('Invalid user / password'))
return identity.filter_user(user_ref.to_dict())
#/keystone/identity/backends/sql.py:Identity
def _get_user(self, session, user_id):
user_ref = session.query(User).get(user_id)
if not user_ref:
raise exception.UserNotFound(user_id=user_id)
return user_ref
#/keystone/identity/backends/sql.py:User
class User(sql.ModelBase, sql.DictBase):
__tablename__ = 'user'
attributes = ['id', 'name', 'domain_id', 'password', 'enabled',
'default_project_id']
id = sql.Column(sql.String(64), primary_key=True)
name = sql.Column(sql.String(255), nullable=False)
domain_id = sql.Column(sql.String(64), nullable=False)
password = sql.Column(sql.String(128))
enabled = sql.Column(sql.Boolean)
extra = sql.Column(sql.JsonBlob())
default_project_id = sql.Column(sql.String(64))
# Unique constraint across two columns to create the separation
# rather than just only 'name' being unique
__table_args__ = (sql.UniqueConstraint('domain_id', 'name'), {})
def to_dict(self, include_extra_dict=False):
d = super(User, self).to_dict(include_extra_dict=include_extra_dict)
if 'default_project_id' in d and d['default_project_id'] is None:
del d['default_project_id']
return d
/keystone/identity/backends/sql.py中的Identity对象中的authenticate函数首先调用_get_user函数，该函数根据user id从keystone数据库中的user表查询user的相关信息，其中查询的user信息中包括加密后的user的密码信息。如我环境中的keystone数据库中的user表数据。
| MariaDB [keystone]> select * from user; +----------------------------------+------------+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------+---------+-----------+----------------------------------+ | id | name | extra | password | enabled | domain_id | default_project_id | +----------------------------------+------------+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------+---------+-----------+----------------------------------+ | 1677a7f58d7d4c98a56f8c2be5e16068 | neutron | {"email": "neutron@localhost"} | $6$rounds=40000$S7Zg3JPb/iFgH0aw$/fnQFBc.pYH8wsOSxBWhDHevIx2zTphl0eSxb6akKNoXUgbUDc9pb5TXNhL3xgGms6RVZm1BFxDK48R2LHPgv0 | 1 | default | c9960b417968496387e6b96a244cc2be | | 7a0fd47f67e94d6e99fe6f0298067102 | cinder | {"email": "cinder@localhost"} | $6$rounds=40000$EZp3iGcBcL9CpXbt$wN/qjTr2m.0PBkXz/HYGcf65QXm0qPqlVTE9n84U1lXqmS1JsdJnxRno8eptuPcVUV91mx39b449FYMfMnuXT/ | 1 | default | c9960b417968496387e6b96a244cc2be | | bde55802097c475592741efc5096d90a | nova | {"email": "nova@localhost"} | $6$rounds=40000$ezqZ86aPM0o1J8DH$hS2cBDqAzminTsL9x7H/kIV6PfLQKude6Z.JQ2ugnW.aBF0n4sevPZVibK30DGrP9/0sY7J5HzBwlmKsCYM8./ | 1 | default | c9960b417968496387e6b96a244cc2be | | d317cfbab4544707a5f323fc1317cf65 | glance | {"email": "glance@localhost"} | $6$rounds=40000$fyCPnCYRF.t6.2Se$PRcMbqExrngnZPAhKiX7BME.MEh7k19VcN.L5GbjeTprXcbODM.ntSd2ezsKuhIx/LMU9ezgPZz0ki92XZTTZ0 | 1 | default | c9960b417968496387e6b96a244cc2be | | e4ea11ef20e0439596b4df43cb7687ac | ceilometer | {"email": "ceilometer@localhost"} | $6$rounds=40000$k9qUckdICo7hY6ZD$4ccb/TkTklY9iRATB9V/DJZoQVigvFGjhJH1nBC19VAW.FIXbCf27yyfIJfpHtCIYR8gsfsQA6cDHNiY3dCHw. | 1 | default | c9960b417968496387e6b96a244cc2be | | f59f17d8e9774eef8730b23ecdc86a4b | admin | {"email": "root@localhost"} | $6$rounds=40000$rPma7p.3qcixVvDW$/rCywiBa74Jg2J7z/m5zH5a.poLSd4haDKgpSJEfa4IK3j.HPbOnClFYO4Nd4ADiTOxX/OUuc2N4Fr8dy/BHx1 | 1 | default | 09e04766c06d477098201683497d3878 | +----------------------------------+------------+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------+---------+-----------+----------------------------------+ 6 rows in set (0.00 sec) |
#/keystone/identity/backends/sql.py:Identity
def _check_password(self, password, user_ref):
"""Check the specified password against the data store.
Note that we'll pass in the entire user_ref in case the subclass
needs things like user_ref.get('name')
For further justification, please see the follow up suggestion at
https://blueprints.launchpad.net/keystone/+spec/sql-identiy-pam
"""
return utils.check_password(password, user_ref.password)
#/keystone/common/utils.py
def check_password(password, hashed):
"""Check that a plaintext password matches hashed.
hashpw returns the salt value concatenated with the actual hash value.
It extracts the actual salt if this value is then passed as the salt.
"""
if password is None or hashed is None:
return False
password_utf8 = verify_length_and_trunc_password(password).encode('utf-8')
return passlib.hash.sha512_crypt.verify(password_utf8, hashed)
上述便是验证HTTP请求中的密码是否正确的代码，当然具体怎么验证的，我也没看懂，感兴趣的读者可以自己再深入研究一下。
验证完成成功后，再将相应的user信息设置成合适的格式返回给调用代码。
再次回到/keystone/token/controllers.py:Auth中的_authenticate_local函数，在将user的信息验证成功后，下面的代码将获取tenant信息，然后提供一个token的过期时间和其他的一些信息返回给authenticate函数(具体获取tenant和token过期时间的代码在这里就不分析了，具体可以参考分析验证user密码的代码进行分析)。
其中，我的环境中_authenticate_local函数返回的信息为：
| auth_info: ({'domain_id': u'default', 'name': u'admin', 'id': u'f59f17d8e9774eef8730b23ecdc86a4b', 'enabled': True, u'email': u'root@localhost', 'default_project_id': u'09e04766c06d477098201683497d3878' },
'description': u'admin tenant', 'enabled': True, 'id': u'09e04766c06d477098201683497d3878', 'parent_id': None, 'domain_id': u'default', 'name': u'admin' },
{'roles': [u'397eaf49b01549dab8be01804bec7972']},
datetime.datetime(2016, 3, 24, 13, 8, 30, 626964), None, None ) |
auth_info中的第1个字典为user的信息，第2个字典为tenant的信息，第4个字典为将为获取的token设置的过期时间。
2.7.2 auth_info格式处理
user_ref = self.v3_to_v2_user(user_ref)
if tenant_ref:
tenant_ref = self.v3_to_v2_project(tenant_ref)
auth_token_data = self._get_auth_token_data(user_ref,
tenant_ref,
metadata_ref,
expiry,
audit_id)
这部分代码较简单，读者可以自行分析，处理后返回auth_token_data，即如下形式：
| auth_token_data: {'expires': datetime.datetime(2016, 3, 24, 13, 8, 30, 626964), 'parent_audit_id': None, 'user': {'username': u'admin', 'name': u'admin', 'id': u'f59f17d8e9774eef8730b23ecdc86a4b', 'enabled': True, u'email': u'root@localhost', 'tenantId': u'09e04766c06d477098201683497d3878' }, 'tenant': {'description': u'admin tenant', 'enabled': True, 'id': u'09e04766c06d477098201683497d3878', 'name': u'admin' }, 'metadata': {'roles': [u'397eaf49b01549dab8be01804bec7972']} } |
2.7.3 获取catalog信息
if tenant_ref:
catalog_ref = self.catalog_api.get_catalog(
user_ref['id'], tenant_ref['id'])
else:
catalog_ref = {}
因为tenant_ref有值，所以执行if中的语句。其中self.catallog_api为/keystone/catalog/core.py中的Manager对象。
#keystone/catalog/core.py:Manager
def get_catalog(self, user_id, tenant_id):
try:
return self.driver.get_catalog(user_id, tenant_id)
except exception.NotFound:
raise exception.NotFound('Catalog not found for user and tenant')
对于self.driver是什么对象，我们可以按照2.7.1中的分析方式进行分析(它们采用类似的构造)。如下
#keystone/catalog/core.py:Manager
@dependency.provider('catalog_api')
class Manager(manager.Manager):
"""Default pivot point for the Catalog backend.
See :mod:`keystone.common.manager.Manager` for more details on how this
dynamically calls the backend.
"""
_ENDPOINT = 'endpoint'
_SERVICE = 'service'
_REGION = 'region'
def __init__(self):
super(Manager, self).__init__(CONF.catalog.driver)
#/keystone/common/manager.py:Manager
class Manager(object):
"""Base class for intermediary request layer.
The Manager layer exists to support additional logic that applies to all
or some of the methods exposed by a service that are not specific to the
HTTP interface.
It also provides a stable entry point to dynamic backends.
An example of a probable use case is logging all the calls.
"""
def __init__(self, driver_name):
self.driver = importutils.import_object(driver_name)
def __getattr__(self, name):
"""Forward calls to the underlying driver."""
f = getattr(self.driver, name)
setattr(self, name, f)
return f
所以self.driver为keystone配置文件中catalog section设置的driver对象(查看/keystone/backends.py的load_backends函数中的定义)。如下
cfg.StrOpt('driver',
default='keystone.catalog.backends.sql.Catalog',
help='Catalog backend driver.'),
该driver为/keystone/catalog/backends/sql中的Catalog对象。
#/keystone/catalog/backends/sql.py:Catalog
def get_catalog(self, user_id, tenant_id):
"""Retrieve and format the V2 service catalog.
:param user_id: The id of the user who has been authenticated for
creating service catalog.
:param tenant_id: The id of the project. 'tenant_id' will be None
in the case this being called to create a catalog to go in a
domain scoped token. In this case, any endpoint that requires
a tenant_id as part of their URL will be skipped (as would a whole
service if, as a consequence, it has no valid endpoints).
:returns: A nested dict representing the service catalog or an
empty dict.
"""
substitutions = dict(
itertools.chain(six.iteritems(CONF),
six.iteritems(CONF.eventlet_server)))
substitutions.update({'user_id': user_id})
silent_keyerror_failures = []
if tenant_id:
substitutions.update({'tenant_id': tenant_id})
else:
silent_keyerror_failures = ['tenant_id']
session = sql.get_session()
endpoints = (session.query(Endpoint).
options(sql.joinedload(Endpoint.service)).
filter(Endpoint.enabled == true()).all())
catalog = {}
for endpoint in endpoints:
if not endpoint.service['enabled']:
continue
try:
formatted_url = core.format_url(
endpoint['url'], substitutions,
silent_keyerror_failures=silent_keyerror_failures)
if formatted_url is not None:
url = formatted_url
else:
continue
except exception.MalformedEndpoint:
continue # this failure is already logged in format_url()
region = endpoint['region_id']
service_type = endpoint.service['type']
default_service = {
'id': endpoint['id'],
'name': endpoint.service.extra.get('name', ''),
'publicURL': ''
}
catalog.setdefault(region, {})
catalog[region].setdefault(service_type, default_service)
interface_url = '%sURL' % endpoint['interface']
catalog[region][service_type][interface_url] = url
return catalog
从上面的代码可以看出，get_catalog函数主要将keystone数据库的endpoint表的信息进行组装，然后将其返回给authenticate函数。其中endpoint表的定义如下
#/keystone/catalog/backends/sql.py:Endpoint
class Endpoint(sql.ModelBase, sql.DictBase):
__tablename__ = 'endpoint'
attributes = ['id', 'interface', 'region_id', 'service_id', 'url',
'legacy_endpoint_id', 'enabled']
id = sql.Column(sql.String(64), primary_key=True)
legacy_endpoint_id = sql.Column(sql.String(64))
interface = sql.Column(sql.String(8), nullable=False)
region_id = sql.Column(sql.String(255),
sql.ForeignKey('region.id',
ondelete='RESTRICT'),
nullable=True,
default=None)
service_id = sql.Column(sql.String(64),
sql.ForeignKey('service.id'),
nullable=False)
url = sql.Column(sql.Text(), nullable=False)
enabled = sql.Column(sql.Boolean, nullable=False, default=True,
server_default=sqlalchemy.sql.expression.true())
extra = sql.Column(sql.JsonBlob())
数据库中endpoint表的信息如下。
| MariaDB [keystone]> select * from endpoint; +----------------------------------+----------------------------------+-----------+----------------------------------+--------------------------------------------+-------+---------+-----------+ | id | legacy_endpoint_id | interface | service_id | url | extra | enabled | region_id | +----------------------------------+----------------------------------+-----------+----------------------------------+--------------------------------------------+-------+---------+-----------+ | 03966fa6606945b985d8ff3ba1912f00 | 5e483b6c64fd44158215b3e285a3614a | admin | f3f4b449f7244b28b65940af94fd8a47 | http://192.168.118.1:8774/v2/%(tenant_id)s | {} | 1 | RegionOne | | 0ab5626e37714ead9c1be3a7fb322d66 | 5e483b6c64fd44158215b3e285a3614a | internal | f3f4b449f7244b28b65940af94fd8a47 | http://192.168.118.1:8774/v2/%(tenant_id)s | {} | 1 | RegionOne | | 11b8c840c71c4258a644bf93ddee8d0f | da4a4ab4857840b0aa9d7c5acc4c82e3 | admin | 9e0e0000b90645cc8a5fcbcc7ad3b02c | http://192.168.118.1:35357/v2.0 | {} | 1 | RegionOne | | 16b0ed99d09044229a7c67992f514aa3 | 5e483b6c64fd44158215b3e285a3614a | public | f3f4b449f7244b28b65940af94fd8a47 | http://192.168.118.1:8774/v2/%(tenant_id)s | {} | 1 | RegionOne | | 1a22050d1c404a1fa257b7de8453f7b5 | 6870936c855741319bdbd915a9284f19 | admin | bcc3368ce9384e63ac5f8f4c894f232a | http://192.168.118.1:8776/v2/%(tenant_id)s | {} | 1 | RegionOne | | 1b6b0ae5906345d3a6ec0352c97e724d | 58abac45c14c44879d39e5c9c65dab61 | internal | 7a4407c4eb7a4f2c97813b0d0a9729ab | http://192.168.118.1:8773/services/Cloud | {} | 1 | RegionOne | | 3572f3870f21445bb82c5e622d87f5cd | 58abac45c14c44879d39e5c9c65dab61 | public | 7a4407c4eb7a4f2c97813b0d0a9729ab | http://192.168.118.1:8773/services/Cloud | {} | 1 | RegionOne | | 3d9c418b79f641a291a885fe20ec6a84 | f5173bb8e2ca4cefb4c0cc123c6a4b58 | admin | 2a2b254ccf6742a2af7f2736c5246b13 | http://192.168.118.1:8777 | {} | 1 | RegionOne | | 4095ec6b1d9c4e3ba87994a90a0d1ed5 | 278489f69a414670b5e66e03db79e580 | internal | c333b7e90e8742c6adf726f199338f04 | http://192.168.118.1:8774/v3 | {} | 1 | RegionOne | | 40a799e5ac0f4731bd2206299f303917 | da4a4ab4857840b0aa9d7c5acc4c82e3 | public | 9e0e0000b90645cc8a5fcbcc7ad3b02c | http://192.168.118.1:5000/v2.0 | {} | 1 | RegionOne | | 589ae871719c46308e7179271c66e43a | 58abac45c14c44879d39e5c9c65dab61 | admin | 7a4407c4eb7a4f2c97813b0d0a9729ab | http://192.168.118.1:8773/services/Admin | {} | 1 | RegionOne | | 63e42f266d9b432abdbf6f62da6bffaa | 6870936c855741319bdbd915a9284f19 | internal | bcc3368ce9384e63ac5f8f4c894f232a | http://192.168.118.1:8776/v2/%(tenant_id)s | {} | 1 | RegionOne | | 65560ec5eed64581a72f4ac3d1fa519d | 4cbcc6153af54f40b4bf5e7fe3e60533 | admin | c90e95ef71164659beb0fa413b4a292f | http://192.168.118.1:8776/v1/%(tenant_id)s | {} | 1 | RegionOne | | 65c0727bdee74006bec9fbd3ca6f90a6 | f5173bb8e2ca4cefb4c0cc123c6a4b58 | internal | 2a2b254ccf6742a2af7f2736c5246b13 | http://192.168.118.1:8777 | {} | 1 | RegionOne | | 6ecc803364da42b7a250bf9fe2e71cc4 | 45bf231a5a9a41eea1b879132b152dda | internal | c75d6dc0a8574c979ccdc1ea44ad41b2 | http://192.168.118.1:9696/ | {} | 1 | RegionOne | | 6fb8352c59f141dfb69725f0a7a81280 | 6870936c855741319bdbd915a9284f19 | public | bcc3368ce9384e63ac5f8f4c894f232a | http://192.168.118.1:8776/v2/%(tenant_id)s | {} | 1 | RegionOne | | 7cd86837358f4c19bb551fe3c36fadf9 | 2e8e4627677f4cec8e4aaee2c61e064b | public | 591dfc7f5b434674bf3566646ffb37ec | http://192.168.118.1:9292 | {} | 1 | RegionOne | | 7f980243a75740378e1ac27c0b9cc8fd | 45bf231a5a9a41eea1b879132b152dda | public | c75d6dc0a8574c979ccdc1ea44ad41b2 | http://192.168.118.1:9696/ | {} | 1 | RegionOne | | 8fb40f64a7024518bce652cbf1bba9cd | 4cbcc6153af54f40b4bf5e7fe3e60533 | public | c90e95ef71164659beb0fa413b4a292f | http://192.168.118.1:8776/v1/%(tenant_id)s | {} | 1 | RegionOne | | a4b0a96daca1412891c299783527f648 | 45bf231a5a9a41eea1b879132b152dda | admin | c75d6dc0a8574c979ccdc1ea44ad41b2 | http://192.168.118.1:9696/ | {} | 1 | RegionOne | | be2bdd1ed6fc4232b46be46548a292e9 | da4a4ab4857840b0aa9d7c5acc4c82e3 | internal | 9e0e0000b90645cc8a5fcbcc7ad3b02c | http://192.168.118.1:5000/v2.0 | {} | 1 | RegionOne | | c38c9bf499b64d8d8eee0a0e226e84df | 2e8e4627677f4cec8e4aaee2c61e064b | admin | 591dfc7f5b434674bf3566646ffb37ec | http://192.168.118.1:9292 | {} | 1 | RegionOne | | c77d7cfc578f4299ae3e6e0c78e23cc9 | 2e8e4627677f4cec8e4aaee2c61e064b | internal | 591dfc7f5b434674bf3566646ffb37ec | http://192.168.118.1:9292 | {} | 1 | RegionOne | | c7c6ca1359ab424e9fdac3c407ce45c9 | f5173bb8e2ca4cefb4c0cc123c6a4b58 | public | 2a2b254ccf6742a2af7f2736c5246b13 | http://192.168.118.1:8777 | {} | 1 | RegionOne | | db973fd1d8cd46699325be0912abda75 | 278489f69a414670b5e66e03db79e580 | admin | c333b7e90e8742c6adf726f199338f04 | http://192.168.118.1:8774/v3 | {} | 1 | RegionOne | | de47b4c075ab4402bf57111bc84dcb50 | 4cbcc6153af54f40b4bf5e7fe3e60533 | internal | c90e95ef71164659beb0fa413b4a292f | http://192.168.118.1:8776/v1/%(tenant_id)s | {} | 1 | RegionOne | | e29c0a5af91b4e32834b5b93629bffe5 | 278489f69a414670b5e66e03db79e580 | public | c333b7e90e8742c6adf726f199338f04 | http://192.168.118.1:8774/v3 | {} | 1 | RegionOne | +----------------------------------+----------------------------------+-----------+----------------------------------+--------------------------------------------+-------+---------+-----------+
MariaDB [keystone]> select count(*) from endpoint; +----------+ | count(*) | +----------+ | 27 | +----------+ 1 row in set (0.05 sec) |
即在数据库中endpoint表有27条信息。每个服务会占3条endpoint(URL)信息，即adminURL, publicURL和internalURL，所以总共有9个服务的endpoint在OpenStack上。这9个service在keystone数据库中的service表中保存。
| MariaDB [keystone]> select * from service; +----------------------------------+-----------+---------+---------------------------------------------------------------------+ | id | type | enabled | extra | +----------------------------------+-----------+---------+---------------------------------------------------------------------+ | 2a2b254ccf6742a2af7f2736c5246b13 | metering | 1 | {"name": "ceilometer", "description": "Openstack Metering Service"} | | 591dfc7f5b434674bf3566646ffb37ec | image | 1 | {"name": "glance", "description": "OpenStack Image Service"} | | 7a4407c4eb7a4f2c97813b0d0a9729ab | ec2 | 1 | {"name": "nova_ec2", "description": "EC2 Service"} | | 9e0e0000b90645cc8a5fcbcc7ad3b02c | identity | 1 | {"name": "keystone", "description": "OpenStack Identity Service"} | | bcc3368ce9384e63ac5f8f4c894f232a | volumev2 | 1 | {"name": "cinderv2", "description": "Cinder Service v2"} | | c333b7e90e8742c6adf726f199338f04 | computev3 | 1 | {"name": "novav3", "description": "Openstack Compute Service v3"} | | c75d6dc0a8574c979ccdc1ea44ad41b2 | network | 1 | {"name": "neutron", "description": "Neutron Networking Service"} | | c90e95ef71164659beb0fa413b4a292f | volume | 1 | {"name": "cinder", "description": "Cinder Service"} | | f3f4b449f7244b28b65940af94fd8a47 | compute | 1 | {"name": "nova", "description": "Openstack Compute Service"} | +----------------------------------+-----------+---------+---------------------------------------------------------------------+
MariaDB [keystone]> select count(*) from service; +----------+ | count(*) | +----------+ | 9 | +----------+ 1 row in set (0.07 sec) |
最终经过get_catalog函数对endpoint信息进行格式化后的返回信息如下：
| {u'RegionOne': { u'compute': {u'adminURL': u'http://192.168.118.1:8774/v2/09e04766c06d477098201683497d3878', 'publicURL': u'http://192.168.118.1:8774/v2/09e04766c06d477098201683497d3878', 'id': u'03966fa6606945b985d8ff3ba1912f00', u'internalURL': u'http://192.168.118.1:8774/v2/09e04766c06d477098201683497d3878', 'name': u'nova' }, u'network': {u'adminURL': u'http://192.168.118.1:9696/', 'publicURL': u'http://192.168.118.1:9696/', 'id': u'6ecc803364da42b7a250bf9fe2e71cc4', u'internalURL': u'http://192.168.118.1:9696/', 'name': u'neutron' }, u'volumev2': {u'adminURL': u'http://192.168.118.1:8776/v2/09e04766c06d477098201683497d3878', 'publicURL': u'http://192.168.118.1:8776/v2/09e04766c06d477098201683497d3878', 'id': u'1a22050d1c404a1fa257b7de8453f7b5', u'internalURL': u'http://192.168.118.1:8776/v2/09e04766c06d477098201683497d3878', 'name': u'cinderv2' }, u'computev3': {u'adminURL': u'http://192.168.118.1:8774/v3', 'publicURL': u'http://192.168.118.1:8774/v3', 'id': u'4095ec6b1d9c4e3ba87994a90a0d1ed5', u'internalURL': u'http://192.168.118.1:8774/v3', 'name': u'novav3' }, u'image': {u'adminURL': u'http://192.168.118.1:9292', 'publicURL': u'http://192.168.118.1:9292', 'id': u'7cd86837358f4c19bb551fe3c36fadf9', u'internalURL': u'http://192.168.118.1:9292', 'name': u'glance' }, u'metering': {u'adminURL': u'http://192.168.118.1:8777', 'publicURL': u'http://192.168.118.1:8777', 'id': u'3d9c418b79f641a291a885fe20ec6a84', u'internalURL': u'http://192.168.118.1:8777', 'name': u'ceilometer' }, u'volume': {u'adminURL': u'http://192.168.118.1:8776/v1/09e04766c06d477098201683497d3878', 'publicURL': u'http://192.168.118.1:8776/v1/09e04766c06d477098201683497d3878', 'id': u'65560ec5eed64581a72f4ac3d1fa519d', u'internalURL': u'http://192.168.118.1:8776/v1/09e04766c06d477098201683497d3878', 'name': u'cinder' }, u'ec2': {u'adminURL': u'http://192.168.118.1:8773/services/Admin', 'publicURL': u'http://192.168.118.1:8773/services/Cloud', 'id': u'1b6b0ae5906345d3a6ec0352c97e724d', u'internalURL': u'http://192.168.118.1:8773/services/Cloud', 'name': u'nova_ec2' }, u'identity': {u'adminURL': u'http://192.168.118.1:35357/v2.0', 'publicURL': u'http://192.168.118.1:5000/v2.0', 'id': u'11b8c840c71c4258a644bf93ddee8d0f', u'internalURL': u'http://192.168.118.1:5000/v2.0', 'name': u'keystone' } } } |
2.7.4 生成token id
生成token id执行的代码如下
(token_id, token_data) = self.token_provider_api.issue_v2_token(
auth_token_data, roles_ref=roles_ref, catalog_ref=catalog_ref)
其中self.toekn_provider_api为/keystone/token/provider中的Manager对象 (查看/keystone/backends.py的load_backends函数中的定义)。
#/keystone/token/provider.py:Manager
def issue_v2_token(self, token_ref, roles_ref=None, catalog_ref=None):
token_id, token_data = self.driver.issue_v2_token(
token_ref, roles_ref, catalog_ref)
if self._needs_persistence:
data = dict(key=token_id,
id=token_id,
expires=token_data['access']['token']['expires'],
user=token_ref['user'],
tenant=token_ref['tenant'],
metadata=token_ref['metadata'],
token_data=token_data,
bind=token_ref.get('bind'),
trust_id=token_ref['metadata'].get('trust_id'),
token_version=self.V2)
self._create_token(token_id, data)
return token_id, token_data
self.driver的定义查看如下代码。
#/keystone/token/provider.py:Manager
@dependency.provider('token_provider_api')
@dependency.requires('assignment_api', 'revoke_api')
class Manager(manager.Manager):
"""Default pivot point for the token provider backend.
See :mod:`keystone.common.manager.Manager` for more details on how this
dynamically calls the backend.
"""
V2 = V2
V3 = V3
VERSIONS = VERSIONS
INVALIDATE_PROJECT_TOKEN_PERSISTENCE = 'invalidate_project_tokens'
INVALIDATE_USER_TOKEN_PERSISTENCE = 'invalidate_user_tokens'
_persistence_manager = None
def __init__(self):
super(Manager, self).__init__(CONF.token.provider)
self._register_callback_listeners()
#/keystone/common/manager.py:Manager
class Manager(object):
"""Base class for intermediary request layer.
The Manager layer exists to support additional logic that applies to all
or some of the methods exposed by a service that are not specific to the
HTTP interface.
It also provides a stable entry point to dynamic backends.
An example of a probable use case is logging all the calls.
"""
def __init__(self, driver_name):
self.driver = importutils.import_object(driver_name)
def __getattr__(self, name):
"""Forward calls to the underlying driver."""
f = getattr(self.driver, name)
setattr(self, name, f)
return f
cfg.StrOpt('provider',
default='keystone.token.providers.uuid.Provider',
help='Controls the token construction, validation, and '
'revocation operations. Core providers are '
'"keystone.token.providers.[fernet|pkiz|pki|uuid].'
'Provider".'),
所以driver为/keystone/token/providers/uuid.py中的Provider对象。
#/keystone/token/providers/uuid.py:Provider
class Provider(common.BaseProvider):
def __init__(self, *args, **kwargs):
super(Provider, self).__init__(*args, **kwargs)
def _get_token_id(self, token_data):
return uuid.uuid4().hex
def needs_persistence(self):
"""Should the token be written to a backend."""
return True
#/keystone/token/providers/common.py:BaseProvider
def issue_v2_token(self, token_ref, roles_ref=None,
catalog_ref=None):
metadata_ref = token_ref['metadata']
trust_ref = None
if CONF.trust.enabled and metadata_ref and 'trust_id' in metadata_ref:
trust_ref = self.trust_api.get_trust(metadata_ref['trust_id'])
token_data = self.v2_token_data_helper.format_token(
token_ref, roles_ref, catalog_ref, trust_ref)
token_id = self._get_token_id(token_data)
token_data['access']['token']['id'] = token_id
return token_id, token_data
最终token id通过_get_token_id函数进行生成，且为16进制的id(uuid.uuid4().hex)。且将生成的token id塞进返回给authenticate函数的token_data中。
最终的token_data信息为：
| {'access': {'token': {'issued_at': '2016-03-24T12:08:30.807133', 'expires': '2016-03-24T13:08:30Z', 'id': 'fcb6cd4b7c3e400baca630adf8d878cf', 'tenant': {'description': u'admin tenant', 'enabled': True, 'id': u'09e04766c06d477098201683497d3878', 'name': u'admin' }, 'audit_ids': ['yjGRG7A-S6asfuvYLESR-g'] }, 'serviceCatalog': [ {'endpoints': [{u'adminURL': u'http://192.168.118.1:8774/v2/09e04766c06d477098201683497d3878', 'region': u'RegionOne', u'internalURL': u'http://192.168.118.1:8774/v2/09e04766c06d477098201683497d3878', 'id': u'03966fa6606945b985d8ff3ba1912f00', 'publicURL': u'http://192.168.118.1:8774/v2/09e04766c06d477098201683497d3878' }], 'endpoints_links': [], 'type': u'compute', 'name': u'nova' }, {'endpoints': [{u'adminURL': u'http://192.168.118.1:9696/', 'region': u'RegionOne', u'internalURL': u'http://192.168.118.1:9696/', 'id': u'6ecc803364da42b7a250bf9fe2e71cc4', 'publicURL': u'http://192.168.118.1:9696/' }], 'endpoints_links': [], 'type': u'network', 'name': u'neutron' }, {'endpoints': [{u'adminURL': u'http://192.168.118.1:8776/v2/09e04766c06d477098201683497d3878', 'region': u'RegionOne', u'internalURL': u'http://192.168.118.1:8776/v2/09e04766c06d477098201683497d3878', 'id': u'1a22050d1c404a1fa257b7de8453f7b5', 'publicURL': u'http://192.168.118.1:8776/v2/09e04766c06d477098201683497d3878' }], 'endpoints_links': [], 'type': u'volumev2', 'name': u'cinderv2' }, {'endpoints': [{u'adminURL': u'http://192.168.118.1:8774/v3', 'region': u'RegionOne', u'internalURL': u'http://192.168.118.1:8774/v3', 'id': u'4095ec6b1d9c4e3ba87994a90a0d1ed5', 'publicURL': u'http://192.168.118.1:8774/v3' }], 'endpoints_links': [], 'type': u'computev3', 'name': u'novav3' }, {'endpoints': [{u'adminURL': u'http://192.168.118.1:9292', 'region': u'RegionOne', u'internalURL': u'http://192.168.118.1:9292', 'id': u'7cd86837358f4c19bb551fe3c36fadf9', 'publicURL': u'http://192.168.118.1:9292' }], 'endpoints_links': [], 'type': u'image', 'name': u'glance' }, {'endpoints': [{u'adminURL': u'http://192.168.118.1:8777', 'region': u'RegionOne', u'internalURL': u'http://192.168.118.1:8777', 'id': u'3d9c418b79f641a291a885fe20ec6a84', 'publicURL': u'http://192.168.118.1:8777' }], 'endpoints_links': [], 'type': u'metering', 'name': u'ceilometer' }, {'endpoints': [{u'adminURL': u'http://192.168.118.1:8776/v1/09e04766c06d477098201683497d3878', 'region': u'RegionOne', u'internalURL': u'http://192.168.118.1:8776/v1/09e04766c06d477098201683497d3878', 'id': u'65560ec5eed64581a72f4ac3d1fa519d', 'publicURL': u'http://192.168.118.1:8776/v1/09e04766c06d477098201683497d3878' }], 'endpoints_links': [], 'type': u'volume', 'name': u'cinder' }, {'endpoints': [{u'adminURL': u'http://192.168.118.1:8773/services/Admin', 'region': u'RegionOne', u'internalURL': u'http://192.168.118.1:8773/services/Cloud', 'id': u'1b6b0ae5906345d3a6ec0352c97e724d', 'publicURL': u'http://192.168.118.1:8773/services/Cloud' }], 'endpoints_links': [], 'type': u'ec2', 'name': u'nova_ec2' }, {'endpoints': [{u'adminURL': u'http://192.168.118.1:35357/v2.0', 'region': u'RegionOne', u'internalURL': u'http://192.168.118.1:5000/v2.0', 'id': u'11b8c840c71c4258a644bf93ddee8d0f', 'publicURL': u'http://192.168.118.1:5000/v2.0' }], 'endpoints_links': [], 'type': u'identity', 'name': u'keystone' } ], 'user': {'username': u'admin', 'roles_links': [], 'id': u'f59f17d8e9774eef8730b23ecdc86a4b', 'roles': [{'name': u'admin'}], 'name': u'admin' }, 'metadata': {'is_admin': 0, 'roles': [u'397eaf49b01549dab8be01804bec7972']} } } |
所以当HTTP请求获取token信息时，keystone将会把token id和 各种服务的endpoint信息也一起返回给HTTP请求者，将上述的token_data(keystone返回的)与《nova list命令的代码流程分析》文章中keystoneclient收到的auth_ref信息做比较可知，两者信息几乎完全相同。
目前，authenticate函数的代码流程分析完成。即验证用户信息，获取endpoint列表，生成token id。
在本文中，我们举例的HTTP请求为获取token信息的请求：
| DEBUG (session:197) REQ: curl -g -i -X POST http://192.168.118.1:5000/v2.0/tokens -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-keystoneclient" -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "admin"}}}' |
最终调用到/keystone/token/controllers.py中Auth类中的authenticate函数。当然keystone还提供了很多其他的api接口，对于如何发送正确的HTTP请求，请具体参看OpenStack官网(http://developer.openstack.org/api-ref-identity-admin-v2.html)
3. 总结
本文分析了keystone的WSGI代码流程。
1. 分析了OpenStack创建WSGI server的代码流程。
WSGI server的父进程监听5000和35357端口获取keystone HTTP请求，然后将HTTP请求下发到合适的WSGIserver上。
2. 分析了keystone的HTTP请求的处理流程
本文通过举例获取token信息的HTTP请求来分析keystone的处理流程，即当HTTP请求到来时，一个所谓的”路由”模块会将请求的URL转换成相应的资源，并路由到合适的操作函数上。