Hue支持Hdp3.0 Oozie

1、问题描述

当在HDP中配置了Oozie的SSO后,会发现在Hue中提交oozie workflow失败异常。在解决问题的过程中,共出现了两个异常。第二个异常是在解决第一个异常的基础上出现的。

2、异常一

2.1异常信息

[13/Sep/2018 18:50:15 +0800] editor2 ERROR Error submitting coordinator
Traceback (most recent call last):
File "/home/hue/hue-4.2.0/apps/oozie/src/oozie/views/editor2.py", line 700, in _submit_coordinator
wf_dir = Submission(request.user, wf, request.fs, request.jt, mapping, local_tz=coordinator.data['properties']['timezone']).deploy()
File "/home/hue/hue-4.2.0/desktop/libs/liboozie/src/liboozie/submission2.py", line 194, in deploy
self._update_properties(jt_address) # Needed for coordinator deploying workflows with credentials
File "/home/hue/hue-4.2.0/desktop/libs/liboozie/src/liboozie/submission2.py", line 351, in _update_properties
credentials.fetch(self.api)
File "/home/hue/hue-4.2.0/desktop/libs/liboozie/src/liboozie/credentials.py", line 40, in fetch
configuration = oozie_api.get_configuration()
File "/home/hue/hue-4.2.0/desktop/libs/liboozie/src/liboozie/oozie_api.py", line 319, in get_configuration
resp = self._root.get('admin/configuration', params)
File "/home/hue/hue-4.2.0/desktop/core/src/desktop/lib/rest/resource.py", line 122, in get
return self.invoke("GET", relpath, params, headers=headers, allow_redirects=True, clear_cookies=clear_cookies)
File "/home/hue/hue-4.2.0/desktop/core/src/desktop/lib/rest/resource.py", line 78, in invoke
log_response=log_response)
File "/home/hue/hue-4.2.0/desktop/core/src/desktop/lib/rest/resource.py", line 97, in _invoke
clear_cookies=clear_cookies)
File "/home/hue/hue-4.2.0/desktop/core/src/desktop/lib/rest/http_client.py", line 211, in execute
raise self._exc_class(ex)
RestException: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",)

2.2解决方法

由于是报Hue证书问题,故在Hue的hue.ini配置文件中增加ssl_cacerts属性。


Hue支持Hdp3.0 Oozie_第1张图片
image.png

3、异常二

3.1异常信息

[18/Sep/2018 09:35:40 +0800] middleware INFO Processing exception: 'str' object has no attribute 'get': Traceback (most recent call last):
File "/home/hue/hue-4.2.0/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/core/handlers/base.py", line 112, in get_response
response = wrapped_callback(request, *callback_args, *callback_kwargs)
File "/home/hue/hue-4.2.0/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/transaction.py", line 371, in inner
return func(
args, **kwargs)
File "/home/hue/hue-4.2.0/apps/oozie/src/oozie/decorators.py", line 113, in decorate
return view_func(request, *args, **kwargs)
File "/home/hue/hue-4.2.0/apps/oozie/src/oozie/decorators.py", line 75, in decorate
return view_func(request, *args, **kwargs)
File "/home/hue/hue-4.2.0/apps/oozie/src/oozie/views/editor2.py", line 367, in submit_workflow
return _submit_workflow_helper(request, workflow, submit_action=reverse('oozie:editor_submit_workflow', kwargs={'doc_id': workflow.id}))
File "/home/hue/hue-4.2.0/apps/oozie/src/oozie/views/editor2.py", line 426, in _submit_workflow_helper
'is_oozie_mail_enabled': _is_oozie_mail_enabled(request.user),
File "/home/hue/hue-4.2.0/apps/oozie/src/oozie/views/editor2.py", line 435, in _is_oozie_mail_enabled
return oozie_conf.get('oozie.email.smtp.host') != 'localhost'
AttributeError: 'str' object has no attribute 'get'

3.2问题定位

在不加SSO的情况下,hue并不会出现该异常。通过在出现异常代码行前新加输出oozie_conf的打印日志,发现输出的是登录页面。由于是Hue的后端调用oozie的Rest API,所以并不需要输出登录页面,解决此问题大致有了一个固定方向就是禁止执行SSO流程。

根据查看官方文档,发现SSO使用的类org.apache.hadoop.security.authentication.server.JWTRedirectAuthenticationHandler大致实现方式为如下图标红所示。


Hue支持Hdp3.0 Oozie_第2张图片
image.png

因此查看了AltKerberosAuthenticationHandler.java中关键方法是根据请求中是否带有User-Agent头来判断是否是浏览器访问。


Hue支持Hdp3.0 Oozie_第3张图片
image.png

通过抓包请求,确实也发现hue的请求传递了User-Agent请求头。


Hue支持Hdp3.0 Oozie_第4张图片
image.png

3.3解决方法

在hue-4.2.0/desktop/core/src/desktop/lib/rest/http_client.py中,通过增加如下标红代码行移除User-Agent请求头。


Hue支持Hdp3.0 Oozie_第5张图片
image.png

你可能感兴趣的:(Hue支持Hdp3.0 Oozie)