OOZIE使用Email提醒详解

　　下班不在公司，让大数据任务在跑着。到底跑完没有呢？我总不能老主动去公司登录来看吧？是不是该有个任务提醒机制？是的，这就是OOZIE的EMAIL action功能。

　【原理】OOZIE并不是自己拥有个邮件服务器，而是得借助远程的邮箱账户来发送邮件。这里我们假设注册了一个[email protected]的邮箱，使用163的smtp服务器（地址是smtp.163.com）。理论上根据目前OOZIE的原理，只能配置一个账户拿来发送邮件。

　【步骤一】首先配置oozie-site.xml文件，添加email发送者的一系列信息：

oozie.email.smtp.host         smtp.163.com                 oozie.email.from.address         [email protected]                 oozie.email.smtp.auth         true                 oozie.email.smtp.username         [email protected]                 oozie.email.smtp.password         123456

　【步骤二】在workflow.xml中编写email任务节点。假设我们要发送邮件给[email protected]

${jobTracker}             ${nameNode}             ${master}             Spark-python             ${nameNode}/user/oozie/${examplesRoot}/apps/pythonApp/lib/spark1.py                                         Workflow failed, error message[${wf:errorMessage(wf:lastErrorNode())}]                              uri:oozie:email-action:0.1">             [email protected]             Email notifications for ${wf:id()}             The wf ${wf:id()} successfully completed.

效果如图：

期间遇到的一些问题：

【1、没提供发件者地址，报错】

2017-03-09 02:15:13,055  INFO SparkActionExecutor:520 - SERVER[bigmaster] USER[hadoop] GROUP[-] TOKEN[] APP[Spark_python] JOB[0000000-170309021355327-oozie-hado-W] ACTION[0000000-170309021355327-oozie-hado-W@spark-node] action completed, external ID [job_1489054353684_0001]

2017-03-09 02:15:13,746  INFO ActionStartXCommand:520 - SERVER[bigmaster] USER[hadoop] GROUP[-] TOKEN[] APP[Spark_python] JOB[0000000-170309021355327-oozie-hado-W] ACTION[0000000-170309021355327-oozie-hado-W@goemail] Start action [0000000-170309021355327-oozie-hado-W@goemail] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]

2017-03-09 02:15:14,945  WARN ActionStartXCommand:523 - SERVER[bigmaster] USER[hadoop] GROUP[-] TOKEN[] APP[Spark_python] JOB[0000000-170309021355327-oozie-hado-W] ACTION[0000000-170309021355327-oozie-hado-W@goemail] Error starting action [goemail]. ErrorType [ERROR], ErrorCode [EM007], Message [EM007: Encountered an error while sending the email message over SMTP.]

org.apache.oozie.action.ActionExecutorException: EM007: Encountered an error while sending the email message over SMTP.

at org.apache.oozie.action.email.EmailActionExecutor.email(EmailActionExecutor.ｊａｖａ:297)

at org.apache.oozie.action.email.EmailActionExecutor.validateAndMail(EmailActionExecutor.ｊａｖａ:171)

at org.apache.oozie.action.email.EmailActionExecutor.start(EmailActionExecutor.ｊａｖａ:110)

at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.ｊａｖａ:234)

at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.ｊａｖａ:65)

at org.apache.oozie.command.XCommand.call(XCommand.ｊａｖａ:287)

at org.apache.oozie.command.wf.SignalXCommand.execute(SignalXCommand.ｊａｖａ:459)

at org.apache.oozie.command.wf.SignalXCommand.execute(SignalXCommand.ｊａｖａ:82)

at org.apache.oozie.command.XCommand.call(XCommand.ｊａｖａ:287)

at org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.ｊａｖａ:280)

at org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.ｊａｖａ:61)

at org.apache.oozie.command.XCommand.call(XCommand.ｊａｖａ:287)

at org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.ｊａｖａ:244)

at org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.ｊａｖａ:56)

at org.apache.oozie.command.XCommand.call(XCommand.ｊａｖａ:287)

at ｊａｖａ.util.concurrent.FutureTask.run(FutureTask.ｊａｖａ:266)

at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.ｊａｖａ:179)

at ｊａｖａ.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ｊａｖａ:1142)

at ｊａｖａ.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ｊａｖａ:617)

at ｊａｖａ.lang.Thread.run(Thread.ｊａｖａ:745)

Caused by: com.sun.mail.smtp.SMTPSendFailedException:  553 Local user only ,163 smtp5, jeifwA8dYwg--.16436S2 1429458765

at com.sun.mail.smtp.SMTPTransport.issueSendCommand(SMTPTransport.ｊａｖａ:1388)

at com.sun.mail.smtp.SMTPTransport.mailFrom(SMTPTransport.ｊａｖａ:959)

at com.sun.mail.smtp.SMTPTransport.sendMessage(SMTPTransport.ｊａｖａ:583)

at ｊａｖａx.mail.Transport.send0(Transport.ｊａｖａ:169)

at ｊａｖａx.mail.Transport.send(Transport.ｊａｖａ:98)

at org.apache.oozie.action.email.EmailActionExecutor.email(EmailActionExecutor.ｊａｖａ:293)

... 19 more

2017-03-09 02:15:14,962  WARN ActionStartXCommand:523 - SERVER[bigmaster] USER[hadoop] GROUP[-] TOKEN[] APP[Spark_python] JOB[0000000-170309021355327-oozie-hado-W] ACTION[0000000-170309021355327-oozie-hado-W@goemail] Setting Action Status to [DONE]

【1、解决】在oozie-site.xml中添加发件人From属性：

   

        oozie.email.from.address

        [email protected]

   

【2、没进行安全认证，报错】

2017-03-09 17:39:11,059  WARN ActionStartXCommand:523 - SERVER[bigmaster] USER[hadoop] GROUP[-] TOKEN[] APP[Spark_python] JOB[0000001-170309173451959-oozie-hado-W] ACTION[0000001-170309173451959-oozie-hado-W@goemail] Error starting action [goemail]. ErrorType [ERROR], ErrorCode [EM007], Message [EM007: Encountered an error while sending the email message over SMTP.]

org.apache.oozie.action.ActionExecutorException: EM007: Encountered an error while sending the email message over SMTP.

at org.apache.oozie.action.email.EmailActionExecutor.email(EmailActionExecutor.ｊａｖａ:297)

at org.apache.oozie.action.email.EmailActionExecutor.validateAndMail(EmailActionExecutor.ｊａｖａ:171)

at org.apache.oozie.action.email.EmailActionExecutor.start(EmailActionExecutor.ｊａｖａ:110)

at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.ｊａｖａ:234)

at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.ｊａｖａ:65)

at org.apache.oozie.command.XCommand.call(XCommand.ｊａｖａ:287)

at org.apache.oozie.command.wf.SignalXCommand.execute(SignalXCommand.ｊａｖａ:459)

at org.apache.oozie.command.wf.SignalXCommand.execute(SignalXCommand.ｊａｖａ:82)

at org.apache.oozie.command.XCommand.call(XCommand.ｊａｖａ:287)

at org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.ｊａｖａ:280)

at org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.ｊａｖａ:61)

at org.apache.oozie.command.XCommand.call(XCommand.ｊａｖａ:287)

at org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.ｊａｖａ:244)

at org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.ｊａｖａ:56)

at org.apache.oozie.command.XCommand.call(XCommand.ｊａｖａ:287)

at ｊａｖａ.util.concurrent.FutureTask.run(FutureTask.ｊａｖａ:266)

at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.ｊａｖａ:179)

at ｊａｖａ.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ｊａｖａ:1142)

at ｊａｖａ.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ｊａｖａ:617)

at ｊａｖａ.lang.Thread.run(Thread.ｊａｖａ:745)

Caused by: com.sun.mail.smtp.SMTPSendFailedException: 553 authentication is required,163 smtp8,DMCoWDAw--.1042S2 14809951

at com.sun.mail.smtp.SMTPTransport.issueSendCommand(SMTPTransport.ｊａｖａ:1388)

at com.sun.mail.smtp.SMTPTransport.mailFrom(SMTPTransport.ｊａｖａ:959)

at com.sun.mail.smtp.SMTPTransport.sendMessage(SMTPTransport.ｊａｖａ:583)

at ｊａｖａx.mail.Transport.send0(Transport.ｊａｖａ:169)

at ｊａｖａx.mail.Transport.send(Transport.ｊａｖａ:98)

at org.apache.oozie.action.email.EmailActionExecutor.email(EmailActionExecutor.ｊａｖａ:293)

... 19 more

2017-03-09 17:39:11,064  WARN ActionStartXCommand:523 - SERVER[bigmaster] USER[hadoop] GROUP[-] TOKEN[] APP[Spark_python] JOB[0000001-170309173451959-oozie-hado-W] ACTION[0000001-170309173451959-oozie-hado-W@goemail] Setting Action Status to [DONE]

【2、解决】在oozie-site.xml中添加认证auth属性并设置为true：

   

        oozie.email.smtp.auth

        true