ELService 服务是用于初始化el计算的服务，oozie的参数计算使用的是 org.apache.commons.el 的 JSP Expression Language Evaluator。功能可以做到，变量值的动态计算，支持oozie这次的计算表达式。这些表达式是固定的，需要在初始化的时候加载到oozie中。现在来看看代码：
oozie将对参数的计算封装成 ELEvaluator：
private Context context;
private ExpressionEvaluatorImpl evaluator = new ExpressionEvaluatorImpl();
ExpressionEvaluatorImpl ** 是common包中的计算实现。context** 继承了
VariableResolver，FunctionMapper
/**
* Provides functions and variables for the EL evaluator. All functions and variables in the context of an EL
* evaluator are accessible from EL expressions.
*/
public static class Context implements VariableResolver, FunctionMapper {
private Map vars;
private Map functions;
/**
* Create an empty context.
*/
public Context() {
vars = new HashMap();
functions = new HashMap();
}
/**
* Add variables to the context.
*
* @param vars variables to add to the context.
*/
public void setVariables(Map vars) {
this.vars.putAll(vars);
}
/**
* Add a variable to the context.
*
* @param name variable name.
* @param value variable value.
*/
public void setVariable(String name, Object value) {
vars.put(name, value);
}
/**
* Return a variable from the context.
*
* @param name variable name.
* @return the variable value.
*/
public Object getVariable(String name) {
return vars.get(name);
}
/**
* Add a function to the context.
*
* @param prefix function prefix.
* @param functionName function name.
* @param method method that will be invoked for the function, it must be a static and public method.
*/
public void addFunction(String prefix, String functionName, Method method) {
if ((method.getModifiers() & (Modifier.PUBLIC | Modifier.STATIC)) != (Modifier.PUBLIC | Modifier.STATIC)) {
throw new IllegalArgumentException(XLog.format("Method[{0}] must be public and static", method));
}
prefix = (prefix.length() > 0) ? prefix + ":" : "";
functions.put(prefix + functionName, method);
}
/**
* Resolve a variable name. Used by the EL evaluator implemenation.
*
* @param name variable name.
* @return the variable value.
* @throws ELException thrown if the variable is not defined in the context.
*/
public Object resolveVariable(String name) throws ELException {
if (!vars.containsKey(name)) {
throw new ELException(XLog.format("variable [{0}] cannot be resolved", name));
}
return vars.get(name);
}
/**
* Resolve a function prefix:name. Used by the EL evaluator implementation.
*
* @param prefix function prefix.
* @param name function name.
* @return the method associated to the function.
*/
public Method resolveFunction(String prefix, String name) {
if (prefix.length() > 0) {
name = prefix + ":" + name;
}
return functions.get(name);
}
}
这里的 Map中的key是方法或者是数值的别名。
/**
* Evaluate an EL expression.
*
* @param expr EL expression to evaluate.
* @param clazz return type of the EL expression.
* @return the object the EL expression evaluated to.
* @throws Exception thrown if an EL function failed due to a transient error or EL expression could not be
* evaluated.
*/
@SuppressWarnings({"unchecked", "deprecation"})
public T evaluate(String expr, Class clazz) throws Exception {
ELEvaluator existing = current.get();
try {
current.set(this);
return (T) evaluator.evaluate(expr, clazz, context, context);
}
catch (ELException ex) {
if (ex.getRootCause() instanceof Exception) {
throw (Exception) ex.getRootCause();
}
else {
throw ex;
}
}
finally {
current.set(existing);
}
}
表达式计算的具体实现方法；
正如前面介绍组件的时候，可以看到oozie内置了大量的表达式运算。现在我们来看看他们的具体实现；
oozie.service.ELService.constants.workflow
KB=org.apache.oozie.util.ELConstantsFunctions#KB,
MB=org.apache.oozie.util.ELConstantsFunctions#MB,
GB=org.apache.oozie.util.ELConstantsFunctions#GB,
TB=org.apache.oozie.util.ELConstantsFunctions#TB,
PB=org.apache.oozie.util.ELConstantsFunctions#PB,
RECORDS=org.apache.oozie.action.hadoop.HadoopELFunctions#RECORDS,
MAP_IN=org.apache.oozie.action.hadoop.HadoopELFunctions#MAP_IN,
MAP_OUT=org.apache.oozie.action.hadoop.HadoopELFunctions#MAP_OUT,
REDUCE_IN=org.apache.oozie.action.hadoop.HadoopELFunctions#REDUCE_IN,
REDUCE_OUT=org.apache.oozie.action.hadoop.HadoopELFunctions#REDUCE_OUT,
GROUPS=org.apache.oozie.action.hadoop.HadoopELFunctions#GROUPS
EL constant declarations, separated by commas, format is [PREFIX:]NAME=CLASS#CONSTANT.
oozie.service.ELService.functions.workflow
firstNotNull=org.apache.oozie.util.ELConstantsFunctions#firstNotNull,
concat=org.apache.oozie.util.ELConstantsFunctions#concat,
replaceAll=org.apache.oozie.util.ELConstantsFunctions#replaceAll,
appendAll=org.apache.oozie.util.ELConstantsFunctions#appendAll,
trim=org.apache.oozie.util.ELConstantsFunctions#trim,
timestamp=org.apache.oozie.util.ELConstantsFunctions#timestamp,
urlEncode=org.apache.oozie.util.ELConstantsFunctions#urlEncode, toJsonStr=org.apache.oozie.util.ELConstantsFunctions#toJsonStr,
toPropertiesStr=org.apache.oozie.util.ELConstantsFunctions#toPropertiesStr,
toConfigurationStr=org.apache.oozie.util.ELConstantsFunctions#toConfigurationStr,
wf:id=org.apache.oozie.DagELFunctions#wf_id,
wf:name=org.apache.oozie.DagELFunctions#wf_name,
wf:appPath=org.apache.oozie.DagELFunctions#wf_appPath,
wf:conf=org.apache.oozie.DagELFunctions#wf_conf,
wf:user=org.apache.oozie.DagELFunctions#wf_user,
wf:group=org.apache.oozie.DagELFunctions#wf_group,
wf:callback=org.apache.oozie.DagELFunctions#wf_callback,
wf:transition=org.apache.oozie.DagELFunctions#wf_transition,
wf:lastErrorNode=org.apache.oozie.DagELFunctions#wf_lastErrorNode,
wf:errorCode=org.apache.oozie.DagELFunctions#wf_errorCode,
wf:errorMessage=org.apache.oozie.DagELFunctions#wf_errorMessage,
wf:run=org.apache.oozie.DagELFunctions#wf_run,
wf:actionData=org.apache.oozie.DagELFunctions#wf_actionData,
wf:actionExternalId=org.apache.oozie.DagELFunctions#wf_actionExternalId,
wf:actionTrackerUri=org.apache.oozie.DagELFunctions#wf_actionTrackerUri,
wf:actionExternalStatus=org.apache.oozie.DagELFunctions#wf_actionExternalStatus,
hadoop:counters=org.apache.oozie.action.hadoop.HadoopELFunctions#hadoop_counters,
hadoop:conf=org.apache.oozie.action.hadoop.HadoopELFunctions#hadoop_conf,
fs:exists=org.apache.oozie.action.hadoop.FsELFunctions#fs_exists,
fs:isDir=org.apache.oozie.action.hadoop.FsELFunctions#fs_isDir,
fs:dirSize=org.apache.oozie.action.hadoop.FsELFunctions#fs_dirSize,
fs:fileSize=org.apache.oozie.action.hadoop.FsELFunctions#fs_fileSize,
fs:blockSize=org.apache.oozie.action.hadoop.FsELFunctions#fs_blockSize,
hcat:exists=org.apache.oozie.coord.HCatELFunctions#hcat_exists
EL functions declarations, separated by commas, format is [PREFIX:]NAME=CLASS#METHOD.
现在看几个具体实现:
wf:user=org.apache.oozie.DagELFunctions#wf_user,
/**
* Return the job owner user name.
*
* @return the job owner user name.
*/
public static String wf_user() {
return getWorkflow().getUser();
}
public static WorkflowJobBean getWorkflow() {
ELEvaluator eval = ELEvaluator.getCurrent();
return (WorkflowJobBean) eval.getVariable(WORKFLOW);
}
hadoop:counters=org.apache.oozie.action.hadoop.HadoopELFunctions#hadoop_counters
public static Map> hadoop_counters(String nodeName) throws ELEvaluationException {
WorkflowInstance instance = DagELFunctions.getWorkflow().getWorkflowInstance();
Object obj = instance.getTransientVar(nodeName + WorkflowInstance.NODE_VAR_SEPARATOR + HADOOP_COUNTERS);
Map> counters = (Map>) obj;
if (counters == null) {
counters = getCounters(nodeName);
// In Hadoop 0.23 they deprecated 'org.apache.hadoop.mapred.Task$Counter' and they REMOVED IT
// Here we are getting the new Name and inserting it using the old name if the old name is not found
if (counters.get(RECORDS) == null) {
counters.put(RECORDS, counters.get(RECORDS_023));
}
instance.setTransientVar(nodeName + WorkflowInstance.NODE_VAR_SEPARATOR + HADOOP_COUNTERS, counters);
}
return counters;
}
fs:exists=org.apache.oozie.action.hadoop.FsELFunctions#fs_exists
/**
* Return if a path exists.
*
* @param pathUri file system path uri.
* @return true if the path exists, false if it does not.
* @throws Exception
*/
public static boolean fs_exists(String pathUri) throws Exception {
URI uri = new URI(pathUri);
String path = uri.getPath();
FileSystem fs = getFileSystem(uri);
return fs.exists(new Path(path));
}
hcat:exists=org.apache.oozie.coord.HCatELFunctions#hcat_exists
/**
* Return true if partitions exists or false if not.
*
* @param uri hcatalog partition uri.
* @return true if the uri exists, false if it does not.
* @throws Exception
*/
public static boolean hcat_exists(String uri) throws Exception {
URI hcatURI = new URI(uri);
URIHandlerService uriService = Services.get().get(URIHandlerService.class);
URIHandler handler = uriService.getURIHandler(hcatURI);
WorkflowJob workflow = DagELFunctions.getWorkflow();
String user = workflow.getUser();
return handler.exists(hcatURI, EMPTY_CONF, user);
}
oozie.service.ELService.constants.wf-sla-submit
MINUTES=org.apache.oozie.util.ELConstantsFunctions#SUBMIT_MINUTES,
HOURS=org.apache.oozie.util.ELConstantsFunctions#SUBMIT_HOURS,
DAYS=org.apache.oozie.util.ELConstantsFunctions#SUBMIT_DAYS
EL constant declarations, separated by commas, format is [PREFIX:]NAME=CLASS#CONSTANT.
oozie.service.ELService.functions.coord-job-submit-freq
coord:days=org.apache.oozie.coord.CoordELFunctions#ph1_coord_days,
coord:months=org.apache.oozie.coord.CoordELFunctions#ph1_coord_months,
coord:hours=org.apache.oozie.coord.CoordELFunctions#ph1_coord_hours,
coord:minutes=org.apache.oozie.coord.CoordELFunctions#ph1_coord_minutes,
coord:endOfDays=org.apache.oozie.coord.CoordELFunctions#ph1_coord_endOfDays,
coord:endOfMonths=org.apache.oozie.coord.CoordELFunctions#ph1_coord_endOfMonths,
coord:conf=org.apache.oozie.coord.CoordELFunctions#coord_conf,
coord:user=org.apache.oozie.coord.CoordELFunctions#coord_user,
hadoop:conf=org.apache.oozie.action.hadoop.HadoopELFunctions#hadoop_conf
EL functions declarations, separated by commas, format is [PREFIX:]NAME=CLASS#METHOD.
现在看几个具体实现:
coord:days=org.apache.oozie.coord.CoordELFunctions#ph1_coord_days
public static int ph1_coord_days(int val) {
val = ParamChecker.checkGTZero(val, "n");
ELEvaluator eval = ELEvaluator.getCurrent();
eval.setVariable("timeunit", TimeUnit.DAY);
eval.setVariable("endOfDuration", TimeUnit.NONE);
return val;
}
oozie.service.ELService.functions.coord-job-wait-timeout
coord:days=org.apache.oozie.coord.CoordELFunctions#ph1_coord_days,
coord:months=org.apache.oozie.coord.CoordELFunctions#ph1_coord_months,
coord:hours=org.apache.oozie.coord.CoordELFunctions#ph1_coord_hours,
coord:minutes=org.apache.oozie.coord.CoordELFunctions#ph1_coord_minutes,
hadoop:conf=org.apache.oozie.action.hadoop.HadoopELFunctions#hadoop_conf
EL functions declarations, separated by commas, format is [PREFIX:]NAME=CLASS#METHOD.
oozie.service.ELService.constants.coord-job-submit-nofuncs
MINUTE=org.apache.oozie.coord.CoordELConstants#SUBMIT_MINUTE,
HOUR=org.apache.oozie.coord.CoordELConstants#SUBMIT_HOUR,
DAY=org.apache.oozie.coord.CoordELConstants#SUBMIT_DAY,
MONTH=org.apache.oozie.coord.CoordELConstants#SUBMIT_MONTH,
YEAR=org.apache.oozie.coord.CoordELConstants#SUBMIT_YEAR
EL constant declarations, separated by commas, format is [PREFIX:]NAME=CLASS#CONSTANT.
oozie.service.ELService.functions.coord-job-submit-nofuncs
coord:conf=org.apache.oozie.coord.CoordELFunctions#coord_conf,
coord:user=org.apache.oozie.coord.CoordELFunctions#coord_user,
hadoop:conf=org.apache.oozie.action.hadoop.HadoopELFunctions#hadoop_conf
EL functions declarations, separated by commas, format is [PREFIX:]NAME=CLASS#METHOD.
oozie.service.ELService.functions.coord-job-submit-instances
coord:hoursInDay=org.apache.oozie.coord.CoordELFunctions#ph1_coord_hoursInDay_echo,
coord:daysInMonth=org.apache.oozie.coord.CoordELFunctions#ph1_coord_daysInMonth_echo,
coord:tzOffset=org.apache.oozie.coord.CoordELFunctions#ph1_coord_tzOffset_echo,
coord:current=org.apache.oozie.coord.CoordELFunctions#ph1_coord_current_echo,
coord:currentRange=org.apache.oozie.coord.CoordELFunctions#ph1_coord_currentRange_echo,
coord:offset=org.apache.oozie.coord.CoordELFunctions#ph1_coord_offset_echo,
coord:latest=org.apache.oozie.coord.CoordELFunctions#ph1_coord_latest_echo,
coord:latestRange=org.apache.oozie.coord.CoordELFunctions#ph1_coord_latestRange_echo,
coord:future=org.apache.oozie.coord.CoordELFunctions#ph1_coord_future_echo,
coord:futureRange=org.apache.oozie.coord.CoordELFunctions#ph1_coord_futureRange_echo,
coord:formatTime=org.apache.oozie.coord.CoordELFunctions#ph1_coord_formatTime_echo,
coord:conf=org.apache.oozie.coord.CoordELFunctions#coord_conf,
coord:user=org.apache.oozie.coord.CoordELFunctions#coord_user,
coord:absolute=org.apache.oozie.coord.CoordELFunctions#ph1_coord_absolute_echo,
hadoop:conf=org.apache.oozie.action.hadoop.HadoopELFunctions#hadoop_conf
EL functions declarations, separated by commas, format is [PREFIX:]NAME=CLASS#METHOD.
现在看几个具体实现:
coord:current=org.apache.oozie.coord.CoordELFunctions#ph1_coord_current_echo
public static String ph1_coord_current_echo(String n) {
return echoUnResolved("current", n);
}
private static String echoUnResolved(String functionName, String n) {
return echoUnResolvedPre(functionName, n, "coord:");
}
private static String echoUnResolvedPre(String functionName, String n, String prefix) {
ELEvaluator eval = ELEvaluator.getCurrent();
eval.setVariable(".wrap", "true");
return prefix + functionName + "(" + n + ")"; // Unresolved
}
oozie.service.ELService.functions.coord-job-submit-data
coord:dataIn=org.apache.oozie.coord.CoordELFunctions#ph1_coord_dataIn_echo,
coord:dataOut=org.apache.oozie.coord.CoordELFunctions#ph1_coord_dataOut_echo,
coord:nominalTime=org.apache.oozie.coord.CoordELFunctions#ph1_coord_nominalTime_echo_wrap,
coord:actualTime=org.apache.oozie.coord.CoordELFunctions#ph1_coord_actualTime_echo_wrap,
coord:dateOffset=org.apache.oozie.coord.CoordELFunctions#ph1_coord_dateOffset_echo,
coord:dateTzOffset=org.apache.oozie.coord.CoordELFunctions#ph1_coord_dateTzOffset_echo,
coord:formatTime=org.apache.oozie.coord.CoordELFunctions#ph1_coord_formatTime_echo,
coord:actionId=org.apache.oozie.coord.CoordELFunctions#ph1_coord_actionId_echo,
coord:name=org.apache.oozie.coord.CoordELFunctions#ph1_coord_name_echo,
coord:conf=org.apache.oozie.coord.CoordELFunctions#coord_conf,
coord:user=org.apache.oozie.coord.CoordELFunctions#coord_user,
coord:databaseIn=org.apache.oozie.coord.HCatELFunctions#ph1_coord_databaseIn_echo,
coord:databaseOut=org.apache.oozie.coord.HCatELFunctions#ph1_coord_databaseOut_echo,
coord:tableIn=org.apache.oozie.coord.HCatELFunctions#ph1_coord_tableIn_echo,
coord:tableOut=org.apache.oozie.coord.HCatELFunctions#ph1_coord_tableOut_echo,
coord:dataInPartitionFilter=org.apache.oozie.coord.HCatELFunctions#ph1_coord_dataInPartitionFilter_echo,
coord:dataInPartitionMin=org.apache.oozie.coord.HCatELFunctions#ph1_coord_dataInPartitionMin_echo,
coord:dataInPartitionMax=org.apache.oozie.coord.HCatELFunctions#ph1_coord_dataInPartitionMax_echo,
coord:dataInPartitions=org.apache.oozie.coord.HCatELFunctions#ph1_coord_dataInPartitions_echo,
coord:dataOutPartitions=org.apache.oozie.coord.HCatELFunctions#ph1_coord_dataOutPartitions_echo,
coord:dataOutPartitionValue=org.apache.oozie.coord.HCatELFunctions#ph1_coord_dataOutPartitionValue_echo,
hadoop:conf=org.apache.oozie.action.hadoop.HadoopELFunctions#hadoop_conf
EL functions declarations, separated by commas, format is [PREFIX:]NAME=CLASS#METHOD.
现在看几个具体实现:
coord:dataIn=org.apache.oozie.coord.CoordELFunctions#ph1_coord_dataIn_echo
public static String ph1_coord_dataIn_echo(String n) {
ELEvaluator eval = ELEvaluator.getCurrent();
String val = (String) eval.getVariable("oozie.dataname." + n);
if (val == null || val.equals("data-in") == false) {
XLog.getLog(CoordELFunctions.class).error("data_in_name " + n + " is not valid");
throw new RuntimeException("data_in_name " + n + " is not valid");
}
return echoUnResolved("dataIn", "'" + n + "'");
}
oozie.service.ELService.constants.coord-sla-submit
MINUTES=org.apache.oozie.coord.CoordELConstants#SUBMIT_MINUTES,
HOURS=org.apache.oozie.coord.CoordELConstants#SUBMIT_HOURS,
DAYS=org.apache.oozie.coord.CoordELConstants#SUBMIT_DAYS
EL constant declarations, separated by commas, format is [PREFIX:]NAME=CLASS#CONSTANT.
oozie.service.ELService.functions.coord-sla-submit
coord:dataOut=org.apache.oozie.coord.CoordELFunctions#ph1_coord_dataOut_echo,
coord:nominalTime=org.apache.oozie.coord.CoordELFunctions#ph1_coord_nominalTime_echo_fixed,
coord:actualTime=org.apache.oozie.coord.CoordELFunctions#ph1_coord_actualTime_echo_wrap,
coord:dateOffset=org.apache.oozie.coord.CoordELFunctions#ph1_coord_dateOffset_echo,
coord:dateTzOffset=org.apache.oozie.coord.CoordELFunctions#ph1_coord_dateTzOffset_echo,
coord:formatTime=org.apache.oozie.coord.CoordELFunctions#ph1_coord_formatTime_echo,
coord:actionId=org.apache.oozie.coord.CoordELFunctions#ph1_coord_actionId_echo,
coord:name=org.apache.oozie.coord.CoordELFunctions#ph1_coord_name_echo,
coord:conf=org.apache.oozie.coord.CoordELFunctions#coord_conf,
coord:user=org.apache.oozie.coord.CoordELFunctions#coord_user,
coord:databaseOut=org.apache.oozie.coord.HCatELFunctions#ph1_coord_databaseOut_echo,
coord:tableOut=org.apache.oozie.coord.HCatELFunctions#ph1_coord_tableOut_echo,
coord:dataOutPartitions=org.apache.oozie.coord.HCatELFunctions#ph1_coord_dataOutPartitions_echo,
coord:dataOutPartitionValue=org.apache.oozie.coord.HCatELFunctions#ph1_coord_dataOutPartitionValue_echo,
hadoop:conf=org.apache.oozie.action.hadoop.HadoopELFunctions#hadoop_conf
EL functions declarations, separated by commas, format is [PREFIX:]NAME=CLASS#METHOD.
oozie.service.ELService.functions.coord-action-create
coord:hoursInDay=org.apache.oozie.coord.CoordELFunctions#ph2_coord_hoursInDay,
coord:daysInMonth=org.apache.oozie.coord.CoordELFunctions#ph2_coord_daysInMonth,
coord:tzOffset=org.apache.oozie.coord.CoordELFunctions#ph2_coord_tzOffset,
coord:current=org.apache.oozie.coord.CoordELFunctions#ph2_coord_current,
coord:currentRange=org.apache.oozie.coord.CoordELFunctions#ph2_coord_currentRange,
coord:offset=org.apache.oozie.coord.CoordELFunctions#ph2_coord_offset,
coord:latest=org.apache.oozie.coord.CoordELFunctions#ph2_coord_latest_echo,
coord:latestRange=org.apache.oozie.coord.CoordELFunctions#ph2_coord_latestRange_echo,
coord:future=org.apache.oozie.coord.CoordELFunctions#ph2_coord_future_echo,
coord:futureRange=org.apache.oozie.coord.CoordELFunctions#ph2_coord_futureRange_echo,
coord:actionId=org.apache.oozie.coord.CoordELFunctions#ph2_coord_actionId,
coord:name=org.apache.oozie.coord.CoordELFunctions#ph2_coord_name,
coord:formatTime=org.apache.oozie.coord.CoordELFunctions#ph2_coord_formatTime,
coord:conf=org.apache.oozie.coord.CoordELFunctions#coord_conf,
coord:user=org.apache.oozie.coord.CoordELFunctions#coord_user,
coord:absolute=org.apache.oozie.coord.CoordELFunctions#ph2_coord_absolute_echo,
coord:absoluteRange=org.apache.oozie.coord.CoordELFunctions#ph2_coord_absolute_range,
hadoop:conf=org.apache.oozie.action.hadoop.HadoopELFunctions#hadoop_conf
EL functions declarations, separated by commas, format is [PREFIX:]NAME=CLASS#METHOD.
现在看几个具体实现:
coord:hoursInDay=org.apache.oozie.coord.CoordELFunctions#ph2_coord_hoursInDay
/**
* Determine how many hours is on the date of n-th dataset instance. It depends on: 1. Data set frequency
* 2. Data set Time unit (day, month, minute) 3. Data set Time zone/DST 4. End Day/Month flag 5.
* Data set initial instance 6. Action Creation Time
*
* @param n instance count domain: n is integer
* @return number of hours on that day returns -1 means n-th instance is earlier than Initial-Instance of DS
* @throws Exception
*/
public static int ph2_coord_hoursInDay(int n) throws Exception {
int datasetFrequency = (int) getDSFrequency();
// /Calendar nominalInstanceCal =
// getCurrentInstance(getActionCreationtime());
Calendar nominalInstanceCal = getEffectiveNominalTime();
if (nominalInstanceCal == null) {
return -1;
}
nominalInstanceCal.add(getDSTimeUnit().getCalendarUnit(), datasetFrequency * n);
/*
* if (nominalInstanceCal.getTime().compareTo(getInitialInstance()) < 0)
* { return -1; }
*/
nominalInstanceCal.setTimeZone(getDatasetTZ());// Use Dataset TZ
// DateUtils.moveToEnd(nominalInstanceCal, getDSEndOfFlag());
return DateUtils.hoursInDay(nominalInstanceCal);
}
coord:current=org.apache.oozie.coord.CoordELFunctions#ph2_coord_current
/**
* Determine the date-time in Oozie processing timezone of n-th dataset instance. It depends on: 1.
* Data set frequency 2.
* Data set Time unit (day, month, minute) 3. Data set Time zone/DST 4. End Day/Month flag 5. Data
* set initial instance 6. Action Creation Time
*
* @param n instance count domain: n is integer
* @return date-time in Oozie processing timezone of the n-th instance returns 'null' means n-th instance is
* earlier than Initial-Instance of DS
* @throws Exception
*/
public static String ph2_coord_current(int n) throws Exception {
if (isSyncDataSet()) {
// For Sync Dataset
return coord_current_sync(n);
}
else {
throw new UnsupportedOperationException("Asynchronous Dataset is not supported yet");
}
}
coord:formatTime=org.apache.oozie.coord.CoordELFunctions#ph2_coord_formatTime
/**
* Convert from standard date-time formatting to a desired format.
*
* @param dateTimeStr - A timestamp in standard (ISO8601) format.
* @param format - A string representing the desired format.
* @return coordinator action creation or materialization date time
* @throws Exception if unable to format the Date object to String */
public static String ph2_coord_formatTime(String dateTimeStr, String format)
throws Exception {
Date dateTime = DateUtils.parseDateOozieTZ(dateTimeStr);
return DateUtils.formatDateCustom(dateTime, format);
}
oozie.service.ELService.functions.coord-action-create-inst
coord:hoursInDay=org.apache.oozie.coord.CoordELFunctions#ph2_coord_hoursInDay,
coord:daysInMonth=org.apache.oozie.coord.CoordELFunctions#ph2_coord_daysInMonth,
coord:tzOffset=org.apache.oozie.coord.CoordELFunctions#ph2_coord_tzOffset,
coord:current=org.apache.oozie.coord.CoordELFunctions#ph2_coord_current_echo,
coord:currentRange=org.apache.oozie.coord.CoordELFunctions#ph2_coord_currentRange_echo,
coord:offset=org.apache.oozie.coord.CoordELFunctions#ph2_coord_offset_echo,
coord:latest=org.apache.oozie.coord.CoordELFunctions#ph2_coord_latest_echo,
coord:latestRange=org.apache.oozie.coord.CoordELFunctions#ph2_coord_latestRange_echo,
coord:future=org.apache.oozie.coord.CoordELFunctions#ph2_coord_future_echo,
coord:futureRange=org.apache.oozie.coord.CoordELFunctions#ph2_coord_futureRange_echo,
coord:formatTime=org.apache.oozie.coord.CoordELFunctions#ph2_coord_formatTime,
coord:conf=org.apache.oozie.coord.CoordELFunctions#coord_conf,
coord:user=org.apache.oozie.coord.CoordELFunctions#coord_user,
coord:absolute=org.apache.oozie.coord.CoordELFunctions#ph2_coord_absolute_echo,
coord:absoluteRange=org.apache.oozie.coord.CoordELFunctions#ph2_coord_absolute_range,
hadoop:conf=org.apache.oozie.action.hadoop.HadoopELFunctions#hadoop_conf
EL functions declarations, separated by commas, format is [PREFIX:]NAME=CLASS#METHOD.
oozie.service.ELService.functions.coord-action-start
coord:hoursInDay=org.apache.oozie.coord.CoordELFunctions#ph3_coord_hoursInDay,
coord:daysInMonth=org.apache.oozie.coord.CoordELFunctions#ph3_coord_daysInMonth,
coord:tzOffset=org.apache.oozie.coord.CoordELFunctions#ph3_coord_tzOffset,
coord:latest=org.apache.oozie.coord.CoordELFunctions#ph3_coord_latest,
coord:latestRange=org.apache.oozie.coord.CoordELFunctions#ph3_coord_latestRange,
coord:future=org.apache.oozie.coord.CoordELFunctions#ph3_coord_future,
coord:futureRange=org.apache.oozie.coord.CoordELFunctions#ph3_coord_futureRange,
coord:dataIn=org.apache.oozie.coord.CoordELFunctions#ph3_coord_dataIn,
coord:dataOut=org.apache.oozie.coord.CoordELFunctions#ph3_coord_dataOut,
coord:nominalTime=org.apache.oozie.coord.CoordELFunctions#ph3_coord_nominalTime,
coord:actualTime=org.apache.oozie.coord.CoordELFunctions#ph3_coord_actualTime,
coord:dateOffset=org.apache.oozie.coord.CoordELFunctions#ph3_coord_dateOffset,
coord:dateTzOffset=org.apache.oozie.coord.CoordELFunctions#ph3_coord_dateTzOffset,
coord:formatTime=org.apache.oozie.coord.CoordELFunctions#ph3_coord_formatTime,
coord:actionId=org.apache.oozie.coord.CoordELFunctions#ph3_coord_actionId,
coord:name=org.apache.oozie.coord.CoordELFunctions#ph3_coord_name,
coord:conf=org.apache.oozie.coord.CoordELFunctions#coord_conf,
coord:user=org.apache.oozie.coord.CoordELFunctions#coord_user,
coord:databaseIn=org.apache.oozie.coord.HCatELFunctions#ph3_coord_databaseIn,
coord:databaseOut=org.apache.oozie.coord.HCatELFunctions#ph3_coord_databaseOut,
coord:tableIn=org.apache.oozie.coord.HCatELFunctions#ph3_coord_tableIn,
coord:tableOut=org.apache.oozie.coord.HCatELFunctions#ph3_coord_tableOut,
coord:dataInPartitionFilter=org.apache.oozie.coord.HCatELFunctions#ph3_coord_dataInPartitionFilter,
coord:dataInPartitionMin=org.apache.oozie.coord.HCatELFunctions#ph3_coord_dataInPartitionMin,
coord:dataInPartitionMax=org.apache.oozie.coord.HCatELFunctions#ph3_coord_dataInPartitionMax,
coord:dataInPartitions=org.apache.oozie.coord.HCatELFunctions#ph3_coord_dataInPartitions,
coord:dataOutPartitions=org.apache.oozie.coord.HCatELFunctions#ph3_coord_dataOutPartitions,
coord:dataOutPartitionValue=org.apache.oozie.coord.HCatELFunctions#ph3_coord_dataOutPartitionValue,
hadoop:conf=org.apache.oozie.action.hadoop.HadoopELFunctions#hadoop_conf
EL functions declarations, separated by commas, format is [PREFIX:]NAME=CLASS#METHOD.
几个具体实现
coord:databaseIn=org.apache.oozie.coord.HCatELFunctions#ph3_coord_databaseIn
public static String ph3_coord_databaseIn(String dataName) {
HCatURI hcatURI = getURIFromResolved(dataName, EventType.input);
if (hcatURI != null) {
return hcatURI.getDb();
}
else {
return "";
}
}
coord:databaseOut=org.apache.oozie.coord.HCatELFunctions#ph3_coord_databaseOut
public static String ph3_coord_databaseOut(String dataName) {
HCatURI hcatURI = getURIFromResolved(dataName, EventType.output);
if (hcatURI != null) {
return hcatURI.getDb();
}
else {
return "";
}
}
private static HCatURI getURIFromResolved(String dataInName, EventType type) {
final XLog LOG = XLog.getLog(HCatELFunctions.class);
StringBuilder uriTemplate = new StringBuilder();
ELEvaluator eval = ELEvaluator.getCurrent();
String uris;
if(type == EventType.input) {
uris = (String) eval.getVariable(".datain." + dataInName);
}
else { //type=output
uris = (String) eval.getVariable(".dataout." + dataInName);
}
if (uris != null) {
String[] uri = uris.split(CoordELFunctions.DIR_SEPARATOR, -1);
uriTemplate.append(uri[0]);
}
else {
LOG.warn("URI is NULL");
return null;
}
LOG.info("uriTemplate [{0}] ", uriTemplate);
HCatURI hcatURI;
try {
hcatURI = new HCatURI(uriTemplate.toString());
}
catch (URISyntaxException e) {
LOG.info("uriTemplate [{0}]. Reason [{1}]: ", uriTemplate, e);
throw new RuntimeException("HCat URI can't be parsed " + e);
}
return hcatURI;
}
这里并没有把所以的表达式都列全，但是大家应该可以了解了表达的计算模式，还有一点可能大家需要清除的，就是在实现方法中 有 ph1/ph2/ph3这样的标识，在ph1阶段可能仅仅是 把需要处理的数据放入上下文，在ph2或者ph3才会 真正计算出值，因为ph3、ph2的时候是action真正启动，实例的时候。
Oozie使用这些key 创建不同的 evaluator 从而到达 隔离的效果；
这里列举了不同的表达式，但是oozie在什么时候使用到他们的呢:
evaluator.evaluate
比如在 ActionStartXCommand 中：
String tmpActionConf = XmlUtils.removeComments(wfAction.getConf());
String actionConf = context.getELEvaluator().evaluate(tmpActionConf, String.class);
这样就可以将 actionConf 中所以的需要处理的表达式都处理了。