完整更新日志:release log
注意:更新日志里面只有 jira 编号和标题,详细信息需要到 jira 网站上依次查找。
A1--->A2--->A3 A7--->A8--->A9
| | | |
S--> F1 J1-->F2 J2--->E
| | | |
A4--->A5--->A6 A10-->A11-->A12
例如,第一个递归选择路径 F1-> A1-> A2-> A3-> J1,选择 F2-> A7-> A8-> A9-> J2;然后返回检查 F2-> A10-> A11-> A12-> J2。
然后,返回到F1,选择第二条路径:F1-> A4-> A5-> A6。但是,它再次重新走了F2-> J2的两条路径。
例如,现在有10个ForkJoin对,每对有4个分支,则总共有4^10条路径。
/**
*
if (conf == null) {
throw new IllegalArgumentException("conf cannot be null");
}
if (scriptFile == null) {
throw new IllegalArgumentException("scriptFile cannot be null");
}
*/
OozieClient.notNull(conf, "conf");
OozieClient.notNull(scriptFile, "scriptFile");
public interface CoordInputDependency {
// public static final String INTERNAL_VERSION_ID = "V=1";
String INTERNAL_VERSION_ID = "V=1";
// public void addInputInstanceList(String inputEventName, List inputInstanceList);
void addInputInstanceList(String inputEventName, List<CoordInputInstance> inputInstanceList);
curl "http://host:11000/oozie/v2/jobs?filter=startCreatedTime%3D2016-04-28T00%3A00Z&jobtype=bundle"
curl "http://host:11000/oozie/v2/jobs?filter=startCreatedTime%3D2016-04-28T00%3A00Z&jobtype=coordinator"
curl "http://host:11000/oozie/v2/jobs?filter=startCreatedTime%3D2016-04-28T00%3A00Z&jobtype=wf"
jdbc:mysql:replication://master_host:port,slave_host:port/database
全局配置参数(oozie.site.xml):oozie.service.LiteWorkflowStoreService.user.retry.policy
工作流配置:
<property>
<name>oozie.jmx_monitoring.enablename>
<value>falsevalue>
<description>
If the oozie functional metrics needs to be exposed via JMX interface,
set it to true.
description>
property>
// executor = new ThreadPoolExecutor(threads, threads, 10, TimeUnit.SECONDS, (BlockingQueue) queue);
executor = new ThreadPoolExecutor(threads, threads, 10, TimeUnit.SECONDS, (BlockingQueue) queue, new NamedThreadFactory("CallableQueue"));
public class NamedThreadFactory implements ThreadFactory {
private final AtomicInteger counter = new AtomicInteger();
private final String threadPrefix;
public NamedThreadFactory(String threadPrefix) {
this.threadPrefix = ParamChecker.notEmpty(threadPrefix, "threadPrefix");
}
@Override
public Thread newThread(Runnable r) {
Thread t = new Thread(r);
t.setName(threadPrefix + "-" + counter.getAndIncrement());
return t;
}
}
示例:timeStamp 为 2009-01-01T00:00Z
millis 为 false, 返回 ‘1230768000’
millis 为 ‘true’,返回 ‘1230768000000’
<property>
<name>oozie.service.JPAService.openjpa.BrokerImplname>
<value>non-finalizingvalue>
<description>
The default OpenJPAEntityManager implementation automatically closes
itself during instance finalization.
This guards against accidental resource leaks that may occur if a
developer fails to explicitly close EntityManagers when finished with
them, but it also incurs a scalability bottleneck, since the JVM must
perform synchronization during instance creation, and since the finalizer
thread will have more instances to monitor.
To avoid this overhead, set the openjpa.BrokerImpl configuration property
to non-finalizing.
To use default implementation set it to empty space.
description>
property>
<property>
<name>oozie.zookeeper.connection.timeoutname>
<value>180value>
<description>
Default ZK connection timeout (in sec).
description>
property>
<property>
<name>oozie.zookeeper.session.timeoutname>
<value>300value>
<description>
Default ZK session timeout (in sec). If connection is lost even
after retry, then Oozie server will shutdown
itself if oozie.zookeeper.server.shutdown.ontimeout is true.
description>
property>
<property>
<name>oozie.zookeeper.max.retriesname>
<value>10value>
<description>
Maximum number of times to retry.
description>
property>
<input-logic>
<or>
<and name="AorB">
<data-in dataset="A"/>
<data-in dataset="B"/>
and>
<and name="CorD">
<data-in dataset="C"/>
<data-in dataset="D"/>
and>
or>
input-logic>
<input-logic>
<combine name="AorB">
<data-in dataset="A"/>
<data-in dataset="B"/>
combine>
input-logic>
<input-logic>
<or name="AorB" min="5" wait="10">
<data-in dataset="A"/>
<data-in dataset="B"/>
or>
input-logic>
22257 [uber-SubtaskRunner] ERROR org.apache.hadoop.hive.ql.Driver - FAILED: IllegalArgumentException java.net.URISyntaxException: Relative path in absolute URI: file:./tmp/mapr/b79b1ef5-c8a4-4da9-a307-b4f9045c7bb9/hive_2015-10-01_11-30$
java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: file:./tmp/mapr/b79b1ef5-c8a4-4da9-a307-b4f9045c7bb9/hive_2015-10-01_11-30-46_978_7012892802177357645-1
at org.apache.hadoop.fs.Path.initialize(Path.java:217)
at org.apache.hadoop.fs.Path.<init>(Path.java:208)
at org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:284)
at org.apache.hadoop.hive.ql.Context.getLocalScratchDir(Context.java:319)
at org.apache.hadoop.hive.ql.Context.getLocalTmpPath(Context.java:414)
...
sharelib create -fs FS_URI [-locallib SHARED_LIBRARY] [-concurrency CONCURRENCY]
private void concurrentCopyFromLocal(final FileSystem fs, int threadPoolSize,
File srcFile, final Path dstPath) throws IOException {
List<Future<Void>> futures = Collections.emptyList();
ExecutorService threadPool = Executors.newFixedThreadPool(threadPoolSize);
try {
futures = copyFolderRecursively(fs, threadPool, srcFile, dstPath);
System.out.println("Running " + futures.size() + " copy tasks on " + threadPoolSize + " threads");
} finally {
try {
threadPool.shutdown();
} finally {
checkCopyResults(futures);
}
}
}
<property>
<name>oozie.external_monitoring.enablename>
<value>falsevalue>
<description>
If the oozie functional metrics needs to be exposed to the metrics-server
backend, set it to true
If set to true, the following properties has to be specified :
oozie.metrics.server.name, oozie.metrics.host, oozie.metrics.prefix,
oozie.metrics.report.interval.sec, oozie.metrics.port
description>
property>
<property>
<name>oozie.external_monitoring.typename>
<value>graphitevalue>
<description>
The name of the server to which we want to send the metrics,
would be graphite or ganglia.
description>
property>
<property>
<name>oozie.external_monitoring.addressname>
<value>http://localhost:2020value>
property>
<property>
<name>oozie.external_monitoring.metricPrefixname>
<value>oozievalue>
property>
<property>
<name>oozie.external_monitoring.reporterIntervalSecsname>
<value>60value>
property>
oozie.credentials.skip
false
This determines if Oozie should skip getting credentials from the
credential providers.
This can be overwritten at a job-level or action-level.