* Please note:
* Must be checked once every time you submit a configuration file is correct!
* Data is priceless! Accidentally deleted the consequences!
*/
public class WordCount {
// *************************************************************************
// PROGRAM
// *************************************************************************
public static void main(String[] args) throws Exception {
final ParameterTool params = ParameterTool.fromArgs(args);
// set up the execution environment
final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
// make parameters available in the web interface
env.getConfig().setGlobalJobParameters(params);
// get input data
DataSet text;
if (params.has("input")) {
// read the text file from given input path
text = env.readTextFile(params.get("input"));
} else {
// get default test text data
System.out.println("Executing WordCountWindow example with default input data set.");
System.out.println("Use --input to specify file input.");
text = WordCountData.getDefaultTextLineDataSet(env);
}
DataSet> counts =
// split up the lines in pairs (2-tuples) containing: (word,1)
text.flatMap(new Tokenizer())
// group by the tuple field "0" and sum up tuple field "1"
.groupBy(0)
.sum(1)
.setParallelism(2)
// 流水线的同时作业数量 https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/parallel.html
// 如果设置了 setParallelism 则可以理解： 这类任务（transformations/operators, data sources, and sinks）都有2个同时执行
;
// emit result
if (params.has("output")) {
counts.writeAsCsv(params.get("output"), "\n", " ");
// execute program
env.execute("WordCountWindow Example");
} else {
System.out.println("Printing result to stdout. Use --output to specify output path.");
counts.print();
}
}
// *************************************************************************
// USER FUNCTIONS
// *************************************************************************
/**
* Implements the string tokenizer that splits sentences into words as a user-defined
* FlatMapFunction. The function takes a line (String) and splits it into
* multiple pairs in the form of "(word,1)" ({@code Tuple2}).
*/
public static final class Tokenizer implements FlatMapFunction> {
@Override
public void flatMap(String value, Collector> out) {
// normalize and split the line
String[] tokens = value.toLowerCase().split("\\W+");
// emit the pairs
for (String token : tokens) {
if (token.length() > 0) {
out.collect(new Tuple2<>(token, 1));
}
try{
Thread.sleep(1000);
}catch (Exception s){
s.printStackTrace();
}
}
}
}
}
WordCountData.java
package cn.acewill.flink.utils;
import org.apache.flink.api.java.DataSet;
import org.apache.flink.api.java.ExecutionEnvironment;
/**
* @author Created by yangjf on 20180920.
* Update date:
* Time: 下午6:14
* Project: dev-flink-1.6
* Package: cn.acewell.flink.utils
* Describe :
* Provides the default data sets used for the WordCountWindow example program.
* The default data sets are used, if no parameters are given to the program.
* Frequency: Calculate once a day.
* Result of Test: test ok
* Command:
*
* Please note:
* Must be checked once every time you submit a configuration file is correct!
* Data is priceless! Accidentally deleted the consequences!
*/
public class WordCountData {
public static final String[] WORDS = new String[]{
"To be, or not to be,--that is the question:--",
"Whether 'tis nobler in the mind to suffer",
"The slings and arrows of outrageous fortune",
"Or to take arms against a sea of troubles,",
"And by opposing end them?--To die,--to sleep,--",
"No more; and by a sleep to say we end",
"The heartache, and the thousand natural shocks",
"That flesh is heir to,--'tis a consummation",
"Devoutly to be wish'd. To die,--to sleep;--",
"To sleep! perchance to dream:--ay, there's the rub;",
"For in that sleep of death what dreams may come,",
"When we have shuffled off this mortal coil,",
"When we have shuffled off this mortal coil,",
"When we have shuffled off this mortal coil,",
"When we have shuffled off this mortal coil,",
"Must give us pause: there's the respect",
"Must give us pause: there's the respect",
"Must give us pause: there's the respect",
"Must give us pause: there's the respect",
"That makes calamity of so long life;",
"For who would bear the whips and scorns of time,",
"The oppressor's wrong, the proud man's contumely,",
"The pangs of despis'd love, the law's delay,",
"The insolence of office, and the spurns",
"The insolence of office, and the spurns",
"The insolence of office, and the spurns",
"That patient merit of the unworthy takes,",
"When he himself might his quietus make",
"With a bare bodkin? who would these fardels bear,",
"To grunt and sweat under a weary life,",
"But that the dread of something after death,--",
"The undiscover'd country, from whose bourn",
"No traveller returns,--puzzles the will,",
"And makes us rather bear those ills we have",
"Than fly to others that we know not of?",
"Thus conscience does make cowards of us all;",
"And thus the native hue of resolution",
"Is sicklied o'er with the pale cast of thought;",
"And enterprises of great pith and moment,",
"With this regard, their currents turn awry,",
"And lose the name of action.--Soft you now!",
"The fair Ophelia!--Nymph, in thy orisons",
"Be all my sins remember'd."
};
public static DataSet getDefaultTextLineDataSet(ExecutionEnvironment env) {
return env.fromElements(WORDS);
}
}
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml&q
Mockito单元测试实例：
public class SettingServiceTest {
private List<PersonDTO> personList = new ArrayList<PersonDTO>();
@InjectMocks
private SettingPojoService settin
public class DeleteExtraSpace {
/**
* 题目：给定字符串，删除开始和结尾处的空格，并将中间的多个连续的空格合并成一个。
* 方法1.用已有的String类的trim和replaceAll方法
* 方法2.全部用正则表达式，这个我不熟
* 方法3.“重新发明轮子”，从头遍历一次
*/
public static v
今天早上打开MyEclipse时，自动关闭！弹出An error has occurred.See the log file错误提示！
很郁闷昨天启动和关闭还好着！！！打开几次依然报此错误，确定不是眼花了！
打开日志文件！找到当日错误文件内容：
--------------------------------------------------------------------------
![[Flink课程]---- 9.1 使用Ambari 搭建Flink 集群_第1张图片](http://img.e-com-net.com/image/info8/12bdc2c951c84d00a9a598a8ff52af38.jpg)
![[Flink课程]---- 9.1 使用Ambari 搭建Flink 集群_第2张图片](http://img.e-com-net.com/image/info8/2f312be6947341cdb27f61c0b1e5b5dd.jpg)
![[Flink课程]---- 9.1 使用Ambari 搭建Flink 集群_第3张图片](http://img.e-com-net.com/image/info8/1317256c4e72433f9f7b2c43b4f1ac83.jpg)
![[Flink课程]---- 9.1 使用Ambari 搭建Flink 集群_第4张图片](http://img.e-com-net.com/image/info8/819f291c3fa04dec9aed4ac78d95bfc1.jpg)
![[Flink课程]---- 9.1 使用Ambari 搭建Flink 集群_第5张图片](http://img.e-com-net.com/image/info8/7add504deab847b2a3d7f3e29aa6ac92.jpg)
![[Flink课程]---- 9.1 使用Ambari 搭建Flink 集群_第6张图片](http://img.e-com-net.com/image/info8/405cfd6335934a288fc9a4d30a06105e.jpg)
![[Flink课程]---- 9.1 使用Ambari 搭建Flink 集群_第7张图片](http://img.e-com-net.com/image/info8/da8442ec7a0b485eba2f3656d27be0b1.jpg)
![[Flink课程]---- 9.1 使用Ambari 搭建Flink 集群_第8张图片](http://img.e-com-net.com/image/info8/5e05b8b7da01463daed4e3db4d3edb98.jpg)
![[Flink课程]---- 9.1 使用Ambari 搭建Flink 集群_第9张图片](http://img.e-com-net.com/image/info8/fd71ce55900348fc896c7ed043b58480.jpg)
![[Flink课程]---- 9.1 使用Ambari 搭建Flink 集群_第10张图片](http://img.e-com-net.com/image/info8/c60a9b7d1d044454a91f91288a9f2c16.jpg)
![[Flink课程]---- 9.1 使用Ambari 搭建Flink 集群_第11张图片](http://img.e-com-net.com/image/info8/43dcfa7c14f94feb95b2a9b233e2ee71.jpg)
![[Flink课程]---- 9.1 使用Ambari 搭建Flink 集群_第12张图片](http://img.e-com-net.com/image/info8/59000f9477dc46bfa5afc47188b1e9e3.jpg)