1.首先将es中所有的操作封装成为一个EsSearchManager，并且使用单例模式，提供一个实例外部调用。
EsSearchManager esSearchManager = EsSearchManager.getInstance();
getInstance()的返回值是这个类的实例,构造函数中也是对client的创建
public static EsSearchManager getInstance(){
if(null == esSearchManager ){
synchronized (EsSearchManager.class){
esSearchManager = new EsSearchManager();
}
}
return esSearchManager;
}
private EsSearchManager(){
getClient();
}
private Client getClient() {
try{
if(client==null){
init();
}
}catch (Exception e){
LOG.error(e.getMessage());
}
return client;
}
private void init() throws Exception {
client = EsClient.getClient();
}
2.得到了client就可以创建索引，存入数据等一系列的索引数据操作了
根据索引和类型创建索引。
/**
* 根据索引和类型创建索引
* @param indexName
* @param type
* @return
* @throws Exception
*/
public Boolean buildIndex(String indexName,String type) throws Exception {
IndicesExistsResponse response = getClient().admin().indices()
.prepareExists(indexName,type).execute().actionGet();
Boolean flag = true;
ResourceBundle rb = ResourceBundle.getBundle("commons");
String replicas = rb.getString("replicas");
String shards = rb.getString("shards");
String refreshInterval = rb.getString("refreshInterval");
if (!response.isExists()) { //需要将配置放置到配置文件中
Settings settings = Settings.settingsBuilder()
.put("number_of_replicas", Integer.parseInt(replicas))
.put("number_of_shards", Integer.parseInt(shards))
.put("index.translog.flush_threshold_ops", 10000000)
.put("refresh_interval", refreshInterval)
.put("index.codec", "best_compression").build();
CreateIndexResponse createIndxeResponse = getClient().admin().indices()
.prepareCreate(indexName).setSettings(settings).addMapping(type).execute()
.actionGet();
flag = createIndxeResponse.isAcknowledged();
LOG.info("返回值" + flag);
}
return flag;
}
/**
* 创建单条索引
* @param indexName
* @param type
* @param json
* @throws Exception
*/
public void buildDocument(String indexName, String type, String json) throws Exception {
getClient().prepareIndex(indexName, type).setSource(json).execute()
.actionGet();
}
/**
* 构造list集合索引数据
* @param indexName
* @param type
* @param list
*/
public void buildList2Documents(String indexName, String type, List> list) throws Exception{
BulkRequestBuilder bulkRequest = getClient().prepareBulk();
for(Map map : list){
bulkRequest.add(getClient().prepareIndex(indexName, type)
.setSource(this.generateJson(map)));
}
BulkResponse bulkIndexResponse = bulkRequest.execute().actionGet();
if (bulkIndexResponse.hasFailures()) {
LOG.error(bulkIndexResponse.buildFailureMessage());
}
}
如何判断索引是否存在，删除索引，下面就是判断和删除索引的方法
/**
* 根据索引名称判断索引是否存在
* @param indexName
* @return
* @throws NumberFormatException
* @throws UnknownHostException
*/
public Boolean existsIndex(String indexName) throws Exception {
IndicesExistsResponse response = getClient().admin().indices()
.prepareExists(indexName).execute().actionGet();
return response.isExists();
}
/**
* 根据索引名称删除索引
* @param indexName
* @return
* @throws NumberFormatException
* @throws UnknownHostException
*/
public Boolean deleteIndex(String indexName) throws Exception {
boolean flag = true;
IndicesExistsResponse response = getClient().admin().indices()
.prepareExists(indexName).execute().actionGet();
if (response.isExists()) {
DeleteIndexResponse response2 = getClient().admin().indices()
.prepareDelete(indexName).execute().actionGet();
flag = response2.isAcknowledged();
}
return flag;
}
根据docId删除某一条索引记录
**
* 根据索引Id删除文档
* @param indexName
* @param type
* @param docId
* @throws NumberFormatException
* @throws UnknownHostException
*/
public void deleteDocument(String indexName, String type, String docId)
throws Exception {
getClient().prepareDelete(indexName, type, docId).execute().actionGet();
}
3.上面给出了一些索引的基本操作，下面来讲讲es中如何根据索引去查询数据呢
```
/** 下面这个方法就是构建全文检索，根据关键词全文检索
* 全文检索查询,多条件类型查询
*/
public PageEntity queryFulltext(List keywords,
List indexs, List types, List fieldNames,
List allColumns, int pagenum, int pagesize)
throws Exception {
BoolQueryBuilder qb = buildFullText(keywords, types, fieldNames);
if (qb == null) {
LOG.info("queryFull Text == null");
return null;
}
LOG.info("Fulltext begin");
long begin = System.currentTimeMillis();
PageEntity result = execute(qb, fieldNames,allColumns, indexs, types,
pagenum, pagesize);
long end = System.currentTimeMillis();
LOG.info("query Fulltext end cost:[{}]ms", end - begin);
return result;
}
/**
* 构造多字段 全文搜索查询条件
* [@param]keywords
* [@param types
* [@param] fieldNames
* [@return
* @throws Exception
*/
private BoolQueryBuilder buildFullText(List keywords,
List types, List fieldNames)
throws Exception {
BoolQueryBuilder qb = null;
if (keywords != null && keywords.size() > 0 && fieldNames != null
&& fieldNames.size() > 0 && types != null && types.size() > 0) {
qb = QueryBuilders.boolQuery();
for (String keyword : keywords) {
QueryBuilder mustCondition = QueryBuilders.multiMatchQuery(
keyword, fieldNames.toArray(new String[0]));
qb.must(mustCondition);
}
}
return qb;
}
上面的方法实现的全文检索，只要某个字段或者某几个字段实现了分词，就可以实现类似数据库中的模糊匹配查询，下面介绍下term查询，只是针对某些特殊字段完全匹配才能够查询到，这些字段往往都是不需要分词的。
/**
* term 查询（在查询的时候不分词，主要针对 人名 地名等特殊的词语）
* [[@param](https://my.oschina.net/u/2303379)] keywords
* [@param) types
* [[@param](https://my.oschina.net/u/2303379)] fieldnames
* [[@param](https://my.oschina.net/u/2303379)] pagenum
* [[@param](https://my.oschina.net/u/2303379)] pagesize
* @throws Exception
*/
public PageEntity queryWithTerm(List keywords,
List indexs, List types, List fieldnames,
List dateFieldnames, List allColumns,
Long startTime, Long endTime, int pagenum, int pagesize)
throws Exception {
BoolQueryBuilder qb = buildTermQuery(keywords, types, fieldnames,
dateFieldnames, startTime, endTime);
if (qb == null) {
LOG.info("queryTerm() QueryBuilder == null");
return new PageEntity();
}
LOG.info("query begin");
long begin = System.currentTimeMillis();
PageEntity result = execute(qb, fieldnames, allColumns,
indexs, types, pagenum, pagesize);
long end = System.currentTimeMillis();
LOG.info("query end cost:[{}]ms", end - begin);
return result;
}
/**
* es query 实现es分页查询
* @param qb
* @param fieldnames
* @param allColumns
* @param indexs
* @param types
* @param pagenum
* @param pagesize
* @return
* @throws Exception
*/
private PageEntity execute(QueryBuilder qb,
List fieldnames, List allColumns,
List indexs, List types, int pagenum, int pagesize)
throws Exception {
PageEntity page = new PageEntity();
String[] typeArry = types.toArray(new String[0]);
String[] indexArry = indexs.toArray(new String[0]);
int startnum = (pagenum - 1) * pagesize;
SearchResponse response = null;
try {
response = getClient().prepareSearch(indexArry).setTypes(typeArry)
.setQuery(qb).setFrom(startnum).setSize(pagesize)
.execute().actionGet();
} catch (Exception e) {
LOG.error("query error", e);
throw e;
}
if (response == null) {
return page;
}
SearchHits hits = response.getHits();
LOG.info("execute hit:" + hits.totalHits());
List resultString = new ArrayList();
if (null != hits && hits.totalHits() > 0) {
for (SearchHit hit : hits) {
JSONObject obj = new JSONObject();
if (allColumns != null && allColumns.size() > 0) {
fieldnames = allColumns;
}
for (String str : fieldnames) {
obj.put(str, hit.getSource().get(str));
}
resultString.add(obj);
}
}
page.setContents(resultString);
page.setPageSize(pagesize);
page.setCurrentPageNo(pagenum);
page.setTotalCount(hits.totalHits());
return page;
}
-
4.下面的例子代码演示如何调用上述代码操作es的
-
创建testIndex中的testType类型的索引
-
EsSearchManager esSearchManager = EsSearchManager.getInstance(); -
esSearchManager.buildIndex("testIndex","testType"); -
//索引数据 -
List -
Map -
map.put("fieldA",100); -
map.put("fieldB",22); -
map.put("fieldC","hoge"); -
map.put("fieldD","huga"); -
list.add(map); -
esSearchManager.buildList2Documents("testindex","testtypes",list); -
//传入某个字段进行不分词精确查找 -
Listkeywords = new ArrayList<>(); -
keywords.add("huga"); -
Listtypes = new ArrayList<>(); -
types.add("testtypes"); -
Listindexs = new ArrayList<>(); -
indexs.add("testindex"); -
ListfieldNames = new ArrayList<>(); -
fieldNames.add("fieldD"); -
PageEntitypg = esSearchManager.queryWithTerm(keywords,indexs, -
types,fieldNames,null,null,null,null,1,10); -
//传入某个字段进行全文检索 -
Listkeywords = new ArrayList<>(); -
keywords.add("huga"); -
Listtypes = new ArrayList<>(); -
types.add("testtypes"); -
Listindexs = new ArrayList<>(); -
indexs.add("testindex"); -
ListfieldNames = new ArrayList<>(); -
fieldNames.add("fieldD"); -
PageEntitypg1 = esSearchManager.queryFulltext(keywords, -
indexs, types, fieldNames, -
null, 1,10);
以上总结了部分es基本操作API和调用demo，详细的代码请查看github地址 https://github.com/winstonelei/BigDataTools ，包括了一些大数据组件的基本操作，包含了hbase，hadoop,es，hive等