Elastic Search框架（上篇，概述+入门语法+API+集群部署）
如何选择POST还是PUT，看是否有幂等性（提交多次结果是否一致，若一致就可以用PUT，每次如果都不一样就用POST）
一、ES概述：技术选型
ElasticSearch：对数据进行大量分析、分布式索引、监控&指标
二、ES入门
1. 安装与使用
点我下载ES 7.8
安装好之后点开 bin/elasticsearch.bat
之后访问 http://localhost:9200/
可以看到json格式的数据就是正常打开
ES数据的发送和返回都是json格式的
2. 倒排索引
为什么叫倒排索引
在没有搜索引擎时，我们是直接输入一个网址，然后获取网站内容，这时我们的行为是：
document -> to -> words
通过文章，获取里面的单词，此谓「正向索引」，forward index.
后来，我们希望能够输入一个单词，找到含有这个单词，或者和这个单词有关系的文章：
word -> to -> documents
于是我们把这种索引，成为inverted index，直译过来，应该叫「反向索引」，国内翻译成「倒排索引」，有点委婉了。
原文：https://blog.csdn.net/u013008898/article/details/116493167
3. HTTP，索引，创建
shopping是索引名称
-
创建索引（等于创建数据库，发送PUT请求）
http://127.0.0.1:9200/shopping -
得到索引（发送GET请求）
http://127.0.0.1:9200/shopping
得到全部：http://127.0.0.1:9200/_cat/indices?v -
删除索引（发送DELETE请求）
http://127.0.0.1:9200/shopping
4. 文档数据的创建
- 往索引里新增数据（不自定义ID：POST，传JSON）
http://127.0.0.1:9200/shopping/_doc/ - 往索引里新增数据（自定义ID：POST、PUT，传JSON）
http://127.0.0.1:9200/shopping/_doc/123
http://127.0.0.1:9200/shopping/_create/123
5. 主键查询 & 全查询
- 查询主键单数据（GET）
http://127.0.0.1:9200/shopping/_doc/123 - 查询全部数据（GET）
http://127.0.0.1:9200/shopping/_search
6. 全量修改 & 局部修改
- 全量修改（PUT、POST）：
http://127.0.0.1:9200/shopping/_doc/123
{
"name":"haige",
"age",123
}
- 局部修改（POST）
http://127.0.0.1:9200/shopping/_update/123
{
"doc" :{
"name":"haige",
}
}
7. 条件查询 & 分页查询 & 查询排序
（查询都是GET）
- 条件查询
http://127.0.0.1:9200/shopping/_search
{
"query":{
"match":{
"name":"哈喽"
}
}
}
- 分页查询
http://127.0.0.1:9200/shopping/_search
{
"query":{
"match":{
"name":"哈喽"
}
},
"from":0,
"size":2
}
表示从第0项开始显示，显示2项
- 查询排序（并且只显示name字段）
{
"query":{
"match":{
"name":"哈喽"
}
},
"from":0,
"size":2,
"_source" : ["name"],
"sort" : {
"age" : {
"order" : "desc"
}
}
}
8. 多条件查询，范围查询
多条件查询
{
"query":{
"bool" :{
"must" :[
{
"match" :{
"name":"测试"
}
},
{
"match" :{
"age":20
}
}
]
}
}
}
多条件查询
{
"query":{
"bool" :{
"must" :[
{
"match" :{
"name":"测试"
}
},
{
"match" :{
"age":20
}
}
]
}
}
}
{
"query":{
"bool" :{
"should" :[
{
"match" :{
"name":"测试"
}
}
],
"filter" :{
"range":{
"age":{
"gt" : 20
}
}
}
}
}
}
全文检索
match默认全文检索
完全匹配
match_phrase
高亮显示
highlight
{
"query":{
"match_phrase":{
"name":"测试3"
}
},
"highlight":{
"fields":{
"name":{}
}
}
}
9. 聚合查询
- 分组查询
（GET）：http://127.0.0.1:9200/shopping/_search
{
"aggs":{ // 聚合操作
"price_group":{ // 分组，随意起名
"terms":{ // 分组
"field":"age" // 分组字段
}
}
},
"size":0 // 不显示原始数据
}
- 平均值
（GET）：http://127.0.0.1:9200/shopping/_search
{
"aggs":{ // 聚合操作
"price_avg":{ // 平均值名字，随意起名
"avg":{ // 平均值
"field":"age" // 分组字段
}
}
},
"size":0 // 不显示原始数据
}
10. 映射关系
{
"properties":{
"name":{
"type":"text",
"index":true
},
"sex":{
"type":"keyword",
"index":true
},
"tel":{
"type":"keyword",
"index":false
}
}
}
三、JavaAPI
0. 导包
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0modelVersion>
<groupId>org.examplegroupId>
<artifactId>esartifactId>
<version>1.0-SNAPSHOTversion>
<properties>
<maven.compiler.source>8maven.compiler.source>
<maven.compiler.target>8maven.compiler.target>
<project.build.sourceEncoding>UTF-8project.build.sourceEncoding>
properties>
<dependencies>
<dependency>
<groupId>org.elasticsearchgroupId>
<artifactId>elasticsearchartifactId>
<version>7.8.0version>
dependency>
<dependency>
<groupId>org.elasticsearch.clientgroupId>
<artifactId>elasticsearch-rest-high-level-clientartifactId>
<version>7.8.0version>
dependency>
<dependency>
<groupId>org.apache.logging.log4jgroupId>
<artifactId>log4j-apiartifactId>
<version>2.8.2version>
dependency>
<dependency>
<groupId>org.apache.logging.log4jgroupId>
<artifactId>log4j-coreartifactId>
<version>2.8.2version>
dependency>
<dependency>
<groupId>com.fasterxml.jackson.coregroupId>
<artifactId>jackson-databindartifactId>
<version>2.9.9version>
dependency>
<dependency>
<groupId>junitgroupId>
<artifactId>junitartifactId>
<version>4.12version>
dependency>
dependencies>
project>
1. 创建索引、查询、删除
- 创建索引
package com.itguigu.es.test;
import org.apache.http.HttpHost;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.indices.CreateIndexRequest;
import org.elasticsearch.client.indices.CreateIndexResponse;
import java.io.IOException;
public class ESTest_Create_index {
public static void main(String[] args) throws IOException {
// 创建客户端
RestHighLevelClient esClient = new RestHighLevelClient(
RestClient.builder(new HttpHost("localhost",9200,"http")));
// 创建索引
CreateIndexRequest request = new CreateIndexRequest("user1");
CreateIndexResponse createIndexResponse = esClient.indices().create(request, RequestOptions.DEFAULT);
// 相应状态
boolean acknowledged = createIndexResponse.isAcknowledged();
System.out.println("索引创建结果："+acknowledged);
// 关闭客户端
esClient.close();
}
}
- 查询索引
// 查询索引
GetIndexRequest request = new GetIndexRequest("user");
GetIndexResponse getIndexResponse = esClient.indices().get(request, RequestOptions.DEFAULT);
// 相应状态
System.out.println(getIndexResponse.getAliases());
System.out.println(getIndexResponse.getDataStreams());
System.out.println(getIndexResponse.getSettings());
- 删除索引
//删除索引
DeleteIndexRequest request = new DeleteIndexRequest("user");
AcknowledgedResponse delete = esClient.indices().delete(request, RequestOptions.DEFAULT);
//返回状态信息
boolean acknowledged = delete.isAcknowledged();
System.out.println("删除状态："+acknowledged);
- 往索引里新增数据
// 创建客户端
RestHighLevelClient esClient = new RestHighLevelClient(
RestClient.builder(new HttpHost("localhost",9200,"http")));
// 创建实体类
User user = new User();
user.setName("小明");
user.setSex("男");
user.setAge(22);
// 创建index连接
IndexRequest request = new IndexRequest("user1").id("123");
// 将实体类转换为json格式字符串
ObjectMapper mapper = new ObjectMapper();
String userJSON = mapper.writeValueAsString(user);
// 绑定到request连接中
request.source(userJSON, XContentType.JSON);
// 查询并打印结果
IndexResponse index = esClient.index(request, RequestOptions.DEFAULT);
System.out.println(index.getResult());
// 关闭客户端
esClient.close();
- 修改数据
// 修改数据
UpdateRequest request = new UpdateRequest();
request.index("user1").id("123");
request.doc(XContentType.JSON,"name","小黄");
// 修改数据
UpdateResponse update = esClient.update(request, RequestOptions.DEFAULT);
System.out.println(update.getResult());
- 查询数据
// 查询数据
GetRequest request = new GetRequest();
request.index("user1").id("123");
GetResponse response = esClient.get(request, RequestOptions.DEFAULT);
//打印结果
System.out.println(response.getSourceAsString());
- 删除数据
DeleteRequest request = new DeleteRequest();
request.index("user1").id("123");
DeleteResponse response = esClient.delete(request, RequestOptions.DEFAULT);
System.out.println(response.getResult());
- 批量添加 & 批量删除
批量添加：
// 用Bulk批量添加
BulkRequest request = new BulkRequest();
request.add(new IndexRequest().index("user1").id("1001").source(XContentType.JSON,"name","zhangsan"));
request.add(new IndexRequest().index("user1").id("1002").source(XContentType.JSON,"name","lisi"));
request.add(new IndexRequest().index("user1").id("1003").source(XContentType.JSON,"name","wangwu"));
BulkResponse response = esClient.bulk(request, RequestOptions.DEFAULT);
System.out.println(response.getIngestTookInMillis());
批量删除：
BulkRequest request = new BulkRequest();
request.add(new DeleteRequest().index("user1").id("1001"));
request.add(new DeleteRequest().index("user1").id("1002"));
request.add(new DeleteRequest().index("user1").id("1003"));
BulkResponse response = esClient.bulk(request, RequestOptions.DEFAULT);
System.out.println(response.getIngestTook());
- 批量查询所有数据：
public static void main(String[] args) throws IOException {
// 创建客户端
RestHighLevelClient esClient = new RestHighLevelClient(
RestClient.builder(new HttpHost("localhost",9200,"http")));
SearchRequest request = new SearchRequest();
request.indices("user1");
request.source(new SearchSourceBuilder().query(QueryBuilders.matchAllQuery()));
SearchResponse response = esClient.search(request, RequestOptions.DEFAULT);
for (SearchHit hit : response.getHits()) {
System.out.println(hit.getSourceAsString());
}
// 关闭客户端
esClient.close();
}
- 批量查询特定条件数据
SearchRequest request = new SearchRequest();
request.indices("user1");
request.source(new SearchSourceBuilder().query(QueryBuilders.termQuery("name","zhangsan1")));
SearchResponse response = esClient.search(request, RequestOptions.DEFAULT);
for (SearchHit hit : response.getHits()) {
System.out.println(hit.getSourceAsString());
}
- 分页查询
SearchRequest request = new SearchRequest();
request.indices("user1");
SearchSourceBuilder query = new SearchSourceBuilder().query(QueryBuilders.matchAllQuery());
query.from(0);
query.size(2);
request.source(query);
SearchResponse response = esClient.search(request, RequestOptions.DEFAULT);
for (SearchHit hit : response.getHits()) {
System.out.println(hit.getSourceAsString());
}
- 对结果排序
SearchRequest request = new SearchRequest();
request.indices("user1");
SearchSourceBuilder query = new SearchSourceBuilder().query(QueryBuilders.matchAllQuery());
query.sort("age", SortOrder.DESC);
request.source(query);
SearchResponse response = esClient.search(request, RequestOptions.DEFAULT);
for (SearchHit hit : response.getHits()) {
System.out.println(hit.getSourceAsString());
}
- 过滤字段
SearchRequest request = new SearchRequest();
request.indices("user1");
SearchSourceBuilder query = new SearchSourceBuilder().query(QueryBuilders.matchAllQuery());
String[] includes = {"name"};
String[] excludes = {};
query.fetchSource(includes,excludes);
request.source(query);
SearchResponse response = esClient.search(request, RequestOptions.DEFAULT);
for (SearchHit hit : response.getHits()) {
System.out.println(hit.getSourceAsString());
}
- 范围查询
//创建请求
SearchRequest request = new SearchRequest();
request.indices("user1");
//搜索源
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
//范围查询类
RangeQueryBuilder rangeQueryBuilder = new RangeQueryBuilder("age");
rangeQueryBuilder.gte(24);
rangeQueryBuilder.lte(50);
sourceBuilder.query(rangeQueryBuilder);
request.source(sourceBuilder);
//返回结果
SearchResponse search = esClient.search(request, RequestOptions.DEFAULT);
for (SearchHit hit : search.getHits()) {
System.out.println(hit.getSourceAsString());
}
- 模糊查询
public static void main(String[] args) throws IOException {
// 创建客户端
RestHighLevelClient esClient = new RestHighLevelClient(
RestClient.builder(new HttpHost("localhost",9200,"http")));
// 创建请求
SearchRequest request = new SearchRequest("user1");
//搜索类型源构建者
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
//添加模糊fuzzyQuery选项
searchSourceBuilder.query(QueryBuilders.fuzzyQuery("name","zhan").fuzziness(Fuzziness.ONE));
request.source(searchSourceBuilder);
SearchResponse search = esClient.search(request, RequestOptions.DEFAULT);
for (SearchHit hit : search.getHits()) {
System.out.println(hit.getSourceAsString());
}
// 关闭客户端
esClient.close();
}
- 高亮查询
public static void main(String[] args) throws IOException {
// 创建客户端
RestHighLevelClient esClient = new RestHighLevelClient(
RestClient.builder(new HttpHost("localhost",9200,"http")));
SearchRequest request = new SearchRequest("user1");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("name", "zhangsan1");
//设置高亮
HighlightBuilder highlightBuilder = new HighlightBuilder();
highlightBuilder.preTags("");
highlightBuilder.postTags("");
searchSourceBuilder.highlighter(highlightBuilder);
searchSourceBuilder.query(termQueryBuilder);
request.source(searchSourceBuilder);
SearchResponse search = esClient.search(request, RequestOptions.DEFAULT);
for (SearchHit hit : search.getHits()) {
System.out.println(hit.getSourceAsString());
}
// 关闭客户端
esClient.close();
}
四、Elasticsearch 环境
1. 集群环境部署
windows下：将es客户复制三份，配置文件写
第一个客户端：
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-1001
node.master: true
node.data: true
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: localhost
#
# Set a custom port for HTTP:
#
http.port: 1001
transport.tcp.port: 9301
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
http.cors.enabled: true
http.cors.allow-origin: "*"
第二个客户端配置文件：
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-1002
node.master: true
node.data: true
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: localhost
#
# Set a custom port for HTTP:
#
http.port: 1002
transport.tcp.port: 9302
discovery.seed_hosts: ["localhost:9301"]
discovery.zen.fd.ping_timeout: 1m
discovery.zen.fd.ping_retries: 5
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
http.cors.enabled: true
http.cors.allow-origin: "*"
第三个客户端配置文件：
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-1003
node.master: true
node.data: true
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: localhost
#
# Set a custom port for HTTP:
#
http.port: 1003
transport.tcp.port: 9303
discovery.seed_hosts: ["localhost:9301","localhost:9302"]
discovery.zen.fd.ping_timeout: 1m
discovery.zen.fd.ping_retries: 5
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
http.cors.enabled: true
http.cors.allow-origin: "*"