elasticsearch入门及高级应用。
Elasticsearch
- 原生安装
1.1下载
Spring boot和es的对应关系，spring boot 1.5.4.RELEASE，官方默认设置版本为2.4.5。
下载链接：
https://www.elastic.co/cn/downloads/past-releases/elasticsearch-2-4-5
官方文档：
https://www.elastic.co/guide/en/elasticsearch/reference/2.4/index.html
Java api:
https://www.elastic.co/guide/en/elasticsearch/client/java-api/2.4/java-api.html
1.2安装
1.创建用户和设置密码（默认不允许使用root运行）
useradd elk
passwd elk
- 创建数据和日志文件存放路径(elk用户)
mkdir -p /home/elk/es/data
mkdir -p /home/elk/es/logs
- 解压文件
tar -zxvf elasticsearch-2.4.5.tar.gz
- 配置es
vi /home/elk/es/elasticsearch-2.4.5/config/elasticsearch.yml
# 集群名称
cluster.name: elk-application
# 集群主节点
cluster.initial_master_nodes: ["node-1"]
# 当前节点名称
node.name: node-1
# 主节点
node.master: true
node.date: true
# 存储数据
path.data: /home/elk/es/data
# 存储日志
path.logs: /home/elk/es/logs
# 绑定本机所有IP
network.host: 0.0.0.0
# 开启http功能
http.enabled: true
http.port: 9200
# 允许跨域访问
http.cors.enabled: true
http.cors.allow-origin: "*"
# 开始tcp数据传输端口
transport.tcp.port: 9300
transport.tcp.compress: true
1.3 运行
su elk (不要使用root用户启动)
/home/elk/es/elasticsearch-2.4.5/bin/elasticsearch >>/home/elk/logs/elasticsearch.log &
运行测试: curl http://ip:9200,若输出信息如下，表示运行成功。
2.docker安装运行
2.1拉取镜像
docker pull elasticsearch:2.4.5
查看镜像信息 docker inspect elasticsearch:2.4.5
2.2配置
添加elk用户（已配置请跳过）
useradd elk
passwd elk
创建日志、数据、配置文件目录
mkdir -p /home/elk/docker/es/data
mkdir -p /home/elk/docker/es/logs
mkdir -p /home/elk/docker/es/config
配置elasticsearch
vi /home/elk/docker/es/config/elasticsearch.yml
cluster.name: elk-application-docker
cluster.initial_master_nodes: ["node-1"]
node.name: node-docker-1
node.master: true
node.date: true
#path.data: /home/elk/es/data
#path.logs: /home/elk/es/logs
network.host: 0.0.0.0
http.enabled: true
http.port: 9200
http.cors.enabled: true
http.cors.allow-origin: "*"
transport.tcp.port: 9300
transport.tcp.compress: true
docker目录挂载：
/home/elk/docker/es/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
/home/elk/docker/es/data/:/usr/share/elasticsearch/data/
/home/elk/docker/es/logs/:/usr/share/elasticsearch/logs/
2.3运行
docker run --name elasticsearch245 \
-v /home/elk/docker/es/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml \
-v /home/elk/docker/es/data/:/usr/share/elasticsearch/data/ \
-v /home/elk/docker/es/logs/:/usr/share/elasticsearch/logs/ \
-p 9200:9200 \
-p 9300:9300 \
-e "ES_JAVA_OPTS=-Xms256m -Xmx512m" \
-d elasticsearch:2.4.5
注意:es5.6默认配置内存为2G,若内存不够将无法启动,可以通过-e参数指定。
运行测试: curl http://ip:9200,若输出信息如下，表示运行成功。
- 高级应用
3.1删除指定日期索引
到es的data目录下查看所有索引库
删除2019.12.23当天所有索引数据。
curl -XDELETE 'http://192.168.0.105:9200/*-2019.12.23*'
删除2019.12当月所有索引数据。
curl -XDELETE 'http://192.168.0.105:9200/*-2019.12*'
3.2 http模糊查询
curl 'localhost:9200/logstash-*/_search?pretty&analyze_wildcard&q=message:*test*'
3.3添加spring-data-elasticsearch
本次搭建的es版本为2.4.5，spring-data-elasticsearch则对应为2.0.4.RELEASE;pom文件中加入如下信息:
|
<dependency> <groupId>org.springframework.bootgroupId> <artifactId>spring-data-elasticsearchartifactId> <version>2.0.4.RELEASEversion> dependency> |
3.4 java添加索引
| Settings settings = Settings.builder().put("cluster.name", clusterName).build(); TransportClient client = TransportClient.builder().settings(settings).build() .addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("192.168.0.105"), 9300));
String indexName = "logstash-2019.12.24"; // 索引库名称 String indexId = "kenick20191224010"; // 索引库中索引的id String indexType = "logs"; // 索引类型 Map sourceJson.put("@timestamp", "2019-12-24T06:01:47.010Z"); sourceJson.put("message", "test010");
IndexResponse indexResponse = client.prepareIndex(indexName, indexType, indexId).setSource(sourceJson, XContentType.JSON).get(); System.out.println("索引创建结果:" + indexResponse.isCreated()); |
注意:添加的索引ID不能重复，否则添加失败。
3.5 java搜索
| TransportClient esClient = ESUtil.getClient(); // 获取es客户端连接
// 添加索引 String indexName = "logstash-2019.12.24"; // 索引库名称 String indexId = "kenick20191224010"; // 索引库中索引的id String indexType = "logs"; // 索引类型
// 根据索引类型和ID查询 GetResponse getResponse = esClient.prepareGet(indexName, indexType, indexId).get(); System.out.println("开始根据索引类型和ID查询！"); System.out.println(getResponse.getSourceAsString());
// 对message进行模糊查询 SearchRequest searchRequest = new SearchRequest(indexName); searchRequest.types(indexType);
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.query(QueryBuilders.wildcardQuery("message", "test*")); searchRequest.source(sourceBuilder);
SearchResponse searchResponse = esClient.search(searchRequest).get(5000, TimeUnit.MILLISECONDS); RestStatus status = searchResponse.status(); System.out.println("模糊查询结果:" + status);
SearchHits hits = searchResponse.getHits(); SearchHit[] searchHits = hits.getHits(); System.out.println("模糊查询详细信息:"); for (SearchHit hit : searchHits) { String sourceAsString = hit.getSourceAsString(); // 取_source字段值，json串 System.out.println("source:" + sourceAsString); } |
执行结果：