2019独角兽企业重金招聘Python工程师标准>>> 
1 es启动时,命令行设置名字
./elasticsearch -Ecluster.name=my_cluster_name -Enode.name=my_node_name
2 REST API可以做什么
1.检查集群,节点,索引的健康,状态,统计
2.管理集群,节点,索引的数据和元数据
3.执行CRUD和搜索操作
4.执行高级搜索操作,比如分页,排序,过滤,脚本,聚合等
2.1 集群健康检测api
http://192.168.0.128:9200/_cat/health?v
epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1498119164 16:12:44 es-cluster yellow 1 1 20 20 0 0 20 0 - 50.0%
说明:集群的状态描述:
green:一切都准备好了,集群功能全部可用;
yellow:数据准备好了,但是副本还没有分配好,集群功能全部可用;
red:有些数据不可用,但是集群部分功能可用;
2.2 集群节点列表api
http://192.168.0.128:9200/_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.0.128 19 72 58 mdi * master
2.3 列出集群中所有的索引
http://192.168.0.128:9200/_cat/indices?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open ttl mPSsvTX3TbSsSQKUcJqtbA 5 1 0 0 795b 795b
yellow open java 3IfBdV_-T8SuvNSK72jBqQ 5 1 0 0 650b 650b
yellow open book -rZ8v4AfTDyPPTm3oZ_qLQ 5 1 16 0 64.7kb 64.7kb
yellow open index 4BAj2ycsSGyosLYPmTQEZw 5 1 0 0 795b 795b
上面health都为yellow是因为只有一个node,es默认创建一个副本,等待其他的节点加入
2.4 创建一个customer的index
http://192.168.0.128:9200/customer?pretty
method: PUT
response:
{
"acknowledged": true,
"shards_acknowledged": true
}
2.5 创建文档索引和查询文档
http://192.168.0.128:9200/customer/person/1?pretty
method: PUT
params: {"name":"张三","age":34,"sex":"男"}
response:
{
"_index": "customer",
"_type": "person",
"_id": "1",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"created": true
}
----------------------------------------------------------
http://192.168.0.128:9200/customer/person?pretty
method: POST
params: {"name":"张三","age":34,"sex":"男"}
response:
{
"_index": "customer",
"_type": "person",
"_id": "AVzTAOlNiSjTxTQlMxWw",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"created": true
}
-----------------------------------------------------------
http://192.168.0.128:9200/customer/person/AVzTAOlNiSjTxTQlMxWw/_update?pretty
method: POST
params:
{
"doc":{"name":"李四","age":44,"sex":"男"}
}
response:
{
"_index": "customer",
"_type": "person",
"_id": "AVzTAOlNiSjTxTQlMxWw",
"_version": 2,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
}
}
ps:AVzTAOlNiSjTxTQlMxWw是文档id
---------------------------------------------------------------
使用脚本更新,ctx._source指向当前source文档
http://192.168.0.128:9200/customer/person/AVzTAOlNiSjTxTQlMxWw/_update?pretty
method: POST
params:
{
"script" : "ctx._source.age += 5"
}
response:
{
"_index": "customer",
"_type": "person",
"_id": "AVzTAOlNiSjTxTQlMxWw",
"_version": 3,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
}
}
-------------------------------------------------------
http://192.168.0.128:9200/customer/person/1?pretty
method: GET
response:
{
"_index": "customer",
"_type": "person",
"_id": "1",
"_version": 1,
"found": true,
"_source": {
"name": "张三",
"age": 34,
"sex": "男"
}
}
2.6 删除索引
http://192.168.0.128:9200/customer
method: DELETE
response:
{
"acknowledged": true
}
2.7 根据文档id,删除文档
http://192.168.0.128:9200/customer/person/AVzTAOlNiSjTxTQlMxWw?pretty
method: DELETE
response:
{
"found": true,
"_index": "customer",
"_type": "person",
"_id": "AVzTAOlNiSjTxTQlMxWw",
"_version": 4,
"result": "deleted",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
}
}
2.8 elasticsearch提供了批处理_bulk API,可以同时处理index,update,delete操作.批处理减少网络请求.批处理时,如果某个动作失败了,不会影响其他的动作;批处理返回结果按执行的顺序返回动作执行状态,可以检测是否失败.
批量添加两个index
http://192.168.0.128:9200/customer/person/_bulk?pretty
method: POST
params:
{"index" : {"_id" : 2}}
{"name" : "赵六","age" : 23}
{"index" : {"_id" : 3}}
{"name" : "王五","age" : 53}
response:
{
"took" : 1541,
"errors" : false,
"items" : [
{
"index" : {
"_index" : "customer",
"_type" : "person",
"_id" : "2",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"created" : true,
"status" : 201
}
},
{
"index" : {
"_index" : "customer",
"_type" : "person",
"_id" : "3",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"created" : true,
"status" : 201
}
}
]
}
------------------------------------------------------
更新文档2,删除文档3
http://192.168.0.128:9200/customer/person/_bulk?pretty
method: POST
params:
{"update" : {"_id" : 2}}
{"doc" : {"age" : 33}}
{"delete" : {"_id" : 3}}
response:
{
"took": 941,
"errors": false,
"items": [
{
"update": {
"_index": "customer",
"_type": "person",
"_id": "2",
"_version": 2,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 200
}
},
{
"delete": {
"found": true,
"_index": "customer",
"_type": "person",
"_id": "3",
"_version": 2,
"result": "deleted",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 200
}
}
]
}
2.9 使用REST API搜索文档
搜索所有文档,结果按account_number升序排序
http://192.168.0.128:9200/bank/_search?q=*&sort=account_number:asc&pretty
等价的写法:
http://192.168.0.128:9200/bank/_search
method: POST
params:
{
"query": { "match_all": {} },
"sort": [
{ "account_number": "asc" }
],
"from" : 5, //从第5条开始,默认是0
"size" : 1 //返回1条,默认是10条
}
response:
{
"took": 32, //搜索时间,单位:毫秒
"timed_out": false, //搜索是否超时
"_shards": { //搜索分片数量,以及成功和失败的数量
"total": 5,
"successful": 5,
"failed": 0
},
"hits": { //搜索结果
"total": 1000, //满足搜索条件的文档数量
"max_score": null,
"hits": [ //真实搜索结果数组,默认显示10条
{
"_index": "bank",
"_type": "account",
"_id": "0",
"_score": null,
"_source": {
"account_number": 0,
"balance": 16623,
"firstname": "Bradshaw",
"lastname": "Mckenzie",
"age": 29,
"gender": "F",
"address": "244 Columbus Place",
"employer": "Euron",
"email": "[email protected]",
"city": "Hobucken",
"state": "CO"
},
"sort": [ //排序的结果
0
]
}
]
}
}
------------------------------------------
搜索所有的文档,返回前2条,并显示指定的fields
http://192.168.0.128:9200/bank/_search
method: POST
params:
{
"query": { "match_all": {} },
"_source" : ["account_number","balance","email"], //返回指定的字段
"size" : 2
}
response:
{
"took": 17,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1000,
"max_score": 1,
"hits": [
{
"_index": "bank",
"_type": "account",
"_id": "25",
"_score": 1,
"_source": {
"account_number": 25,
"balance": 40540,
"email": "[email protected]"
}
},
{
"_index": "bank",
"_type": "account",
"_id": "44",
"_score": 1,
"_source": {
"account_number": 44,
"balance": 34487,
"email": "[email protected]"
}
}
]
}
}
------------------------------------
搜索account_number为20的文档
http://192.168.0.128:9200/bank/_search
method: POST
params:
{
"query": { "match": {"account_number" : 20} }
}
------------------------------
搜索address中含有mill的所有文档
http://192.168.0.128:9200/bank/_search
method: POST
params:
{
"query": { "match": {"address" : "mill"} }
}
---------------------------------
使用match_phrase匹配address中含有"mill lane"短语的文档
http://192.168.0.128:9200/bank/_search
method: POST
params:
{
"query": { "match_phrase": {"address" : "mill lane"} }
}
---------------------------------
使用bool query匹配address中同时含有"mill "和"lane"短语的文档,must:and
http://192.168.0.128:9200/bank/_search
method: POST
params:
{
"query": {
"bool": {
"must": [
{ "match": { "address": "mill" } },
{ "match": { "address": "lane" } }
]
}
}
}
与之类似的:
should:or关系
must_not:即不含"mill",也不含"lane"的文档
bool query可以同时包含must,should,must_not组成复杂的查询
2.10 文档score:根据搜索条件估算一个文档匹配程度的相对的数值.得分越高,文档越有价值;反之,价值越低.有些情况不需要score(比如"filter""),es会检测自动优化查询,不计算得分.
http://192.168.0.128:9200/bank/_search
method: POST
params:
{
"query": {
"bool": {
"must": { "match_all": {} }, //查询所有的文档
"filter": { //过滤,不计算得分,从结果可以查出所有的score都为1,是个常量
"range": { //范围查询,适用于numeric或deta 类型
"balance": {
"gte": 20000,
"lte": 30000
}
}
}
}
}
}
2.11 执行聚合:es提供了分组和统计的能力,这就是聚合.可以认为就是sql中的group by和aggregate 功能.es在聚合时同时返回搜索的文档和聚合两部分.
按state分组聚合,不返回搜索的文档
http://192.168.0.128:9200/bank/_search
method: POST
params:
{
"size": 0,//不返回搜索的文档
"aggs": {//聚合
"group_by_state": {
"terms": {
"field": "state.keyword" //按state分组,降序排序
}
}
}
}
response:
{
"took": 58,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1000,
"max_score": 0,
"hits": []
},
"aggregations": {
"group_by_state": {
"doc_count_error_upper_bound": 20,
"sum_other_doc_count": 770,
"buckets": [
{
"key": "ID",
"doc_count": 27
},
...,
{
"key": "MO",
"doc_count": 20
}
]
}
}
}
--------------------------------------------------
按state分组,统计每个state的平均工资,并降序排序
http://192.168.0.128:9200/bank/_search
method: POST
params:
{
"size": 0,
"aggs": {
"group_by_state": {
"terms": {
"field": "state.keyword",
"order": {
"average_balance": "desc"
}
},
"aggs": {
"average_balance": {
"avg": {
"field": "balance"
}
}
}
}
}
}
-------------------------------------------------
按年龄段分组,然后按性别分组,统计每个年龄段中不同性别的平均工资
http://192.168.0.128:9200/bank/_search
method: POST
params:
{
"size": 0,
"aggs": {
"group_by_age": {
"range": {
"field": "age",
"ranges": [
{
"from": 20,
"to": 30
},
{
"from": 30,
"to": 40
},
{
"from": 40,
"to": 50
}
]
},
"aggs": {
"group_by_gender": {
"terms": {
"field": "gender.keyword"
},
"aggs": {
"average_balance": {
"avg": {
"field": "balance"
}
}
}
}
}
}
}
}