Elasticsearch DB笔记

作者：刘宾， [email protected]
转载请注明出处，谢谢！

---

参考：
http://www.ruanyifeng.com/blog/2017/08/elasticsearch.html
https://es.xiaoleilu.com/index.html

1. 安装与配置，最新版本6.2.2

1.1 dockercompose.yml

elasticsearch:
  build: elasticsearch/
  #cpuset: 3,4,5
  volumes:
    - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml:ro
    - /srv/docker/elk/elastic/data:/usr/share/elasticsearch/data
  ports:
    - "9200:9200"
    - "9300:9300"
  environment:
    ES_JAVA_OPTS: "-Xmx256m -Xms256m"



kibana:
  build: kibana/
  #cpuset: 3,4,5
  volumes:
    - ./kibana/config/:/opt/kibana/config/
  ports:
    - "5601:5601"
  links:
    - elasticsearch
~

1.2 配置文件， ./elasticsearch/config/elasticsearch.yml

## Default Elasticsearch configuration from elasticsearch-docker.
## from https://github.com/elastic/elasticsearch-docker/blob/master/build/elasticsearch/elasticsearch.yml
#
cluster.name: "docker-cluster"
network.host: 0.0.0.0

# minimum_master_nodes need to be explicitly set when bound on a public IP
# set to 1 to allow single node clusters
# Details: https://github.com/elastic/elasticsearch/pull/17288
discovery.zen.minimum_master_nodes: 1

## Use single node discovery in order to disable production mode and avoid bootstrap checks
## see https://www.elastic.co/guide/en/elasticsearch/reference/current/bootstrap-checks.html
#
discovery.type: single-node

## Disable X-Pack
## see https://www.elastic.co/guide/en/x-pack/current/xpack-settings.html
##     https://www.elastic.co/guide/en/x-pack/current/installing-xpack.html#xpack-enabling
#
xpack.security.enabled: false
xpack.monitoring.enabled: false
xpack.ml.enabled: false
xpack.graph.enabled: false
xpack.watcher.enabled: false

1.3 设置

1. 永久修改vm.max_map_count

追加vm.max_map_count=262144到/etc/sysctl.conf最后一行
sysctl -p

2. 临时修改vm.max_map_count

sysctl -w vm.max_map_count=262144

3. 设置数据卷必须可写

sudo chmod -R 777 /srv/docker/elk/elastic/data

1.4 验证

1. 查询当前elasticsearch系统信息

GET http://192.168.32.28:9200

2. REST接口

2.1 clustur

GET http://192.168.32.28:9200/_cat/health?v

2.2 node

GET http://192.168.32.28:9200/_cat/nodes?v

2.3 shards

GET http://192.168.32.28:9200/_cat/shards

2.1 index

• 查询所有index

GET http://192.168.32.28:9200/_cat/indices?v

• 增加index

PUT http://192.168.32.28:9200/customer?pretty

• 删除index

DELETE http://192.168.32.28:9200/customer?pretty

2.2 type

在6.0后限制使用type, 7.0后正式移除type

• 查询index(weather)中所有types

GET http://localhost:9200/_mapping?pretty
GET http://localhost:9200/weather/_mapping?pretty

2.3 文档

index和type可以自动创建

• 增加文档

PUT /customer/_doc/1?pretty
{
  "name": "John Doe"
}
POST /customer/_doc?pretty
{
  "name": "Jane Doe"
}

• 查询文档

GET /customer/_doc/1?pretty
{
  "_index" : "customer",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 1,
  "found" : true,
  "_source" : { "name": "John Doe" }
}

• 修改数据

PUT /customer/_doc/1?pretty
{
  "name": "John Doe"
}

• 修改文档
  es删除旧的文档，添加新的文档。

POST /customer/_doc/1/_update?pretty
{
  "doc": { "name": "Jane Doe", "age": 20 }
}

• 删除文档

DELETE /customer/_doc/2?pretty

2.6 中文分词

1. 安装对应的分词插件， 上面代码安装的是5.5.1版的插件，与 Elastic 5.5.1 配合使用

./bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v5.5.1/elasticsearch-analysis-ik-5.5.1.zip

2. 重新启动 Elastic
3. 创建type

$ curl -X PUT 'localhost:9200/accounts' -d '
{
  "mappings": {
    "person": {
      "properties": {
        "user": {
          "type": "text",
          "analyzer": "ik_max_word",
          "search_analyzer": "ik_max_word"
        },
        "title": {
          "type": "text",
          "analyzer": "ik_max_word",
          "search_analyzer": "ik_max_word"
        },
        "desc": {
          "type": "text",
          "analyzer": "ik_max_word",
          "search_analyzer": "ik_max_word"
        }
      }
    }
  }
}'

4. 数据查询

4.1 查询index/type下所有数据

localhost:9200/accounts/_search
localhost:9200/accounts/person/_search？pretty

4.2. 查询参数方式查询

http://192.168.32.21:9200/tests/user/_search?q=user:liu&size=2&from=1&sort=account_number:asc
http://192.168.32.21:9200/tests/user/_search?q=user:liu&size=2&from=1&sort=account_number:desc

4.3. DSL查询

• 或条件查询

$ curl 'localhost:9200/accounts/person/_search'  -d '
{
  "query" : { "match" : { "desc" : "软件 系统" }}
}'

• 与条件查询

$ curl 'localhost:9200/accounts/person/_search'  -d '
{
  "query": {
    "bool": {
      "must": [
        { "match": { "desc": "软件" } },
        { "match": { "desc": "系统" } }
      ]
    }
  }
}'

• rock climbing组合查询：

GET /megacorp/employee/_search
{
    "query" : {
        "match_phrase" : {
            "about" : "rock climbing"
        }
    }
}

• 完全匹配：

GET /megacorp/employee/_search
{
  "query": { "match_all": {} },
  "from": 10,
  "size": 10
}
GET /bank/_search
{
  "query": { "match_all": {} },
  "sort": { "balance": { "order": "desc" } }
}

• 指定返回信息

GET /bank/_search
{
  "query": { "match_all": {} },
  "_source": ["account_number", "balance"]
}

• bool query

与条件
GET /bank/_search
{
  "query": {
    "bool": {
      "must": [
        { "match": { "address": "mill" } },
        { "match": { "address": "lane" } }
      ]
    }
  }
}
或条件
GET /bank/_search
{
  "query": {
    "bool": {
      "should": [
        { "match": { "address": "mill" } },
        { "match": { "address": "lane" } }
      ]
    }
  }
}
GET /bank/_search
{
  "query": {
    "bool": {
      "must_not": [
        { "match": { "address": "mill" } },
        { "match": { "address": "lane" } }
      ]
    }
  }
}
GET /bank/_search
{
  "query": {
    "bool": {
      "must": [
        { "match": { "age": "40" } }
      ],
      "must_not": [
        { "match": { "state": "ID" } }
      ]
    }
  }
}

• filter
  return all accounts with balances between 20000 and 30000,

GET /bank/_search
{
  "query": {
    "bool": {
      "must": { "match_all": {} },
      "filter": {
        "range": {
          "balance": {
            "gte": 20000,
            "lte": 30000
          }
        }
      }
    }
  }
}

• 集合运算

GET /bank/_search
{
  "size": 0,
  "aggs": {
    "group_by_state": {
      "terms": {
        "field": "state.keyword"
      },
      "aggs": {
        "average_balance": {
          "avg": {
            "field": "balance"
          }
        }
      }
    }
  }
}

4.4 高亮输出：

GET /megacorp/employee/_search
{
    "query" : {
        "match_phrase" : {
            "about" : "rock climbing"
        }
    },
    "highlight": {
        "fields" : {
            "about" : {}
        }
    }
}

{
   ...
   "hits": {
      "total":      1,
      "max_score":  0.23013961,
      "hits": [
         {
            ...
            "_score":         0.23013961,
            "_source": {
               "first_name":  "John",
               "last_name":   "Smith",
               "age":         25,
               "about":       "I love to go rock climbing",
               "interests": [ "sports", "music" ]
            },
            "highlight": {
               "about": [
                  "I love to go rock climbing" 
               ]
            }
         }
      ]
   }
}