ElasticSearch7.4.1 ，laravel ，scout，ik全文搜索实战

elasticsearch 安装：本实验使用 7.4.1

• 需先安装 java ， java -version
• elasticsearch 7.4.1 内含了jdk 不用单独安装

wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-7.4.1.tar.gz
tar -xvf elasticsearch-7.4.1.tar.gz
cd elasticsearch-$version/bin
./elasticsearch #运行

deb安装：
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.4.1-amd64.deb  #下载
sudo dpkg -i elasticsearch-6.2.3.deb #安装
程序位置 `/usr/share/elasticsearch/`
配置文件 `/etc/elasticsearch/elasticsearch.yml`

启动：
$path/bin/elasticsearch -d  |  service elasticsearch start

• 访问 curl 127.0.0.1:9200 返回

{
    "name": "homestead",
    "cluster_name": "elasticsearch",
    "cluster_uuid": "FC1WotHLTo6JKIsj-Y2wNg",
    "version": {
        "number": "7.4.1",
        "build_flavor": "default",
        "build_type": "deb",
        "build_hash": "fc0eeb6e2c25915d63d871d344e3d0b45ea0ea1e",
        "build_date": "2019-10-22T17:16:35.176724Z",
        "build_snapshot": false,
        "lucene_version": "8.2.0",
        "minimum_wire_compatibility_version": "6.8.0",
        "minimum_index_compatibility_version": "6.0.0-beta1"
    },
    "tagline": "You Know, for Search"
}

列出所有索引 `curl 127.0.0.1:9200/_cat/indices?v

laravel 中使用 ， 需要用到的包

composer require laravel/scout
composer require tamayo/laravel-scout-elastic

1. 配置

// config/app.php
'providers' => [
    // ...
    Laravel\Scout\ScoutServiceProvider::class,
    // ...
    ScoutEngines\Elasticsearch\ElasticsearchProvider::class,
],

2. 生成scout配置文件 config/scout.php
   php artisan vendor:publish --provider="Laravel\Scout\ScoutServiceProvider"

3. 在config/scout.php中加入elasticsearch的配置

 'elasticsearch' => [
        'index' => env('ELASTICSEARCH_INDEX', 'laravel'),
        'hosts' => [
            env('ELASTICSEARCH_HOST', 'http://localhost:9200'),
        ],
    ],

4. 在 .env文件，加入scout和elasticsearch的配置

# scout配置
SCOUT_DRIVER=elasticsearch
SCOUT_PREFIX=

# elasticsearch 配置
ELASTICSEARCH_INDEX=esdemo
# elasticsearch 地址
ELASTICSEARCH_HOST=http://127.0.0.1:9200

5. 在模型Orm中支持 索引

 $this->field, // 与映射里定义的需要相同才能按照映射的分词方式处理；
        ];
    }
}

6. 导入数据据到 elasticsearch 索引里
   php artisan scout:import "App\Student"
   删除索引数据
   php artisan scout:flush "App\Student"

7. laravel中使用

$data = Sdudent::search("key word”)->get();
return $data;

8. 中文无法搜索时的处理：
   修改 ErickTamayo/laravel-scout-elastic/src/ElasticsearchEngine.php

    protected function performSearch(Builder $builder, array $options = [])
    {
        $params = [
            'index' => $this->index,
            'type' => $builder->index ?: $builder->model->searchableAs(),
            'body' => [
                'query' => [
                    'bool' => [
                       // 'must' => [['query_string' => [ 'query' => "*{$builder->query}*"]]]
                        'must' => [['query_string' => [ 'query' => "{$builder->query}"]]]

                    ]
                ]
            ]
        ];

---

中文分词

默认使用的分词方式， 中文会切割成一个一个字符，搜索结果偏差大， ik分词可以很好的解决这个问题
ik插件安装：

$path/bin/elasticsearch-plugin list #查看

/usr/share/elasticsearch/bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.4.1/elasticsearch-analysis-ik-7.4.1.zip #安装

$path/bin/elasticsearch-plugin remove pluginName #删除

分词实例： elasticseach 7.4.1

1. 创建索引
   curl -X PUT "localhost:9200/test"

2. 查看索引
   curl -X GET "localhost:9200/test"

3. 创建映射 这个是决定php 使用时所使用的字段是否使用ik分词的关键

curl -X POST "localhost:9200/test/_mapping" -H 'Content-Type: application/json' -d'
{
    "properties": {
        "content": {  
            "type": "text",
            "analyzer": "ik_max_word",
            "search_analyzer": "ik_max_word"
        }
    }
}'

4. 查看映射
   curl -X GET "localhost:9200/test/_mapping?pretty"

5. 索引文档

curl -X PUT "localhost:9200/test/_doc/1" -H 'Content-Type: application/json' -d '
{ "content":"今天学习搜索分词"}'

curl -X PUT "localhost:9200/test/_doc/2" -H 'Content-Type: application/json' -d '
{ "content":"today study search explode words"}'

索引分词测试：

curl -H  'Content-Type: application/json' -XPOST 'http://localhost:9200/laravel/_analyze?pretty' -d'
{
  "text": "分词测试"
}'
response：
{
    "tokens": [
        {
            "token": "分",
            "start_offset": 0,
            "end_offset": 1,
            "type": "",
            "position": 0
        },
        {
            "token": "词",
            "start_offset": 1,
            "end_offset": 2,
            "type": "",
            "position": 1
        },
        {
            "token": "测",
            "start_offset": 2,
            "end_offset": 3,
            "type": "",
            "position": 2
        },
        {
            "token": "试",
            "start_offset": 3,
            "end_offset": 4,
            "type": "",
            "position": 3
        }
    ]
}

curl -H  'Content-Type: application/json' -XPOST 'http://localhost:9200/laravel/_analyze?pretty' -d'
{
  "analyzer":"ik_max_word",
  "text": "分词测试"
}'
response
{
    "tokens": [
        {
            "token": "分词",
            "start_offset": 0,
            "end_offset": 2,
            "type": "CN_WORD",
            "position": 0
        },
        {
            "token": "测试",
            "start_offset": 2,
            "end_offset": 4,
            "type": "CN_WORD",
            "position": 1
        }
    ]
}

6. 查看索引列表
   curl -X GET localhost:9200/test/_search?q=*&pretty

7. 搜索测试

curl -X POST "localhost:9200/test/_doc/_search" -H 'Content-Type: application/json' -d'
{
    "query": {
        "match": {
            "content": "搜索"
        }
    },
    "highlight": {
        "pre_tags": [
            "",
            ""
        ],
        "post_tags": [
            "",
            ""
        ],
        "fields": {
            "content": {}
        }
    }
}'

response

{
    "took": 3,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1,
            "relation": "eq"
        },
        "max_score": 1.4523084,
        "hits": [
            {
                "_index": "test",
                "_type": "_doc",
                "_id": "1",
                "_score": 1.4523084,
                "_source": {
                    "content": "今天学习搜索分词"
                },
                "highlight": {
                    "content": [
                        "今天学习搜索分词"
                    ]
                }
            }
        ]
    }
}

8. elasticsearch7 移除了type， 对于多表搜索 的处理模式是 每个表建立一个索引， scout 中索引是写在配置文件中固定的 config/scout.php

    'elasticsearch' => [
        'index' => env('ELASTICSEARCH_INDEX', 'laravel'),
        'hosts' => [
            env('ELASTICSEARCH_HOST', 'http://localhost:9200'),
        ],

scout 多个索引可以通过修改ORM 的__construnct(){} 重构 配置索引 index

class Orm extends Model
{
    use Searchable;
    protected $table = 'test';
    protected $fillable = [
        "code",
        "name_en",
        "name_zh",

        "is_delete",
        "created_at",
        "updated_at",
        ];

    protected $hidden = [
    ];

    public function __construct($attribute = [])
    {
        \Config::set('scout.elasticsearch.index',$this->getTable()); #使用表名做为索引
        parent::__construct($attribute);
    }

创建 command 创建索以及map , 导入数据
php artisan es:init modelOrm
php artisan scout:import modelOrm

argument('model');

        $model = new $class;

        $client = new \GuzzleHttp\Client();
        $url = config('scout.elasticsearch.hosts')[0] . '/' . $model->getTable().'/_mapping';

        $client->delete(config('scout.elasticsearch.hosts')[0] . '/' . $model->getTable());
        $client->put(config('scout.elasticsearch.hosts')[0] . '/' . $model->getTable());

        $data = [];
        foreach ($model->toSearchableArray() as $key => $value){
            $data[$key] =  [
                "type" =>  "text",
                "analyzer" =>  "ik_max_word",
                "search_analyzer"=>  "ik_max_word"

            ];
        }

        $client->post($url, [
            \GuzzleHttp\RequestOptions::JSON => [
                "properties" => $data
            ]
        ]);
    }
}

关键词高亮

修改 ErickTamayo/laravel-scout-elastic/src/ElasticsearchEngine.php

protected function performSearch(Builder $builder, array $options = [])
{
    $fields = [];
    foreach ($builder->model->toSearchableArray() as $key => $value){
        $fields[$key] = (object)[];
    }

    $params = [
        'index' => $this->model->getTable(),
        'type' => $builder->index ?: $builder->model->searchableAs(),
        'body' => [
            'query' => [
                'bool' => [
                    'must' => [['query_string' => [ 'query' => "{$builder->query}"]]]
                ]
            ],
            "highlight" => [
                "fields" => $fields
            ]
        ]
    ];

public function map(Builder $builder, $results, $model)
{
    if ($results['hits']['total'] === 0) {
        return $model->newCollection();
    }

    $keys = collect($results['hits']['hits'])->pluck('_id')->values()->all();

    $source = [];

    foreach ($results['hits']['hits'] as $row){
        $source[$row['_id']] = $row['highlight'];
    }

    return $model->getScoutModelsByIds(
            $builder, $keys
        )->filter(function ($model) use ($keys) {
            return in_array($model->getScoutKey(), $keys);
        })->map(function ($item,$key) use($source,$model) {
            $item['highlight'] = $source[$item[$model->getKeyName()]];
            return $item;
        });
}