ElasticSearch7.4.1 ,laravel ,scout,ik全文搜索實戰(zhàn)

elasticsearch 安裝:本實驗使用 7.4.1

  • 需先安裝 java , java -version
  • elasticsearch 7.4.1 內(nèi)含了jdk 不用單獨安裝
wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-7.4.1.tar.gz
tar -xvf elasticsearch-7.4.1.tar.gz
cd elasticsearch-$version/bin
./elasticsearch #運行

deb安裝:
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.4.1-amd64.deb  #下載
sudo dpkg -i elasticsearch-6.2.3.deb #安裝
程序位置 `/usr/share/elasticsearch/`
配置文件 `/etc/elasticsearch/elasticsearch.yml`

啟動:
$path/bin/elasticsearch -d  |  service elasticsearch start


  • 訪問 curl 127.0.0.1:9200 返回
{
    "name": "homestead",
    "cluster_name": "elasticsearch",
    "cluster_uuid": "FC1WotHLTo6JKIsj-Y2wNg",
    "version": {
        "number": "7.4.1",
        "build_flavor": "default",
        "build_type": "deb",
        "build_hash": "fc0eeb6e2c25915d63d871d344e3d0b45ea0ea1e",
        "build_date": "2019-10-22T17:16:35.176724Z",
        "build_snapshot": false,
        "lucene_version": "8.2.0",
        "minimum_wire_compatibility_version": "6.8.0",
        "minimum_index_compatibility_version": "6.0.0-beta1"
    },
    "tagline": "You Know, for Search"
}

列出所有索引 `curl 127.0.0.1:9200/_cat/indices?v

laravel 中使用 , 需要用到的包

composer require laravel/scout
composer require tamayo/laravel-scout-elastic

  1. 配置
// config/app.php
'providers' => [
    // ...
    Laravel\Scout\ScoutServiceProvider::class,
    // ...
    ScoutEngines\Elasticsearch\ElasticsearchProvider::class,
],
  1. 生成scout配置文件 config/scout.php
    php artisan vendor:publish --provider="Laravel\Scout\ScoutServiceProvider"

  2. 在config/scout.php中加入elasticsearch的配置

 'elasticsearch' => [
        'index' => env('ELASTICSEARCH_INDEX', 'laravel'),
        'hosts' => [
            env('ELASTICSEARCH_HOST', 'http://localhost:9200'),
        ],
    ],
  1. 在 .env文件,加入scout和elasticsearch的配置
# scout配置
SCOUT_DRIVER=elasticsearch
SCOUT_PREFIX=

# elasticsearch 配置
ELASTICSEARCH_INDEX=esdemo
# elasticsearch 地址
ELASTICSEARCH_HOST=http://127.0.0.1:9200
  1. 在模型Orm中支持 索引
<?php
namespace App;

use Illuminate\Database\Eloquent\Model;
use Laravel\Scout\Searchable;

/**
 * 學(xué)生模型
 */
class Student extends Model
{
    use Searchable;

    public function searchableAs()
    {
        return '_doc';  // 這里只能填_doc, 定義索引里面的類型,上文我們說過,可以把type理解成一個數(shù)據(jù)表。我們現(xiàn)在要做的就是把我們所有的要全文搜索的字段都存入到es中的一個叫'_doc'的表中。
    }

    /**
     * 可搜索的數(shù)據(jù)索引
     * @return array
     */
    public function toSearchableArray()
    {
        return [
              'indexName' => $this->field, // 與映射里定義的需要相同才能按照映射的分詞方式處理;
        ];
    }
}
  1. 導(dǎo)入數(shù)據(jù)據(jù)到 elasticsearch 索引里
    php artisan scout:import "App\Student"
    刪除索引數(shù)據(jù)
    php artisan scout:flush "App\Student"

  2. laravel中使用

$data = Sdudent::search("key word”)->get();
return $data;

  1. 中文無法搜索時的處理:
    修改 ErickTamayo/laravel-scout-elastic/src/ElasticsearchEngine.php
    protected function performSearch(Builder $builder, array $options = [])
    {
        $params = [
            'index' => $this->index,
            'type' => $builder->index ?: $builder->model->searchableAs(),
            'body' => [
                'query' => [
                    'bool' => [
                       // 'must' => [['query_string' => [ 'query' => "*{$builder->query}*"]]]
                        'must' => [['query_string' => [ 'query' => "{$builder->query}"]]]

                    ]
                ]
            ]
        ];


中文分詞

默認使用的分詞方式, 中文會切割成一個一個字符,搜索結(jié)果偏差大, ik分詞可以很好的解決這個問題
ik插件安裝:

$path/bin/elasticsearch-plugin list #查看

/usr/share/elasticsearch/bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.4.1/elasticsearch-analysis-ik-7.4.1.zip #安裝

$path/bin/elasticsearch-plugin remove pluginName #刪除

分詞實例: elasticseach 7.4.1

  1. 創(chuàng)建索引
    curl -X PUT "localhost:9200/test"

  2. 查看索引
    curl -X GET "localhost:9200/test"

  3. 創(chuàng)建映射 這個是決定php 使用時所使用的字段是否使用ik分詞的關(guān)鍵

curl -X POST "localhost:9200/test/_mapping" -H 'Content-Type: application/json' -d'
{
    "properties": {
        "content": {  
            "type": "text",
            "analyzer": "ik_max_word",
            "search_analyzer": "ik_max_word"
        }
    }
}'
  1. 查看映射
    curl -X GET "localhost:9200/test/_mapping?pretty"

  2. 索引文檔

curl -X PUT "localhost:9200/test/_doc/1" -H 'Content-Type: application/json' -d '
{ "content":"今天學(xué)習(xí)搜索分詞"}'

curl -X PUT "localhost:9200/test/_doc/2" -H 'Content-Type: application/json' -d '
{ "content":"today study search explode words"}'

索引分詞測試:

curl -H  'Content-Type: application/json' -XPOST 'http://localhost:9200/laravel/_analyze?pretty' -d'
{
  "text": "分詞測試"
}'
response:
{
    "tokens": [
        {
            "token": "分",
            "start_offset": 0,
            "end_offset": 1,
            "type": "<IDEOGRAPHIC>",
            "position": 0
        },
        {
            "token": "詞",
            "start_offset": 1,
            "end_offset": 2,
            "type": "<IDEOGRAPHIC>",
            "position": 1
        },
        {
            "token": "測",
            "start_offset": 2,
            "end_offset": 3,
            "type": "<IDEOGRAPHIC>",
            "position": 2
        },
        {
            "token": "試",
            "start_offset": 3,
            "end_offset": 4,
            "type": "<IDEOGRAPHIC>",
            "position": 3
        }
    ]
}

curl -H  'Content-Type: application/json' -XPOST 'http://localhost:9200/laravel/_analyze?pretty' -d'
{
  "analyzer":"ik_max_word",
  "text": "分詞測試"
}'
response
{
    "tokens": [
        {
            "token": "分詞",
            "start_offset": 0,
            "end_offset": 2,
            "type": "CN_WORD",
            "position": 0
        },
        {
            "token": "測試",
            "start_offset": 2,
            "end_offset": 4,
            "type": "CN_WORD",
            "position": 1
        }
    ]
}
  1. 查看索引列表
    curl -X GET localhost:9200/test/_search?q=*&pretty

  2. 搜索測試

curl -X POST "localhost:9200/test/_doc/_search" -H 'Content-Type: application/json' -d'
{
    "query": {
        "match": {
            "content": "搜索"
        }
    },
    "highlight": {
        "pre_tags": [
            "<tag1>",
            "<tag2>"
        ],
        "post_tags": [
            "</tag1>",
            "</tag2>"
        ],
        "fields": {
            "content": {}
        }
    }
}'

response

{
    "took": 3,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1,
            "relation": "eq"
        },
        "max_score": 1.4523084,
        "hits": [
            {
                "_index": "test",
                "_type": "_doc",
                "_id": "1",
                "_score": 1.4523084,
                "_source": {
                    "content": "今天學(xué)習(xí)搜索分詞"
                },
                "highlight": {
                    "content": [
                        "今天學(xué)習(xí)<tag1>搜索</tag1><tag1>分詞</tag1>"
                    ]
                }
            }
        ]
    }
}
  1. elasticsearch7 移除了type, 對于多表搜索 的處理模式是 每個表建立一個索引, scout 中索引是寫在配置文件中固定的 config/scout.php
    'elasticsearch' => [
        'index' => env('ELASTICSEARCH_INDEX', 'laravel'),
        'hosts' => [
            env('ELASTICSEARCH_HOST', 'http://localhost:9200'),
        ],

scout 多個索引可以通過修改ORM 的__construnct(){} 重構(gòu) 配置索引 index

class Orm extends Model
{
    use Searchable;
    protected $table = 'test';
    protected $fillable = [
        "code",
        "name_en",
        "name_zh",

        "is_delete",
        "created_at",
        "updated_at",
        ];

    protected $hidden = [
    ];

    public function __construct($attribute = [])
    {
        \Config::set('scout.elasticsearch.index',$this->getTable()); #使用表名做為索引
        parent::__construct($attribute);
    }

創(chuàng)建 command 創(chuàng)建索以及map , 導(dǎo)入數(shù)據(jù)
php artisan es:init modelOrm
php artisan scout:import modelOrm

<?php

namespace App\Console\Commands;

use Illuminate\Console\Command;

class EsInit extends Command
{
    /**
     * The name and signature of the console command.
     *
     * @var string
     */
    protected $signature = 'es:init {model}';

    /**
     * The console command description.
     *
     * @var string
     */
    protected $description = 'elasticsearch create mapping';

    /**
     * Create a new command instance.
     *
     * @return void
     */
    public function __construct()
    {
        parent::__construct();
    }

    /**
     * Execute the console command.
     *
     * @return mixed
     */
    public function handle()
    {
        $class = $this->argument('model');

        $model = new $class;

        $client = new \GuzzleHttp\Client();
        $url = config('scout.elasticsearch.hosts')[0] . '/' . $model->getTable().'/_mapping';

        $client->delete(config('scout.elasticsearch.hosts')[0] . '/' . $model->getTable());
        $client->put(config('scout.elasticsearch.hosts')[0] . '/' . $model->getTable());

        $data = [];
        foreach ($model->toSearchableArray() as $key => $value){
            $data[$key] =  [
                "type" =>  "text",
                "analyzer" =>  "ik_max_word",
                "search_analyzer"=>  "ik_max_word"

            ];
        }

        $client->post($url, [
            \GuzzleHttp\RequestOptions::JSON => [
                "properties" => $data
            ]
        ]);
    }
}

關(guān)鍵詞高亮

修改 ErickTamayo/laravel-scout-elastic/src/ElasticsearchEngine.php

protected function performSearch(Builder $builder, array $options = [])
{
    $fields = [];
    foreach ($builder->model->toSearchableArray() as $key => $value){
        $fields[$key] = (object)[];
    }

    $params = [
        'index' => $this->model->getTable(),
        'type' => $builder->index ?: $builder->model->searchableAs(),
        'body' => [
            'query' => [
                'bool' => [
                    'must' => [['query_string' => [ 'query' => "{$builder->query}"]]]
                ]
            ],
            "highlight" => [
                "fields" => $fields
            ]
        ]
    ];

public function map(Builder $builder, $results, $model)
{
    if ($results['hits']['total'] === 0) {
        return $model->newCollection();
    }

    $keys = collect($results['hits']['hits'])->pluck('_id')->values()->all();

    $source = [];

    foreach ($results['hits']['hits'] as $row){
        $source[$row['_id']] = $row['highlight'];
    }

    return $model->getScoutModelsByIds(
            $builder, $keys
        )->filter(function ($model) use ($keys) {
            return in_array($model->getScoutKey(), $keys);
        })->map(function ($item,$key) use($source,$model) {
            $item['highlight'] = $source[$item[$model->getKeyName()]];
            return $item;
        });
}
最后編輯于
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時請結(jié)合常識與多方信息審慎甄別。
平臺聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點,簡書系信息發(fā)布平臺,僅提供信息存儲服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容