- The Elastic Stack,包括 Elasticsearch、Kibana、Beats 和 Logstash(也稱為 ELK Stack),能夠安全可靠地獲取任何來源、任何格式的數(shù)據(jù),然后實時地對數(shù)據(jù)進行搜索、分析和可視化。
- Elaticsearch(ES)是一個開源的高擴展的分布式全文搜索引擎,是整個 Elastic
Stack 技術棧的核心。它可以近乎實時的存儲、檢索數(shù)據(jù);本身擴展性很好,可以擴展到上百臺服務器,處理 PB 級別的數(shù)據(jù)。 - Lucene 是 Apache 軟件基金會 Jakarta 項目組的一個子項目,提供了一個簡單卻強大的應用程式接口,能夠做全文索引和搜尋。但它只是一個提供全文搜索功能類庫的核心工具包,而真正使用它還需要一個完善的服務框架搭建起來進行應用。
- 目前市面上流行的搜索引擎軟件,主流的就兩款:Elasticsearch 和 Solr,這兩款都是基于 Lucene 搭建的,可以獨立部署啟動的搜索引擎服務軟件。由于內(nèi)核相同,所以兩者除了服務器安裝、部署、管理、集群以外,對于數(shù)據(jù)的操作 修改、添加、保存、查詢等等都十分類似。

- 應用實例:GitHub、維基百科、SoundCloud、百度、新浪、阿里、Stack Overflow等。
- 安裝包下載地址:傳送門,下載window或其它版本解壓后進入bin目錄,點擊
elasticsearch.bat文件啟動 ES 服務。在此之前,先安裝對應的JDK版本。若不能啟動,可能是分配的內(nèi)存不足,則可修改配置文件:config/jvm.options。 - 注意:
9300端口為 Elasticsearch 集群間組件的通信端口,9200端口為瀏覽器訪問的 http協(xié)議 RESTful端口。


es服務啟動成功

Elasticsearch與MySQL的類比
- 注意:這里 Types 的概念已經(jīng)逐漸被弱化,Elasticsearch 6.X 中,一個 Index 下已經(jīng)只能包含一個type,Elasticsearch 7.X 中,Type 的概念已經(jīng)被刪除了。
- 創(chuàng)建索引:
http://127.0.0.1:9200/playing,method:put
# response
{
"acknowledged": true, # 操作成功
"shards_acknowledged": true, # 分片操作成功
"index": "playing" #索引名稱
}
- 注意:創(chuàng)建索引庫的分片數(shù)默認為1片,在ES 7.0.0 之前的版本中,默認為 5 片。
- 由于
put操作具有冪等性,若重復添加索引,則會返回錯誤信息:
{
"error": {
"root_cause": [
{
"type": "resource_already_exists_exception",
"reason": "index [playing/2Rmw7bI-SZ-ntKHIbJGDoA] already exists",
"index_uuid": "2Rmw7bI-SZ-ntKHIbJGDoA",
"index": "playing"
}
],
"type": "resource_already_exists_exception",
"reason": "index [playing/2Rmw7bI-SZ-ntKHIbJGDoA] already exists",
"index_uuid": "2Rmw7bI-SZ-ntKHIbJGDoA",
"index": "playing"
},
"status": 400
}
-
post操作沒有冪等性,意味著兩次操作結(jié)果可能不一樣,這種操作是不允許的:
{
"error": "Incorrect HTTP method for uri [/playing] and method [POST], allowed: [HEAD, DELETE, GET, PUT]",
"status": 405
}
- 查看某個索引:
http://127.0.0.1:9200/playing,method:get
{
"playing": {
"aliases": {},
"mappings": {},
"settings": {
"index": {
"creation_date": "1630457890587",
"number_of_shards": "1",
"number_of_replicas": "1",
"uuid": "2Rmw7bI-SZ-ntKHIbJGDoA",
"version": {
"created": "7060099"
},
"provided_name": "playing"
}
}
}
}
- 查看所有索引:
http://127.0.0.1:9200/_cat/indices?v,method:get
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open playing 2Rmw7bI-SZ-ntKHIbJGDoA 1 1 0 0 283b 283b
| 表頭 | 含義 |
|---|---|
| health | 當前服務器健康狀態(tài):green(集群完整) yellow(單點正常、集群不完整) red(單點不正常) |
| status | 索引打開、關閉狀態(tài) |
| index | 索引名 |
| uuid | 索引統(tǒng)一編號 |
| pri | 主分片數(shù)量 |
| rep | 副本數(shù)量 |
| docs.count | 可用文檔數(shù)量 |
| docs.deleted | 文檔刪除狀態(tài)(邏輯刪除) |
| store.size | 主分片和副分片整體占空間大小 |
| pri.store.size | 主分片占空間大小 |
- 刪除某個索引:
http://127.0.0.1:9200/playing,method:delete
{
"acknowledged": true
}
- 重復訪問索引時,服務器返回響應:索引不存在
{
"error": {
"root_cause": [
{
"type": "index_not_found_exception",
"reason": "no such index [playing]",
"resource.type": "index_or_alias",
"resource.id": "playing",
"index_uuid": "_na_",
"index": "playing"
}
],
"type": "index_not_found_exception",
"reason": "no such index [playing]",
"resource.type": "index_or_alias",
"resource.id": "playing",
"index_uuid": "_na_",
"index": "playing"
},
"status": 404
}
- 創(chuàng)建文檔:
http://127.0.0.1:9200/playing/_doc,method:post,請求體內(nèi)容:
{
"title":"一加phone",
"category":"手機",
"price":3999.00
}
- 服務器返回內(nèi)容:
{
"_index": "playing",
"_type": "_doc", # 類型-文檔
"_id": "ua3ynnsB4tbB67MkkEHa", #唯一標識
"_version": 1, # 版本
"result": "created", # 表示創(chuàng)建成功
"_shards": {
"total": 2, # 分片 - 總數(shù)
"successful": 1, # 分片 - 成功
"failed": 0 #分片 - 失敗
},
"_seq_no": 0,
"_primary_term": 1
}
- 若采用
put操作,則返回的是不允許這樣操作的提示,因為同樣的post請求發(fā)送出去,都返回不用_id值唯一標識的文檔信息,這就說明post不是冪等性的。
{
"error": "Incorrect HTTP method for uri [/playing/_doc] and method [PUT], allowed: [POST]",
"status": 405
}
- 由于沒有指定數(shù)據(jù)唯一性標識(ID),默認情況下,ES 服務器會隨機生成一個。若想要自定義唯一性標識,則需要在創(chuàng)建時指定:
http://127.0.0.1:9200/playing/_doc/1,method:post
{
"_index": "playing",
"_type": "_doc",
"_id": "1", # 自定義的id值
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 1,
"_primary_term": 1
}
- 多次發(fā)送上面的請求,會發(fā)現(xiàn)返回的結(jié)果中版本號、結(jié)果和序列號改變了:
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_version": 2,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 2,
"_primary_term": 1
}
- 注意:若增加數(shù)據(jù)時明確數(shù)據(jù)主鍵,則請求方式也可以為
PUT,更改上一個操作的method請求為put,返回結(jié)果如下:
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_version": 3, # 改變
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 3, # 改變
"_primary_term": 1
}
- 查詢文檔:
http://127.0.0.1:9200/playing/_doc/1,method:GET
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_version": 3,
"_seq_no": 3,
"_primary_term": 1,
"found": true, # 表示找到
"_source": { # 文檔源信息
"title": "一加phone",
"category": "手機",
"price": 3999
}
}
- 若查詢一個不存在的文檔(id=12),則返回以下內(nèi)容:
{
"_index": "playing",
"_type": "_doc",
"_id": "12",
"found": false
}
- 查詢某個索引下的全部文檔:
http://127.0.0.1:9200/playing/_search,method:GET
{
"took": 2763,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "playing",
"_type": "_doc",
"_id": "ua3ynnsB4tbB67MkkEHa",
"_score": 1,
"_source": {
"title": "一加phone",
"category": "手機",
"price": 3999
}
},
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_score": 1,
"_source": {
"title": "一加phone",
"category": "手機",
"price": 3999
}
}
]
}
}
- 修改文檔:①完全覆蓋:
http://127.0.0.1:9200/playing/_doc/1,method:PUT,請求體內(nèi)容:
{
"title":"一加phone",
"category":"手機",
"price":4999.00
}
- 服務器返回內(nèi)容:
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_version": 4,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 4,
"_primary_term": 1
}
- 查詢該文檔:
http://127.0.0.1:9200/playing/_doc/1,method:GET,可以發(fā)現(xiàn)版本號和序列號都發(fā)生改變了。
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_version": 4,
"_seq_no": 4,
"_primary_term": 1,
"found": true,
"_source": {
"title": "一加phone",
"category": "手機",
"price": 4999
}
}
- 局部更新:
http://127.0.0.1:9200/playing/_update/1,method:POST,請求體內(nèi)容:
{
"doc": {
"title": "小米phone"
}
}
- 服務器返回內(nèi)容:
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_version": 5,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 5,
"_primary_term": 1
}
- 查詢該文檔:
http://127.0.0.1:9200/playing/_doc/1,method:GET,可以發(fā)現(xiàn)版本和序列號都發(fā)生了改變。
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_version": 5,
"_seq_no": 5,
"_primary_term": 1,
"found": true,
"_source": {
"title": "小米phone",
"category": "手機",
"price": 4999
}
}
- 刪除文檔:
http://127.0.0.1:9200/playing/_doc/1,method:DELETE
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_version": 6,
"result": "deleted",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 6,
"_primary_term": 1
}
- 再刪除一次,則返回找不到的提示:
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_version": 7,
"result": "not_found", #找不到
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 7,
"_primary_term": 1
}
- 條件查詢:
http://127.0.0.1:9200/playing/_search?q=category:手機,method:GET
{
"took": 1251,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 0.21072102,
"hits": [
{
"_index": "playing",
"_type": "_doc",
"_id": "ua3ynnsB4tbB67MkkEHa",
"_score": 0.21072102,
"_source": {
"title": "一加phone",
"category": "手機",
"price": 3999
}
},
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_score": 0.21072102,
"_source": {
"title": "一加phone",
"category": "手機",
"price": 3999
}
}
]
}
}
- 一般不推薦放在query_string中,而是放在請求體中,全量查詢:
http://127.0.0.1:9200/playing/_search,method:POST,請求體內(nèi)容為:
{
"query":{
"match":{
"category": "手機"
}
}
}
- 服務器返回內(nèi)容:
{
"took": 22,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 0.21072102,
"hits": [
{
"_index": "playing",
"_type": "_doc",
"_id": "ua3ynnsB4tbB67MkkEHa",
"_score": 0.21072102,
"_source": {
"title": "一加phone",
"category": "手機",
"price": 3999
}
},
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_score": 0.21072102,
"_source": {
"title": "一加phone",
"category": "手機",
"price": 3999
}
}
]
}
}
- 分頁查詢:
http://127.0.0.1:9200/playing/_search,method:POST,請求體內(nèi)容:
{
"query":{
"match_all":{
}
},
"from": 0,
"size": 1
}
- 服務器返回內(nèi)容:
{
"took": 9,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "playing",
"_type": "_doc",
"_id": "ua3ynnsB4tbB67MkkEHa",
"_score": 1,
"_source": {
"title": "一加phone",
"category": "手機",
"price": 3999
}
}
]
}
}
- 分頁投影排序查詢:
http://127.0.0.1:9200/playing/_search,method:POST,請求體內(nèi)容:
{
"query":{
"match_all":{
}
},
"from": 1,
"size": 1,
"_source": ["title"], #投影字段
"sort": {
"price": {
"order": "desc"
}
}
}
- 服務器返回內(nèi)容:
{
"took": 10,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": null,
"hits": [
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_score": null,
"_source": {
"title": "一加phone"
},
"sort": [
3999
]
}
]
}
}
- 多條件查詢must操作:
http://127.0.0.1:9200/playing/_search,method:POST,請求體內(nèi)容:
{
"query":{
"bool": {
"must": [ #表示 and 操作,必須同時滿足
{
"match": {
"category": "手機"
}
}
]
}
}
}
- 多條件查詢should和filter操作:
http://127.0.0.1:9200/playing/_search,method:POST,請求體內(nèi)容:
{
"query":{
"bool": {
"should": [ #表示 or操作,只要符合條件都可以
{
"match": {
"title": "小米"
}
},
{
"match": {
"title": "一加"
}
}
],
"filter": { #按價格過濾
"range": {
"price": {
"gt": 4000
}
}
}
}
}
}
- 服務器返回內(nèi)容:
{
"took": 8,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 1.3862942,
"hits": [
{
"_index": "playing",
"_type": "_doc",
"_id": "2",
"_score": 1.3862942,
"_source": {
"title": "小米phone",
"category": "手機",
"price": 4999
}
}
]
}
}
- 對結(jié)果進行高亮顯示:
http://127.0.0.1:9200/playing/_search,method:POST,請求體內(nèi)容:
{
"query":{
"match_phrase": { #短詞匹配,這里還能查詢到估計是es分詞了
"title": "phone"
}
},
"highlight": {
"fields": {
"title": {}
}
}
}
- 服務器返回內(nèi)容:
{
"took": 9,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 0.18232156,
"hits": [
{
"_index": "playing",
"_type": "_doc",
"_id": "1",
"_score": 0.18232156,
"_source": {
"title": "一加phone",
"category": "手機",
"price": 3999
},
"highlight": {
"title": [
"一加<em>phone</em>"
]
}
},
{
"_index": "playing",
"_type": "_doc",
"_id": "2",
"_score": 0.18232156,
"_source": {
"title": "小米phone",
"category": "手機",
"price": 4999
},
"highlight": {
"title": [
"小米<em>phone</em>"
]
}
}
]
}
}
- 聚合查詢:
http://127.0.0.1:9200/playing/_search,method:POST,請求體內(nèi)容:
{
"aggs": { # 聚合操作
"price_group": { #分組名
"terms": { #分組操作
"field": "price" #分組字段
}
}
},
"size": 0 # 不用返回原始數(shù)據(jù)
}
- 服務器返回內(nèi)容:
{
"took": 146,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": null,
"hits": [] # 對應上面的 size 字段值
},
"aggregations": {
"price_group": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": 3999,
"doc_count": 1
},
{
"key": 4999,
"doc_count": 1
}
]
}
}
}
- 平均值查詢:
http://127.0.0.1:9200/playing/_search,method:POST,請求體內(nèi)容:
{
"aggs": {
"price_avg": {
"avg": {
"field": "price"
}
}
},
"size": 0
}
- 服務器返回內(nèi)容:
{
"took": 383,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": null,
"hits": []
},
"aggregations": {
"price_avg": {
"value": 4499
}
}
}
- 創(chuàng)建映射:
http://127.0.0.1:9200/user/_mapping,method:PUT
{
"properties": {
"name":{
"type": "text",
"index": true
},
"sex":{
"type": "keyword", # 表示完全匹配
"index": true
},
"age":{
"type": "long",
"index": false # 表示不能被索引,也不能被查詢
}
}
}
- 查詢映射:
http://127.0.0.1:9200/user/_mapping,method:GET
{
"user": {
"mappings": {
"properties": {
"age": {
"type": "long",
"index": false
},
"name": {
"type": "text"
},
"sex": {
"type": "keyword",
"index": true
}
}
}
}
}