Elasticsearch 是目前流行的大數(shù)據(jù)處理框架之一,詳細(xì)的說(shuō)明可以自行百度等.
以下文章采用es縮寫作為代表Elasticsearch
es有官方的php api接口,可以采用github下載或者composer.
1.創(chuàng)建索引($client->indices()->create())
$params = [
'index' => 'user_index', //索引名(相當(dāng)于mysql的數(shù)據(jù)庫(kù))
'body' => [
'mappings' => [
'normal_type' => [ //類型名(相當(dāng)于mysql的表)
'_all'=>[ // 是否開(kāi)啟所有字段的檢索
'enabled' => 'false'
],
'properties' => [ //文檔類型設(shè)置(相當(dāng)于mysql的數(shù)據(jù)類型)
'uid' => [
'type' => 'integer' // 字段類型為整型
],
'username' => [
'type' => 'keyword' // 字段類型為關(guān)鍵字,如果需要全文檢索,則修改為text,注意keyword字段為整體查詢,不能作為模糊搜索
],
'platform' => [
'type' => 'keyword'
],
'mobile' => [
'type' => 'integer'
],
'sex' => [
'type' => 'integer'
],
'source' => [
'type' => 'keyword'
],
'province' => [
'type' => 'keyword'
],
'city' => [
'type' => 'keyword'
],
'tag' => [
'properties' => [
'tagName' =>[
'type' => 'text'
],
'tagWeight' => [
'type' => 'integer',
]
]
],
]
]
]
]
];
$data = $client->indices()->create($params);
簡(jiǎn)單的索引就創(chuàng)建完畢了,現(xiàn)在我們可以獲取下剛創(chuàng)建的索引信息.
2.獲取索引信息
2.1 獲取索引信息($client->indices()->getSettings())
$params = [
'index' => 'user_index',
'client' => [
'ignore' => 404
]
];
$res = $client->indices()->getSettings($params);//獲取庫(kù)索引設(shè)置
信息
返回的結(jié)果如下:
array (
'user_index' =>
array (
'settings' =>
array (
'index' =>
array (
'creation_date' => '1498116455327',
'number_of_shards' => '5',
'number_of_replicas' => '1',
'uuid' => '8pkONoI7RAuw5HjnFa4UaQ',
'version' =>
array (
'created' => '5020299',
),
'provided_name' => 'user_index',
),
),
),
)
2.2 獲取Mapping信息
- 接口:$client->indices()->getMapping
$params = [
'index' => 'user_index',
'client' => [
'ignore' => 404
]
];
$res = $client->indices()->getMapping($params); //獲取mapping信息
返回的結(jié)果如下:
array (
'user_index' =>
array (
'mappings' =>
array (
'normal_type' =>
array (
'_all' =>
array (
'enabled' => false,
),
'properties' =>
array (
'city' =>
array (
'type' => 'keyword',
),
'mobile' =>
array (
'type' => 'integer',
),
'platform' =>
array (
'type' => 'keyword',
),
'province' =>
array (
'type' => 'keyword',
),
'sex' =>
array (
'type' => 'integer',
),
'source' =>
array (
'type' => 'keyword',
),
'uid' =>
array (
'type' => 'integer',
),
'username' =>
array (
'type' => 'keyword',
),
),
),
),
),
)
這里獲取的相關(guān)信息就是我們剛剛創(chuàng)建的索引es可以動(dòng)態(tài)的修改以及添加相關(guān)的信息.
2.3 修改/添加mapping信息
- 注意事項(xiàng):已經(jīng)建立好的字段類型是不能更改的!!
- 接口:$client->indices()->putMapping();
$params = [
'index' => 'user_index', //索引名(相當(dāng)于mysql的數(shù)據(jù)庫(kù))
'type' => 'normal_type',
'body' => [
'normal_type' => [
'properties' =>[
'tag' =>[
'type' => 'text'
]
]
]
]
];
$client->indices()->putMapping($params);
2.4 刪除索引
- 接口:$client->indices()->delete($params);
$params = [
'index' => 'user_index', //索引名(相當(dāng)于mysql的數(shù)據(jù)庫(kù))
];
3 es的增刪改查
3.1 增加數(shù)據(jù)
- 單條數(shù)據(jù)插入
- 接口:$client->index()
$params = [
'index' => 'my_index',
'type' => 'my_type',
'id' => 'my_id', // 不填則es會(huì)自動(dòng)生成唯一的id
'body' => [ 'testField' => 'abc']
];
$response = $client->index($params);
- 批量數(shù)據(jù)插入
- 接口:$client->bulk();
foreach($userList as $value){
$params['body'][] = [
'index' => [
'_index' => 'user_index',
'_type' => 'normal_type',
'_id' =>$value['uid']
]
];
$params['body'][] = [
'uid' => $value['uid'],
'username' => $value['username'],
'platform' => $value['platform'],
'mobile' => $value['mobile'],
'province' => $value['province'],
'city' => $value['city'],
'sex' => $value['gender '],
'source'=>'xxx'
];
}
$responses = $client->bulk($params);
- ps:當(dāng)循環(huán)插入大量數(shù)據(jù)時(shí),需要注意php的內(nèi)存超出限制的問(wèn)題.(還不清楚這個(gè)框架哪部分操作之后沒(méi)有釋放內(nèi)存)
3.2 搜索數(shù)據(jù)
- 單條數(shù)據(jù)搜搜,但是需要知道id
- 接口:$client->get()
$params = [
'index' => 'my_index',
'type' => 'my_type',
'id' => 'my_id' // es 自動(dòng)生成/自己錄入的
];
$response = $client->get($params);
- 批量搜索
- 接口:$client->search()
$params = [
'index' => 'my_index', //['my_index1', 'my_index2'],可以通過(guò)這種形式進(jìn)行跨庫(kù)查詢
'type' => 'my_type',//['my_type1', 'my_type2'],
'body' => [
'query' => [
'match' => [ // 搜索條件
'age' => '24' ,
'tag.tagName' =>'tag' // 如果字段存儲(chǔ)的是object.
//這里搜索必須要這樣才能搜索出結(jié)果.
]
],
'from' => '0', // 分頁(yè)
'size' => '200' // 每頁(yè)數(shù)量
'sort' => [ // 排序
'age' => 'desc' //對(duì)age字段進(jìn)行降序排序
]
]
];
$res = $client->search($params);
返回的數(shù)據(jù)大致如下:
array (size=4)
'took' => int 2
'timed_out' => boolean false
'_shards' =>
array (size=3)
'total' => int 5
'successful' => int 5
'failed' => int 0
'hits' => // 搜索到數(shù)據(jù)
array (size=3)
'total' => int 2
'max_score' => float 1
'hits' => // 具體數(shù)據(jù)包
array (size=1)
0 =>
array (size=5)
...
- 很多時(shí)候需要用到類似mysql的and/or 查詢es搜索也能做到.
- 這個(gè)時(shí)候我們就要用到es的組合過(guò)濾器
- https://www.elastic.co/guide/cn/elasticsearch/guide/current/combining-filters.html 官方文檔如下
- 以下是測(cè)試過(guò)的一個(gè)例子
{
"index": "xxx_index",
"type": "normal_type",
"body": {
"size": 20,
"from": 0,
"query": {
"bool": {
"must": [
{
"match": {
"tag.tagName": "xxxx"
}
}
]
}
},
"sort": {
"uid": "desc"
}
}
}
3.3 刪除數(shù)據(jù)
- 注意:數(shù)據(jù)刪除的時(shí)候有一定的緩存更新時(shí)間,在沒(méi)更新的時(shí)候還是能被搜索出來(lái).
- 接口:$client->delete();
$param = [
'index' => 'my_index',
'type' => 'my_type',
'id' => 'my_id'
];
$ret = $this->client->delete($param);
返回?cái)?shù)據(jù)如下:
array (size=7)
'found' => boolean true
'_index' => string 'user_index' (length=10)
'_type' => string 'normal_type' (length=11)
'_id' => string 'AVzh_QnIZvcaH3gE7MvE' (length=20)
'_version' => int 2
'result' => string 'deleted' (length=7)
'_shards' =>
array (size=3)
'total' => int 2
'successful' => int 1
'failed' => int 0
3.4 更新數(shù)據(jù)
- 接口:$client->update()
$params = [
'index' => 'my_index',
'type' => 'my_type',
'id' => '3416a75f4cea9109507cacd8e2f2aefc',
'body' => [
'doc' => [ // 必須帶上這個(gè).表示是文檔操作
'age' => 150
]
]
];
$res = $client->update($params);
返回結(jié)果
array (size=6)
'_index' => string 'user_index' (length=10)
'_type' => string 'normal_type' (length=11)
'_id' => string 'AVzh_QnIZvcaH3gE7Mw5' (length=20)
'_version' => int 2
'result' => string 'updated' (length=7)
'_shards' =>
array (size=3)
'total' => int 2
'successful' => int 1 // 當(dāng)數(shù)據(jù)沒(méi)有發(fā)生變化時(shí)這里是為0
'failed' => int 0
4.不同索引之間進(jìn)行倒數(shù)據(jù)
- 注意:不同索引之間倒數(shù)據(jù)時(shí),需要注意字段的類型,類型不一樣會(huì)導(dǎo)致導(dǎo)入失敗,這個(gè)時(shí)候可以使用script進(jìn)行處理
- 此接口不能再phpapi中運(yùn)行,作為記錄存在
- 接口:POST _reindex
POST _reindex
{
"source": {
"index": "user_profile"
},
"dest": {
"index": "user_prfile_v1"
},
"script": {
"inline": " ctx._source.tag=params.new_tags",
"params": {
"new_tags":{
"tagName":null,
"tagWeight":0
}
}
}
}