比如說,現(xiàn)在要下載大批量的數(shù)據(jù),從es,放到excel中,我們說,月度,或者年度,銷售記錄,很多,比如幾千條,幾萬條,幾十萬條
其實(shí)就要用到我們之前講解的es scroll api,對(duì)大量數(shù)據(jù)批量的獲取和處理
PUT /car_shop/cars/4
{
"brand": "寶馬",
"name": "寶馬320",
"price": 320000,
"produce_date": "2017-01-01",
"sale_price": 280000,
"sale_date": "2017-01-25"
}
就是要看寶馬的銷售記錄
做一個(gè)演示,每個(gè)批次下載一條寶馬的銷售記錄,分幾個(gè)批次給它下載完
Settings settings = Settings.builder()
.put("cluster.name", "my-application")
.put("client.transport.sniff", true)
.build();
TransportClient client = new PreBuiltTransportClient(settings).
addTransportAddress(new TransportAddress(InetAddress.getByName("192.168.80.188"), 9300));
SearchResponse searchResponse = client.prepareSearch("car_shop")
.setTypes("cars")
.setQuery(QueryBuilders.termQuery("brand", "寶馬"))
.setScroll(new TimeValue(60000))
.setSize(1)
.get();
int batchCount = 0;
do {
for (SearchHit searchHit : searchResponse.getHits().getHits()) {
System.out.println("batch: " + ++batchCount);
System.out.println(searchHit.getSourceAsString());
}
searchResponse = client.prepareSearchScroll(searchResponse.getScrollId())
.setScroll(new TimeValue(60000))
.execute().actionGet();
} while (searchResponse.getHits().getHits().length != 0);
client.close();