環(huán)境配置
- ES版本:5.4.3
- 系統(tǒng):CentOS release 6.x
- spring-data-elasticsearch:3.0.0.RELEASE
- spring:5.0.1.RELEASE
問題
1.本地訪問es的restful接口不通
需要更改下面配置(elasticsearch.yml):
network.bind_host:0.0.0.0
network.host: <服務(wù)器ip>
如果系統(tǒng)是CentOS release 6.x的,需要在(elasticsearch.yml)增加下面語句
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
2.max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
需要修改linux的內(nèi)核設(shè)置,在/etc/sysctl.conf文件加入
vm.max_map_count=262144
然后系統(tǒng)執(zhí)行:
sysctl -p
3.availableProcessors is already set to
Caused by: java.lang.IllegalStateException: availableProcessors is already set to [4], rejecting [4]
at io.netty.util.NettyRuntime$AvailableProcessorsHolder.setAvailableProcessors(NettyRuntime.java:51)
at io.netty.util.NettyRuntime.setAvailableProcessors(NettyRuntime.java:87)
at org.elasticsearch.transport.netty4.Netty4Utils.setAvailableProcessors(Netty4Utils.java:82)
at org.elasticsearch.transport.netty4.Netty4Transport.<init>(Netty4Transport.java:138)
at org.elasticsearch.transport.Netty4Plugin.lambda$getTransports$0(Netty4Plugin.java:93)
at org.elasticsearch.client.transport.TransportClient.buildTemplate(TransportClient.java:174)
at org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:265)
at org.elasticsearch.transport.client.PreBuiltTransportClient.<init>(PreBuiltTransportClient.java:130)
at org.elasticsearch.transport.client.PreBuiltTransportClient.<init>(PreBuiltTransportClient.java:116)
at org.elasticsearch.transport.client.PreBuiltTransportClient.<init>(PreBuiltTransportClient.java:106)
at org.springframework.data.elasticsearch.client.TransportClientFactoryBean.buildClient(TransportClientFactoryBean.java:91)
at org.springframework.data.elasticsearch.client.TransportClientFactoryBean.afterPropertiesSet(TransportClientFactoryBean.java:86)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1763)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1700)
... 82 more
解決辦法:
@Slf4j
@Component
public class ElasticSearchConfiguration implements InitializingBean {
static {
System.setProperty("es.set.netty.runtime.available.processors", "false");
}
@Override
public void afterPropertiesSet() throws Exception {
log.info("*****************es_config*************************");
log.info("es.set.netty.runtime.available.processors:{}", System.getProperty("es.set.netty.runtime.available.processors"));
log.info("***************************************************");
}
}
原因是Lucene改進(jìn)了數(shù)值類型的索引數(shù)據(jù)結(jié)構(gòu),使用了block k-d 樹。這種樹的葉結(jié)點(diǎn)上文檔不是有序排列的,所以查詢的時(shí)候需要獲取所有的文檔id,訪問 docValues構(gòu)造 bitset,進(jìn)行其它條件的聯(lián)合查詢。當(dāng)查詢的結(jié)果集很大的時(shí)候,大量時(shí)間會(huì)用于構(gòu)造bitset,導(dǎo)致了慢查詢。
解決方式:在不需要進(jìn)行 rangeQuery 的時(shí)候,盡量不要選擇數(shù)值類型作為存儲(chǔ)格式,使用 keyword 替代,尤其是在重復(fù)值數(shù)量很大的時(shí)候。