問題描述

某日，同事找到我，說他們的程序卡死了，不能連接zookeeper了。

錯誤情況

2019-03-27 14:28:56 - INFO org.apache.zookeeper.ClientCnxn$SendThread.logStartConnect(ClientCnxn.java:1032) : Opening socket connection to server 192.168.6.1/192.168.6.1:2181. Will not attempt to authenticate using SASL (unknown error)
2019-03-27 14:28:56 -ERROR org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:289) : Unable to open socket to 192.168.6.1/192.168.6.1:2181
2019-03-27 14:28:56 - WARN org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1162)   : Session 0x169bce56a6b043f for server null, unexpected error, closing socket connection and attempting reconnect
java.net.SocketException: 網(wǎng)絡不可達
    at sun.nio.ch.Net.connect0(Native Method)
    at sun.nio.ch.Net.connect(Net.java:454)
    at sun.nio.ch.Net.connect(Net.java:446)
    at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
    at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:277)
    at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:287)
    at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1024)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
2019-03-27 14:28:58 - INFO org.apache.zookeeper.ClientCnxn$SendThread.logStartConnect(ClientCnxn.java:1032) : Opening socket connection to server 192.168.6.1/192.168.6.1:2181. Will not attempt to authenticate using SASL (unknown error)

問題處理

1. 先排除是否zookeeper的問題，在這個機器上面，telnet 192.168.6.1 2181，發(fā)現(xiàn)不通
2. 登錄zookeeper機器，使用bin/zkServer.sh status,查看zookeeper狀態(tài)，發(fā)現(xiàn)zookeeper集群是正常的，那么zookeeper集群本身是沒有問題的。
3. 查看zookeeper的日志（話說zookeeper的日志真夠亂的，服務器上面有好幾個zookeeper.out都不知道是哪個，只能一個個打開，通過最近的時間判別。），通過日志發(fā)現(xiàn)了問題的原因,如下：

2019-03-27 09:24:38,191 [myid:1] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357] - caught end of stream exception
2019-03-27 09:24:38,078 [myid:1] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too many connections from /192.168.15.63 - max is 60
2019-03-27 09:24:38,003 [myid:1] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too many connections from /192.168.15.63 - max is 60

日志說的很清楚，連接數(shù)太多了，超過了默認配置的60個連接數(shù)。

解決辦法

方法一. 修改zookeeper默認的連接數(shù)，并重啟zookeeper集群。

vi ZOOKEEPER_HOME/conf/zoo.cfg

image.png

注意：
maxClientCnxns=0，則代表不限制連接數(shù)。

方法二. 優(yōu)化程序，減少zookeeper的連接數(shù)。