http.Agent理解

http.Agent官方文檔

附圖,個人對http.Agent的理解:

image.png

一個 Agent是在client端用來管理鏈接的持久性和重用。對于一個host+port維持著一個請求隊列,這些請求重復(fù)使用一個socket,直到這個隊列空,這時,這個socket會被destroy或者放到pool里,在pool里時這個socket將會被再次重用(這兩個行為取決于keepAlive的配置)

keepAlive
keepAliveMsecs
maxSockets
maxFreeSockets

在pool中的鏈接已經(jīng)開啟了tcp的Keep-Alive,然而在server端會有如下行為影響pool中的鏈接:

測試代碼準(zhǔn)備:

client.js


const http = require('http');

const agent = new http.Agent({
    keepAlive: true,
    keepAliveMsecs: 1000,
    maxSockets: 4,
    maxFreeSockets: 2
});

const test = () => {
    return new Promise((resolve, reject) => {
        const option = {
            protocol: 'http:',
            host: 'localhost',
            port: 9990,
            path: `/`,
            agent: agent,
            // agent: agent,
            headers: {"Connection": "keep-alive"},
            method: 'GET'
        };


        const req = http.request(option, function(res) {
            res.setEncoding('utf8');
            let body = '';
            res.on('data', (chunk) => {
                body += chunk;
            });
            res.on('end', () => {
                resolve(body)
            });
        });

        req.on('error', (e) => {
            console.error(`problem with request: ${e.message}`);
            console.log(e.stack)
        });


        req.end();
    })
};




const sendReq = (count) => {
    let arr = [];
    for (let i=0;i<count;i++) arr.push(test())
    Promise.all(arr).then(function(){
        console.log('======end======')
    })
}


server.js

const http = require('http');

let server = http.createServer(function(req, res) {

    console.log(req.connection.remotePort);
    res.end('200');

}).listen(9990);

server.keepAliveTimeout = 5000; // 這個值默認(rèn)就是5s,可以直接賦值修改

  • server端主動關(guān)閉空閑鏈接:client收到通知后,當(dāng)前socket會從pool中移除,下一次請求時會創(chuàng)建一個新的socket

Pooled connections have TCP Keep-Alive enabled for them, but servers may still close idle connections, in which case they will be removed from the pool and a new connection will be made when a new HTTP request is made for that host and port.

client.js補(bǔ)充

sendReq(1);  // 先發(fā)送一個req

setTimeout(() => {sendReq(1)}, 10 * 1000); //隔10s后再次發(fā)送一次req

server.js輸出如下:

 // console.log(req.connection.remotePort);

 53957 // 發(fā)送第一個請求的socket port
 54011 // 隔10s后發(fā)送第二個請求的socket port。port不同,說明第一個socket已經(jīng)被關(guān)閉

wireshark抓包如下:

image.png

可以看到每隔1s發(fā)送向server端發(fā)送了一次TCP Keep-Alive探測。由于server端設(shè)置的keepAliveTimeout為5s(默認(rèn)就是5s),所以在5s后關(guān)閉了這個tcp鏈接,相應(yīng)的,client端收到關(guān)閉的信號就會close到當(dāng)前的socket,并從pool中移除這個socket

_http_agent.js

Agent.prototype.removeSocket = function removeSocket(s, options) {
  var name = this.getName(options);
  debug('removeSocket', name, 'writable:', s.writable);
  var sets = [this.sockets];

  // If the socket was destroyed, remove it from the free buffers too.
  if (!s.writable)
    sets.push(this.freeSockets);

  for (var sk = 0; sk < sets.length; sk++) {
    var sockets = sets[sk];

    if (sockets[name]) {
      var index = sockets[name].indexOf(s);
      if (index !== -1) {
        sockets[name].splice(index, 1);
        // Don't leak
        if (sockets[name].length === 0)
          delete sockets[name];
      }
    }
  }

  // 省略其他代碼
};
  • server端拒絕多個請求共用一個tcp鏈接,在這種情況下,在每次請求時鏈接都會建立并且不能被pool。agent仍然會處理請求的發(fā)送,只是每個請求都會建立在一個新的tcp鏈接上

Servers may also refuse to allow multiple requests over the same connection, in which case the connection will have to be remade for every request and cannot be pooled. The Agent will still make the requests to that server, but each one will occur over a new connection.

client.js不變

server.js添加如下代碼

    res.shouldKeepAlive = false; // 禁用shouldkeepAlive
    res.end('200');

wireshark抓包如下:


image.png

可以看到,請求結(jié)束后,server就會關(guān)閉socket

When a connection is closed by the client or the server, it is removed from the pool. Any unused sockets in the pool will be unrefed so as not to keep the Node.js process running when there are no outstanding requests. (see socket.unref()).

當(dāng)想要保持一個http請求很長時間并不在pool中,可以調(diào)用“agentRemove”(這個時間取決于server端socket close的時間)

Sockets are removed from an agent when the socket emits either a 'close' event or an 'agentRemove' event. When intending to keep one HTTP request open for a long time without keeping it in the agent, something like the following may be done:

client.js

      // new
        req.on('socket', (socket) => {
            socket.emit('agentRemove');
        });

server.js

server.keepAliveTimeout = 20000; // 為了清楚,服務(wù)端設(shè)置20s后再關(guān)閉

wireshark抓包如下:


image.png

可以看到,觸發(fā)“agentRemove”后,當(dāng)前socket并沒有發(fā)送探測包,并且知道server端通知關(guān)閉才關(guān)閉。

當(dāng)agent參數(shù)設(shè)置為false時,client將會為每一個http請求都創(chuàng)建一個鏈接。


node keep-alive還是很有必要開啟的,尤其時作為中間層代理,當(dāng)有如下模式時:高并發(fā)下優(yōu)勢更明顯

browser瀏覽器 -> nginx -> node -> nginx -> java

當(dāng)nginx和node都開啟keep-alive時,性能測試如下:


image.png

參考資料mark:
http://tldp.org/HOWTO/TCP-Keepalive-HOWTO/overview.html

https://stackoverflow.com/questions/30365250/what-will-happen-if-i-use-socket-setkeepalive-in-node-js-server

https://stackoverflow.com/questions/19043355/how-to-use-request-js-node-js-module-pools

https://github.com/nodejs/node/issues/10774

NodeJS的底層通信

這篇文章很詳細(xì),贊一個
https://www.zhuxiaodong.net/2018/tcp-http-keepalive/

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時請結(jié)合常識與多方信息審慎甄別。
平臺聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡書系信息發(fā)布平臺,僅提供信息存儲服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容