1、httpclient

核心模塊有
ManagedHttpClientConnection
PoolingHttpClientConnectionManager

鏈接池是由StrictConnPool維護(hù)， PoolingHttpClientConnectionManager不直接維護(hù)鏈接池

2、httpclient實(shí)現(xiàn)類

3、HttpClientConnectionManager實(shí)現(xiàn)類

• BasicHttpClientConnectionManager
• PoolingHttpClientConnectionManager

3.1、BasicHttpClientConnectionManager

單個(gè)連接的連接管理器，只支持一個(gè)鏈接。多線程使用時(shí)也只能一個(gè)線程執(zhí)行，其他線程會(huì)等待

3.1、PoolingHttpClientConnectionManager

維護(hù)連接請(qǐng)求池，并能夠?yàn)槎鄠€(gè)執(zhí)行線程連接請(qǐng)求提供服務(wù)。根據(jù)路由進(jìn)行區(qū)分，每個(gè)路由（域名+端口）默認(rèn)最大鏈接數(shù)為2，總共鏈接數(shù)默認(rèn)為20;

• 默認(rèn)最大鏈接數(shù)20
• 默認(rèn)最大路由鏈接數(shù)2
• 默認(rèn)空閑鏈接存活時(shí)間 10s
• 默認(rèn)空閑鏈接存活時(shí)間單位 s
• 默認(rèn)鏈接存活時(shí)間 -1 (永久存活)
• 默認(rèn)鏈接存活時(shí)間單位 s

備注：http中的鏈接不主動(dòng)關(guān)閉，鏈接池中的鏈接由IdleConnectionEvictor進(jìn)行維護(hù)管理；在未過(guò)期會(huì)一直保持鏈接狀態(tài)；

   CloseableHttpClient httpClient = HttpClients.custom().setConnectionManager(cm).build();
// 在build時(shí)設(shè)置很多初始化配置，build方法是入口

1. httpclient

1.1、httpclient實(shí)現(xiàn)類

1.2、httpclient創(chuàng)建方式說(shuō)明

通過(guò)上述說(shuō)明創(chuàng)建client只有兩種方式

• 通過(guò)HttpClientBuilder build();
• 直接new 自定義實(shí)現(xiàn)CloseableHttpClient 抽象類方法；

httpClients提供快速創(chuàng)建的方式

• createDefault() 創(chuàng)建默認(rèn)的連接器，默認(rèn)為InternalHttpClient客戶端，包含多個(gè)功能。常用方式
• createSystem() 通過(guò)系統(tǒng)參數(shù)創(chuàng)建連接器
• createMinimal() 創(chuàng)建默認(rèn)最小客戶端
• createMinimal(HttpClientConnectionManager) 創(chuàng)建默認(rèn)帶有鏈接池的客戶端

創(chuàng)建demo 1

PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
// 整個(gè)連接池最大連接數(shù)
cm.setMaxTotal(100);
// 每路由最大連接數(shù)，默認(rèn)值是2
cm.setDefaultMaxPerRoute(5);

CloseableHttpClient httpClient = HttpClients.custom().setConnectionManager(cm).disableAutomaticRetries().build();

創(chuàng)建demo 2

CloseableHttpClient httpClient1 = HttpClients.createDefault();

1.3、創(chuàng)建流程說(shuō)明

創(chuàng)建client 必備的要素

• ClientExecChain execChain
• HttpClientConnectionManager connManager
• HttpRoutePlanner routePlanner
• Lookup<CookieSpecProvider> cookieSpecRegistry
• Lookup<AuthSchemeProvider> authSchemeRegistry
• CookieStore cookieStore
• CredentialsProvider credentialsProvider
• RequestConfig defaultConfig
• List<Closeable> closeables

最佳實(shí)踐

1. 單線程測(cè)試場(chǎng)景

public class HttpClientPerformanceTest {
    // 測(cè)試參數(shù)配置
    private static final String TARGET_URL = "https://www.baidu.com";
    private static final int TOTAL_REQUESTS = 1; // 總請(qǐng)求次數(shù)
    private static final int WARMUP_ROUNDS = 10;   // 預(yù)熱輪次

    public static void main(String[] args) throws Exception {
        // 預(yù)熱（避免首次請(qǐng)求的初始化影響）
        // warmup();

        // 測(cè)試共用Client模式
        System.out.println("=== 測(cè)試共用HttpClient實(shí)例 ===");
        testSharedClient();

        // 測(cè)試每次新建Client模式
        System.out.println("\n=== 測(cè)試每次新建HttpClient ===");
        testNewClientEachTime();
    }

    private static void warmup() throws Exception {
        CloseableHttpClient client = HttpClients.createDefault();
        for (int i = 0; i < WARMUP_ROUNDS; i++) {
            executeRequest(client);
        }
    }

    private static void testSharedClient() throws Exception {
        long totalTime = 0;
        long startTime = System.currentTimeMillis();

        CloseableHttpClient client = HttpClients.createDefault();
        for (int i = 0; i < TOTAL_REQUESTS; i++) {
            long requestStart = System.currentTimeMillis();
            executeRequest(client);
            long duration = (System.currentTimeMillis() - requestStart); // 毫秒
            totalTime += duration;

            if ((i + 1) % 20 == 0) {
                System.out.printf("已完成 %d 次請(qǐng)求，平均耗時(shí) %.2f ms%n",
                        i + 1, (double) totalTime / (i + 1));
            }
        }

        long overallDuration = System.currentTimeMillis() - startTime;
        printResult("共用實(shí)例", TOTAL_REQUESTS, overallDuration, totalTime);
    }

    private static void testNewClientEachTime() throws Exception {
        long totalTime = 0;
        long startTime = System.currentTimeMillis();

        for (int i = 0; i < TOTAL_REQUESTS; i++) {
            long requestStart = System.currentTimeMillis();
            try (CloseableHttpClient client = HttpClients.createDefault()) {
                executeRequest(client);
            }
            long duration = (System.currentTimeMillis() - requestStart); // 毫秒
            totalTime += duration;

            if ((i + 1) % 10 == 0) {
                System.out.printf("已完成 %d 次請(qǐng)求，平均耗時(shí) %.2f ms%n",
                        i + 1, (double) totalTime / (i + 1));
            }
        }

        long overallDuration = System.currentTimeMillis() - startTime;
        printResult("新建實(shí)例", TOTAL_REQUESTS, overallDuration, totalTime);
    }

    private static void executeRequest(CloseableHttpClient client) throws Exception {
        HttpGet request = new HttpGet(TARGET_URL);
        CloseableHttpResponse response = client.execute(request);
        // 確保消費(fèi)實(shí)體內(nèi)容，才能釋放鏈接資產(chǎn)，鏈接不會(huì)斷開
        // EntityUtils.consume(response.getEntity());
        String result = parseHttpEntity(response.getEntity());
        // 關(guān)閉連接，即斷開tcp鏈接，釋放linux層面的鏈接資源，這樣的話，每次請(qǐng)求都需要重新建立鏈接，效率很低
        // response.close();
    }

    private static void printResult(String mode, int count, long totalDuration, long sumOfIndividual) {
        System.out.println("\n測(cè)試結(jié)果 [" + mode + "]:");
        System.out.println("總請(qǐng)求次數(shù): " + count);
        System.out.println("總耗時(shí): " + totalDuration + " ms");
        System.out.printf("平均每次請(qǐng)求耗時(shí): %.2f ms%n", (double) totalDuration / count);
        System.out.printf("平均網(wǎng)絡(luò)耗時(shí): %.2f ms%n", (double) sumOfIndividual / count);
        System.out.printf("吞吐量: %.2f 請(qǐng)求/秒%n",
                count / (totalDuration / 1000.0));
    }

    public static String parseHttpEntity(HttpEntity httpEntity, String charset) throws IOException {
        charset = charset == null ? "utf-8" : charset;
        String entityStr = null;
        if (httpEntity == null) {
            return entityStr;
        } else {
            entityStr = EntityUtils.toString(httpEntity, charset);
            return entityStr;
        }
    }

    public static String parseHttpEntity(HttpEntity httpEntity) throws IOException {
        return parseHttpEntity(httpEntity, "utf-8");
    }
}

通過(guò)多次對(duì)比看，共享實(shí)例在單線程時(shí)耗時(shí)在請(qǐng)求量大時(shí)，共享實(shí)例的效率更高；

2、多線程測(cè)試

package com.tianzehaoSpring;

import com.alibaba.fastjson.JSON;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.util.EntityUtils;

import java.io.IOException;
import java.util.*;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.FutureTask;

/**
 * Hello world!
 *
 */
public class HttpClientUtilThreadTest
{
    public static void main( String[] args ) throws InterruptedException {
        for(int i = 0; i < 1; i++){
            httpClientTest();
        }
    }


    private static void httpClientTest() throws InterruptedException {
        int maxThreadTotal = 80;
        // List<String> urls = Arrays.asList("https://www.baidu.com","https://cn.bing.com/");
        List<String> urls = Arrays.asList("http://10.1.106.177");
        // CloseableHttpClient httpClient = getHttpClient(maxThreadTotal,urls.size());

        Map<String,List<Long>> wasteTimeMapWithHttpClient = new HashMap<>();
        Map<String,List<Long>> wasteTimeMapNoHttpClient = new HashMap<>();
        long startTime = System.currentTimeMillis();

        CountDownLatch withShareClientCountDown = new CountDownLatch(maxThreadTotal);
        CountDownLatch withNoShareClientCountDown = new CountDownLatch(maxThreadTotal);
        for (String url : urls) {

            List<Long> times = getBlankList(maxThreadTotal);

            wasteTimeMapWithHttpClient.put(url, times);

            CloseableHttpClient httpClient = getHttpClient(maxThreadTotal,urls.size());
            for(int i = 0; i < maxThreadTotal; i++){
                int index = i;
                Thread thread = new Thread(()->{
                    long start = System.currentTimeMillis();
                    HttpGet httpGet = new HttpGet(url);
                    try (CloseableHttpResponse response = httpClient.execute(httpGet)) {
                        EntityUtils.consume(response.getEntity());
                    } catch (IOException e) {
                        throw new RuntimeException(e);
                    }
                    long wasteTime = System.currentTimeMillis() - start;
                    times.set(index, wasteTime);
                    withShareClientCountDown.countDown();
                });
                thread.setName("withShareClientCountDown" + i);
                thread.start();
            }
        }
        withShareClientCountDown.await();
        System.out.println("With HttpClient: " + (System.currentTimeMillis() - startTime) + "ms. " + "avg:" +  ((System.currentTimeMillis() - startTime)/maxThreadTotal) + "ms." );
        for (String url : urls) {
            List<Long> times = wasteTimeMapWithHttpClient.get(url);
            long totalTime = times.stream().filter(wasteTime-> wasteTime!=0).mapToLong(Long::longValue).sum();
            long avgTime = totalTime/times.size();
            System.out.println(JSON.toJSONString(times));
            System.out.println("With HttpClient: " + url + " avgTime: " + avgTime);
        }



        for (String url : urls) {
            List<Long> times = getBlankList(maxThreadTotal);
            wasteTimeMapNoHttpClient.put(url, times);

            for(int i = 0; i < maxThreadTotal; i++){
                int index = i;
                new Thread(()->{
                    long start = System.currentTimeMillis();
                    HttpGet httpGet = new HttpGet(url);
                    CloseableHttpClient httpClient = HttpClients.createDefault();
                    try (CloseableHttpResponse response = httpClient.execute(httpGet)) {
                        EntityUtils.consume(response.getEntity());
                    } catch (IOException e) {
                        throw new RuntimeException(e);
                    }
                    long wasteTime = System.currentTimeMillis() - start;
                    times.set(index, wasteTime);
                    withNoShareClientCountDown.countDown();
                }).start();
            }
        }
        withNoShareClientCountDown.await();
        System.out.println("no HttpClient: " + (System.currentTimeMillis() - startTime) + "ms. " + "avg:" +  ((System.currentTimeMillis() - startTime)/maxThreadTotal) + "ms." );
        for (String url : urls) {
            List<Long> times = wasteTimeMapNoHttpClient.get(url);
            long totalTime = times.stream().filter(wasteTime-> wasteTime!=0).mapToLong(Long::longValue).sum();
            long avgTime = totalTime/times.size();
            System.out.println(JSON.toJSONString(times));
            System.out.println("no HttpClient: " + url + " avgTime: " + avgTime);
        }
        System.out.println("   ");
    }
    private static List<Long> getBlankList(int size){
        List<Long> list = new ArrayList<>();
        for(int i = 0; i < size; i++){
            list.add(0L);
        }
        return list;
    }


    public static CloseableHttpClient getHttpClient(Integer maxThread,Integer maxRoute){

        int MAX_TOTAL_CONNECTIONS = maxThread * maxRoute * 2;
        int DEFAULT_MAX_PER_ROUTE = maxThread * 2;
        int CONNECT_TIMEOUT = 5000;  // 建立與目標(biāo)主機(jī)連接的最大時(shí)間 默認(rèn) -1不限制
        int SOCKET_TIMEOUT = 10000;  // 等待數(shù)據(jù)傳輸?shù)淖畲髸r(shí)間 默認(rèn) -1不限制
        int CONNECTION_REQUEST_TIMEOUT = 2000; // 從連接池中獲取連接的超時(shí)時(shí)間 默認(rèn) -1不限制
        PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();
        connectionManager.setMaxTotal(MAX_TOTAL_CONNECTIONS);
        connectionManager.setDefaultMaxPerRoute(DEFAULT_MAX_PER_ROUTE);

        // 配置全局請(qǐng)求參數(shù)
        RequestConfig requestConfig = RequestConfig.custom()
                // .setConnectTimeout(CONNECT_TIMEOUT)
                // .setSocketTimeout(SOCKET_TIMEOUT)
                // .setConnectionRequestTimeout(CONNECTION_REQUEST_TIMEOUT)
                .build();

        // 創(chuàng)建 HttpClient 實(shí)例
        return HttpClients.custom()
                .setConnectionManager(connectionManager)
                .setDefaultRequestConfig(requestConfig)
                .build();
    }
}

開啟多線程請(qǐng)求

結(jié)論： 不管是多線程，還是單線程，共享實(shí)例的效率更高，因?yàn)楣蚕韺?shí)例可以復(fù)用tcp鏈接，而每次創(chuàng)建實(shí)例則會(huì)重新建立tcp鏈接，效率較低。

常見問(wèn)題

1、內(nèi)存泄漏檢查

tcp 鏈接管理器會(huì)自動(dòng)關(guān)閉鏈接，httpclient對(duì)象在失去tcp鏈接引用后可被自動(dòng)回收；

2、共享實(shí)例阻塞人問(wèn)題

response 不消費(fèi)，會(huì)導(dǎo)致鏈接無(wú)法釋放；無(wú)法再發(fā)起請(qǐng)求；

3、每次創(chuàng)建實(shí)例請(qǐng)求耗時(shí)高

共享實(shí)例測(cè)試時(shí) 關(guān)閉了鏈接，導(dǎo)致共享實(shí)例與每次創(chuàng)建實(shí)例測(cè)試結(jié)果相同，因?yàn)楣蚕韺?shí)例測(cè)試時(shí)，關(guān)閉了鏈接

// 執(zhí)行該語(yǔ)句，會(huì)關(guān)閉鏈接，即斷開tcp鏈接，釋放linux層面的鏈接資源，這樣的話，每次請(qǐng)求都需要重新建立鏈接，效率很低。
response.close();

附錄: httpClientUtil

import org.apache.http.HttpEntity;
import org.apache.http.NameValuePair;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.methods.HttpRequestBase;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.util.EntityUtils;

import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;

public class HttpClientUtil {

    private static final String DEFAULT_CHARSET = "UTF-8";
    private static final int CONNECT_TIMEOUT = 5000; // 連接超時(shí)時(shí)間(毫秒)
    private static final int SOCKET_TIMEOUT = 5000;  // 讀取數(shù)據(jù)超時(shí)時(shí)間(毫秒)

    /**
     * 發(fā)送GET請(qǐng)求
     * @param url 請(qǐng)求地址
     * @param params 請(qǐng)求參數(shù)
     * @return 響應(yīng)內(nèi)容
     */
    public static String doGet(String url, Map<String, String> params) {
        return doGet(null, url, params, null, DEFAULT_CHARSET);
    }

    /**
     * 發(fā)送GET請(qǐng)求
     * @param httpClient 可選的HttpClient實(shí)例
     * @param url 請(qǐng)求地址
     * @param params 請(qǐng)求參數(shù)
     * @param headers 自定義請(qǐng)求頭
     * @param charset 字符編碼
     * @return 響應(yīng)內(nèi)容
     */
    public static String doGet(CloseableHttpClient httpClient, String url,
                               Map<String, String> params, Map<String, String> headers,
                               String charset) {
        // 構(gòu)建帶參數(shù)的URL
        if (params != null && !params.isEmpty()) {
            url = buildUrlWithParams(url, params);
        }

        HttpGet httpGet = new HttpGet(url);


        // 設(shè)置自定義Header
        if (headers != null && !headers.isEmpty()) {
            setHeaders(httpGet, headers);
        }

        boolean isShareHttpClient = httpClient != null;
        // 使用傳入的HttpClient或創(chuàng)建新的
        if (httpClient == null) {
            httpClient = HttpClients.createDefault();
        }

        try {
            CloseableHttpResponse response = httpClient.execute(httpGet)
            HttpEntity entity = response.getEntity();
            if (entity != null) {
                return EntityUtils.toString(entity, charset);
            }
            return "";
        } catch (IOException e) {
            throw new RuntimeException("HTTP GET請(qǐng)求失敗: " + e.getMessage(), e);
        }finally {
            if(!isShareHttpClient){
                //非共享HttpClient，則關(guān)閉
                try {
                    httpClient.close();
                } catch (IOException e) {
                    // 忽略關(guān)閉異常
                }
            }
        }
    }

    /**
     * 發(fā)送POST請(qǐng)求(表單形式)
     * @param url 請(qǐng)求地址
     * @param params 請(qǐng)求參數(shù)
     * @return 響應(yīng)內(nèi)容
     */
    public static String doPost(String url, Map<String, String> params) {
        return doPost(null, url, params, null, DEFAULT_CHARSET);
    }

    /**
     * 發(fā)送POST請(qǐng)求(表單形式)
     * @param httpClient 可選的HttpClient實(shí)例
     * @param url 請(qǐng)求地址
     * @param params 請(qǐng)求參數(shù)
     * @param headers 自定義請(qǐng)求頭
     * @param charset 字符編碼
     * @return 響應(yīng)內(nèi)容
     */
    public static String doPost(CloseableHttpClient httpClient, String url,
                                Map<String, String> params, Map<String, String> headers,
                                String charset) {
        HttpPost httpPost = new HttpPost(url);

        // 設(shè)置自定義Header
        if (headers != null && !headers.isEmpty()) {
            setHeaders(httpPost, headers);
        }

        // 設(shè)置表單參數(shù)
        if (params != null && !params.isEmpty()) {
            try {
                List<NameValuePair> paramList = new ArrayList<>();
                for (Map.Entry<String, String> entry : params.entrySet()) {
                    paramList.add(new BasicNameValuePair(entry.getKey(), entry.getValue()));
                }
                httpPost.setEntity(new UrlEncodedFormEntity(paramList, charset));
            } catch (UnsupportedEncodingException e) {
                throw new RuntimeException("不支持的編碼格式: " + charset, e);
            }
        }

        boolean isShareHttpClient = httpClient != null;
        // 使用傳入的HttpClient或創(chuàng)建新的
        if (httpClient == null) {
            httpClient = HttpClients.createDefault();
        }

        try {
            CloseableHttpResponse response = httpClient.execute(httpPost)
            HttpEntity entity = response.getEntity();
            if (entity != null) {
                return EntityUtils.toString(entity, charset);
            }
            return "";
        } catch (IOException e) {
            throw new RuntimeException("HTTP POST請(qǐng)求失敗: " + e.getMessage(), e);
        }
        finally {
            if(!isShareHttpClient){
                //非共享HttpClient，則關(guān)閉
                try {
                    httpClient.close();
                } catch (IOException e) {
                    // 忽略關(guān)閉異常
                }
            }
        }
    }

    /**
     * 發(fā)送POST請(qǐng)求(JSON形式)
     * @param url 請(qǐng)求地址
     * @param json JSON字符串
     * @return 響應(yīng)內(nèi)容
     */
    public static String doPostJson(String url, String json) {
        return doPostJson(null, url, json, null, DEFAULT_CHARSET);
    }

    /**
     * 發(fā)送POST請(qǐng)求(JSON形式)
     * @param httpClient 可選的HttpClient實(shí)例
     * @param url 請(qǐng)求地址
     * @param json JSON字符串
     * @param headers 自定義請(qǐng)求頭
     * @param charset 字符編碼
     * @return 響應(yīng)內(nèi)容
     */
    public static String doPostJson(CloseableHttpClient httpClient, String url,
                                    String json, Map<String, String> headers,
                                    String charset) {
        HttpPost httpPost = new HttpPost(url);
        httpPost.setHeader("Content-Type", "application/json");

        // 設(shè)置自定義Header
        if (headers != null && !headers.isEmpty()) {
            setHeaders(httpPost, headers);
        }

        // 設(shè)置JSON請(qǐng)求體
        if (json != null) {
            httpPost.setEntity(new StringEntity(json, charset));
        }

        boolean isShareHttpClient = httpClient != null;
        // 使用傳入的HttpClient或創(chuàng)建新的
        if (httpClient == null) {
            httpClient = HttpClients.createDefault();
        }

        try (CloseableHttpResponse response = httpClient.execute(httpPost)) {
            HttpEntity entity = response.getEntity();
            if (entity != null) {
                return EntityUtils.toString(entity, charset);
            }
            return "";
        } catch (IOException e) {
            throw new RuntimeException("HTTP POST JSON請(qǐng)求失敗: " + e.getMessage(), e);
        }
        finally {
            if(!isShareHttpClient){
                //非共享HttpClient，則關(guān)閉
                try {
                    httpClient.close();
                } catch (IOException e) {
                    // 忽略關(guān)閉異常
                }
            }
        }
    }

    /**
     * 構(gòu)建帶參數(shù)的URL
     */
    private static String buildUrlWithParams(String url, Map<String, String> params) {
        StringBuilder paramStr = new StringBuilder();
        boolean first = true;
        for (Map.Entry<String, String> entry : params.entrySet()) {
            if (first) {
                first = false;
                paramStr.append("?");
            } else {
                paramStr.append("&");
            }
            paramStr.append(entry.getKey()).append("=").append(entry.getValue());
        }
        return url + paramStr.toString();
    }

    /**
     * 設(shè)置請(qǐng)求頭
     */
    private static void setHeaders(HttpRequestBase request, Map<String, String> headers) {
        for (Map.Entry<String, String> entry : headers.entrySet()) {
            request.setHeader(entry.getKey(), entry.getValue());
        }
    }
}