詳解proxy_pass、upstream與resolver

應(yīng)用場(chǎng)景

這里列舉幾個(gè)應(yīng)用場(chǎng)景，下文會(huì)針對(duì)這幾個(gè)場(chǎng)景并結(jié)合代碼進(jìn)行分析。

  1. proxy_pass + upstream
    upstream foo.example.com {
        server 127.0.0.1:8001;
    }

    server {
        listen       80;
        server_name  localhost;

        location /foo {
            proxy_pass http://foo.example.com;
        }
    }

訪問(wèn)http://localhost/foo，proxy模塊會(huì)將請(qǐng)求轉(zhuǎn)發(fā)到127.0.0.1的8001端口上。

  1. 只有proxy_pass，沒(méi)有upstream與resolver
    server {
        listen       80;
        server_name  localhost;

        location /foo {
            proxy_pass http://foo.example.com;
        }
    }

實(shí)際上是隱式創(chuàng)建了upstream，upstream名字就是foo.example.com。upstream模塊利用本機(jī)設(shè)置的DNS服務(wù)器（或/etc/hosts），將foo.example.com解析成IP，訪問(wèn)http://localhost/foo，proxy模塊會(huì)將請(qǐng)求轉(zhuǎn)發(fā)到解析后的IP上。

如果本機(jī)未設(shè)置DNS服務(wù)器，或者DNS服務(wù)器無(wú)法解析域名，則nginx啟動(dòng)時(shí)會(huì)報(bào)類似如下錯(cuò)誤：

nginx: [emerg] host not found in upstream "foo.example.com" in /path/nginx/conf/nginx.conf:110

  1. proxy_pass + resolver（變量設(shè)置域名）
    server {
        listen       80;
        server_name  localhost;

        resolver 114.114.114.114;
        location /foo {
            set $foo foo.example.com;
            proxy_pass http://$foo;
        }
    }

訪問(wèn)http://localhost/foo，nginx會(huì)動(dòng)態(tài)利用resolver設(shè)置的DNS服務(wù)器（本機(jī)設(shè)置的DNS服務(wù)器或/etc/hosts無(wú)效），將域名解析成IP，proxy模塊會(huì)將請(qǐng)求轉(zhuǎn)發(fā)到解析后的IP上。

  1. proxy_pass + upstream（顯式） + resolver（變量設(shè)置域名）
    upstream foo.example.com {
        server 127.0.0.1:8001;
    }

    server {
        listen       80;
        server_name  localhost;

        resolver 114.114.114.114;
        location /foo {
            set $foo foo.example.com;
            proxy_pass http://$foo;
        }
    }

訪問(wèn)http://localhost/foo時(shí)，upstream模塊會(huì)優(yōu)先查找是否有定義upstream后端服務(wù)器，如果有定義則直接利用，不再走DNS解析。所以proxy模塊會(huì)將請(qǐng)求轉(zhuǎn)發(fā)到127.0.0.1的8001端口上。

  1. proxy_pass + upstream（隱式） + resolver（變量設(shè)置域名）
    server {
        listen       80;
        server_name  localhost;

        resolver 114.114.114.114;
        location /foo {
            set $foo foo.example.com;
            proxy_pass http://$foo;
        }

        location /foo2 {
            proxy_pass http://foo.example.com;
        }
    }

location /foo2實(shí)際上是隱式定義了upstream foo.example.com，并由本地DNS服務(wù)器進(jìn)行了域名解析，訪問(wèn)http://localhost/foo時(shí)，upstream模塊會(huì)優(yōu)先查找upstream，即隱式定義的foo.example.com，proxy模塊會(huì)將請(qǐng)求轉(zhuǎn)發(fā)到解析后的IP上。

  1. proxy_pass + resolver（不用變量設(shè)置域名）
    server {
        listen       80;
        server_name  localhost;

        resolver 114.114.114.114;
        location /foo {
            proxy_pass http://foo.example.com;
        }
    }

不使用變量設(shè)置域名，則resolver的設(shè)置不起作用，此時(shí)相當(dāng)于場(chǎng)景2，只有proxy_pass的場(chǎng)景。

  1. proxy_pass + upstream + resolver（不用變量設(shè)置域名）
    upstream foo.example.com {
        server 127.0.0.1:8001;
    }

    server {
        listen       80;
        server_name  localhost;

        resolver 114.114.114.114;
        location /foo {
            proxy_pass http://foo.example.com;
        }
    }

不使用變量設(shè)置域名，則resolver的設(shè)置不起作用，此時(shí)相當(dāng)于場(chǎng)景1 proxy_pass + upstream。

  1. proxy_pass 直接指定IP加端口號(hào)
    server {
        listen       80;
        server_name  localhost;

        location /foo {
            proxy_pass http://127.0.0.1:8001/;
        }
    }

實(shí)際上是隱式創(chuàng)建了upstream，proxy_pass會(huì)將請(qǐng)求轉(zhuǎn)發(fā)到127.0.0.1的8001端口上。

主要代碼

解析proxy_pass指令的代碼：

static char *
ngx_http_proxy_pass(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
{
    ngx_http_proxy_loc_conf_t *plcf = conf;

    size_t                      add;
    u_short                     port;
    ngx_str_t                  *value, *url;
    ngx_url_t                   u;
    ngx_uint_t                  n;
    ngx_http_core_loc_conf_t   *clcf;
    ngx_http_script_compile_t   sc;

    if (plcf->upstream.upstream || plcf->proxy_lengths) {
        return "is duplicate";
    }

    clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module);

    clcf->handler = ngx_http_proxy_handler;

    if (clcf->name.data[clcf->name.len - 1] == '/') {
        clcf->auto_redirect = 1;
    }

    value = cf->args->elts;

    url = &value[1];

    /* 查找指令中$符號(hào)的位置，判斷是否使用了變量 */
    n = ngx_http_script_variables_count(url);

    if (n) {
        /* 使用變量設(shè)置域名 */
        ngx_memzero(&sc, sizeof(ngx_http_script_compile_t));

        sc.cf = cf;
        sc.source = url;
        sc.lengths = &plcf->proxy_lengths;
        sc.values = &plcf->proxy_values;
        sc.variables = n;
        sc.complete_lengths = 1;
        sc.complete_values = 1;

        if (ngx_http_script_compile(&sc) != NGX_OK) {
            return NGX_CONF_ERROR;
        }

#if (NGX_HTTP_SSL)
        plcf->ssl = 1;
#endif

        return NGX_CONF_OK;
    }

    if (ngx_strncasecmp(url->data, (u_char *) "http://", 7) == 0) {
        add = 7;
        port = 80;

    } else if (ngx_strncasecmp(url->data, (u_char *) "https://", 8) == 0) {

#if (NGX_HTTP_SSL)
        plcf->ssl = 1;

        add = 8;
        port = 443;
#else
        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                           "https protocol requires SSL support");
        return NGX_CONF_ERROR;
#endif

    } else {
        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid URL prefix");
        return NGX_CONF_ERROR;
    }

    ngx_memzero(&u, sizeof(ngx_url_t));

    u.url.len = url->len - add;
    u.url.data = url->data + add;
    u.default_port = port;
    u.uri_part = 1;
    u.no_resolve = 1;

    plcf->upstream.upstream = ngx_http_upstream_add(cf, &u, 0);
    if (plcf->upstream.upstream == NULL) {
        return NGX_CONF_ERROR;
    }

    plcf->vars.schema.len = add;
    plcf->vars.schema.data = url->data;
    plcf->vars.key_start = plcf->vars.schema;

    ngx_http_proxy_set_vars(&u, &plcf->vars);

    plcf->location = clcf->name;

    if (clcf->named
#if (NGX_PCRE)
        || clcf->regex
#endif
        || clcf->noname)
    {
        if (plcf->vars.uri.len) {
            ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                               "\"proxy_pass\" cannot have URI part in "
                               "location given by regular expression, "
                               "or inside named location, "
                               "or inside \"if\" statement, "
                               "or inside \"limit_except\" block");
            return NGX_CONF_ERROR;
        }

        plcf->location.len = 0;
    }

    plcf->url = *url;

    return NGX_CONF_OK;
}

upstream定義的后端服務(wù)器的處理邏輯，包括顯式定義的和隱式定義的。隱式定義，即proxy_pass指定的后端服務(wù)器的地址沒(méi)有顯式用upstream定義，nginx內(nèi)部會(huì)定義一個(gè)。

ngx_int_t
ngx_http_upstream_init_round_robin(ngx_conf_t *cf,
    ngx_http_upstream_srv_conf_t *us)
{
    ngx_url_t                      u;
    ngx_uint_t                     i, j, n, w;
    ngx_http_upstream_server_t    *server;
    ngx_http_upstream_rr_peer_t   *peer, **peerp;
    ngx_http_upstream_rr_peers_t  *peers, *backup;

    us->peer.init = ngx_http_upstream_init_round_robin_peer;

    /* 
     * 使用upstream指令顯式定義upstream 
     * 或者proxy_pass直接指定IP的場(chǎng)景
     */
    if (us->servers) {
        server = us->servers->elts;

        n = 0;
        w = 0;

        for (i = 0; i < us->servers->nelts; i++) {
            if (server[i].backup) {
                continue;
            }

            n += server[i].naddrs;
            w += server[i].naddrs * server[i].weight;
        }

        if (n == 0) {
            ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                          "no servers in upstream \"%V\" in %s:%ui",
                          &us->host, us->file_name, us->line);
            return NGX_ERROR;
        }

        peers = ngx_pcalloc(cf->pool, sizeof(ngx_http_upstream_rr_peers_t));
        if (peers == NULL) {
            return NGX_ERROR;
        }

        peer = ngx_pcalloc(cf->pool, sizeof(ngx_http_upstream_rr_peer_t) * n);
        if (peer == NULL) {
            return NGX_ERROR;
        }

        peers->single = (n == 1);
        peers->number = n;
        peers->weighted = (w != n);
        peers->total_weight = w;
        peers->name = &us->host;

        n = 0;
        peerp = &peers->peer;

        for (i = 0; i < us->servers->nelts; i++) {
            /* 設(shè)置sockaddr、weight、max_fails、fail_timeout等屬性 */
        }

        us->peer.data = peers;

        /* 處理backup servers相關(guān)邏輯 */

        return NGX_OK;
    }

    /* 未使用upstream指令，使用proxy_pass隱式定義upstream */
    /* an upstream implicitly defined by proxy_pass, etc. */

    if (us->port == 0) {
        ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                      "no port in upstream \"%V\" in %s:%ui",
                      &us->host, us->file_name, us->line);
        return NGX_ERROR;
    }

    ngx_memzero(&u, sizeof(ngx_url_t));

    u.host = us->host;
    u.port = us->port;

    if (ngx_inet_resolve_host(cf->pool, &u) != NGX_OK) {
        if (u.err) {
            ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                          "%s in upstream \"%V\" in %s:%ui",
                          u.err, &us->host, us->file_name, us->line);
        }

        return NGX_ERROR;
    }

    n = u.naddrs;

    peers = ngx_pcalloc(cf->pool, sizeof(ngx_http_upstream_rr_peers_t));
    if (peers == NULL) {
        return NGX_ERROR;
    }

    peer = ngx_pcalloc(cf->pool, sizeof(ngx_http_upstream_rr_peer_t) * n);
    if (peer == NULL) {
        return NGX_ERROR;
    }

    peers->single = (n == 1);
    peers->number = n;
    peers->weighted = 0;
    peers->total_weight = n;
    peers->name = &us->host;

    peerp = &peers->peer;

    for (i = 0; i < u.naddrs; i++) {
        /* 設(shè)置sockaddr、weight、max_fails、fail_timeout等屬性 */
    }

    us->peer.data = peers;

    /* implicitly defined upstream has no backup servers */

    return NGX_OK;
}

upstream模塊初始化請(qǐng)求時(shí)的邏輯：

static void
ngx_http_upstream_init_request(ngx_http_request_t *r)
{
    ngx_str_t                      *host;
    ngx_uint_t                      i;
    ngx_resolver_ctx_t             *ctx, temp;
    ngx_http_cleanup_t             *cln;
    ngx_http_upstream_t            *u;
    ngx_http_core_loc_conf_t       *clcf;
    ngx_http_upstream_srv_conf_t   *uscf, **uscfp;
    ngx_http_upstream_main_conf_t  *umcf;

    if (r->aio) {
        return;
    }

    u = r->upstream;

    /* NGX_HTTP_CACHE 等其他處理 */

    cln->handler = ngx_http_upstream_cleanup;
    cln->data = r;
    u->cleanup = &cln->handler;

    if (u->resolved == NULL) {
        /* 如果沒(méi)有使用resolver設(shè)置DNS，直接取upstream的設(shè)置 */
        uscf = u->conf->upstream;

    } else {

#if (NGX_HTTP_SSL)
        u->ssl_name = u->resolved->host;
#endif

        host = &u->resolved->host;

        if (u->resolved->sockaddr) {

            if (u->resolved->port == 0
                && u->resolved->sockaddr->sa_family != AF_UNIX)
            {
                ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                              "no port in upstream \"%V\"", host);
                ngx_http_upstream_finalize_request(r, u,
                                               NGX_HTTP_INTERNAL_SERVER_ERROR);
                return;
            }

            if (ngx_http_upstream_create_round_robin_peer(r, u->resolved)
                != NGX_OK)
            {
                ngx_http_upstream_finalize_request(r, u,
                                               NGX_HTTP_INTERNAL_SERVER_ERROR);
                return;
            }

            ngx_http_upstream_connect(r, u);

            return;
        }

        umcf = ngx_http_get_module_main_conf(r, ngx_http_upstream_module);

        uscfp = umcf->upstreams.elts;

        /* 在顯式/隱式定義的upstream中查找 */
        for (i = 0; i < umcf->upstreams.nelts; i++) {

            uscf = uscfp[i];

            if (uscf->host.len == host->len
                && ((uscf->port == 0 && u->resolved->no_port)
                     || uscf->port == u->resolved->port)
                && ngx_strncasecmp(uscf->host.data, host->data, host->len) == 0)
            {
                goto found;
            }
        }

        if (u->resolved->port == 0) {
            ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                          "no port in upstream \"%V\"", host);
            ngx_http_upstream_finalize_request(r, u,
                                               NGX_HTTP_INTERNAL_SERVER_ERROR);
            return;
        }

        temp.name = *host;

        ctx = ngx_resolve_start(clcf->resolver, &temp);
        if (ctx == NULL) {
            ngx_http_upstream_finalize_request(r, u,
                                               NGX_HTTP_INTERNAL_SERVER_ERROR);
            return;
        }

        if (ctx == NGX_NO_RESOLVER) {
            ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                          "no resolver defined to resolve %V", host);

            ngx_http_upstream_finalize_request(r, u, NGX_HTTP_BAD_GATEWAY);
            return;
        }

        ctx->name = *host;
        ctx->handler = ngx_http_upstream_resolve_handler;
        ctx->data = r;
        ctx->timeout = clcf->resolver_timeout;

        u->resolved->ctx = ctx;

        if (ngx_resolve_name(ctx) != NGX_OK) {
            u->resolved->ctx = NULL;
            ngx_http_upstream_finalize_request(r, u,
                                               NGX_HTTP_INTERNAL_SERVER_ERROR);
            return;
        }

        return;
    }

found:

    if (uscf == NULL) {
        ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0,
                      "no upstream configuration");
        ngx_http_upstream_finalize_request(r, u,
                                           NGX_HTTP_INTERNAL_SERVER_ERROR);
        return;
    }

#if (NGX_HTTP_SSL)
    u->ssl_name = uscf->host;
#endif

    if (uscf->peer.init(r, uscf) != NGX_OK) {
        ngx_http_upstream_finalize_request(r, u,
                                           NGX_HTTP_INTERNAL_SERVER_ERROR);
        return;
    }

    u->peer.start_time = ngx_current_msec;

    if (u->conf->next_upstream_tries
        && u->peer.tries > u->conf->next_upstream_tries)
    {
        u->peer.tries = u->conf->next_upstream_tries;
    }

    ngx_http_upstream_connect(r, u);
}

詳細(xì)分析

場(chǎng)景1

解析proxy_pass的函數(shù)ngx_http_proxy_pass中，沒(méi)有找到$符號(hào)（即，變量設(shè)置域名），走ngx_http_proxy_pass后半部分的處理邏輯。ngx_http_upstream_init_round_robin初始化upstream時(shí)，走顯式定義upstream的邏輯。proxy_pass轉(zhuǎn)發(fā)請(qǐng)求初始化時(shí)，ngx_http_upstream_init_request中直接使用upstream中的后端server建立連接。

場(chǎng)景2

ngx_http_upstream_init_round_robin初始化upstream時(shí)，走隱式定義upstream的邏輯，會(huì)調(diào)用ngx_inet_resolve_host對(duì)proxy_pass中的域名進(jìn)行解析，設(shè)置upstream。proxy_pass轉(zhuǎn)發(fā)請(qǐng)求初始化時(shí)，ngx_http_upstream_init_request中直接使用upstream中的設(shè)置，也就是利用本地設(shè)置的DNS服務(wù)器解析出的IP，建立連接。

場(chǎng)景3

解析proxy_pass指令時(shí)，找到了$符號(hào)，設(shè)置ngx_http_script_compile_t，并利用ngx_http_script_compile進(jìn)行編譯，不走后半部分邏輯。配置文件沒(méi)有顯式/隱式定義upstream，所以不會(huì)調(diào)用ngx_http_upstream_init_round_robin方法。proxy_pass轉(zhuǎn)發(fā)請(qǐng)求初始化時(shí)，ngx_http_upstream_init_request中發(fā)現(xiàn)沒(méi)有顯式也沒(méi)有隱式定義的upstream，隨后調(diào)用ngx_resolve_start，對(duì)域名進(jìn)行解析，之后將請(qǐng)求轉(zhuǎn)發(fā)過(guò)去。

場(chǎng)景4

解析proxy_pass指令時(shí)，找到了$符號(hào)，設(shè)置ngx_http_script_compile_t，并利用ngx_http_script_compile進(jìn)行編譯，不走后半部分邏輯。顯式調(diào)用了upstream，所以調(diào)用ngx_http_upstream_init_round_robin方法中的顯式upstream的處理邏輯。proxy_pass轉(zhuǎn)發(fā)請(qǐng)求初始化時(shí)，ngx_http_upstream_init_request中優(yōu)先查找upstream，如果找到了，直接將請(qǐng)求轉(zhuǎn)發(fā)到upstream中的后端server上。如果upstream中沒(méi)有找到，則對(duì)域名進(jìn)行解析，然后將請(qǐng)求轉(zhuǎn)發(fā)到解析后的IP上。

場(chǎng)景5

基本與場(chǎng)景4相同，不同之處在于調(diào)用ngx_http_upstream_init_round_robin方法時(shí)，走隱式upstream部分的處理邏輯。

場(chǎng)景6

與場(chǎng)景2相同。

場(chǎng)景7

與場(chǎng)景1相同。

場(chǎng)景8

實(shí)際上是隱式創(chuàng)建了upstream，但是因?yàn)閜roxy_pass中指定了IP和端口號(hào)，所以ngx_http_upstream_init_round_robin初始化upstream時(shí)，us->servers不為空，所以走該函數(shù)的上半部分邏輯。與場(chǎng)景1有些類似。

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡(jiǎn)書(shū)系信息發(fā)布平臺(tái)，僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容