6.1-Bucket&Metric聚合分析及嵌套聚合

Bucket & Metric Aggregation

image.png

  • Metric - ?些系列的統(tǒng)計(jì)?法

  • Bucket - ?組滿?條件的?檔

Aggregation 的語(yǔ)法

  • Aggregation 屬于Search的一部分。一般情況下，建議將其 Size 指定為0
image.png

?個(gè)例?：?資統(tǒng)計(jì)信息

# 多個(gè) Metric 聚合，找到最低最高和平均工資
POST employees/_search
{
  "size": 0,
  "aggs": {
    "max_salary": {
      "max": {
        "field": "salary"
      }
    },
    "min_salary": {
      "min": {
        "field": "salary"
      }
    },
    "avg_salary": {
      "avg": {
        "field": "salary"
      }
    }
  }
}

res:
{
  "took" : 64,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 20,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "max_salary" : {
      "value" : 50000.0
    },
    "avg_salary" : {
      "value" : 24700.0
    },
    "min_salary" : {
      "value" : 9000.0
    }
  }
}

Metric Aggregation

  • 單值分析：只輸出?個(gè)分析結(jié)果

    • min, max, avg, sum

    • Cardinality （類似 distinct Count）

  • 多值分析：輸出多個(gè)分析結(jié)果

    • stats, extended stats

    • percentile, percentile rank

    • top hits （排在前?的示例）

Metric 聚合的具體 Demo

  • 查看最低?資

  • 查看最??資

  • ?個(gè)聚合輸出多個(gè)值

  • ?次查詢包含多個(gè)聚合

    • 同時(shí)查看最低，最?和平均?資

Bucket

image.png

  • 按照?定的規(guī)則，將?檔分配到不同的 桶中，從?達(dá)到分類的?的。ES 提供的 ?些常?的 Bucket Aggregation

    • Terms

    • 數(shù)字類型

    • Range / Data Range

    • Histogram / Date Histogram

    • ?持嵌套：也就在桶?再做分桶

Terms Aggregation

  • 字段需要打開(kāi) fielddata，才能進(jìn)? Terms Aggregation

    • Keyword 默認(rèn)?持 doc_values

    • Text 需要在 Mapping 中 enable。會(huì)按照分詞后的結(jié)果進(jìn)?分

  • Demo

    • 對(duì) job 和 job.keyword 進(jìn)?聚合

    • 對(duì)性別進(jìn)? Terms 聚合

    • 指定 bucket size

Cardinality

  • 類似 SQL 中的 Distinct
POST employees/_search
{
  "size": 0,
  "aggs": {
    "cardinate": {
      "cardinality": {
        "field": "job.keyword"
      }
    }
  }
}

res:
{
  "took" : 133,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 20,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "cardinate" : {
      "value" : 7
    }
  }
}

Bucket Size & Top Hits Demo

  • 應(yīng)?場(chǎng)景：當(dāng)獲取分桶后，桶內(nèi)最匹配的頂部?檔列表

  • Size：按年齡分桶，找出指定數(shù)據(jù)量的分桶信息

  • Top Hits：查看各個(gè)?種中，年紀(jì)最?的 3 名員?

優(yōu)化 Terms 聚合的性能

image.png

Range & Histogram 聚合

  • 按照數(shù)字的范圍，進(jìn)?分桶

  • 在 Range Aggregation 中，可以?定義 Key

  • Demo：

    • 按照?資的 Range 分桶

    • 按照?資的間隔（Histogram）分桶

Bucket + Metric Aggregation

  • Bucket 聚合分析允許通過(guò)添加?聚合分析來(lái)進(jìn)?步分析，?聚合分析可以是

    • Bucket

    • Metric

  • Demo

    • 按照?作類型進(jìn)?分桶，并統(tǒng)計(jì)?資信息

    • 先按照?作類型分桶，然后按性別分桶，并統(tǒng)計(jì)?資信息

本節(jié)知識(shí)點(diǎn)

  • 聚合分析的具體語(yǔ)法

    • ?個(gè)聚合查詢中可以包含多個(gè)聚合； 每個(gè) Bucket 聚合可以包含?聚合

  • Metrix

    • 單值輸出 & 多值輸出

  • Bucket

    • Terms & 數(shù)字范圍

demos

DELETE /employees
PUT /employees/
{
  "mappings" : {
      "properties" : {
        "age" : {
          "type" : "integer"
        },
        "gender" : {
          "type" : "keyword"
        },
        "job" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 50
            }
          }
        },
        "name" : {
          "type" : "keyword"
        },
        "salary" : {
          "type" : "integer"
        }
      }
    }
}

PUT /employees/_bulk
{ "index" : {  "_id" : "1" } }
{ "name" : "Emma","age":32,"job":"Product Manager","gender":"female","salary":35000 }
{ "index" : {  "_id" : "2" } }
{ "name" : "Underwood","age":41,"job":"Dev Manager","gender":"male","salary": 50000}
{ "index" : {  "_id" : "3" } }
{ "name" : "Tran","age":25,"job":"Web Designer","gender":"male","salary":18000 }
{ "index" : {  "_id" : "4" } }
{ "name" : "Rivera","age":26,"job":"Web Designer","gender":"female","salary": 22000}
{ "index" : {  "_id" : "5" } }
{ "name" : "Rose","age":25,"job":"QA","gender":"female","salary":18000 }
{ "index" : {  "_id" : "6" } }
{ "name" : "Lucy","age":31,"job":"QA","gender":"female","salary": 25000}
{ "index" : {  "_id" : "7" } }
{ "name" : "Byrd","age":27,"job":"QA","gender":"male","salary":20000 }
{ "index" : {  "_id" : "8" } }
{ "name" : "Foster","age":27,"job":"Java Programmer","gender":"male","salary": 20000}
{ "index" : {  "_id" : "9" } }
{ "name" : "Gregory","age":32,"job":"Java Programmer","gender":"male","salary":22000 }
{ "index" : {  "_id" : "10" } }
{ "name" : "Bryant","age":20,"job":"Java Programmer","gender":"male","salary": 9000}
{ "index" : {  "_id" : "11" } }
{ "name" : "Jenny","age":36,"job":"Java Programmer","gender":"female","salary":38000 }
{ "index" : {  "_id" : "12" } }
{ "name" : "Mcdonald","age":31,"job":"Java Programmer","gender":"male","salary": 32000}
{ "index" : {  "_id" : "13" } }
{ "name" : "Jonthna","age":30,"job":"Java Programmer","gender":"female","salary":30000 }
{ "index" : {  "_id" : "14" } }
{ "name" : "Marshall","age":32,"job":"Javascript Programmer","gender":"male","salary": 25000}
{ "index" : {  "_id" : "15" } }
{ "name" : "King","age":33,"job":"Java Programmer","gender":"male","salary":28000 }
{ "index" : {  "_id" : "16" } }
{ "name" : "Mccarthy","age":21,"job":"Javascript Programmer","gender":"male","salary": 16000}
{ "index" : {  "_id" : "17" } }
{ "name" : "Goodwin","age":25,"job":"Javascript Programmer","gender":"male","salary": 16000}
{ "index" : {  "_id" : "18" } }
{ "name" : "Catherine","age":29,"job":"Javascript Programmer","gender":"female","salary": 20000}
{ "index" : {  "_id" : "19" } }
{ "name" : "Boone","age":30,"job":"DBA","gender":"male","salary": 30000}
{ "index" : {  "_id" : "20" } }
{ "name" : "Kathy","age":29,"job":"DBA","gender":"female","salary": 20000}

# Metric 聚合，找到最低的工資
POST employees/_search
{
  "size": 0,
  "aggs": {
    "min_salary": {
      "min": {
        "field":"salary"
      }
    }
  }
}

# Metric 聚合，找到最高的工資
POST employees/_search
{
  "size": 0,
  "aggs": {
    "max_salary": {
      "max": {
        "field":"salary"
      }
    }
  }
}

# 多個(gè) Metric 聚合，找到最低最高和平均工資
POST employees/_search
{
  "size": 0,
  "aggs": {
    "max_salary": {
      "max": {
        "field": "salary"
      }
    },
    "min_salary": {
      "min": {
        "field": "salary"
      }
    },
    "avg_salary": {
      "avg": {
        "field": "salary"
      }
    }
  }
}

# 一個(gè)聚合，輸出多值
POST employees/_search
{
  "size": 0,
  "aggs": {
    "stats_salary": {
      "stats": {
        "field":"salary"
      }
    }
  }
}




# 對(duì)keword 進(jìn)行聚合
POST employees/_search
{
  "size": 0,
  "aggs": {
    "jobs": {
      "terms": {
        "field":"job.keyword"
      }
    }
  }
}


# 對(duì) Text 字段進(jìn)行 terms 聚合查詢，失敗
POST employees/_search
{
  "size": 0,
  "aggs": {
    "jobs": {
      "terms": {
        "field":"job"
      }
    }
  }
}

# 對(duì) Text 字段打開(kāi) fielddata，支持terms aggregation
PUT employees/_mapping
{
  "properties" : {
    "job":{
       "type":     "text",
       "fielddata": true
    }
  }
}


# 對(duì) Text 字段進(jìn)行 terms 分詞。分詞后的terms
POST employees/_search
{
  "size": 0,
  "aggs": {
    "jobs": {
      "terms": {
        "field":"job"
      }
    }
  }
}

POST employees/_search
{
  "size": 0,
  "aggs": {
    "jobs": {
      "terms": {
        "field":"job.keyword"
      }
    }
  }
}


# 對(duì)job.keyword 和 job 進(jìn)行 terms 聚合，分桶的總數(shù)并不一樣
POST employees/_search
{
  "size": 0,
  "aggs": {
    "cardinate": {
      "cardinality": {
        "field": "job"
      }
    }
  }
}


# 對(duì) 性別的 keyword 進(jìn)行聚合
POST employees/_search
{
  "size": 0,
  "aggs": {
    "gender": {
      "terms": {
        "field":"gender"
      }
    }
  }
}


#指定 bucket 的 size
POST employees/_search
{
  "size": 0,
  "aggs": {
    "ages_5": {
      "terms": {
        "field":"age",
        "size":3
      }
    }
  }
}



# 指定size，不同工種中，年紀(jì)最大的3個(gè)員工的具體信息
POST employees/_search
{
  "size": 0,
  "aggs": {
    "jobs": {
      "terms": {
        "field":"job.keyword"
      },
      "aggs":{
        "old_employee":{
          "top_hits":{
            "size":3,
            "sort":[
              {
                "age":{
                  "order":"desc"
                }
              }
            ]
          }
        }
      }
    }
  }
}



#Salary Ranges 分桶，可以自己定義 key
POST employees/_search
{
  "size": 0,
  "aggs": {
    "salary_range": {
      "range": {
        "field":"salary",
        "ranges":[
          {
            "to":10000
          },
          {
            "from":10000,
            "to":20000
          },
          {
            "key":">20000",
            "from":20000
          }
        ]
      }
    }
  }
}


#Salary Histogram,工資0到10萬(wàn)，以 5000一個(gè)區(qū)間進(jìn)行分桶
POST employees/_search
{
  "size": 0,
  "aggs": {
    "salary_histrogram": {
      "histogram": {
        "field":"salary",
        "interval":5000,
        "extended_bounds":{
          "min":0,
          "max":100000

        }
      }
    }
  }
}


# 嵌套聚合1，按照工作類型分桶，并統(tǒng)計(jì)工資信息
POST employees/_search
{
  "size": 0,
  "aggs": {
    "Job_salary_stats": {
      "terms": {
        "field": "job.keyword"
      },
      "aggs": {
        "salary": {
          "stats": {
            "field": "salary"
          }
        }
      }
    }
  }
}

# 多次嵌套。根據(jù)工作類型分桶，然后按照性別分桶，計(jì)算工資的統(tǒng)計(jì)信息
POST employees/_search
{
  "size": 0,
  "aggs": {
    "Job_gender_stats": {
      "terms": {
        "field": "job.keyword"
      },
      "aggs": {
        "gender_stats": {
          "terms": {
            "field": "gender"
          },
          "aggs": {
            "salary_stats": {
              "stats": {
                "field": "salary"
              }
            }
          }
        }
      }
    }
  }
}

相關(guān)閱讀

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡(jiǎn)書(shū)系信息發(fā)布平臺(tái)，僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容