pod的親和度調(diào)度

image.png

前言:
由于Affinity對(duì)pod的調(diào)度更加精細(xì),我們?cè)谑褂弥兄饾u代替了NodeSelector??梢苑譃閚ode親和性調(diào)度和pod親和性調(diào)度。
1)node親和性調(diào)度:不僅有NodeSelector的硬限制,而且可以在軟限制中定義權(quán)重。
2)pod親和性調(diào)度:它可以使得pod根據(jù)在節(jié)點(diǎn)上正在運(yùn)行的pod的標(biāo)簽(而不是節(jié)點(diǎn)的標(biāo)簽)進(jìn)行調(diào)度,要求對(duì)節(jié)點(diǎn)和pod兩個(gè)條件進(jìn)行匹配。

1. Node Affinity

1.1 node節(jié)點(diǎn)的預(yù)制標(biāo)簽

有一些預(yù)置的標(biāo)簽,我們可以直接使用

  • 查看master上的標(biāo)簽
[root@DoM01 ~]# kubectl get node dom03 --show-labels
NAME    STATUS   ROLES    AGE   VERSION   LABELS
dom03   Ready    master   52d   v1.15.2   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=dom03,kubernetes.io/os=linux,node-role.kubernetes.io/master=
  • 格式不是很友好,我們用describe看一下
[root@DoM01 ~]# kubectl describe node dom01
Name:               dom01
Roles:              master
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=dom01
                    kubernetes.io/os=linux
                    node-role.kubernetes.io/master=
……

說明:
"beta.kubernetes.io/arch=amd64","beta.kubernetes.io/os=linux" 這兩個(gè)在1.18中棄用。
看名字使用,也沒什么解釋的。

1.2 自定義標(biāo)簽

1.2.1 給node增加標(biāo)簽

  • 語法
# kubectl label node node名  鍵=值
  • 示例
[root@DoM01 ~]# kubectl label node don01 zone=east
node/don01 labeled
  • 驗(yàn)證
    如下可見 don01 設(shè)置了標(biāo)簽 zone=east
[root@DoM01 ~]# kubectl describe node don01
Name:               don01
Roles:              <none>
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=don01
                    kubernetes.io/os=linux
                    zone=east

1.2.2 修改label

說明:加 --overwrite參數(shù)

# kubectl label node don01 zone=south --overwrite

1.2.3 修改label

說明:刪除一個(gè)key為zone的標(biāo)簽,只需把key的后邊加一個(gè)減號(hào)即會(huì)刪除該key

# kubectl label node don01 zone-

1.3 Require

  • 概述:
    equiredDuringSchedulingIgnoredDuringExecution是硬限制,必須滿足此條件才可以調(diào)度pod到該node上(功能和nodeSelector很像)

  • 示例

apiVersion: v1
kind: Pod
metadata:
  name: nginxtest
  namespace: test
spec:
  affinity:
     # 說明是"節(jié)點(diǎn)親和性調(diào)度"
    nodeAffinity:
      # 說明是"節(jié)點(diǎn)親和性調(diào)度"
      requiredDuringSchedulingIgnoredDuringExecution:
        #說明要選擇節(jié)點(diǎn)了
        nodeSelectorTerms:
        - matchExpressions:
          - key: zone
            operator: In
            values:
            - "east"
  containers:
    - name: nginxtest
      image: harbocto.boe.com.cn/public/nginx
  • 創(chuàng)建并查看結(jié)果
[root@DoM01 test]# kubectl create -f nginx.yml
pod/nginxtest created
[root@DoM01 test]# kubectl get pod -n test -o wide
NAME        READY   STATUS    RESTARTS   AGE   IP             NODE    NOMINATED NODE   READINESS GATES
nginxtest   1/1     Running   0          91s   10.244.5.166   don03   <none>           <none>

說明:可以看到,pod被調(diào)度到了一個(gè)zone=east的節(jié)點(diǎn)don01上

  • 更改調(diào)度
    將yml文件修改成調(diào)度到zone=south的節(jié)點(diǎn),然后更新pod如下
[root@DoM01 test]# kubectl apply -f nginx.yml
The Pod "nginxtest" is invalid: spec: Forbidden: pod updates may not change fields other than `spec.containers[*].image`, `spec.initContainers[*].image`, `spec.activeDeadlineSeconds` or `spec.tolerations` (only additions to existing tolerations)
  core.PodSpec{
        ……

說明:以上報(bào)錯(cuò)為了引出下邊兩個(gè)重要規(guī)則。

  • 規(guī)則
    1)親和度的值是不能直接修改的。
    2)如果此時(shí)修改了node標(biāo)簽,使得節(jié)點(diǎn)不滿足要求,這個(gè)改變也將被系統(tǒng)忽略。(即pod仍會(huì)在該節(jié)點(diǎn)上運(yùn)行)

1.4 Perferred

  • 概述
    preferredDuringSchedulingIgnoredDuringExecution 是軟限制,強(qiáng)調(diào)優(yōu)先滿足制定規(guī)則,多個(gè)優(yōu)先級(jí)可以設(shè)置權(quán)重。

  • 示例

apiVersion: v1
kind: Pod
metadata:
  name: nginxtest
  namespace: test
spec:
  affinity:
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 60
          preference:
            matchExpressions:
            - key: zone
              operator: In
              values:
              - "east"
        - weight: 80
          preference:
            matchExpressions:
            - key: zone
              operator: In
              values:
              - "south"
  containers:
    - name: nginxtest
      image: harbocto.boe.com.cn/public/nginx  
  • 創(chuàng)建并查看結(jié)果
[root@DoM01 test]# kubectl create -f nginx.yml
pod/nginxtest created
[root@DoM01 test]# kubectl get pod -n test  -o wide
NAME        READY   STATUS    RESTARTS   AGE   IP             NODE    NOMINATED NODE   READINESS GATES
nginxtest   1/1     Running   0          23s   10.244.7.190   don05   <none>           <none>

說明:雖然 zone=east的權(quán)重是60,但是仍可以調(diào)度到上邊

  • 修改east的權(quán)重為20,刪除pod再啟動(dòng)一下
[root@DoM01 test]# kubectl create -f nginx.yml
pod/nginxtest created
[root@DoM01 ~]# kubectl get pod -n test  -o wide
NAME        READY   STATUS    RESTARTS   AGE   IP            NODE    NOMINATED NODE   READINESS GATES
nginxtest   1/1     Running   0          18m   10.244.3.33   don01   <none>           <none>
[root@DoM01 ~]#

說明:可以看到此時(shí)該pod被調(diào)度到zone=south的節(jié)點(diǎn)上了,權(quán)重的作用可見一斑。

1.3 注意事項(xiàng)

  • 如果設(shè)置了nodeSelector和nodeAffinity,則需同時(shí)滿足。
  • 如果設(shè)置多個(gè)nodeSelectorTerms,有一個(gè)滿足即可。
  • 如果nodeSelectorTerms下有多個(gè) matchExpressions,則必須滿足所有條件才可以。

2. Pod Affinity

說明:
根據(jù)pod1的標(biāo)簽選是否在某一組(或一個(gè))node節(jié)點(diǎn)上部署pod2。
這一組node上用來限制親和度的標(biāo)簽的key 稱為 topologyKey。
因此pod2需要兩個(gè)標(biāo)簽來確定親和度:
(1)限制在那個(gè)范圍內(nèi)(topologyKey)。( 2)和哪個(gè)pod親和(相應(yīng)pod的標(biāo)簽)。
關(guān)于topologyKey,我們不需要指明值,因?yàn)橹灰粋€(gè)值的一組node下親和就可以了。

2.1 Pod Affinity

說明:
同樣分為
"requiredDuringSchedulingIgnoredDuringExecution"
"preferredDuringSchedulingIgnoredDuringExecution"
兩種

2.1.1 required

  • 參照目標(biāo)pod
apiVersion: v1
kind: Pod
metadata:
  name: nginx-flag
  namespace: test
labels:
  security: "S1"
  app: "nginx-flag"
spec:       
  containers:
    - name: nginx-flag
      image: nginx
  • pod的親和度調(diào)度
apiVersion: v1
kind: Pod
metadata:
  name: nginxtest
  namespace: test
spec:
  affinity:
    podAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: security
            operator: In
            values:
            - "S1"
        topologyKey: kubernetes.io/hostname
  containers:
  - name: nginxtest
    image: harbocto.boe.com.cn/public/nginx

  • 啟動(dòng)并查看結(jié)果
[root@DoM01 test]# kubectl create -f nginx-flag.yml
[root@DoM01 test]# kubectl create -f nginx.yml
pod/nginxtest created
[root@DoM01 test]# kubectl get pod -n test -o wide
NAME         READY   STATUS    RESTARTS   AGE   IP             NODE    NOMINATED NODE   READINESS GATES
nginx-flag   1/1     Running   1          14m   10.244.7.191   don05   <none>           <none>
nginxtest    1/1     Running   0          58s   10.244.7.192   don05   <none>           <none>

說明:可見nginxtest調(diào)度到了nginx-flag上

  • 找不到合適節(jié)點(diǎn)
    如果沒有啟動(dòng)nginx-flag而直接啟動(dòng)nginxtest,系統(tǒng)找不到合適的節(jié)點(diǎn)調(diào)度,會(huì)一直處于pending狀態(tài)。
[root@DoM01 test]# kubectl create -f nginx.yml
pod/nginxtest created
[root@DoM01 test]# kubectl get pod -n test
NAME        READY   STATUS    RESTARTS   AGE
nginxtest   0/1     Pending   0          3s

如下可見,nginx-flag啟動(dòng)之后,nginxtest被調(diào)度到了有nginx-flag的節(jié)點(diǎn)上。

[root@DoM01 test]# kubectl create -f nginx-flag.yml
pod/nginx-flag created
NAME         READY   STATUS    RESTARTS   AGE     IP             NODE    NOMINATED NODE   READINESS GATES
nginx-flag   1/1     Running   0          17s     10.244.5.169   don03   <none>           <none>
nginxtest    1/1     Running   0          2m33s   10.244.5.168   don03   <none>           <none>

2.1.2 preferred

  • 參照目標(biāo)pod
    同上
  • pod的親和度調(diào)度
    創(chuàng)建nginx.yml文件如下:
apiVersion: v1
kind: Pod
metadata:
  name: nginxtest
  namespace: test
spec:
  affinity:
    podAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 20
        podAffinityTerm:
          labelSelector:
            matchExpressions:
            - key: security
              operator: In
              values:
              - "S1"
          topologyKey: kubernetes.io/hostname
      - weight: 80
        podAffinityTerm:
          labelSelector:
            matchExpressions:
            - key: security
              operator: In
              values:
              - "S2"
          topologyKey: kubernetes.io/hostname
  containers:
    - name: nginxtest
      image: harbocto.boe.com.cn/public/nginx
  • 啟動(dòng)和查看
[root@DoM01 test]# kubectl create -f nginx.yml
pod/nginxtest created
[root@DoM01 test]# kubectl get pod -n test  -o wide
NAME         READY   STATUS    RESTARTS   AGE   IP            NODE    NOMINATED NODE   READINESS GATES
nginx-flag   1/1     Running   0          56s   10.244.3.34   don01   <none>           <none>
nginxtest    1/1     Running   0          8s    10.244.3.35   don01   <none>           <none>

如上,可見nginxtest被調(diào)度到了nginx-flag所在的節(jié)點(diǎn)上

  • 親和度是相互的

測(cè)試:
再啟動(dòng)一個(gè)nginx-flag-02,lable設(shè)置為security=S2。由前邊可知,nginxtest和它的親和度是80,但是它會(huì)主動(dòng)選擇nginxtest。

apiVersion: v1
kind: Pod
metadata:
  name: nginx-flag-02
  namespace: test
  labels:
    security: "S2"
    app: "nginx-flag"
spec:
  containers:
    - name: nginx-flag-02
      image: harbocto.boe.com.cn/public/nginx

刪除nginxtest,在重新啟動(dòng)

[root@DoM01 test]# kubectl create -f nginx-flag02.yml
pod/nginx-flag-02 created
[root@DoM01 test]# kubectl get pod -n test  -o wide
NAME            READY   STATUS    RESTARTS   AGE     IP            NODE    NOMINATED NODE   READINESS GATES
nginx-flag      1/1     Running   0          3m18s   10.244.3.34   don01   <none>           <none>
nginx-flag-02   1/1     Running   0          6s      10.244.3.36   don01   <none>           <none>
nginxtest       1/1     Running   0          2m30s   10.244.3.35   don01   <none>           <none>

如上:
發(fā)現(xiàn)它們竟然會(huì)粘在一起,nginx-flag-02居然也會(huì)啟動(dòng)在(看了一下node資源,如果沒有親和度的話nginx-flag-02應(yīng)該啟動(dòng)在don03上。)

  • 刪除ngintest,在刪除nginx-flag-02。再啟動(dòng)nginx-flag-02,果然如上邊預(yù)期被調(diào)度到了don03上。
[root@DoM01 test]# kubectl delete -n test pod nginxtest
pod "nginxtest" deleted
[root@DoM01 test]# kubectl delete -n test pod nginx-flag-02
pod "nginx-flag-02" deleted
[root@DoM01 test]# kubectl create -f nginx-flag02.yml
pod/nginx-flag-02 created
[root@DoM01 test]# kubectl get pod -n test  -o wide
NAME            READY   STATUS    RESTARTS   AGE     IP             NODE    NOMINATED NODE   READINESS GATES
nginx-flag      1/1     Running   0          5m36s   10.244.3.34    don01   <none>           <none>
nginx-flag-02   1/1     Running   0          2s      10.244.5.191   don03   <none>           <none>
  • 測(cè)試親和度權(quán)重
[root@DoM01 test]# kubectl create -f nginx.yml
pod/nginxtest created
[root@DoM01 test]# kubectl get pod -n test -o wide
NAME            READY   STATUS    RESTARTS   AGE     IP             NODE    NOMINATED NODE   READINESS GATES
nginx-flag      1/1     Running   0          4h24m   10.244.3.34    don01   <none>           <none>
nginx-flag-02   1/1     Running   0          4h19m   10.244.5.191   don03   <none>           <none>
nginxtest       1/1     Running   0          14s     10.244.5.194   don03   <none>           <none>

說明:如上可見,nginxtest被調(diào)度到權(quán)重更高的nginx-flag-02的節(jié)點(diǎn)上了。

2.2 Pod Anti Affinity

  • 參照pod
    復(fù)制一下剛才的參照pod,把五個(gè)節(jié)點(diǎn)的四個(gè)都占了
apiVersion: v1
kind: Pod
metadata:
  name: nginx-flag-01
  namespace: test
  labels:
    security: "S1"
    app: "nginx-flag"
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: kubernetes.io/hostname
            operator: In
            values:
            - "don01"
  containers:
    - name: nginx-flag-01
      image: harbocto.boe.com.cn/public/nginx
---
apiVersion: v1
kind: Pod
metadata:
  name: nginx-flag-02
  namespace: test
  labels:
    security: "S2"
    app: "nginx-flag"
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: kubernetes.io/hostname
            operator: In
            values:
            - "don02"
  containers:
    - name: nginx-flag-02
      image: harbocto.boe.com.cn/public/nginx
---
apiVersion: v1
kind: Pod
metadata:
  name: nginx-flag-03
  namespace: test
  labels:
    security: "S3"
    app: "nginx-flag"
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: kubernetes.io/hostname
            operator: In
            values:
            - "don03"
  containers:
    - name: nginx-flag-03
      image: harbocto.boe.com.cn/public/nginx
---
apiVersion: v1
kind: Pod
metadata:
  name: nginx-flag-04
  namespace: test
  labels:
    security: "S4"
    app: "nginx-flag"
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: kubernetes.io/hostname
            operator: In
            values:
            - "don04"
  containers:
    - name: nginx-flag-04
      image: harbocto.boe.com.cn/public/nginx
  • pod的反親和調(diào)度
apiVersion: v1
kind: Pod
metadata:
  name: nginxtest-02
  namespace: test
spec:
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - "nginx-flag"
        topologyKey: kubernetes.io/hostname
  containers:
  - name: nginxtest-02
    image: harbocto.boe.com.cn/public/nginx
  • 創(chuàng)建和查看
[root@DoM01 test]# kubectl get pod -n test -o wide
NAME            READY   STATUS    RESTARTS   AGE   IP             NODE    NOMINATED NODE   READINESS GATES
nginx-flag-01   1/1     Running   0          15m   10.244.3.38    don01   <none>           <none>
nginx-flag-02   1/1     Running   0          15m   10.244.4.35    don02   <none>           <none>
nginx-flag-03   1/1     Running   0          15m   10.244.5.201   don03   <none>           <none>
nginx-flag-04   1/1     Running   0          15m   10.244.6.27    don04   <none>           <none>
nginxtest-02    1/1     Running   0          14m   10.244.7.210   don05   <none>           <none>

如上可見,nginxtest-02被調(diào)度到最后剩下的一個(gè)節(jié)點(diǎn)上了

2.3 注意事項(xiàng)

  • topology
    1)反親和性 requiredDuringScheduling 中topologyKey 不能為空
    2)反親和性 preferredDuringScheduling 中topologyKey 為空,則被認(rèn)為是如下的組合:
    kubernetes.io/hostname
    failure-domain.beta.kubernetes.io/zone
    failure-domain.beta.kubernetes.io/region
    3)如果admission controller 設(shè)置了LimitPodHardAntiAffinityTopology ,則互斥性被限制在 kubernetes.io/hostname

  • namespace限制
    1)位置:和topologyKey同級(jí)
    2)未定義namespace:表示和參照目標(biāo)的pod相同
    3)設(shè)置為空:表示所有namespace


最后編輯于
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡(jiǎn)書系信息發(fā)布平臺(tái),僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容