鍛骨境-第8層 k8s自建SLB 負(fù)載均衡服務(wù)

我需要一個負(fù)載均衡 -Metallb

Metalllb官方地址:https://metallb.universe.tf/installation/
在學(xué)習(xí)的過程中,會發(fā)現(xiàn)不是所有的需要都要走7層協(xié)議提供http api 接口,我們還需要有一個loadbalance ,做tcp 的服務(wù)。
在k8s 對外提供服務(wù)能力的介紹里,說明了三種方式: LoadBalance,ClusterIp,NodePort .


三種方式的區(qū)別

LoadBalance借助于負(fù)載IP通過端口達(dá)到Ip;port 的目的,這里的port 基本無限制。
ClusterIP 是集群內(nèi)部 服務(wù)之間訪問的方式,如果需要對外提供服務(wù) ,還需要 Ingress做 路由轉(zhuǎn)發(fā)。
NodePort 的端口有限制3000-32767,其實也是可以擴大 范圍的,ApiServer 配置參數(shù):–service-node-port-range=1-65535
但是這種暴露的方式不可取,因為他要暴露的是我們node 節(jié)點的對外IP+Port . 測試使用還行

不論是想自己搭建Ingress Controller 還是自由端口映射,我們 都需要一個LoadBalance ,這里使用Metallb 實現(xiàn)。

設(shè)置Master污點

設(shè)置master 污點的原因是因為機器內(nèi)存不夠,為了 能跑容器服務(wù),只能在master 上跑了。默認(rèn)master 不做為worker 。

讓 master節(jié)點參與POD負(fù)載的命令為
kubectl taint nodes --all node-role.kubernetes.io/master-
讓 master節(jié)點恢復(fù)不參與POD負(fù)載的命令為 
kubectl taint nodes <node-name> node-role.kubernetes.io/master=:NoSchedule

具體的就是:

[root@k8s-master etcd]# 
[root@k8s-master etcd]# kubectl  taint    nodes   k8s-master   node-role.kubernetes.io/master-
node/k8s-master untainted
[root@k8s-master etcd]# kubectl  taint    nodes   k8s-node1   node-role.kubernetes.io/master-
node/k8s-node1 untainted
[root@k8s-master etcd]# 

搭建metallb

1. 配置文件: config.yaml

主要下面的 address 是一個ip 范圍。

metallb-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: my-ip-space
      protocol: layer2
      addresses:
      - 192.168.10.133-192.168.10.134

如果要更改ip地址,直接修改即可,然后apply之后,metallb會自動更新。
創(chuàng)建命名空間 metallb-system

[root@k8s-master metallb]# 
[root@k8s-master metallb]# kubectl create ns  metallb-system 
namespace/metallb-system created
[root@k8s-master metallb]# 

2. 配置安裝文件:metallb.yaml

---
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: metallb-system
  name: controller
  labels:
    app: metallb
---
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: metallb-system
  name: speaker
  labels:
    app: metallb

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: metallb-system:controller
  labels:
    app: metallb
rules:
- apiGroups: [""]
  resources: ["services"]
  verbs: ["get", "list", "watch", "update"]
- apiGroups: [""]
  resources: ["services/status"]
  verbs: ["update"]
- apiGroups: [""]
  resources: ["events"]
  verbs: ["create", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: metallb-system:speaker
  labels:
    app: metallb
rules:
- apiGroups: [""]
  resources: ["services", "endpoints", "nodes"]
  verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: metallb-system
  name: config-watcher
  labels:
    app: metallb
rules:
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources: ["events"]
  verbs: ["create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: metallb-system:controller
  labels:
    app: metallb
subjects:
- kind: ServiceAccount
  name: controller
  namespace: metallb-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: metallb-system:controller
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: metallb-system:speaker
  labels:
    app: metallb
subjects:
- kind: ServiceAccount
  name: speaker
  namespace: metallb-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: metallb-system:speaker
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  namespace: metallb-system
  name: config-watcher
  labels:
    app: metallb
subjects:
- kind: ServiceAccount
  name: controller
- kind: ServiceAccount
  name: speaker
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: config-watcher
---
apiVersion: apps/v1beta2
kind: DaemonSet
metadata:
  namespace: metallb-system
  name: speaker
  labels:
    app: metallb
    component: speaker
spec:
  selector:
    matchLabels:
      app: metallb
      component: speaker
  template:
    metadata:
      labels:
        app: metallb
        component: speaker
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "7472"
    spec:
      serviceAccountName: speaker
      terminationGracePeriodSeconds: 0
      hostNetwork: true
      containers:
      - name: speaker
        image: metallb/speaker:v0.7.3
        imagePullPolicy: IfNotPresent
        args:
        - --port=7472
        - --config=config
        env:
        - name: METALLB_NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        ports:
        - name: monitoring
          containerPort: 7472
        resources:
          limits:
            cpu: 100m
            memory: 100Mi
          
        securityContext:
          allowPrivilegeEscalation: false
          readOnlyRootFilesystem: true
          capabilities:
            drop:
            - all
            add:
            - net_raw

---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  namespace: metallb-system
  name: controller
  labels:
    app: metallb
    component: controller
spec:
  revisionHistoryLimit: 3
  selector:
    matchLabels:
      app: metallb
      component: controller
  template:
    metadata:
      labels:
        app: metallb
        component: controller
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "7472"
    spec:
      serviceAccountName: controller
      terminationGracePeriodSeconds: 0
      securityContext:
        runAsNonRoot: true
        runAsUser: 65534 # nobody
      containers:
      - name: controller
        image: metallb/controller:v0.7.3
        imagePullPolicy: IfNotPresent
        args:
        - --port=7472
        - --config=config
        ports:
        - name: monitoring
          containerPort: 7472
        resources:
          limits:
            cpu: 100m
            memory: 100Mi
          
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - all
          readOnlyRootFilesystem: true


3. 部署

[root@k8s-master metallb]# vim metallb.yaml
[root@k8s-master metallb]# 
[root@k8s-master metallb]# 
[root@k8s-master metallb]# 
[root@k8s-master metallb]# kubectl  apply -f config.yaml 
configmap/config created
[root@k8s-master metallb]# kubectl  apply -f metallb.yaml  
serviceaccount/controller created
serviceaccount/speaker created
clusterrole.rbac.authorization.k8s.io/metallb-system:controller created
clusterrole.rbac.authorization.k8s.io/metallb-system:speaker created
role.rbac.authorization.k8s.io/config-watcher created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker created
rolebinding.rbac.authorization.k8s.io/config-watcher created
daemonset.apps/speaker created
deployment.apps/controller created
[root@k8s-master metallb]# 

查看結(jié)果:

[root@k8s-master metallb]# kubectl  get  po -n metallb-system -owide
NAME                          READY   STATUS    RESTARTS   AGE   IP               NODE         NOMINATED NODE   READINESS GATES
controller-7cc9c87cfb-fn6wh   1/1     Running   0          92s   10.244.1.85      k8s-node1    <none>           <none>
speaker-np4qr                 1/1     Running   0          92s   192.168.10.133   k8s-master   <none>           <none>
speaker-w2wb7                 1/1     Running   0          92s   192.168.10.134   k8s-node1    <none>           <none>
[root@k8s-master metallb]# 

示例: 使用LoadBalance 部署 DashBoard WEB UI ,其他文章大部分 都是 NodePort 的,其實不可取 。

官網(wǎng)的部署 文件:https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/
修改后的,主要修改了RBAC權(quán)限 和 Service 類型為LoadBalance ,端口為 7443

dashboard/yaml 內(nèi)容如下:
kubectl apply -f dashboard.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard

---

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 7443
      targetPort: 8443
  selector:
    k8s-app: kubernetes-dashboard
  type: LoadBalancer

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-certs
  namespace: kubernetes-dashboard
type: Opaque

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-csrf
  namespace: kubernetes-dashboard
type: Opaque
data:
  csrf: ""

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-key-holder
  namespace: kubernetes-dashboard
type: Opaque

---

kind: ConfigMap
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-settings
  namespace: kubernetes-dashboard

---

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
rules:

  - apiGroups: [""]
    resources: ["secrets"]
    resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"]
    verbs: ["get", "update", "delete"]
  
  - apiGroups: [""]
    resources: ["configmaps"]
    resourceNames: ["kubernetes-dashboard-settings"]
    verbs: ["get", "update"]
   
  - apiGroups: [""]
    resources: ["services"]
    resourceNames: ["heapster", "dashboard-metrics-scraper"]
    verbs: ["proxy"]
  - apiGroups: [""]
    resources: ["services/proxy"]
    resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"]
    verbs: ["get"]

---

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
rules:
  - apiGroups: ["metrics.k8s.io"]
    resources: ["pods", "nodes"]
    verbs: ["get", "list", "watch"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: kubernetes-dashboard
subjects:
  - kind: ServiceAccount
    name: kubernetes-dashboard
    namespace: kubernetes-dashboard

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: kubernetes-dashboard
    namespace: kubernetes-dashboard

---

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: kubernetes-dashboard
  template:
    metadata:
      labels:
        k8s-app: kubernetes-dashboard
    spec:
      containers:
        - name: kubernetes-dashboard
          image: kubernetesui/dashboard:v2.0.0-beta4
          imagePullPolicy: Always
          ports:
            - containerPort: 8443
              protocol: TCP
          args:
            - --auto-generate-certificates
            - --namespace=kubernetes-dashboard
          volumeMounts:
            - name: kubernetes-dashboard-certs
              mountPath: /certs
              
            - mountPath: /tmp
              name: tmp-volume
          livenessProbe:
            httpGet:
              scheme: HTTPS
              path: /
              port: 8443
            initialDelaySeconds: 30
            timeoutSeconds: 30
      volumes:
        - name: kubernetes-dashboard-certs
          secret:
            secretName: kubernetes-dashboard-certs
        - name: tmp-volume
          emptyDir: {}
      serviceAccountName: kubernetes-dashboard


---

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: dashboard-metrics-scraper
  name: dashboard-metrics-scraper
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 8000
      targetPort: 8000
  selector:
    k8s-app: dashboard-metrics-scraper

---

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: dashboard-metrics-scraper
  name: dashboard-metrics-scraper
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: dashboard-metrics-scraper
  template:
    metadata:
      labels:
        k8s-app: dashboard-metrics-scraper
    spec:
      containers:
        - name: dashboard-metrics-scraper
          image: kubernetesui/metrics-scraper:v1.0.1
          ports:
            - containerPort: 8000
              protocol: TCP
          livenessProbe:
            httpGet:
              scheme: HTTP
              path: /
              port: 8000
            initialDelaySeconds: 30
            timeoutSeconds: 30
          volumeMounts:
          - mountPath: /tmp
            name: tmp-volume
      serviceAccountName: kubernetes-dashboard
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule
      volumes:
        - name: tmp-volume
          emptyDir: {}


結(jié)果如下,可以 看到通過192.168.10.133:7443可以訪問dashboard UI .

[root@k8s-master k8s]# kubectl  get svc   -n  kubernetes-dashboard 
NAME                        TYPE           CLUSTER-IP       EXTERNAL-IP      PORT(S)          AGE
dashboard-metrics-scraper   ClusterIP      10.108.208.131   <none>           8000/TCP         6m38s
kubernetes-dashboard        LoadBalancer   10.100.184.70    192.168.10.133   7443:32765/TCP   6m38s
[root@k8s-master k8s]# 

必須使用 火狐瀏覽器,打開登錄頁,后面 用ingress 之后,就不是必須 了:

打開: https://192.168.10.133:7443
選擇接受 風(fēng)險 并繼續(xù),然后需要填寫 token:
獲取登錄 token

[root@k8s-node1 .kube]# kubectl get  secret  -n kubernetes-dashboard 
NAME                               TYPE                                  DATA   AGE
default-tls-cert                   kubernetes.io/tls                     2      18m
default-token-n8kvc                kubernetes.io/service-account-token   3      44m
kubernetes-dashboard-certs         Opaque                                0      44m
kubernetes-dashboard-csrf          Opaque                                1      44m
kubernetes-dashboard-key-holder    Opaque                                2      44m
kubernetes-dashboard-token-t8bz5   kubernetes.io/service-account-token   3      44m
[root@k8s-node1 .kube]# kubectl describe   secret  kubernetes-dashboard-token-t8bz5   -n kubernetes-dashboard 
Name:         kubernetes-dashboard-token-t8bz5
Namespace:    kubernetes-dashboard
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: kubernetes-dashboard
              kubernetes.io/service-account.uid: f96e760d-fe04-11e9-96d3-000c29a4e4b2

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1025 bytes
namespace:  20 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC10b2tlbi10OGJ6NSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImY5NmU3NjBkLWZlMDQtMTFlOS05NmQzLTAwMGMyOWE0ZTRiMiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDprdWJlcm5ldGVzLWRhc2hib2FyZCJ9.hyKSFITlSWyZn-TmJimQiWhUOsO38yNGnq-k3IREBhxWRjwx-Y-OjkCd1RUaRgW-ocGqYHrKqXWMsv1_Nv9UR1-CIAfdNzFwkb_RTf2UVIB6C098WizSTJeUzodUsGJDPh9QhWnSrZIFbOkKxzjll2mFEnhvnbmZil_VNYRo-Oi0rGLcKdChCkfq7RWinZL4xGlH8g3xbktuFGHSxrxHVr7If5yhSms82qD4WA5ePiJbDRIZHdBUQJM53VprG9CrzRFLMmWYOPlnf5CnSoQWbT9zgDGRGMCU04rZXRKRbvGw1pGbVHK2PKSmesddw_iVJDfRBA5o-MzOgozunsl7JQ
[root@k8s-node1 .kube]# 

登錄成功:

login.png
最后編輯于
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時請結(jié)合常識與多方信息審慎甄別。
平臺聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點,簡書系信息發(fā)布平臺,僅提供信息存儲服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容