Kubernetes + Dashboard + Heapster (三) nginx-ingress負(fù)載均衡

kubectl 基本命令

檢查節(jié)點(diǎn)狀態(tài)
kubectl get nodes
檢查namespace
kubectl get namespace
刪除節(jié)點(diǎn)
etcdctl ls /registry/minions
或者
kubectl delete node  xxxx
更改 rc - pods 數(shù)量

kubectl scale rc rc_name --replicas=number

nginx-ingress

原理

一般來(lái)說(shuō)，svc和pod擁有的IP只能在集群內(nèi)部使用。集群外部請(qǐng)求需要通過(guò)負(fù)載均衡轉(zhuǎn)發(fā)到svc在node上暴露的NodePort,然后再由kube-proxy 將其轉(zhuǎn)發(fā)給相關(guān)的pod。
[圖片上傳失敗...(image-162def-1516283321671)]

而Ingress就是為進(jìn)入集群的請(qǐng)求提供路由規(guī)則的集合，如圖所示。
[圖片上傳失敗...(image-8fd5c7-1516283321671)]
Ingress可以給service提供集群外部訪問(wèn)的URL、負(fù)載均衡、SSL終止、HTTP路由等。為了配置這些Ingress規(guī)則，集群管理員需要部署一個(gè)Ingress controller，它監(jiān)聽(tīng)I(yíng)ngress和service的變化，并根據(jù)規(guī)則配置負(fù)載均衡并提供訪問(wèn)入口。

配置規(guī)則

每個(gè)Ingress都需要配置rules，目前Kubernetes僅支持http規(guī)則。上面的示例表示請(qǐng)求/testpath時(shí)轉(zhuǎn)發(fā)到服務(wù)test的80端口。
根據(jù)Ingress Spec配置的不同，Ingress可以分為以下幾種類型：

單服務(wù)Ingress

單服務(wù)Ingress即該Ingress僅指定一個(gè)沒(méi)有任何規(guī)則的后端服務(wù)。

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: test-ingress
spec:
  backend:
    serviceName: testsvc
    servicePort: 80

路由到多服務(wù)的Ingress

路由到多服務(wù)的Ingress即根據(jù)請(qǐng)求路徑的不同轉(zhuǎn)發(fā)到不同的后端服務(wù)上

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: test
spec:
  rules:
  - host: foo.bar.com
    http:
      paths:
      - path: /foo
        backend:
          serviceName: s1
          servicePort: 80
      - path: /bar
        backend:
          serviceName: s2
          servicePort: 80

虛擬主機(jī)Ingress

虛擬主機(jī)Ingress即根據(jù)名字的不同轉(zhuǎn)發(fā)到不同的后端服務(wù)上，而他們共用同一個(gè)的IP地址，如下所示

foo.bar.com --|                 |-> foo.bar.com s1:80
              | 178.91.123.132  |
bar.foo.com --|                 |-> bar.foo.com s2:80

下面是一個(gè)基于Host header路由請(qǐng)求的Ingress：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: test
spec:
  rules:
  - host: foo.bar.com
    http:
      paths:
      - backend:
          serviceName: s1
          servicePort: 80
  - host: bar.foo.com
    http:
      paths:
      - backend:
          serviceName: s2
          servicePort: 80

總結(jié)

對(duì)于我來(lái)說(shuō)，最常用的還是虛擬主機(jī)模式。就像nginx中conf.d多個(gè)主機(jī)似的一回事。但是對(duì)于一個(gè)大型網(wǎng)站的話，或許使用路由到多服務(wù)模式會(huì)更好一些。

前面是對(duì)模式的整理，但還有一個(gè)組件。其中default-http-backend作為 Nginx Ingress Controller默認(rèn)的后端，處理所有404請(qǐng)求。

安裝配置

default-http-backend

首先配置 default-http-backend
當(dāng)前pod作為Nginx Ingress Controller默認(rèn)的后端，處理所有404請(qǐng)求。當(dāng)前還沒(méi)有做任何配置，此時(shí)請(qǐng)求的時(shí)候?qū)⒂纱藀od響應(yīng)。

鏡像準(zhǔn)備

如果有加速器

docker pull gcr.io/google_containers/defaultbackend:1.4

如果沒(méi)有

docker pull registry.cn-beijing.aliyuncs.com/cloudexp/defaultbackend:latest

yml 文件

curl -o default-http-backend.yml https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/default-backend.yaml

或者

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: default-http-backend
  labels:
    k8s-app: default-http-backend
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        k8s-app: default-http-backend
    spec:
      terminationGracePeriodSeconds: 60
      containers:
      - name: default-http-backend
        # Any image is permissable as long as:
        # 1. It serves a 404 page at /
        # 2. It serves 200 on a /healthz endpoint
        image: registry.cn-beijing.aliyuncs.com/cloudexp/defaultbackend:latest
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 30
          timeoutSeconds: 5
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: 10m
            memory: 20Mi
          requests:
            cpu: 10m
            memory: 20Mi

部署

kubectl create -f default-http-backend.yml
#返回結(jié)果
deployment "default-http-backend" created
service "default-http-backend" created

nginx-ingress-controller

鏡像準(zhǔn)備

docker pull registry.cn-hangzhou.aliyuncs.com/google-containers/nginx-ingress-controller:0.9.0

yml文件

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: kube-system
  labels:
    k8s-app: nginx-ingress-controller
spec:
  replicas: 1
  template:
    metadata:
      labels:
        k8s-app: nginx-ingress-controller
    spec:
      # hostNetwork makes it possible to use ipv6 and to preserve the source IP correctly regardless of docker configuration
      # however, it is not a hard dependency of the nginx-ingress-controller itself and it may cause issues if port 10254 already is taken on the host
      # that said, since hostPort is broken on CNI (https://github.com/kubernetes/kubernetes/issues/31307) we have to use hostNetwork where CNI is used
      # like with kubeadm
      # hostNetwork: true
      terminationGracePeriodSeconds: 60
      containers:
      - image: registry.cn-hangzhou.aliyuncs.com/google-containers/nginx-ingress-controller:0.9.0
        name: nginx-ingress-controller
        readinessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
        livenessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          timeoutSeconds: 1
        ports:
        - containerPort: 80
          hostPort: 80
        - containerPort: 443
          hostPort: 443
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
        args:
        - /nginx-ingress-controller
        - --default-backend-service=kube-system/default-http-backend
        #- --publish-service=kube-system/nginx-ingress-lb
        - --apiserver-host=http://10.10.30.102:8080

注意如果啟動(dòng)報(bào)錯(cuò)找不到APIServer，就加上這一行。
如果github上的鏡像無(wú)法pull，可以更換為阿里云。

部署

部署成功后創(chuàng)建以下兩個(gè)pod：

[root@localhost kubernetes]# kubectl get pods  --namespace=kube-system -l k8s-app=nginx-ingress-controller -o wide
NAME                                        READY     STATUS    RESTARTS   AGE       IP            NODE
nginx-ingress-controller-2867543418-f4t1t   1/1       Running   0          46s       172.30.14.4   centos-minion-1
[root@localhost kubernetes]# kubectl get pods  --namespace=kube-system -l k8s-app=nginx-ingress-controller -o wide
NAME                                        READY     STATUS    RESTARTS   AGE       IP            NODE
nginx-ingress-controller-2867543418-f4t1t   1/1       Running   0          49s       172.30.14.4   centos-minion-1

Ingress

創(chuàng)建服務(wù)

部署兩個(gè)版本的nginx。

nginx1
apiVersion: v1
kind: Service
metadata:
  name: nginx1-8
spec:
  ports:
    - port: 80
      targetPort: 80
  selector:
    app: nginx1-8
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx1-8-deployment
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx1-8
    spec:
      containers:
      - name: nginx
        image: docker.io/nginx:latest
        ports:
        - containerPort: 80
nginx2

為了更好區(qū)分，還了個(gè)別的示例

apiVersion: v1
kind: Service
metadata:
  name: nginx2-8
spec:
  ports:
    - port: 80
      targetPort: 80
  selector:
    app: nginx2-8
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx2-8-deployment
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx2-8
    spec:
      containers:
      - name: nginx
        image: docker.io/kubeguide/guestbook-php-frontend
        ports:
        - containerPort: 80

創(chuàng)建完成后，檢查

kubectl get pods -o wide

配置ingress

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: test
spec:
  rules:
  - host: test.xxx1.com
    http:
      paths:
      - backend:
          serviceName: nginx1-8
          servicePort: 80
  - host: test.xxx2.com
    http:
      paths:
      - backend:
          serviceName: nginx2-8
          servicePort: 80

配置hosts文件和host對(duì)應(yīng)，訪問(wèn)即可

#主機(jī) 域名
10.10.30.102    test.xxx1.com
10.10.30.102    test.xxx2.com

測(cè)試

curl -I test.xxx1.com

curl -I test.xxx2.com

自定義上傳文件大小

nginx ingress controller默認(rèn)定義的上傳大小為1M，因此需要更改nginx配置中client_max_body_size的大小，具體修改ingress文件，如下

cat nginx-ingress-cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-configuration
  namespace: kube-system
  labels:
    k8s-app: nginx-ingress-controller
data:
  proxy-body-size: "50m"

nginx-ingress-controller.yml 需要匹配官方文件
https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/without-rbac.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ingress-nginx
  template:
    metadata:
      labels:
        app: ingress-nginx
      annotations:
        prometheus.io/port: '10254'
        prometheus.io/scrape: 'true'
    spec:
      nodeName: centos-minion-1
      #hostNetwork: true
      containers:
        - name: nginx-ingress-controller
          #image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.10.0
          image: registry.cn-hangzhou.aliyuncs.com/google-containers/nginx-ingress-controller:0.9.0
          args:
            - /nginx-ingress-controller
            - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --annotations-prefix=nginx.ingress.kubernetes.io
            - --apiserver-host=http://10.10.30.102:8080
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          volumeMounts:
          - name: logs
            mountPath: /var/log/nginx/
          ports:
          - name: http
            containerPort: 80
          - name: https
            containerPort: 443
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
      volumes:
        - name: logs
          hostPath:
            path: /data/logs/nginx

更多參考：
Kubernetes + Dashboard + Heapster (一) 安裝配置
Kubernetes + Dashboard + Heapster (二) 監(jiān)控部署
Kubernetes + Dashboard + Heapster (三) ingress負(fù)載均衡
Kubernetes + Dashboard + Heapster (四) 慢慢填坑

最后編輯于 ：
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡(jiǎn)書(shū)系信息發(fā)布平臺(tái)，僅提供信息存儲(chǔ)服務(wù)。

友情鏈接更多精彩內(nèi)容