kubernetes中常用的Ingress Controller

之前文章介紹了常用一些Ingress Controller,如下：

  • HAProxy Ingress Controller
  • Nginx Ingress Controller
  • Traefik Ingress Controller
  • Kong Ingress Controller

今天來詳細(xì)介紹兩種常用的Ingress Controller.

二、Nginx Ingress

2.1、安裝

2.1.1 在線安裝

在線安裝直接執(zhí)行以下命令：

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml

然后安裝NodePort：

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/baremetal/service-nodeport.yaml

2.1.2、離線安裝

到https://github.com/kubernetes/ingress-nginx/tree/master/deploy/static這個(gè)下面下載對(duì)應(yīng)的YAML文件，有configmap.yaml，namespace.yaml，rbac.yaml，with-rbac.yaml，可以寫一個(gè)如下循環(huán)下載：

for yaml in configmap.yaml namespace.yaml rbac.yaml with-rbac.yaml; do wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/$yaml;done

然后執(zhí)行如下命令安裝，前提先執(zhí)行namespace.yaml這個(gè)YAML文件：

kubectl apply -f namespace.yaml
kubectl apply -f .

然后我們可以執(zhí)行以下命令查看：

[root@master nginx]# kubectl get pod -n ingress-nginx
NAME                                        READY   STATUS              RESTARTS   AGE
nginx-ingress-controller-799dbf6fbd-zxvmp   0/1     ContainerCreating   0          12s

然后安裝nodePort:

wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/baremetal/service-nodeport.yaml
kubectl apply -f service-nodeport.yaml

然后我修改了nodePort的端口，如下：

apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  type: NodePort
  ports:
    - name: http
      port: 80
      targetPort: 80
      nodePort: 30080
      protocol: TCP
    - name: https
      port: 443
      targetPort: 443
      nodePort: 30443
      protocol: TCP
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

然后查看結(jié)果：

[root@master nginx]# kubectl get pod -n ingress-nginx
NAME                                        READY   STATUS    RESTARTS   AGE
nginx-ingress-controller-799dbf6fbd-zxvmp   1/1     Running   0          52m
[root@master nginx]# kubectl get svc -n ingress-nginx
NAME            TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx   NodePort   10.68.194.177   <none>        80:30080/TCP,443:30443/TCP   46m

2.2、使用

先創(chuàng)建Pod，service，定義如下YAML：
[root@master test]# cat nginx-ingress-demo.yaml

---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-deploy
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx-demo
      release: canary
  template:
    metadata:
      name: my-nginx
      labels:
        app: nginx-demo
        release: canary
    spec:
      containers:
      - name: my-nginx
        image: nginx
        imagePullPolicy: IfNotPresent
        ports:
        - name: http
          containerPort: 80

---
apiVersion: v1
kind: Service
metadata:
  name: nginx-svc
  namespace: default
spec:
  selector:
    app: nginx-demo
    release: canary
  ports:
  - name: http
    port: 80
    targetPort: 80

然后執(zhí)行kubectl apply -f nginx-ingress-demo.yaml，查看結(jié)果：

[root@master test]# kubectl get pod
NAME                            READY   STATUS    RESTARTS   AGE
nginx-deploy-549cb5c7ff-65fbb   1/1     Running   0          21s
nginx-deploy-549cb5c7ff-75fmt   1/1     Running   0          21s
nginx-deploy-549cb5c7ff-wg4w7   1/1     Running   0          17s
[root@master test]# kubectl get svc
NAME                     TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
kubernetes               ClusterIP   10.68.0.1      <none>        443/TCP          5d2h
nginx-svc                ClusterIP   10.68.214.28   <none>        80/TCP           6m21s

然后定義Ingress Nginx，YAML文件如下：
[root@master test]# cat ingress-nginx.yaml

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress-nginx
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: 
    http:
      paths:
      - path:
        backend:
          serviceName: nginx-svc
          servicePort: 80

然后查看結(jié)果：

[root@master test]# kubectl describe ingresses ingress-nginx 
Name:             ingress-nginx
Namespace:        default
Address:          10.68.194.177
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host  Path  Backends
  ----  ----  --------
  *     
           nginx-svc:80 (172.20.1.61:80,172.20.1.62:80,172.20.2.87:80)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"networking.k8s.io/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-nginx","namespace":"default"},"spec":{"rules":[{"host":null,"http":{"paths":[{"backend":{"serviceName":"nginx-svc","servicePort":80},"path":null}]}}]}}

  kubernetes.io/ingress.class:  nginx
Events:
  Type    Reason  Age                From                      Message
  ----    ------  ----               ----                      -------
  Normal  CREATE  11m                nginx-ingress-controller  Ingress default/ingress-nginx
  Normal  UPDATE  77s (x3 over 11m)  nginx-ingress-controller  Ingress default/ingress-nginx

在流量器訪問：


image.png
image.png

2.3、TLS

Nginx Ingress不止支持HTTP，還支持HTTPS。我們使用如下命令手動(dòng)創(chuàng)建證書：

openssl genrsa -out tls.key 2048
openssl req -new -x509  -key tls.key -out tls.crt -subj "/C=CN/ST=Chongqing/L=Chongqing/OU=DevOps/CN=nginx.joker.com"

然后配置Secrect：

kubectl create secret tls tls-secret --key tls.key --cert tls.crt

然后查看：

[root@master test]# kubectl get secret
NAME                  TYPE                                  DATA   AGE
default-token-gmdbb   kubernetes.io/service-account-token   3      5d3h
tls-secret            kubernetes.io/tls                     2      20s
[root@master test]# kubectl describe secrets tls-secret
Name:         tls-secret
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  kubernetes.io/tls

Data
====
tls.crt:  1302 bytes
tls.key:  1679 bytes

配置Ingress nginx:
[root@master test]# cat ingress-nginx-tls.yaml

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress-nginx-tls
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
  - host:
    secretName: tls-secret
  rules:
  - host: 
    http:
      paths:
      - path:
        backend:
          serviceName: nginx-svc
          servicePort: 80

然后查看結(jié)果：

[root@master test]# kubectl get ingresses.
NAME                HOSTS   ADDRESS         PORTS     AGE
ingress-nginx-tls   *                       80, 443   4s

瀏覽器訪問：


image.png
image.png

三、Traefik Ingress

3.1、安裝

創(chuàng)建namespace（namespace.yaml）：

apiVersion: v1
kind: Namespace
metadata:
  name: ingress-traefik
  labels:
    app.kubernetes.io/name: ingress-traefik
    app.kubernetes.io/part-of: ingress-traefik
---

創(chuàng)建RBAC安全認(rèn)證（rdac.yaml）：

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: ingress-traefik 
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: ingress-traefik

配置traefik（traefik.yaml）：

---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: traefik-ingress-controller
  namespace: ingress-traefik 
  labels:
    k8s-app: traefik-ingress-lb
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: traefik-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
#      tolerations:
#      - operator: "Exists"
#      nodeSelector:
#        kubernetes.io/hostname: master
      containers:
      - image: traefik:v1.7.17
        name: traefik-ingress-lb
        ports:
        - name: http
          containerPort: 80
        - name: admin
          containerPort: 8080
        args:
        - --api
        - --kubernetes
        - --logLevel=INFO
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: ingress-traefik 
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 80
      name: web
      nodePort: 38000
    - protocol: TCP
      port: 8080
      nodePort: 38080
      name: admin
  type: NodePort

然后執(zhí)行如下命令創(chuàng)建：

kubectl apply -f .

查看其結(jié)果：

[root@master traefik]# kubectl get svc -n ingress-traefik 
NAME                      TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)                       AGE
traefik-ingress-service   NodePort   10.68.173.196   <none>        80:38000/TCP,8080:38080/TCP   9m19s
[root@master traefik]# kubectl get pod -n ingress-traefik 
NAME                                          READY   STATUS    RESTARTS   AGE
traefik-ingress-controller-86769d5d99-fvf82   1/1     Running   0          7m10s

Traefik提供一個(gè)WEB UI工具，就是上面8080端口，可以通過其映射的NodePort端口在瀏覽器訪問，如下：


image.png
image.png

3.2、使用

使用和上面nginx ingress一樣，只是在annotations里配置的kubernetes.io/ingress.class: "traefik"。

3.3、配置

生成SSL證書：

openssl req -newkey rsa:2048 -nodes -keyout tls.key -x509 -days 365 -out tls.crt

創(chuàng)建Secret來存儲(chǔ)證書：

# kubectl create secret generic traefik-cert --from-file=tls.crt --from-file=tls.key -n ingress-traefik

配置traefik，讓其支持https，配置文件如下：

defaultEntryPoints = ["http", "https"]

[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
      entryPoint = "https"
  [entryPoints.https]
  address = ":443"
    [entryPoints.https.tls]
      [[entryPoints.https.tls.certificates]]
      CertFile = "/ssl/tls.crt"
      KeyFile = "/ssl/tls.key"

將上面的 traefik.toml 配置文件通過一個(gè) ConfigMap 對(duì)象掛載到 traefik pod 中去。
創(chuàng)建configMap：

# kubectl create configmap traefik-conf --from-file=traefik.toml -n ingress-traefik

更改traefik的YAML文件：

---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: traefik-ingress-controller
  namespace: ingress-traefik 
  labels:
    k8s-app: traefik-ingress-lb
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: traefik-ingress-lb
      name: traefik-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
#      tolerations:
#      - operator: "Exists"
#      nodeSelector:
#        kubernetes.io/hostname: master
      containers:
      - image: traefik:v1.7.17
        name: traefik-ingress-lb
        volumeMounts:
        - name: ssl
          mountPath: "/ssl"
        - name: config
          mountPath: "/config"
        ports:
        - name: http
          containerPort: 80
        - name: https
          containerPort: 443
        - name: admin
          containerPort: 8080
        args:
        - --configfile=/config/traefik.toml
        - --api
        - --kubernetes
        - --logLevel=INFO
      volumes:
      - name: ssl
        secret:
          secretName: traefik-cert
      - name: config
        configMap:
          name: traefik-conf
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: ingress-traefik 
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 80
      name: web
      nodePort: 38000
    - protocol: TCP
      port: 8080
      nodePort: 38080
      name: admin
  type: NodePort

然后生成配置清單，并查看日志觀察其是否生效；

# kubectl apply -f traefik.yaml
# kubectl logs traefik-ingress-controller-867f5668dd-7p7tv -n ingress-traefik 
time="2019-11-25T08:25:30Z" level=info msg="Using TOML configuration file /config/traefik.toml"
time="2019-11-25T08:25:30Z" level=info msg="No tls.defaultCertificate given for https: using the first item in tls.certificates as a fallback."
time="2019-11-25T08:25:30Z" level=info msg="Traefik version v1.7.17 built on 2019-09-23_06:22:08PM"
time="2019-11-25T08:25:30Z" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://docs.traefik.io/v1.7/basics/#collected-data\n"
time="2019-11-25T08:25:30Z" level=info msg="Preparing server https &{Address::443 TLS:0xc0002b22d0 Redirect:<nil> Auth:<nil> WhitelistSourceRange:[] WhiteList:<nil> Compress:false ProxyProtocol:<nil> ForwardedHeaders:0xc00050a100} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s"
time="2019-11-25T08:25:30Z" level=info msg="Preparing server traefik &{Address::8080 TLS:<nil> Redirect:<nil> Auth:<nil> WhitelistSourceRange:[] WhiteList:<nil> Compress:false ProxyProtocol:<nil> ForwardedHeaders:0xc00050a120} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s"
time="2019-11-25T08:25:30Z" level=info msg="Starting server on :443"
time="2019-11-25T08:25:30Z" level=info msg="Preparing server http &{Address::80 TLS:<nil> Redirect:0xc000843fc0 Auth:<nil> WhitelistSourceRange:[] WhiteList:<nil> Compress:false ProxyProtocol:<nil> ForwardedHeaders:0xc00050a0e0} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s"
time="2019-11-25T08:25:30Z" level=info msg="Starting server on :8080"
time="2019-11-25T08:25:30Z" level=info msg="Starting provider configuration.ProviderAggregator {}"
time="2019-11-25T08:25:30Z" level=info msg="Starting server on :80"
time="2019-11-25T08:25:30Z" level=info msg="Starting provider *kubernetes.Provider {\"Watch\":true,\"Filename\":\"\",\"Constraints\":[],\"Trace\":false,\"TemplateVersion\":0,\"DebugLogGeneratedTemplate\":false,\"Endpoint\":\"\",\"Token\":\"\",\"CertAuthFilePath\":\"\",\"DisablePassHostHeaders\":false,\"EnablePassTLSCert\":false,\"Namespaces\":null,\"LabelSelector\":\"\",\"IngressClass\":\"\",\"IngressEndpoint\":null,\"ThrottleDuration\":0}"
time="2019-11-25T08:25:30Z" level=info msg="ingress label selector is: \"\""
time="2019-11-25T08:25:30Z" level=info msg="Creating in-cluster Provider client"
time="2019-11-25T08:25:30Z" level=info msg="Server configuration reloaded on :443"
time="2019-11-25T08:25:30Z" level=info msg="Server configuration reloaded on :8080"
time="2019-11-25T08:25:30Z" level=info msg="Server configuration reloaded on :80"

3.4、Rewrite

測(cè)試demo：

apiVersion: v1
kind: Service
metadata:
  name: nginx-demo
spec:
  selector:
    app: nginx
  ports:
  - name: nginx
    port: 80
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        imagePullPolicy: IfNotPresent

正常訪問的ingress如下：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ingress-demo
  annotations:
    kubernetes.io/ingress.class: "traefik"
spec:
  rules:
  - host:
    http:
      paths:
      - backend:
          serviceName: nginx-demo
          servicePort: 80
        path: /

但是如果我們把path改為如下：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ingress-demo
  annotations:
    kubernetes.io/ingress.class: "traefik"
spec:
  rules:
  - host:
    http:
      paths:
      - backend:
          serviceName: nginx-demo
          servicePort: 80
        path: /app

那么這時(shí)候訪問就會(huì)報(bào)404.


image.png
image.png

這時(shí)候我們可以在配置里加上rewrite規(guī)則，如下：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ingress-demo
  annotations:
    kubernetes.io/ingress.class: "traefik"
    traefik.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host:
    http:
      paths:
      - backend:
          serviceName: nginx-demo
          servicePort: 80
        path: /app

然后就可以正常訪問了，如下：


image.png
image.png

為了更友好化，我們可以定義如下：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ingress-demo
  annotations:
    kubernetes.io/ingress.class: "traefik"
    traefik.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
  - host:
    http:
      paths:
      - backend:
          serviceName: nginx-demo
          servicePort: 80
        path: /app(/|$)(.*)

但是這種還是會(huì)有問題，比如有的靜態(tài)資源無法正常顯示，比如靜態(tài)資源的路徑在/static下面，現(xiàn)在我們做了 url rewrite 過后，要正常訪問也需要帶上前綴才可以：http://xxx.xxx/static/screen.css，對(duì)于圖片或者其他靜態(tài)資源也是如此，當(dāng)然我們?nèi)ジ捻?yè)面引入靜態(tài)資源的方式為相對(duì)路徑也是可以的，但是畢竟要修改代碼，這個(gè)時(shí)候我們可以借助 ingress-traefik 中的 configuration-snippet 來對(duì)靜態(tài)資源做一次跳轉(zhuǎn)，如下所示：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ingress-demo
  annotations:
    kubernetes.io/ingress.class: "traefik"
    traefik.ingress.kubernetes.io/rewrite-target: /$2
    traefik.ingress.kubernetes.io/configuration-snippet: |
      rewrite ^/static/(.*)$ /app/static/$1 redirect;
      rewrite ^/image/(.*)$ /app/image/$1 redirect;
spec:
  rules:
  - host:
    http:
      paths:
      - backend:
          serviceName: nginx-demo
          servicePort: 80
        path: /app(/|$)(.*)

這時(shí)候就可以正常加載靜態(tài)資源了。

但是我們還想直接訪問主域名，但是會(huì)報(bào)404，要解決這種問題，可以app-root的注解，這樣我們?cè)L問主域名的時(shí)候就可以跳轉(zhuǎn)到app-root目錄（真實(shí)存在的路徑）下面，如下：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ingress-demo
  annotations:
    kubernetes.io/ingress.class: "traefik"
    traefik.ingress.kubernetes.io/app-root: /app/
    traefik.ingress.kubernetes.io/rewrite-target: /$2
    traefik.ingress.kubernetes.io/configuration-snippet: |
      rewrite ^/static/(.*)$ /app/static/$1 redirect;
      rewrite ^/image/(.*)$ /app/image/$1 redirect;
spec:
  rules:
  - host:
    http:
      paths:
      - backend:
          serviceName: nginx-demo
          servicePort: 80
        path: /app(/|$)(.*)

但是還有一個(gè)問題是我們的 path 路徑其實(shí)也匹配了 /app 這樣的路徑，可能我們更加希望我們的應(yīng)用在最后添加一個(gè) / 這樣的 slash，同樣我們可以通過 configuration-snippet 配置來完成，如下 Ingress 對(duì)象：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ingress-demo
  annotations:
    kubernetes.io/ingress.class: "traefik"
    traefik.ingress.kubernetes.io/app-root: /app/
    traefik.ingress.kubernetes.io/rewrite-target: /$2
    traefik.ingress.kubernetes.io/configuration-snippet: |
      rewrite ^(/app)$ $1/ redirect;
      rewrite ^/static/(.*)$ /app/static/$1 redirect;
      rewrite ^/image/(.*)$ /app/image/$1 redirect;
spec:
  rules:
  - host:
    http:
      paths:
      - backend:
          serviceName: nginx-demo
          servicePort: 80
        path: /app(/|$)(.*)

-----------------------

公眾號(hào)：?jiǎn)踢吂适拢↖D：qiaobiangushi）

知乎: 喬邊故事

頭條號(hào)：?jiǎn)踢吂适?/strong>

只要臉皮夠厚，整個(gè)世界都將被你踩在腳下。

-----------------------

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡(jiǎn)書系信息發(fā)布平臺(tái)，僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容