前言:

什么是Ingress？

官方的解釋是:
Ingress 是對集群中服務的外部訪問進行管理的 API 對象，典型的訪問方式是 HTTP。
Ingress 可以提供負載均衡、SSL 終結(jié)和基于名稱的虛擬托管。

• Ingress簡介
  Ingress對象，其實就是對“反向代理”的一種抽象，簡單的說就是一個全局的負載均衡器，可以通過訪問URL定位到后端的Service
  有了Ingress這個抽象，K8S就不需要關(guān)心Ingress的細節(jié)了，實際使用時，只需要選擇一個具體的Ingress Controller部署就行了，業(yè)界常用的反向代理項目有：Nginx、HAProxy、Envoy、Traefik 都已經(jīng)成為了K8S專門維護的Ingress Controller
  一個Ingress對象的主要內(nèi)容，就類似Nginx的配置文件描述，對應的轉(zhuǎn)發(fā)規(guī)則就是ingressRule，
  有了Ingress這個對象，用戶就可以根據(jù)自己的需求選擇Ingress Controller，例如，如果應用對代理服務的中斷非常敏感，可以使用Treafik這樣的Ingress Controller

Ingress工作在七層，Service工作在四層，當想要在Kubernetes里為應用進行TLS配置等HTTPS相關(guān)操作時，都必須通過Ingress來進行

• ingress-nginx 簡單的理解就是你原來需要改 Nginx 配置，然后配置各種域名對應哪個 Service，現(xiàn)在把這個動作抽象出來，變成一個 Ingress 對象，你可以用 yaml 創(chuàng)建，每次不要去改 Nginx 了，直接改 yaml 然后創(chuàng)建/更新就行了；那么問題來了：”Nginx 該怎么處理？”

• Ingress Controller 這東西就是解決 “Nginx 的處理方式” 的；Ingress Controoler 通過與 Kubernetes API 交互，動態(tài)的去感知集群中 Ingress 規(guī)則變化，然后讀取他，按照他自己模板生成一段 Nginx 配置，再寫到 Nginx Pod 里，最后 reload 一下，工作流程如下圖：

  
  

• 實際上Ingress也是Kubernetes API的標準資源類型之一，它其實就是一組基于DNS名稱（host）或URL路徑把請求轉(zhuǎn)發(fā)到指定的Service資源的規(guī)則。用于將集群外部的請求流量轉(zhuǎn)發(fā)到集群內(nèi)部完成的服務發(fā)布。我們需要明白的是，Ingress資源自身不能進行“流量穿透”，僅僅是一組規(guī)則的集合，這些集合規(guī)則還需要其他功能的輔助，比如監(jiān)聽某套接字，然后根據(jù)這些規(guī)則的匹配進行路由轉(zhuǎn)發(fā)，這些能夠為Ingress資源監(jiān)聽套接字并將流量轉(zhuǎn)發(fā)的組件就是Ingress Controller

• Ingress 兩種路由方式
  1.虛擬主機
  2.URL 路徑

Ingree-nginx部署

• ingress-nginx 很多全局配置或修改默認配置都是通過annotations 注釋來加載配置文件具體參數(shù)詳細解釋可
  參考官方文檔：

https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/

• 選擇NodePort部署方式

https://kubernetes.github.io/ingress-nginx/deploy/

• extensions/v1beta1 Ingress資源規(guī)范 1.22+版本后會被徹底棄用

apiVersion: extensions/v1betal #資源所屬的API群組和版本
Kind: Ingress#資源類型標識符
metadata: #元數(shù)據(jù)
  name <string> #資源名稱
  annotationsl #資源注解,v1betal使用下面的注解來指定要解析該資源的控制器類型
    kubernetes.io/ingress.class: <string> #適配的Ingress控制器類別
  namespace <string> #名稱空間
spec:
  rules <[]Object> #Ingress規(guī)則列表;
  - host <string>  #虛擬主機的FQDN，支持“*"前綴通配，不支持IP，不支持指定端口
    http <object>
      paths<[]0bject>#虛擬主機PATH定義的列表，由path和backend組成
      - path <string> #流量匹配的HTTP PATH，必須以/開頭
        pathType <string> #匹配機制，支持Exact、_Prefix和ImplementationSpecific
        backend <object> #匹配到的流量轉(zhuǎn)發(fā)到的目標后端
          resource <Object> #引用的同一名稱空間下的資源，與下面兩個字段互斥
          serviceName <string>#引用的Service資源的名稱
          servicePort <string># Service用于提供服務的端口
  tls <[]object> #TLS配置，用于指定上rules中定義的哪些host需要工作HTTPS模式
  - hosts <[]string>  #使用同一組證書的主機名稱列表
    secretName <string>  #保存于數(shù)字證書和私鑰信息的secret資源名稱
  backend <object>   #默認backend的定義，可嵌套字段及使用格式跟rules字段中的相同
  ingressClassName <string> #ingress類名稱，用于指定適配的控制器

• v1 Ingress資源規(guī)范

apiVersion: networking.k8s.io/v1 #資源所屬的API群組和版本
kind: Ingress #資源類型標識符
metadata: #元數(shù)據(jù)
  name <string> #資源名稱
  annotations: #資源注解,vlbetal使用下面的注解來指定要解析該資源的控制器類型
    kubernetes.io/ingress.class:<string> #適配的Ingress控制器類別
  namespace <string>  #名稱空間
spec:
  rules <[]object> #Ingress規(guī)則列表
  - host <string>  #虛擬主機的FQDN，支持“*"前綴通配，不支持IP，不支持指定端口
    http <object>
      paths <[]object>#虛擬主機PATH定義的列表，由path和backend組成
      - path <string> #流量匹配的HTTP PATH，必須以/開頭
        pathType <string> #支持Exact、Prefix和ImplementationSpecific，必選
        backend <Object>#匹配到的流量轉(zhuǎn)發(fā)到的目標后端
          resource <object>  #引用的同一名稱空間下的資源，與下面兩個字段互斥
          service <object> #關(guān)聯(lián)的后端Service對象
            name <string> #后端Service的名稱
            port bject> #后端Service上的端口對象
              name <string> #端口名稱
              number <integer>  #端口號
  tls <[]object>  #TLS配置，用于指定上rules中定義的哪些host需要工作HTTPS模式
  - hosts <[]string>  #使用同一組證書的主機名稱列表
    secretName <string> #保存于數(shù)字證書和私鑰信息的secret資源名稱
  backend <object>  #默認backend的定義，可嵌套字段及使用格式跟rules字段中的相同
  ingressClassName <string> #ingress類名稱，用于指定適配的控制器

• 添加externalIPs:節(jié)點3 IP非必須步驟，方便記憶使用 而不是記憶NodePort端口 NodePort與externalIPs可同時訪問

[root@k8s-master Ingress]# vim deploy.yaml
...
---
# Source: ingress-nginx/templates/controller-service.yaml
apiVersion: v1
kind: Service
metadata:
  annotations:
  labels:
    helm.sh/chart: ingress-nginx-4.0.1
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.0.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  type: NodePort
  externalIPs: [192.168.54.173]   #添加externalIPs字段 固定訪問IP
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
      appProtocol: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: https
      appProtocol: https
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/component: controller
...

• 部署Ingress-nginx

[root@k8s-master Ingress]# kubectl apply -f deploy.yaml 
namespace/ingress-nginx unchanged
serviceaccount/ingress-nginx unchanged
configmap/ingress-nginx-controller configured
clusterrole.rbac.authorization.k8s.io/ingress-nginx unchanged
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx unchanged
role.rbac.authorization.k8s.io/ingress-nginx unchanged
rolebinding.rbac.authorization.k8s.io/ingress-nginx unchanged
service/ingress-nginx-controller-admission unchanged
service/ingress-nginx-controller configured
deployment.apps/ingress-nginx-controller configured
ingressclass.networking.k8s.io/nginx unchanged
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission configured
serviceaccount/ingress-nginx-admission unchanged
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission unchanged
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission unchanged
role.rbac.authorization.k8s.io/ingress-nginx-admission unchanged
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission unchanged
job.batch/ingress-nginx-admission-create unchanged
job.batch/ingress-nginx-admission-patch unchanged
[root@k8s-master Ingress]# kubectl get pod
NAME                              READY   STATUS    RESTARTS   AGE
etcd-operator-646cbffdb6-brbn6    1/1     Running   0          19h
example-etcd-cluster-5fb5d9d6n8   1/1     Running   0          49m
example-etcd-cluster-nc8pdgjrjr   1/1     Running   0          19h
example-etcd-cluster-svgdngq28k   1/1     Running   0          48m
[root@k8s-master Ingress]# kubectl get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP     EXTERNAL-IP      PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.109.24.78   192.168.54.173   80:32493/TCP,443:30239/TCP   120m
ingress-nginx-controller-admission   ClusterIP   10.110.72.52   <none>           443/TCP                      120m

示例1：創(chuàng)建 Ingress-nginx虛擬主機

• 創(chuàng)建 Deployment 和與之對應的SVC

[root@k8s-master Ingress]# cat deployment-demo.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-demo
  namespace: default
spec:
  replicas: 4
  selector:
    matchLabels:
      app: demoapp
      release: stable
  template:
    metadata:
      labels :
        app: demoapp
        release: stable
    spec:
      containers:
      - name: demoapp
        image: ikubernetes/demoapp:v1.1
        ports:
        - containerPort: 80
          name: http

---
apiVersion: v1
kind: Service
metadata:
  name: demoapp-deploy
  namespace: default
spec:
  selector:
    app: demoapp
    release: stable
  ports:
  - name: http
    port: 80
    targetPort: 80

• 創(chuàng)建 ingress-nginx

[root@k8s-master Ingress]# cat  demoapp-ingress.yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-demo
  annotations:
    kubernetes.io/ingress.class: "nginx"
  namespace: default
spec:
  rules:
  - host: www.ik8s.io  #虛擬主機
    http:
      paths:
      - path: /
        pathType: Prefix  #前綴匹配
        backend:
          service:
            name: demoapp-deploy
            port:
              number: 80

[root@k8s-master Ingress]# kubectl apply -f deployment-demo.yaml
[root@k8s-master Ingress]# kubectl apply -f demoapp-ingress.yaml

[root@k8s-master Ingress]# kubectl get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP     EXTERNAL-IP      PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.109.24.78   192.168.54.173   80:32493/TCP,443:30239/TCP   5h50m
ingress-nginx-controller-admission   ClusterIP   10.110.72.52   <none>           443/TCP                      5h50m

[root@k8s-master Ingress]# kubectl get ingress
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
NAME           CLASS    HOSTS         ADDRESS         PORTS   AGE
ingress-demo   <none>   www.ik8s.io   192.168.4.171   80      3h11m

• 訪問測試

[root@bigyong ~]# cat /etc/hosts   
# ::1       localhost localhost.localdomain localhost6 localhost6.localdomain6
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4

...
192.168.54.173   www.ik8s.io   #寫hosts

[root@bigyong ~]# curl 192.168.54.173  #直接訪問只能到ingress-nginx 沒到轉(zhuǎn)發(fā)到后端
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
[root@bigyong ~]# curl -H  "Host:www.ik8s.io" 192.168.54.173  #訪問成功
iKubernetes demoapp v1.1 !! ClientIP: 192.168.113.37, ServerName: deployment-demo-867c7d9d55-gw6qp, ServerIP: 192.168.113.39
[root@bigyong ~]# curl -H  "Host:www.ik8s.io" 192.168.54.173
iKubernetes demoapp v1.1 !! ClientIP: 192.168.113.37, ServerName: deployment-demo-867c7d9d55-9lnpq, ServerIP: 192.168.12.39!
[root@bigyong ~]# curl -H  "Host:www.ik8s.io" 192.168.54.173
iKubernetes demoapp v1.1 !! ClientIP: 192.168.113.37, ServerName: deployment-demo-867c7d9d55-2zcr5, ServerIP: 192.168.51.61!
[root@bigyong ~]# curl -H  "Host:www.ik8s.io" 192.168.54.173
iKubernetes demoapp v1.1 !! ClientIP: 192.168.113.37, ServerName: deployment-demo-867c7d9d55-9lnpq, ServerIP: 192.168.12.39!

示例2： 創(chuàng)建TLS Ingress HTTPS

[root@k8s-master Ingress]# cat   demoapp-ingress.yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-demo
  annotations:
    kubernetes.io/ingress.class: "nginx"
  namespace: default
spec:
  rules:
  - host: www.ik8s.io
    http:
      paths:
      - path: /
        pathType: Prefix  #前綴匹配
        backend:
          service:
            name: demoapp-deploy
            port:
              number: 80
  tls:  #添加tls
  - hosts:
    - www.ik8s.io
    secretName: ik8s-tls
    
[root@k8s-master Ingress]# kubectl  apply -f demoapp-ingress.yaml 
ingress.networking.k8s.io/ingress-demo configured

[root@k8s-master Ingress]# kubectl describe ingress ingress-demo
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
Name:             ingress-demo
Namespace:        default
Address:          192.168.4.171
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
TLS:
  ik8s-tls terminates www.ik8s.io
Rules:
  Host         Path  Backends
  ----         ----  --------
  www.ik8s.io  
               /   demoapp-deploy:80 (192.168.113.39:80,192.168.12.39:80)
Annotations:   kubernetes.io/ingress.class: nginx
Events:
  Type    Reason  Age                   From                      Message
  ----    ------  ----                  ----                      -------
  Normal  Sync    113s (x3 over 6h36m)  nginx-ingress-controller  Scheduled for sync

• 創(chuàng)建tls自簽證書

[root@k8s-master Ingress]# (umask 077); openssl  genrsa -out tls.key 2048
Generating RSA private key, 2048 bit long modulus
............................+++
.......................+++
e is 65537 (0x10001)
[root@k8s-master Ingress]# openssl req -new -x509 -key tls.key -out tls.crt -subj "/CN=www.ik8s.io" -days 365
[root@k8s-master Ingress]# ls
demoapp-ingress.yaml    deployment-demo.yaml  tls.crt
deploy-externalIP.yaml  deploy.yaml           tls.key

• 創(chuàng)建Secret

[root@k8s-master Ingress]# kubectl create secret tls ik8s-tls --cert=./tls.crt --key=./tls.key
secret/ik8s-tls created

• 訪問測試

[root@k8s-master Ingress]# curl -H  "Host:www.ik8s.io" 192.168.54.173   #308已經(jīng)被重定向
<html>
<head><title>308 Permanent Redirect</title></head>
<body>
<center><h1>308 Permanent Redirect</h1></center>
<hr><center>nginx</center>
</body>
</html>

#通過https訪問  提示證書無效不被信任
[root@k8s-master Ingress]# curl -H  "Host:www.ik8s.io" https://192.168.54.173 
curl: (60) Issuer certificate is invalid.
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

• 忽略風險 訪問成功

[root@k8s-master Ingress]# curl -k -H  "Host:www.ik8s.io" https://192.168.54.173  
iKubernetes demoapp v1.1 !! ClientIP: 192.168.113.37, ServerName: deployment-demo-867c7d9d55-9lnpq, ServerIP: 192.168.12.39!

示例3：為dashboard 添加ingress

https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/

[root@k8s-master Ingress]# cat ingress-kubernetes-dashboard.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: dashboard
  annotations:
    kubernetes.io/ingress.class: "nginx" #指定控制器
    ingress.kubernetes.io/ssl-passthrough: "true"   #tcp代理 因為后端dashboard必須通過https訪問  這里通過4層轉(zhuǎn)發(fā)直接轉(zhuǎn)發(fā)到后端Pod
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" #后端直接通過https協(xié)議訪問
    nginx.ingress.kubernetes.io/rewrite-target: /$2 #重寫標記 path:/dashboard(/|$)(.*) 這里的第2部分
  namespace: kubernetes-dashboard
spec:  #這里沒有配置host 相當于*號通配所有主機 
  rules:
  - http:
      paths:
      - path: /dashboard(/|$)(.*)
        backend:
          serviceName: kubernetes-dashboard
          servicePort: 443

[root@k8s-master Ingress]# kubectl apply -f ingress-kubernetes-dashboard.yaml 
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
ingress.extensions/dashboard created

[root@k8s-master Ingress]# kubectl describe ingress dashboard -n kubernetes-dashboard
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
Name:             dashboard
Namespace:        kubernetes-dashboard
Address:          192.168.4.171
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host        Path  Backends
  ----        ----  --------
  *           
              /dashboard(/|$)(.*)   kubernetes-dashboard:443 (192.168.51.64:8443)  #后端pod
Annotations:  ingress.kubernetes.io/ssl-passthrough: true
              kubernetes.io/ingress.class: nginx
              nginx.ingress.kubernetes.io/backend-protocol: HTTPS
              nginx.ingress.kubernetes.io/rewrite-target: /$2
Events:
  Type    Reason  Age                   From                      Message
  ----    ------  ----                  ----                      -------
  Normal  Sync    103s (x2 over 2m41s)  nginx-ingress-controller  Scheduled for sync

• 訪問測試

注意：https://192.168.54.173/dashboard/ 一定要帶上最后的/ $2為/后面部分 不然無法訪問

示例4：longhorn添加ingress-nginx basic認證

參考官方文檔

https://longhorn.io/docs/1.2.0/deploy/accessing-the-ui/longhorn-ingress/

• 默認longhorn沒有登錄認證暴露到公網(wǎng)會存在風險 首先添加basic認證

[root@k8s-master Ingress]# kubectl get pod -n longhorn-system 
NAME                                        READY   STATUS    RESTARTS   AGE
....
longhorn-manager-cc8sp                      1/1     Running   0          149m
longhorn-manager-fs5tx                      1/1     Running   2          149m
longhorn-manager-vwbzn                      1/1     Running   1          149m
longhorn-ui-79f8976fbf-c44ct                1/1     Running   1          149m
[root@k8s-master Ingress]# kubectl get svc  -n longhorn-system 
NAME                TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)     AGE
csi-attacher        ClusterIP   10.108.190.205   <none>        12345/TCP   142m
csi-provisioner     ClusterIP   10.99.216.181    <none>        12345/TCP   142m
csi-resizer         ClusterIP   10.96.161.192    <none>        12345/TCP   141m
csi-snapshotter     ClusterIP   10.101.216.72    <none>        12345/TCP   141m
longhorn-backend    ClusterIP   10.108.67.31     <none>        9500/TCP    149m
longhorn-frontend   ClusterIP   10.107.71.176    <none>        80/TCP      149m

[root@k8s-master Ingress]# USER=user.com; PASSWORD=passwd.com; echo "${USER}:$(openssl passwd -stdin -apr1 <<< ${PASSWORD})" >> auth
[root@k8s-master Ingress]# ls
auth                  deploy-externalIP.yaml  deploy.yaml                        tls.crt
demoapp-ingress.yaml  deployment-demo.yaml    ingress-kubernetes-dashboard.yaml  tls.key

[root@k8s-master Ingress]# kubectl -n longhorn-system create secret generic basic-auth --from-file=authsecret/basic-auth created

[root@k8s-master Ingress]# cat longhorn-ui-ingress.yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: longhorn-ingress
  namespace: longhorn-system
  annotations:
    kubernetes.io/ingress.class: "nginx"   #指定控制器
    # type of authentication
    nginx.ingress.kubernetes.io/auth-type: basic  #認證類型
    # prevent the controller from redirecting (308) to HTTPS
    nginx.ingress.kubernetes.io/ssl-redirect: 'false'     #http通信
    # name of the secret that contains the user/password definitions
    nginx.ingress.kubernetes.io/auth-secret: basic-auth    #secret名稱
    # message to display with an appropriate context why the authentication is required
    nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required '   #輸入提示
    # custom max body size for file uploading like backing image uploading
    nginx.ingress.kubernetes.io/proxy-body-size: 10000m
    nginx.ingress.kubernetes.io/rewrite-target: /$2 
spec:
  rules:
  - http:
      paths:
      - pathType: Prefix
        path: /longhorn(/|$)(.*)
        backend:
          service:
            name: longhorn-frontend
            port:
              number: 80

[root@k8s-master Ingress]# kubectl apply -f longhorn-ui-ingress.yaml 
ingress.networking.k8s.io/longhorn-ingress configured

• 訪問 http://IP/longhorn/ #最后的 "/" 一定要加

• 以下圖片IP 重新部署后的服務IP