第13章 Kubernetes容器持久化存儲(PersistentVolume)

一. PV與PVC的關系

? PersistenVolume(PV): 對存儲資源創(chuàng)建和使用的抽象，使得存儲作為集群中的資源管理
? PersistentVolumeClaim(PVC): 讓用戶不需要關心具體的Volume實現(xiàn)細節(jié)
PV是提供容量，PVC是消費容量，消費這個過程稱為綁定
PV分為：靜態(tài)和動態(tài)

二. PV靜態(tài)供給及應用案例

1. 創(chuàng)建PV

創(chuàng)建名為my-pv的PV，使用的是網絡存儲NFS

# cat pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: my-pv
spec:
  capacity:
    storage: 5Gi
  accessModes:
    - ReadWriteMany
  nfs:
    path: /data/nfs/www
    server: 10.40.6.214

注意：10.40.6.214 這臺nfs服務器必須存在/data/nfs/www目錄，否則容器啟動失敗
# kubectl create -f pv.yaml
# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM                  STORAGECLASS          REASON   AGE
my-pv                                      5Gi        RWX            Retain           Available                                                         48s

2. 創(chuàng)建PVC

創(chuàng)建PVC也就是卷需求模板, 這yaml配置建議和創(chuàng)建Deployment pod的yaml 文件統(tǒng)一到一個文件，使用"---"分割即可

# cat pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 5Gi

# kubectl create -f pvc.yaml
# kubectl get pvc
NAME                           STATUS   VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/my-pvc   Bound    my-pv    5Gi        RWX                           8m47s

3. PV與PVC綁定

啟動容器應用PVC，這個消費過程稱為綁定，綁定是否成功主要有三要素：名稱、容量和訪問模式。

# cat pod-pvc.yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: 10.40.6.165/library/nginx:v1
        volumeMounts:
        - name: www
          mountPath: /usr/share/nginx/html
        ports:
        - containerPort: 80
      volumes:
      - name: www
        persistentVolumeClaim:
          claimName: my-pvc


# kubectl create -f pod-pvc.yaml
# kubectl get pv,pvc
NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                  STORAGECLASS          REASON   AGE
persistentvolume/my-pv                                      5Gi        RWX            Retain           Bound    default/my-pvc                                        12m

NAME                           STATUS   VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/my-pvc   Bound    my-pv    5Gi        RWX                           8m47s

# kubectl get pod

Kubernetes支持持久卷的存儲插件:
https://kubernetes.io/docs/concepts/storage/persistent-volumes/

三. PV動態(tài)供給及應用案例

Dynamic Provisioning 機制工作的核心在于StorageClass的API對象。
StorageClass 聲明存儲插件，用于自動創(chuàng)建PVC。

1. 創(chuàng)建PV動態(tài)供給流程

image.png

NFS 默認是不支持PV動態(tài)供給，需要插件完成, 這插件也是一個pod
Kubernetes支持動態(tài)供給的存儲插件:
https://kubernetes.io/docs/concepts/storage/storage-classes/

NFS 動態(tài)供給存儲插件：
https://github.com/kubernetes-incubator/external-storage/tree/master/nfs-client

2. 創(chuàng)建存儲類(StorageClass)

# cat storageclass-nfs.yaml 
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
  archiveOnDelete: "true"

# kubectl create -f storageclass-nfs.yaml
# kubectl get storageclass
NAME                  PROVISIONER      AGE
managed-nfs-storage   fuseim.pri/ifs   120m

3. PV動態(tài)供給apiserver授權

NFS動態(tài)供給插件動態(tài)創(chuàng)建PV，要連接apiserver, 所以需要ServiceAccount授權

# cat rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["list", "watch", "create", "update", "patch"]

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

# kubectl create -f rbac.yaml

4. 創(chuàng)建NFS動態(tài)供給插件(Pod)

動態(tài)供給插件（Pod）動態(tài)創(chuàng)建PV，需要訪問apiverser, 所以要指定剛創(chuàng)建的serviceAccount: nfs-client-provisioner

# cat deployment-nfs.yaml 
apiVersion: apps/v1beta1 
kind: Deployment
metadata:
  name: nfs-client-provisioner
spec:
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccount: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: quay.io/external_storage/nfs-client-provisioner:latest
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: fuseim.pri/ifs
            - name: NFS_SERVER
              value: 10.40.6.214
            - name: NFS_PATH
              value: /data/nfs
      volumes:
        - name: nfs-client-root
          nfs:
            server: 10.40.6.214
            path: /data/nfs


# kubectl create -f deployment-nfs.yaml
# kubectl get deploy,pod
NAME                                           DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.extensions/nfs-client-provisioner   1         1         1            1           117m

NAME                                          READY   STATUS    RESTARTS   AGE
pod/nfs-client-provisioner-6c6f8f9bd6-nclwf   1/1     Running   0          117m

5. PV動態(tài)供給應用案例

創(chuàng)建一個statefulset 和 headless service 的mysql服務，在yaml文件定義使用 managed-nfs-storage StorageClass創(chuàng)建PVC。

# cat mysql-statefulset.yaml 
apiVersion: v1
kind: Service
metadata:
  name: mysql
  labels:
    app: mysql
spec:
  ports:
  - port: 3306
    name: mysql
  clusterIP: None
  selector:
    app: mysql-public

---

apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: db
spec:
  serviceName: mysql
  template:
    metadata:
      labels:
        app: mysql-public
    spec:
      containers:
      - name: mysql
        image: mysql:5.7
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: "123456"
        - name: MYSQL_DATABASE
          value: test
        ports:
        - containerPort: 3306
        volumeMounts:
        - mountPath: "/var/lib/mysql"
          name: mysql-data
      volumes:
      - name: mysql-data
        persistentVolumeClaim:
          claimName: mysql-pvc

---

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: managed-nfs-storage
  resources:
    requests:
      storage: 8Gi

# kubectl create -f mysql-statefulset.yaml
# kubectl get svc,pod,pv,pvc
NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
service/mysql        ClusterIP   None         <none>        3306/TCP   58s

NAME                                          READY   STATUS    RESTARTS   AGE
pod/db-0                                      1/1     Running   0          58s

NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                  STORAGECLASS          REASON   AGE
persistentvolume/pvc-a81a2372-9652-11e9-95a4-005056b66bc1   8Gi        RWX            Delete           Bound    default/mysql-pvc      managed-nfs-storage            44s

NAME                              STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS          AGE
persistentvolumeclaim/mysql-pvc   Bound    pvc-a81a2372-9652-11e9-95a4-005056b66bc1   8Gi        RWX            managed-nfs-storage   58s


# kubectl run -it --image mysql:5.7 mysql-client --restart Never --rm /bin/bash
root@mysql-client:/# mysql -h db-0.mysql -uroot -p123456
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
| test               |
+--------------------+
5 rows in set (0.01 sec)

mysql>

五. StatefulSet 存儲狀態(tài)管理機制

創(chuàng)建一個nginx headless service 和StatefulSet pod，并創(chuàng)建掛載pv與pvc
https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/

# cat nginx-statefulset-pv.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  ports:
  - port: 80
    name: web
  clusterIP: None
  selector:
    app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
spec:
  selector:
    matchLabels:
      app: nginx
  serviceName: "nginx"
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: nginx
        image: nginx 
        ports:
        - containerPort: 80
          name: web
        volumeMounts:
        - name: www
          mountPath: /usr/share/nginx/html
  volumeClaimTemplates:
  - metadata:
      name: www
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "managed-nfs-storage"
      resources:
        requests:
          storage: 1Gi

# kubectl create -f nginx-statefulset-pv.yaml
# kubectl get pod
NAME                                      READY   STATUS    RESTARTS   AGE
web-0                                     1/1     Running   0          21s
web-1                                     1/1     Running   0          19s

查看有狀態(tài)應用對存儲做了哪些維護，及查看如何動態(tài)創(chuàng)建PV ：

# kubectl get pv,pvc
NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                  STORAGECLASS          REASON   AGE
persistentvolume/pvc-96003cc3-9655-11e9-95a4-005056b66bc1   1Gi        RWO            Delete           Bound    default/www-web-0      managed-nfs-storage            2m18s
persistentvolume/pvc-977007f8-9655-11e9-95a4-005056b66bc1   1Gi        RWO            Delete           Bound    default/www-web-1      managed-nfs-storage            2m16s

NAME                              STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS          AGE
persistentvolumeclaim/www-web-0   Bound    pvc-96003cc3-9655-11e9-95a4-005056b66bc1   1Gi        RWO            managed-nfs-storage   2m18s
persistentvolumeclaim/www-web-1   Bound    pvc-977007f8-9655-11e9-95a4-005056b66bc1   1Gi        RWO            managed-nfs-storage   2m16s

到nfs 服務器查看 /data/nfs/目錄，會自動創(chuàng)建pv相應的目錄
# ll /data/nfs/
total 4
drwxrwxrwx 2 root root  6 Jun 24 15:56 default-www-web-0-pvc-96003cc3-9655-11e9-95a4-005056b66bc1
drwxrwxrwx 2 root root  6 Jun 24 15:56 default-www-web-1-pvc-977007f8-9655-11e9-95a4-005056b66bc1

這里的PVC有唯一標識: 0、1，www-web-（0|1），為每個pod提供不同的存儲，當某個pod 宕機重新拉起會自動去找對應PVC，比如web-0 pod宕機，重新拉起時會去找到www-web-0，保持pod對應的PVC 是唯一的

最后編輯于 ：
?著作權歸作者所有,轉載或內容合作請聯(lián)系作者
【社區(qū)內容提示】社區(qū)部分內容疑似由AI輔助生成，瀏覽時請結合常識與多方信息審慎甄別。
平臺聲明：文章內容（如有圖片或視頻亦包括在內）由作者上傳并發(fā)布，文章內容僅代表作者本人觀點，簡書系信息發(fā)布平臺，僅提供信息存儲服務。

相關閱讀更多精彩內容

友情鏈接更多精彩內容