實(shí)戰(zhàn)：用“廉價(jià)”的NFS作為K8S后端存儲(chǔ)

K8S后端存儲(chǔ)：NFS

大家都知道，NFS是一種基于網(wǎng)絡(luò)的文件系統(tǒng)協(xié)議，允許在不同的機(jī)器之間共享文件系統(tǒng)資源。在K8S中，可以使用NFS作為后端存儲(chǔ)，以提供持久化存儲(chǔ)和共享存儲(chǔ)卷。但是否適合在生產(chǎn)環(huán)境使用NFS作為后端存儲(chǔ)，這取決于具體的應(yīng)用程序和使用場景。如果應(yīng)用程序?qū)π阅芎涂煽啃砸蟊容^高，可能需要選擇其他更適合的存儲(chǔ)方案，比如ceph。如果只是在測試或者開發(fā)環(huán)境中，我覺得使用NFS可以更方便地實(shí)現(xiàn)共享存儲(chǔ)卷，提高測試或者開發(fā)的效率。

搭建NFS

# step1 安裝yum install nfs-utils -y
yum install nfs-utils -y

# step2 創(chuàng)建NFS共享目錄
mkdir /data/nfs_k8s_storage_share

# step3 配置NFS共享，編輯/etc/exports文件，將要共享的目錄添加到文件中
/data/nfs_k8s_storage_share *(rw,sync,no_root_squash) # 這將允許任何客戶端以讀寫模式訪問共享目錄。

# step4 重新加載NFS配置
exportfs -r

# step5 啟動(dòng)NFS服務(wù)
systemctl start nfs-server
systemctl enable nfs-server

# step6 確認(rèn)NFS服務(wù)正在運(yùn)行
systemctl status nfs-server

# step7 注意防火墻，可以關(guān)閉或添加允許策略
firewall-cmd --permanent --zone=public --add-service=nfs
firewall-cmd --reload

# step8 驗(yàn)證是否共享成功
showmount -e 192.168.11.254

k8s工作節(jié)點(diǎn)安裝nfs-utils

安裝完即可，無需做任何配置

yum install nfs-utils -y

“

具體在哪個(gè)工作節(jié)點(diǎn)安裝得看在下面的deployment.yaml中拉起的NFS客戶端配置程序pod跑在哪個(gè)節(jié)點(diǎn)上，可以讓該pod調(diào)度到指定的節(jié)點(diǎn)，那么該節(jié)點(diǎn)就要安裝好nfs-utils。如果不干涉調(diào)度行為，或者考慮到后續(xù)要跑多個(gè)副本就可以將全部工作節(jié)點(diǎn)都安裝好nfs-utils。

”

在k8s master節(jié)點(diǎn)上安裝nfs動(dòng)態(tài)供給插件

倉庫： https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner

相關(guān)yaml下載鏈接： https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/blob/master/deploy/rbac.yaml https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/blob/master/deploy/deployment.yaml https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/blob/master/deploy/class.yaml

  • rbac.yaml：授權(quán)訪問ApiServer
  • deployment.yaml：部署插件，部署之前要修改里面的指向的nfs服務(wù)器地址和共享目錄
  • class.yaml：創(chuàng)建存儲(chǔ)類
開始部署插件
  1. 部署rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

kubectl create -f rbac.yaml
  1. 部署deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  labels:
    app: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: tantianran/nfs-subdir-external-provisioner:v4.0.1 # 使用到鏡像已經(jīng)轉(zhuǎn)存到我的倉庫
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: k8s-sigs.io/nfs-subdir-external-provisioner
            - name: NFS_SERVER
              value: 192.168.11.254 #此處修改成nfs服務(wù)器的ip地址
            - name: NFS_PATH
              value: /data/nfs_k8s_storage_share # 此處修改成nfs的共享目錄
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.11.254 #此處修改成nfs服務(wù)器的ip地址
            path: /data/nfs_k8s_storage_share # 此處修改成nfs的共享目錄

kubectl create -f deployment.yaml
  1. 創(chuàng)建存儲(chǔ)類class.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-client
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
  archiveOnDelete: "false"

kubectl create -f class.yaml
  1. 查看
# 查看授權(quán)
kubectl get sa

# 查看存儲(chǔ)類
kubectl get sc

使用

  1. 創(chuàng)建一個(gè)PVC
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  storageClassName: nfs-client

在這里，PVC名稱為my-pvc，請(qǐng)求5GB的存儲(chǔ)空間，并將存儲(chǔ)類設(shè)置為nfs-client

  1. 查看pvc
# pvc已創(chuàng)建
[root@k8s-a-master nfs-storage-yaml]# kubectl get pvc
NAME     STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
my-pvc   Bound    pvc-ce18ae0a-d4b4-4fa9-8241-b8b9551baa61   5Gi        RWO            nfs-client     2m6s

# 對(duì)應(yīng)的pv已經(jīng)自動(dòng)創(chuàng)建
[root@k8s-a-master nfs-storage-yaml]# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM            STORAGECLASS   REASON   AGE
pvc-ce18ae0a-d4b4-4fa9-8241-b8b9551baa61   5Gi        RWO            Delete           Bound    default/my-pvc   nfs-client              2m7s
[root@k8s-a-master nfs-storage-yaml]#
  1. 創(chuàng)建一個(gè)應(yīng)用pod
apiVersion: v1
kind: Pod
metadata:
  name: test-nginx-pod
spec:
  containers:
  - name: test-nginx-container
    image: nginx
    volumeMounts:
    - name: storage
      mountPath: /data
  volumes:
  - name: storage
    persistentVolumeClaim:
      claimName: my-pvc
  1. 查看pod是否掛載成功
[root@k8s-a-master nfs-storage-yaml]# kubectl exec -it test-nginx-pod -- df -h
...                                                                                            3.9G     0  3.9G   0% /sys/fs/cgroup
192.168.11.254:/data/nfs_k8s_storage_share/default-my-pvc-pvc-ce18ae0a-d4b4-4fa9-8241-b8b9551baa61  500G  1.8G  499G   1% /data
...

本文轉(zhuǎn)載于WX公眾號(hào)：不背鍋運(yùn)維（喜歡的盆友關(guān)注我們）：https://mp.weixin.qq.com/s/f8eCJNxztMFc3uQ_YJzwgg

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡書系信息發(fā)布平臺(tái)，僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容