K8S 安裝
環(huán)境要求
- 服務(wù)器:Centos7 2核2G內(nèi)存
節(jié)點(diǎn)規(guī)劃
| 節(jié)點(diǎn) | 配置 | 說明 |
|---|---|---|
| master | 2c 2G | master 節(jié)點(diǎn) |
| k8swork1 | 2c 2G | work 節(jié)點(diǎn) |
服務(wù)器環(huán)境準(zhǔn)備
-
配置服務(wù)器主機(jī)名(各主機(jī)的服務(wù)器時間要同步)
# master節(jié)點(diǎn) vi /etc/hostname 設(shè)置成 master # work 節(jié)點(diǎn) vi /etc/hostname 設(shè)置成 k8swork1 -
防火墻設(shè)置
sed -i "s/^SELINUX\=enforcing/SELINUX\=disabled/g" /etc/selinux/config setenforce 0 systemctl stop firewalld systemctl disable firewalld -
關(guān)閉swap
sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab -
內(nèi)核參數(shù)設(shè)置
echo "net.bridge.bridge-nf-call-ip6tables = 1" >>/etc/sysctl.conf echo "net.bridge.bridge-nf-call-iptables = 1" >> /etc/sysctl.conf echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf sysctl -p -
倉庫配置(可復(fù)制到其他機(jī)器)
5.1 備份舊的配置
cd /etc/yum.repos.d/ mkdir bak mv *.repo bak5.2 下載阿里云倉庫
cd /etc/yum.repos.d/ curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo5.3 下載kubernetes yaml 文件
cd /etc/yum.repos.d/ cat <<EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF -
docker 配置
# 安裝docker yum install -y docker # 設(shè)置docker 開機(jī)自動啟動 systemctl enable docker.service # 啟動docker service docker restart
安裝k8s組件
-
安裝組件
yum install -y kubelet kubeadm kubectl kubernetes-cni -
因?yàn)閗8s的組件在啟動時,會依賴于 gcr.io 下的很多鏡像,國內(nèi)訪問不了,先提前下載這些鏡像
# 可以通過以下命令來看 依賴的幾個組件的版本要求 kubeadm config images list# 輸出以下內(nèi)容 k8s.gcr.io/kube-apiserver:v1.18.20 k8s.gcr.io/kube-controller-manager:v1.18.20 k8s.gcr.io/kube-scheduler:v1.18.20 k8s.gcr.io/kube-proxy:v1.18.20 k8s.gcr.io/pause:3.2 k8s.gcr.io/etcd:3.4.3-0 k8s.gcr.io/coredns:1.6.7 -
因?yàn)樵L問不了外網(wǎng),可以從dockerhub 上搜索對應(yīng)的組件進(jìn)行使用
docker pull gotok8s/kube-apiserver:v1.18.5 docker pull gotok8s/kube-controller-manager:v1.18.5 docker pull gotok8s/kube-scheduler:v1.18.5 docker pull gotok8s/kube-proxy:v1.18.5 docker pull gotok8s/pause:3.2 docker pull gotok8s/etcd:3.4.3-0 docker pull gotok8s/coredns:1.6.7 docker tag docker.io/gotok8s/kube-proxy:v1.18.5 k8s.gcr.io/kube-apiserver:v1.18.20 docker tag docker.io/gotok8s/kube-controller-manager:v1.18.5 k8s.gcr.io/kube-controller-manager:v1.18.20 docker tag docker.io/gotok8s/kube-scheduler:v1.18.5 k8s.gcr.io/kube-scheduler:v1.18.20 docker tag docker.io/gotok8s/coredns:1.6.7 k8s.gcr.io/coredns:1.6.7 docker tag docker.io/gotok8s/etcd:3.4.3-0 k8s.gcr.io/etcd:3.4.3-0 docker tag docker.io/gotok8s/pause:3.2 k8s.gcr.io/pause:3.2 -
開始通過 kubeadm 安裝master
kubeadm init --apiserver-advertise-address=192.168.136.133 --kubernetes-version v1.18.0 --service-cidr=10.1.0.0/16 --pod-network-cidr=10.244.0.0/16 --image-repository=registry.aliyuncs.com/google_containers如果沒有異常的情況下,可以看到 master 創(chuàng)建好了
但是正常是會有 master 處理 NotReady的情況,因?yàn)閗8s有依賴于網(wǎng)絡(luò)組件,會發(fā)現(xiàn) coredns 組件一直是處于未啟動的狀態(tài)
-
安裝 flannel
5.1 下載 flannel.yaml 文件,可拷貝如下內(nèi)容:
--- apiVersion: policy/v1beta1 kind: PodSecurityPolicy metadata: name: psp.flannel.unprivileged annotations: seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default spec: privileged: false volumes: - configMap - secret - emptyDir - hostPath allowedHostPaths: - pathPrefix: "/etc/cni/net.d" - pathPrefix: "/etc/kube-flannel" - pathPrefix: "/run/flannel" readOnlyRootFilesystem: false # Users and groups runAsUser: rule: RunAsAny supplementalGroups: rule: RunAsAny fsGroup: rule: RunAsAny # Privilege Escalation allowPrivilegeEscalation: false defaultAllowPrivilegeEscalation: false # Capabilities allowedCapabilities: ['NET_ADMIN', 'NET_RAW'] defaultAddCapabilities: [] requiredDropCapabilities: [] # Host namespaces hostPID: false hostIPC: false hostNetwork: true hostPorts: - min: 0 max: 65535 # SELinux seLinux: # SELinux is unused in CaaSP rule: 'RunAsAny' --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: flannel rules: - apiGroups: ['extensions'] resources: ['podsecuritypolicies'] verbs: ['use'] resourceNames: ['psp.flannel.unprivileged'] - apiGroups: - "" resources: - pods verbs: - get - apiGroups: - "" resources: - nodes verbs: - list - watch - apiGroups: - "" resources: - nodes/status verbs: - patch --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: flannel roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: flannel subjects: - kind: ServiceAccount name: flannel namespace: kube-system --- apiVersion: v1 kind: ServiceAccount metadata: name: flannel namespace: kube-system --- kind: ConfigMap apiVersion: v1 metadata: name: kube-flannel-cfg namespace: kube-system labels: tier: node app: flannel data: cni-conf.json: | { "name": "cbr0", "cniVersion": "0.3.1", "plugins": [ { "type": "flannel", "delegate": { "hairpinMode": true, "isDefaultGateway": true } }, { "type": "portmap", "capabilities": { "portMappings": true } } ] } net-conf.json: | { "Network": "10.168.0.0/16", "Backend": { "Type": "vxlan" } } --- apiVersion: apps/v1 kind: DaemonSet metadata: name: kube-flannel-ds-amd64 namespace: kube-system labels: tier: node app: flannel spec: selector: matchLabels: app: flannel template: metadata: labels: tier: node app: flannel spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/os operator: In values: - linux - key: kubernetes.io/arch operator: In values: - amd64 hostNetwork: true priorityClassName: system-node-critical tolerations: - operator: Exists effect: NoSchedule serviceAccountName: flannel initContainers: - name: install-cni image: quay.io/coreos/flannel:v0.12.0-amd64 command: - cp args: - -f - /etc/kube-flannel/cni-conf.json - /etc/cni/net.d/10-flannel.conflist volumeMounts: - name: cni mountPath: /etc/cni/net.d - name: flannel-cfg mountPath: /etc/kube-flannel/ containers: - name: kube-flannel image: quay.io/coreos/flannel:v0.12.0-amd64 command: - /opt/bin/flanneld args: - --ip-masq - --kube-subnet-mgr resources: requests: cpu: "100m" memory: "50Mi" limits: cpu: "100m" memory: "50Mi" securityContext: privileged: false capabilities: add: ["NET_ADMIN", "NET_RAW"] env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: - name: run mountPath: /run/flannel - name: flannel-cfg mountPath: /etc/kube-flannel/ volumes: - name: run hostPath: path: /run/flannel - name: cni hostPath: path: /etc/cni/net.d - name: flannel-cfg configMap: name: kube-flannel-cfg --- apiVersion: apps/v1 kind: DaemonSet metadata: name: kube-flannel-ds-arm64 namespace: kube-system labels: tier: node app: flannel spec: selector: matchLabels: app: flannel template: metadata: labels: tier: node app: flannel spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/os operator: In values: - linux - key: kubernetes.io/arch operator: In values: - arm64 hostNetwork: true priorityClassName: system-node-critical tolerations: - operator: Exists effect: NoSchedule serviceAccountName: flannel initContainers: - name: install-cni image: quay.io/coreos/flannel:v0.12.0-arm64 command: - cp args: - -f - /etc/kube-flannel/cni-conf.json - /etc/cni/net.d/10-flannel.conflist volumeMounts: - name: cni mountPath: /etc/cni/net.d - name: flannel-cfg mountPath: /etc/kube-flannel/ containers: - name: kube-flannel image: quay.io/coreos/flannel:v0.12.0-arm64 command: - /opt/bin/flanneld args: - --ip-masq - --kube-subnet-mgr resources: requests: cpu: "100m" memory: "50Mi" limits: cpu: "100m" memory: "50Mi" securityContext: privileged: false capabilities: add: ["NET_ADMIN", "NET_RAW"] env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: - name: run mountPath: /run/flannel - name: flannel-cfg mountPath: /etc/kube-flannel/ volumes: - name: run hostPath: path: /run/flannel - name: cni hostPath: path: /etc/cni/net.d - name: flannel-cfg configMap: name: kube-flannel-cfg --- apiVersion: apps/v1 kind: DaemonSet metadata: name: kube-flannel-ds-arm namespace: kube-system labels: tier: node app: flannel spec: selector: matchLabels: app: flannel template: metadata: labels: tier: node app: flannel spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/os operator: In values: - linux - key: kubernetes.io/arch operator: In values: - arm hostNetwork: true priorityClassName: system-node-critical tolerations: - operator: Exists effect: NoSchedule serviceAccountName: flannel initContainers: - name: install-cni image: quay.io/coreos/flannel:v0.12.0-arm command: - cp args: - -f - /etc/kube-flannel/cni-conf.json - /etc/cni/net.d/10-flannel.conflist volumeMounts: - name: cni mountPath: /etc/cni/net.d - name: flannel-cfg mountPath: /etc/kube-flannel/ containers: - name: kube-flannel image: quay.io/coreos/flannel:v0.12.0-arm command: - /opt/bin/flanneld args: - --ip-masq - --kube-subnet-mgr resources: requests: cpu: "100m" memory: "50Mi" limits: cpu: "100m" memory: "50Mi" securityContext: privileged: false capabilities: add: ["NET_ADMIN", "NET_RAW"] env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: - name: run mountPath: /run/flannel - name: flannel-cfg mountPath: /etc/kube-flannel/ volumes: - name: run hostPath: path: /run/flannel - name: cni hostPath: path: /etc/cni/net.d - name: flannel-cfg configMap: name: kube-flannel-cfg --- apiVersion: apps/v1 kind: DaemonSet metadata: name: kube-flannel-ds-ppc64le namespace: kube-system labels: tier: node app: flannel spec: selector: matchLabels: app: flannel template: metadata: labels: tier: node app: flannel spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/os operator: In values: - linux - key: kubernetes.io/arch operator: In values: - ppc64le hostNetwork: true priorityClassName: system-node-critical tolerations: - operator: Exists effect: NoSchedule serviceAccountName: flannel initContainers: - name: install-cni image: quay.io/coreos/flannel:v0.12.0-ppc64le command: - cp args: - -f - /etc/kube-flannel/cni-conf.json - /etc/cni/net.d/10-flannel.conflist volumeMounts: - name: cni mountPath: /etc/cni/net.d - name: flannel-cfg mountPath: /etc/kube-flannel/ containers: - name: kube-flannel image: quay.io/coreos/flannel:v0.12.0-ppc64le command: - /opt/bin/flanneld args: - --ip-masq - --kube-subnet-mgr resources: requests: cpu: "100m" memory: "50Mi" limits: cpu: "100m" memory: "50Mi" securityContext: privileged: false capabilities: add: ["NET_ADMIN", "NET_RAW"] env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: - name: run mountPath: /run/flannel - name: flannel-cfg mountPath: /etc/kube-flannel/ volumes: - name: run hostPath: path: /run/flannel - name: cni hostPath: path: /etc/cni/net.d - name: flannel-cfg configMap: name: kube-flannel-cfg --- apiVersion: apps/v1 kind: DaemonSet metadata: name: kube-flannel-ds-s390x namespace: kube-system labels: tier: node app: flannel spec: selector: matchLabels: app: flannel template: metadata: labels: tier: node app: flannel spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/os operator: In values: - linux - key: kubernetes.io/arch operator: In values: - s390x hostNetwork: true priorityClassName: system-node-critical tolerations: - operator: Exists effect: NoSchedule serviceAccountName: flannel initContainers: - name: install-cni image: quay.io/coreos/flannel:v0.12.0-s390x command: - cp args: - -f - /etc/kube-flannel/cni-conf.json - /etc/cni/net.d/10-flannel.conflist volumeMounts: - name: cni mountPath: /etc/cni/net.d - name: flannel-cfg mountPath: /etc/kube-flannel/ containers: - name: kube-flannel image: quay.io/coreos/flannel:v0.12.0-s390x command: - /opt/bin/flanneld args: - --ip-masq - --kube-subnet-mgr resources: requests: cpu: "100m" memory: "50Mi" limits: cpu: "100m" memory: "50Mi" securityContext: privileged: false capabilities: add: ["NET_ADMIN", "NET_RAW"] env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: - name: run mountPath: /run/flannel - name: flannel-cfg mountPath: /etc/kube-flannel/ volumes: - name: run hostPath: path: /run/flannel - name: cni hostPath: path: /etc/cni/net.d - name: flannel-cfg configMap: name: kube-flannel-cfg5.2 執(zhí)行安裝
kubectl apply -f flannel.yaml執(zhí)行查看flannel 啟動情況
kubectl get pods --all-namespaces輸出
NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-7ff77c879f-fzfgj 1/1 Running 0 12h kube-system coredns-7ff77c879f-mc6h4 1/1 Running 0 12h kube-system etcd-master 1/1 Running 0 12h kube-system kube-apiserver-master 1/1 Running 0 12h kube-system kube-controller-manager-master 1/1 Running 3 12h kube-system kube-flannel-ds-amd64-nc6wh 1/1 Running 0 11h # flannel 是daemonset ,在所以節(jié)點(diǎn)都會啟動 kube-system kube-flannel-ds-amd64-nl59d 1/1 Running 0 11h kube-system kube-proxy-fg6l5 1/1 Running 2 12h kube-system kube-proxy-ph5m6 1/1 Running 0 12h kube-system kube-scheduler-master 1/1 Running 3 12h -
至此,查看Node節(jié)點(diǎn)的狀態(tài)
[root@master deployment]# kubectl get nodes | grep master master Ready master 12h v1.18.0 -
work 節(jié)點(diǎn)加入
kubeadm join 192.168.136.133:6443 --token yy2huh.9e20jcil00z4rhwf --discovery-token-ca-cert-hash sha256:3336bb808ec8b8f1d1482a52cbfee2f2cb8252b1902b7dcf83df191d1e7ca669注意:
-
token 的生成
在master 上 kubeadm token list 查看目前有效的token
如果沒有,通過 kubeadm token create 進(jìn)行創(chuàng)建
-
discovery-token-ca-cert-hash 如果沒有,在master 上執(zhí)行以下方式生成
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //' -
如果加入后,work節(jié)點(diǎn)出現(xiàn) NotReady的情況
# 首先確認(rèn)not ready 的原因 kubectl describe node k8swork1Conditions: Type Status LastHeartbeatTime LastTransitionTime Reason Message ---- ------ ----------------- ------------------ ------ ------- NetworkUnavailable False Tue, 14 Sep 2021 22:22:28 +0800 Tue, 14 Sep 2021 22:22:28 +0800 FlannelIsUp Flannel is running on this node MemoryPressure False Wed, 15 Sep 2021 09:50:21 +0800 Wed, 15 Sep 2021 09:24:39 +0800 KubeletHasSufficientMemory kubelet has sufficient memory available DiskPressure False Wed, 15 Sep 2021 09:50:21 +0800 Wed, 15 Sep 2021 09:24:39 +0800 KubeletHasNoDiskPressure kubelet has no disk pressure PIDPressure False Wed, 15 Sep 2021 09:50:21 +0800 Wed, 15 Sep 2021 09:24:39 +0800 KubeletHasSufficientPID kubelet has sufficient PID available Ready false Wed, 15 Sep 2021 09:50:21 +0800 Wed, 15 Sep 2021 09:24:39 +0800 KubeletNotReady 錯誤原因消息注:確認(rèn)問題原因
一般是因?yàn)樵趙ork 節(jié)點(diǎn)上不能下載flannel 鏡像問題,導(dǎo)致網(wǎng)絡(luò)異常
通過在work節(jié)點(diǎn)上找到對應(yīng)的容器,確認(rèn)啟動問題,而在本地測試的時候,是因?yàn)榉阑饓Φ膯栴},導(dǎo)致flannel在啟動時出現(xiàn)異常
重新設(shè)置下
## 如果已經(jīng)按步驟下來,在環(huán)境準(zhǔn)備第2步已經(jīng)做了 vi /etc/selinux/config # 設(shè)置 SELINUX=disabled
-
-
查看節(jié)點(diǎn)狀態(tài)
kubectl get nodes[root@master deployment]# kubectl get nodes NAME STATUS ROLES AGE VERSION k8swork1 Ready <none> 12h v1.18.0 master Ready master 12h v1.18.0通過以上,說明節(jié)點(diǎn)都已經(jīng)Ready 狀態(tài)