1、集群拓?fù)鋱D

image.png

2、環(huán)境準(zhǔn)備,至少要3臺master

vip ：192.168.0.162 keeplive
master01：192.168.0.163 centos7
master02：192.168.0.164 centos7
master03：192.168.0.165 centos7
node01: 192.168.0.166 centos7

3、修改各個主機(jī)之間hosts解析

image.png

4、配置好基礎(chǔ)環(huán)境、參考http://www.itdecent.cn/p/feda1f429526 （到初始化master上一步）

5、 配置 docker 啟動參數(shù)

cat > /etc/docker/daemon.json <<EOF
{
  "registry-mirrors": ["https://av0eyibf.mirror.aliyuncs.com"],
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
  "max-size": "100m"
  },
    "storage-driver": "overlay2"
  }
EOF

6、所有機(jī)器開啟ssh免密登陸，網(wǎng)上很多有教程，這里就不寫了

7、 在三個master節(jié)點安裝keepalived軟件

# yum install -y socat keepalived ipvsadm conntrack

8、 創(chuàng)建如下keepalived的配置文件

# cat /etc/keepalived/keepalived.conf
global_defs {
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state MASTER                  #聲明角色，其他兩臺也設(shè)置MASTER
    interface ens33                  #根據(jù)自己實際的網(wǎng)卡名稱來寫
    virtual_router_id 80            #ID是唯一的，必須一致
    priority 100                          #權(quán)重100 ，根據(jù)權(quán)重來選舉虛擬ip，其他兩臺權(quán)重不能一樣
    advert_int 1
    authentication {                    #認(rèn)證方式，必須統(tǒng)一密碼
        auth_type PASS              
        auth_pass just0kk              
    }
    virtual_ipaddress { 
        192.168.0.162                   #創(chuàng)建一個虛擬IP
    }
}

virtual_server 192.168.0.162 6443 {     #用于k8s-maser集群注冊 的虛擬地址
    delay_loop 6
    lb_algo loadbalance
    lb_kind DR
    net_mask 255.255.255.0
    persistence_timeout 0
    protocol TCP

real_server 192.168.0.163 6443 {      #后端真實的服務(wù)
        weight 1
        SSL_GET {
            url {
              path /healthz
              status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}

real_server 192.168.0.164 6443 {
        weight 1
        SSL_GET {
            url {
              path /healthz
              status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}

real_server 192.168.0.165 6443 {
        weight 1
        SSL_GET {
            url {
              path /healthz
              status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}

9、 創(chuàng)建k8s集群初始化配置文件

cat /etc/kubernetes/kubeadm-config.yaml
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
kubernetesVersion: v1.19.0
controlPlaneEndpoint: "192.168.0.162:6443"                     #這里的注冊地址要寫keeplive的虛擬IP
apiServer:
  certSANs:
  - 192.168.0.162
  - 192.168.0.163
  - 192.168.0.164
  - 192.168.0.165
networking:
  podSubnet: 10.244.0.0/16
imageRepository: "registry.aliyuncs.com/google_containers"
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: ipvs

10、啟動keepalived服務(wù) (三臺master)

# systemctl enable keepalived
# systemctl start keepalived
# systemctl status keepalived

檢查無問題就下一步

11、啟動docker和kubectl

# systemctl enable docker && systemctl enable kubelet
# systemctl daemon-reload
# systemctl restart docker
# systemctl status docker  && systemctl status kubelet

檢查無問題，下一步

12、初始化k8s集群

#  kubeadm init --config /etc/kubernetes/kubeadm-config.yaml

安裝網(wǎng)絡(luò)插件

#kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
如果不能在線安裝就把kube-flannel.yml下載到服務(wù)器來安裝

13、檢查集群狀態(tài)

#kubectl get cs

image.png

沒問題就下一步

14、拷貝證書到各個master節(jié)點，拷貝完自動加入集群，腳本如下，前提做好ssh免密登陸

# cat k8s-cluster-other-init.sh
#!/bin/bash
IPS=(192.168.0.164 192.168.0.164)
JOIN_CMD=`kubeadm token create --print-join-command 2> /dev/null`

for index in 0 1; do
  ip=${IPS[${index}]}
  ssh $ip "mkdir -p /etc/kubernetes/pki/etcd; mkdir -p ~/.kube/"
  scp /etc/kubernetes/pki/ca.crt $ip:/etc/kubernetes/pki/ca.crt
  scp /etc/kubernetes/pki/ca.key $ip:/etc/kubernetes/pki/ca.key
  scp /etc/kubernetes/pki/sa.key $ip:/etc/kubernetes/pki/sa.key
  scp /etc/kubernetes/pki/sa.pub $ip:/etc/kubernetes/pki/sa.pub
  scp /etc/kubernetes/pki/front-proxy-ca.crt $ip:/etc/kubernetes/pki/front-proxy-ca.crt
  scp /etc/kubernetes/pki/front-proxy-ca.key $ip:/etc/kubernetes/pki/front-proxy-ca.key
  scp /etc/kubernetes/admin.conf $ip:/etc/kubernetes/admin.conf
  scp /etc/kubernetes/admin.conf $ip:~/.kube/config

  ssh ${ip} "${JOIN_CMD} --control-plane"
done

加入之后，檢查一下

image.png

已經(jīng)成功加入了，在把node01也加入集群

 kubeadm join 192.168.0.162:6443 --token 0omn7n.03r4ogczlsqey2u1     --discovery-token-ca-cert-hash sha256:3caf6f90feeb1933e91c9a07abeac4f7d01132634fe5ae131cfb226bd45926d0

查看集群節(jié)點報錯了

# kubectl get node
Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")

是因為證書沒有拷貝過來，把master的證書復(fù)制一份過來

scp $HOME/.kube/config root@node01:$HOME/.kube/config

在查看一下

image.png

OK了

15、接下來，創(chuàng)建一個nginx測試pod

 #vim nginx-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress-test
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx-ingress-test
  template:
    metadata:
      labels:
        app: nginx-ingress-test
    spec:
      containers:
        - name: nginx
          image: nginx
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 80

---
apiVersion: v1
kind: Service
metadata:
  name: nginx-svc
spec:
  type: NodePort
  ports:
    - name: http
      port: 80
      targetPort: 80
      protocol: TCP
      nodePort: 80
  selector:
    app: nginx-ingress-test

執(zhí)行創(chuàng)建

 kubectl apply -f nginx-deployment.yaml

image.png

16、測試master高可用，現(xiàn)在vip在master01上面

image.png

把master01節(jié)點down掉，觀察一下

image.png

vip 已經(jīng)飄逸到master02了,在驗證一下集群是否正常

image.png

在所有節(jié)點檢查都是正常的，在把master01起來，vip又會漂移到master01上面，因為master01的權(quán)重是最高的

至此完成了master的高可用部署