kubernetes之四 搭建k8s集群(rpm + macvlan + ipam)

上篇kubernetes之三 搭建k8s集群是用二進制文件安裝，在離線環(huán)境下，需要用到安裝包安裝。本節(jié)使用RPM的方式安裝kubernetes，且網(wǎng)路使用macvlan，ip地址管理使用ipam

準備

1). 版本信息
組件 版本號 補充說明
docker 18.03.0-ce 無
kubernetes 1.18.12 無
etcd 3.4.7 API VERSION 3.4
linux centos 3.10.0-1127.8.2.el7.x86_64
2). 選擇安裝節(jié)點

資源有限，這里用了三臺機器，除了kubernetes的組件外，etcd集群也共享了相同的資源。

IP地址 角色 部署的組件
173.119.126.200 master kube-proxy,kubelet,etcd,flanneld,kube-apiserver,kube-controller-manager,kube-scheduler
173.119.126.199 node kube-proxy,kubelet,etcd,flanneld
173.119.126.198 node kube-proxy,kubelet,etcd,flanneld
3). 修改host，3臺機器都要修改
#在200機器執(zhí)行
echo "k8s-master-216-200" > /etc/hosts
#或者
vim /etc/hosts
173.119.126.200 k8s-master-216-200
173.119.126.199 k8s-worker-216-199
173.119.126.198 k8s-worker-216-198
4). 確認mac地址和product_uuid的唯一性
ifconfig -a
cat /sys/class/dmi/id/product_uuid
5). 關(guān)閉防火墻
systemctl stop firewalld # 關(guān)閉服務(wù)
systemctl disable firewalld
6). 禁用SELinux
sestatus    # 查看SELinux狀態(tài)
vi /etc/sysconfig/selinux
SELINUX=disabled
7). 禁止交換分區(qū)
vim /etc/fstab 
#以下這行注釋掉
/dev/mapper/rhel-swap   swap    swap    defaults        0 0
8).安裝ETCD

此步驟請參照其他文檔吧

9). 在每臺虛擬機上配置子網(wǎng)接口
#開啟網(wǎng)卡混雜
ip link set ens160 promisc on    //開啟網(wǎng)卡混雜模塊
ip link show ens160  | grep PROMISC     //驗證

#配置vlan
ip link add link ens160 name ens160.125 type vlan id 125      //給網(wǎng)卡打vlan
ip link set ens160.125 up                 //啟動ens160.125
如果宿主機在同一個vlan中，可以執(zhí)行一下給ens160.125加上ip，驗證連通性，正式使用無需這一步
#ip addr add 173.16.125.250/24 dev ens160.125 brd +          //給ens160.125網(wǎng)卡配置ip，檢驗
#ip -f inet addr delete 173.16.125.250/24  dev ens160.125      //刪除ens160.125網(wǎng)卡的ip
#ip link delete ens160.125 type vlan                    //刪除vlan

#創(chuàng)建網(wǎng)卡子接口
docker network create -d macvlan --subnet=173.16.125.0/24 --gateway=173.16.125.254   -o parent=ens160.125     macvlan-125    //創(chuàng)建macvlan網(wǎng)橋
docker network ls | grep macvlan-125     //驗證

docker安裝

kubernetes安裝

我們需要在所有的節(jié)點上安裝kubeadm, kubelet, kubectl，版本需要一致。在可以連外網(wǎng)的機器上下載組件.
添加kubernetes yum源

cat > /etc/yum.repos.d/kubernetes.repo <<EOF
[kubernetes]
name=Kubernetes Repo
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
enabled=1
EOF

下載
下載kubeadm-1.20.4-0
yum install --downloadonly --downloaddir /tools/kubernetes/rpm kubeadm-1.20.4-0
下載的安裝包文件列表
cri-tools-1.13.0-0.x86_64.rpm
kubeadm-1.20.4-0.x86_64.rpm
kubectl-1.20.4-0.x86_64.rpm
kubelet-1.20.4-0.x86_64.rpm
kubernetes-cni-0.8.7-0.x86_64.rpm
libnetfilter_cthelper-1.0.0-11.el7.x86_64.rpm
libnetfilter_cttimeout-1.0.0-7.el7.x86_64.rpm
libnetfilter_queue-1.0.2-2.el7_2.x86_64.rpm
socat-1.7.3.2-2.el7.x86_64.rpm
下載對應(yīng)版本的kubelet，kubectl

yum install --downloadonly --downloaddir /tools/kubernetes/rpm kubelet-1.20.4-0
yum install --downloadonly --downloaddir /tools/kubernetes/rpm kubectl-1.20.4-0

將以上安裝包通過scp分發(fā)到不同的服務(wù)器節(jié)點上

scp /tools/kubernetes/rpm/* root@173.119.126.199:/tools/kubernetes/rpm
scp /tools/kubernetes/rpm/* root@173.119.126.198:/tools/kubernetes/rpm

安裝kubernetes組件
這里有個安裝順序

rpm -ivh socat-1.7.3.2-2.el7.x86_64.rpm
rpm -ivh libnetfilter_cthelper-1.0.0-11.el7.x86_64.rpm
rpm -ivh libnetfilter_cttimeout-1.0.0-7.el7.x86_64.rpm
rpm -ivh libnetfilter_queue-1.0.2-2.el7_2.x86_64.rpm
rpm -ivh cri-tools-1.13.0-0.x86_64.rpm
rpm -ivh kubernetes-cni-0.8.7-0.x86_64.rpm --nodeps
rpm -ivh kubelet-1.20.4-0.x86_64.rpm
rpm -ivh kubectl-1.20.4-0.x86_64.rpm
rpm -ivh kubeadm-1.20.4-0.x86_64.rpm

啟動kubelet

systemctl enable kubelet

生成kube-config.yaml文件

cat <<EOF > ./kubeadm-config.yaml
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.15.4
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
controlPlaneEndpoint: "173.119.126.200:6443"
networking:
  dnsDomain: "cluster.local"
EOF

初始化master節(jié)點
kubeadm init
--apiserver-advertise-address=173.119.126.200 --pod-network-cidr=173.16.0.0/16

執(zhí)行后會收到如下的信息：
Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of the control-plane node running the following command on each as root:

  kubeadm join 173.119.126.200:6443 --token i2swzj.mbljm7wwsw3tfffs \
    --discovery-token-ca-cert-hash sha256:ae87b8259873818d048f9e096552b91cf61c6cc227456edf2f6c4169baa4ff35 \
    --control-plane --certificate-key 796b87ea14db4cf8c6e9de1dbcc899552add98dfb37975fd82b3087c51e57906

Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.

Then you can join any number of worker nodes by running the following on each as root:
(請注意下面這行)
kubeadm join 173.119.126.200:6443 --token i2swzj.mbljm7wwsw3tfffs \
    --discovery-token-ca-cert-hash sha256:ae87b8259873818d048f9e096552b91cf61c6cc227456edf2f6c4169baa4ff35

按照提示，先執(zhí)行

mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

在其他節(jié)點上安裝完kubelet,kubectl，kubeadm后執(zhí)行如下腳本

kubeadm join 173.119.126.200:6443 --token i2swzj.mbljm7wwsw3tfffs \
    --discovery-token-ca-cert-hash sha256:ae87b8259873818d048f9e096552b91cf61c6cc227456edf2f6c4169baa4ff35

執(zhí)行完后看到，在maser節(jié)點執(zhí)行 kubectl get nodes 得到如下的結(jié)果

NAME                 STATUS   ROLES                  AGE    VERSION
k8s-master-126-200   Ready    control-plane,master   7d1h   v1.20.4
k8s-worker-126-198   Ready    <none>                 5d2h   v1.20.4
k8s-worker-126-199   Ready    <none>                 5d3h   v1.20.4

接下來就是配置macvlan, 我們采用的是k8s多集群的ip地址統(tǒng)一管理方式，有位大佬提供了這種方式
cni-ipam-etcd，編譯完后將文件命名為ipam-etcd，將其上傳到所有的節(jié)點的目錄下/opt/cni/bin/,
在目錄/etc/cni/net.d/下創(chuàng)建文件00-macvlan.conflist，文件內(nèi)容

{
        "name": "myetcd-ipam",
        "cniVersion": "0.3.1",
        "plugins": [
            {
                "name": "mymacvlan",
                "type": "macvlan",
                "master": "ens160",
                "ipam": {
                    "name": "myetcd-ipam",
                    "type": "ipam-etcd",
                    "etcdConfig": {
                        "etcdURL": "https://173.119.126.199:2379",
                        "etcdCertFile": "/tools/etcd/ssl/server.pem",
                        "etcdKeyFile": "/tools/etcd/ssl/server-key.pem",
                        "etcdTrustedCAFileFile": "/tools/etcd/ssl/ca.pem"
                    },
                    "subnet": "173.16.125.0/24",
                    "rangeStart": "173.16.125.10",
                    "rangeEnd": "173.16.125.100",
                    "gateway": "173.16.125.254",
                    "routes": [{
                        "dst": "0.0.0.0/0"
                    }]
                }
            }
        ]
    }

macvlan配置完畢，需要啟用一個應(yīng)用加以測試,創(chuàng)建一個deploy文件busybox.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: app
spec:
  selector:
    matchLabels:
      app: app
  replicas: 2
  template:
    metadata:
      labels:
        app: app
    spec:
      containers:
      - name: app
        image: busybox:latest #如果不可聯(lián)網(wǎng)，需指向本地hubbor鏡像可以
        args:
        - /bin/sh
        - -c
        - sleep 10; touch /tmp/healthy; sleep 30000

執(zhí)行

kubectl apply -f busybox.yaml

通過命令

kubectl get pods

結(jié)果

NAME                   READY   STATUS              RESTARTS   AGE    IP              NODE                 NOMINATED NODE   READINESS GATES
app-5f997ff969-77rzl   1/1     Running             14         5d1h   173.16.125.10   k8s-worker-126-199   <none>           <none>
app-5f997ff969-cl8wg   1/1     Running             14         5d1h   173.16.125.11   k8s-worker-126-198   <none>           <none>
最后編輯于 ：
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時請結(jié)合常識與多方信息審慎甄別。
平臺聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點，簡書系信息發(fā)布平臺，僅提供信息存儲服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容