環(huán)境說明:
| 主機(jī)名 | 操作系統(tǒng)版本 | ip | docker version | kubelet version | kubeadm version | kubectl version | flannel version | 備注 |
|---|---|---|---|---|---|---|---|---|
| master | Centos 7.6 | 172.27.9.131 | Docker 18.09.6 | V1.14.2 | V1.14.2 | V1.14.2 | V0.11.0 | master主機(jī) |
| node01 | Centos 7.6 | 172.27.9.135 | Docker 18.09.6 | V1.14.2 | V1.14.2 | V1.14.2 | V0.11.0 | node節(jié)點(diǎn) |
| node02 | Centos 7.6 | 172.27.9.136 | Docker 18.09.6 | V1.14.2 | V1.14.2 | V1.14.2 | V0.11.0 | node節(jié)點(diǎn) |
Kubernetes這個(gè)名字源自希臘語,意思是“舵手”,也是“管理者”,“治理者”等詞的源頭。k8s是Kubernetes的簡稱(用數(shù)字『8』替代中間的8個(gè)字母『ubernete』)
前言:本文通過kudeadm方式在centos7.6上安裝kubernetes v1.14.2集群(),共分為六個(gè)部分:
一、Docker安裝;
二、k8s安裝準(zhǔn)備工作;
三、Master節(jié)點(diǎn)安裝;
四、Node節(jié)點(diǎn)安裝;
五、Dashboard安裝;
六、集群測試。
一、Docker安裝
所有節(jié)點(diǎn)都需要安裝docker
1. 安裝依賴包
[root@centos7 ~]# yum install -y yum-utils device-mapper-persistent-data lvm2
2. 設(shè)置Docker源
[root@centos7 ~]# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
3. 安裝Docker
3.1 docker安裝版本查看
[root@centos7 ~]# yum list docker-ce --showduplicates | sort -r
3.2 安裝docker
[root@centos7 ~]# yum install docker-ce-18.09.6 docker-ce-cli-18.09.6 containerd.io
指定安裝的docker版本為18.09.6,由于該版本目前為最新版,故可以直接安裝,不用指定版本:
yum install -y docker-ce docker-ce-cli containerd.io
4. 啟動(dòng)Docker
[root@centos7 ~]# systemctl start docker
[root@centos7 ~]# systemctl enable docker
5. 命令補(bǔ)全
5.1 安裝bash-completion
[root@centos7 ~]# yum -y install bash-completion
5.2 加載bash-completion
[root@centos7 /]# source /etc/profile.d/bash_completion.sh
6. 鏡像加速
由于Docker Hub的服務(wù)器在國外,下載鏡像會(huì)比較慢,可以配置鏡像加速器。主要的加速器有:Docker官方提供的中國registry mirror、阿里云加速器、DaoCloud 加速器,本文以阿里加速器配置為例。
6.1 登陸阿里云容器模塊
登陸地址為: https://cr.console.aliyun.com ,未注冊的可以先注冊阿里云賬戶

6.2 配置鏡像加速器
配置daemon.json文件
[root@centos7 ~]# mkdir -p /etc/docker
[root@centos7 ~]# tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors": ["https://v16stybc.mirror.aliyuncs.com"]
}
EOF
重啟服務(wù)
[root@centos7 ~]# systemctl daemon-reload
[root@centos7 ~]# systemctl restart docker
加速器配置完成
7. 驗(yàn)證
[root@centos7 ~]# docker --version
[root@centos7 ~]# docker run hello-world
通過查詢docker版本和運(yùn)行容器hello-world來驗(yàn)證docker是否安裝成功。
二、k8s安裝準(zhǔn)備工作
安裝Centos是已經(jīng)禁用了防火墻和selinux并設(shè)置了阿里源。master和node節(jié)點(diǎn)都執(zhí)行本部分操作。
1. 配置主機(jī)名
1.1 主機(jī)名可以修改也可以不修改
[root@centos7 ~]# hostnamectl set-hostname master
[root@centos7 ~]# more /etc/hostname
master
退出重新登陸即可顯示新設(shè)置的主機(jī)名master
1.2 修改hosts文件
[root@master ~]# cat >> /etc/hosts << EOF
172.27.9.131 master
172.27.9.135 node01
172.27.9.136 node02
EOF
2. 驗(yàn)證mac地址uuid
[root@master ~]# cat /sys/class/net/ens33/address
[root@master ~]# cat /sys/class/dmi/id/product_uuid
保證各節(jié)點(diǎn)mac和uuid唯一
3. 禁用swap
3.1 臨時(shí)禁用
[root@master ~]# swapoff -a
3.2 永久禁用
若需要重啟后也生效,在禁用swap后還需修改配置文件/etc/fstab,注釋swap
[root@master ~]# sed -i.bak '/swap/s/^/#/' /etc/fstab
4. 內(nèi)核參數(shù)修改
4.1 臨時(shí)修改
[root@master ~]# sysctl net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-iptables = 1
[root@master ~]# sysctl net.bridge.bridge-nf-call-ip6tables=1
net.bridge.bridge-nf-call-ip6tables = 1
4.2 永久修改
[root@master ~]# cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
[root@master ~]# sysctl -p /etc/sysctl.d/k8s.conf
5. 修改Cgroup Driver
5.1 修改daemon.json
修改daemon.json,新增‘“exec-opts”: [“native.cgroupdriver=systemd”’
[root@master ~]# more /etc/docker/daemon.json
{
"registry-mirrors": ["https://v16stybc.mirror.aliyuncs.com"],
"exec-opts": ["native.cgroupdriver=systemd"]
}
5.2 重新加載docker
[root@master ~]# systemctl daemon-reload
[root@master ~]# systemctl restart docker
修改cgroupdriver是為了消除告警:
[WARNING IsDockerSystemdCheck]: detected “cgroupfs” as the Docker cgroup driver. The recommended driver is “systemd”. Please follow the guide at https://kubernetes.io/docs/setup/cri/
6. 設(shè)置kubernetes源
6.1 新增kubernetes源
[root@master ~]# cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
- [] 中括號(hào)中的是repository id,唯一,用來標(biāo)識(shí)不同倉庫
- name 倉庫名稱,自定義
- baseurl 倉庫地址
- enable 是否啟用該倉庫,默認(rèn)為1表示啟用
- gpgcheck 是否驗(yàn)證從該倉庫獲得程序包的合法性,1為驗(yàn)證
- repo_gpgcheck 是否驗(yàn)證元數(shù)據(jù)的合法性 元數(shù)據(jù)就是程序包列表,1為驗(yàn)證
- gpgkey=URL 數(shù)字簽名的公鑰文件所在位置,如果gpgcheck值為1,此處就需要指定gpgkey文件的位置,如果gpgcheck值為0就不需要此項(xiàng)了
6.2 更新緩存
[root@master ~]# yum clean all
[root@master ~]# yum -y makecache
三、Master節(jié)點(diǎn)安裝
1. 版本查看
[root@master ~]# yum list kubelet --showduplicates | sort -r
目前最新版是1.14.2,該版本支持的docker版本為1.13.1, 17.03, 17.06, 17.09, 18.06, 18.09。
2. 安裝kubelet、kubeadm和kubectl
2.1 安裝三個(gè)包
[root@master ~]# yum install -y kubelet-1.14.2 kubeadm-1.14.2 kubectl-1.14.2
若不指定版本直接運(yùn)行‘yum install -y kubelet kubeadm kubectl’則默認(rèn)安裝最新版即1.14.2,兩種方式結(jié)果一致。
2.2 安裝包說明
- kubelet 運(yùn)行在集群所有節(jié)點(diǎn)上,用于啟動(dòng)Pod和容器等對象的工具
- kubeadm 用于初始化集群,啟動(dòng)集群的命令工具
- kubectl 用于和集群通信的命令行,通過kubectl可以部署和管理應(yīng)用,查看各種資源,創(chuàng)建、刪除和更新各種組件
2.3 啟動(dòng)kubelet
啟動(dòng)kubelet并設(shè)置開機(jī)啟動(dòng)
[root@master ~]# systemctl enable kubelet && systemctl start kubelet
2.4 kubelet命令補(bǔ)全
[root@master ~]# echo "source <(kubectl completion bash)" >> ~/.bash_profile
[root@master ~]# source .bash_profile
3. 下載鏡像
3.1 鏡像下載的腳本
Kubernetes幾乎所有的安裝組件和Docker鏡像都放在goolge自己的網(wǎng)站上,直接訪問可能會(huì)有網(wǎng)絡(luò)問題,這里的解決辦法是從阿里云鏡像倉庫下載鏡像,拉取到本地以后改回默認(rèn)的鏡像tag。
[root@master ~]# more image.sh
#!/bin/bash
url=registry.cn-hangzhou.aliyuncs.com/google_containers
version=v1.14.2
images=(`kubeadm config images list --kubernetes-version=$version|awk -F '/' '{print $2}'`)
for imagename in ${images[@]} ; do
docker pull $url/$imagename
docker tag $url/$imagename k8s.gcr.io/$imagename
docker rmi -f $url/$imagename
done
url為阿里云鏡像倉庫地址,version為安裝的kubernetes版本。
3.2 下載鏡像
運(yùn)行腳本image.sh,下載指定版本的鏡像,運(yùn)行腳本前先賦權(quán)。
[root@master ~]# chmod u+x image.sh
[root@master ~]# ./image.sh
[root@master ~]# docker images
4. 初始化Master
4.1 初始化
[root@master ~]# kubeadm init --apiserver-advertise-address 172.27.9.131 --pod-network-cidr=10.244.0.0/16
apiserver-advertise-address指定master的interface,pod-network-cidr指定Pod網(wǎng)絡(luò)的范圍,這里使用flannel網(wǎng)絡(luò)方案。
記錄kubeadm join的輸出,后面需要這個(gè)命令將各個(gè)節(jié)點(diǎn)加入集群中。
4.2 加載環(huán)境變量
[root@master ~]# echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile
[root@master ~]# source .bash_profile
本文所有操作都在root用戶下執(zhí)行,若為非root用戶,則執(zhí)行如下操作:
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
5. 安裝pod網(wǎng)絡(luò)
[root@master ~]# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
6. master節(jié)點(diǎn)配置
taint:污點(diǎn)的意思。如果一個(gè)節(jié)點(diǎn)被打上了污點(diǎn),那么pod是不允許運(yùn)行在這個(gè)節(jié)點(diǎn)上面的
6.1 刪除master節(jié)點(diǎn)默認(rèn)污點(diǎn)
默認(rèn)情況下集群不會(huì)在master上調(diào)度pod,如果偏想在master上調(diào)度Pod,可以執(zhí)行如下操作:
查看污點(diǎn):
[root@master ~]# kubectl describe node master|grep -i taints
Taints: node-role.kubernetes.io/master:NoSchedule
刪除默認(rèn)污點(diǎn):
[root@master ~]# kubectl taint nodes master node-role.kubernetes.io/master-
node/master untainted
6.2 污點(diǎn)機(jī)制
語法:
kubectl taint node [node] key=value[effect]
其中[effect] 可取值: [ NoSchedule | PreferNoSchedule | NoExecute ]
NoSchedule: 一定不能被調(diào)度
PreferNoSchedule: 盡量不要調(diào)度
NoExecute: 不僅不會(huì)調(diào)度, 還會(huì)驅(qū)逐Node上已有的Pod
打污點(diǎn)
[root@master ~]# kubectl taint node master key1=value1:NoSchedule
node/master tainted
[root@master ~]# kubectl describe node master|grep -i taints
Taints: key1=value1:NoSchedule
key為key1,value為value1(value可以為空),effect為NoSchedule表示一定不能被調(diào)度
刪除污點(diǎn):
[root@master ~]# kubectl taint nodes master key1-
node/master untainted
[root@master ~]# kubectl describe node master|grep -i taints
Taints: <none>
刪除指定key所有的effect,‘-’表示移除所有以key1為鍵的污點(diǎn)
四、Node節(jié)點(diǎn)安裝
1. 安裝kubelet、kubeadm和kubectl
同master節(jié)點(diǎn)
2. 下載鏡像
同master節(jié)點(diǎn)
3. 加入集群
以下操作master上執(zhí)行
3.1 查看令牌
[root@master ~]# kubeadm token list
TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS
j5eoyz.zu0x6su7wzh752b3 <invalid> 2019-06-04T17:40:41+08:00 authentication,signing The default bootstrap token generated by 'kubeadm init'. system:bootstrappers:kubeadm:default-node-token
發(fā)現(xiàn)之前初始化時(shí)的令牌已過期
3.2 生成新的令牌
[root@master ~]# kubeadm token create
1zl3he.fxgz2pvxa3qkwxln
3.3 生成新的加密串
[root@master ~]# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | \
openssl dgst -sha256 -hex | sed 's/^.* //'
3.4 node節(jié)點(diǎn)加入集群
在node節(jié)點(diǎn)上分別執(zhí)行如下操作:
[root@node01 ~]# kubeadm join 172.27.9.131:6443 --token 1zl3he.fxgz2pvxa3qkwxln --discovery-token-ca-cert-hash sha256:5f656ae26b5e7d4641a979cbfdffeb7845cc5962bbfcd1d5435f00a25c02ea50
五、Dashboard安裝
1. 下載yaml
[root@master ~]# wget https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/recommended/kubernetes-dashboard.yaml
如果連接超時(shí),可以多試幾次
2. 配置yaml
2.1 修改鏡像地址
sed -i 's/k8s.gcr.io/registry.cn-hangzhou.aliyuncs.com\/kuberneters/g' kubernetes-dashboard.yaml
由于默認(rèn)的鏡像倉庫網(wǎng)絡(luò)訪問不通,故改成阿里鏡像
2.2 外網(wǎng)訪問
sed -i '/targetPort:/a\ \ \ \ \ \ nodePort: 30001\n\ \ type: NodePort' kubernetes-dashboard.yaml
配置NodePort,外部通過https://NodeIp:NodePort 訪問Dashboard,此時(shí)端口為30001
2.3 新增管理員帳號(hào)
cat >> kubernetes-dashboard.yaml << EOF
---
# ------------------- dashboard-admin ------------------- #
apiVersion: v1
kind: ServiceAccount
metadata:
name: dashboard-admin
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: dashboard-admin
subjects:
- kind: ServiceAccount
name: dashboard-admin
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
EOF
創(chuàng)建超級管理員的賬號(hào)用于登錄Dashboard
3. 部署訪問
3.1 部署Dashboard
[root@master ~]# kubectl apply -f kubernetes-dashboard.yaml
[圖片上傳失敗...(image-52ad49-1629448684447)]
3.2 狀態(tài)查看
[root@master ~]# kubectl get deployment kubernetes-dashboard -n kube-system
[root@master ~]# kubectl get pods -n kube-system -o wide
[root@master ~]# kubectl get services -n kube-system
[圖片上傳失敗...(image-f3e9bb-1629448684447)]
3.3 令牌查看
kubectl describe secrets -n kube-system dashboard-admin
令牌為:
eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4ta3Q5eDciLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiYzI2MjZiYTUtOTE5Ny0xMWU5LTk2OTQtMDAwYzI5ZDk5YmEzIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.XnBTkkMNQx_hKlTml7B-D1Ip-hp-WFi1mgjgx2E_SHbaRJluLN5qXWLsBAPl1Cgp-IY5ujg9pYePAZL2GDYyjnCaMiFmQsRW-zbNfkyYfPWje8MtCxJqoILTlCxsa5apwWsKdbsW_X8moMC4cM92hApQshn_-x-V7cxydEKuQPrB3PfQ7ReNMM5VCj4rexAh9Qr7I1wEHFa0KX1XaDRiedjWgXFKU7kWX2VQ3vwnfRdpKq_r0vlDSQRVdrMDPT1BHuBaUn8Gz-EdLR3qUy5vlZbzo1UFvxuL2enoZxntHZmnP1vg4552HROeEJhigdtRSbDvxHTEGWT3A67faha-Kg
3.4 訪問
https://NodeIp:30001
[圖片上傳失敗...(image-9c60e-1629448684446)]
請使用火狐瀏覽器訪問
[圖片上傳失敗...(image-667b78-1629448684446)]
通過令牌方式登錄
[圖片上傳失敗...(image-4e35aa-1629448684446)]
Dashboard提供了可以實(shí)現(xiàn)集群管理、工作負(fù)載、服務(wù)發(fā)現(xiàn)和負(fù)載均衡、存儲(chǔ)、字典配置、日志視圖等功能。
六、集群測試
1. 部署應(yīng)用
1.1 命令方式
[root@master ~]# kubectl run httpd-app --image=httpd --replicas=3
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
deployment.apps/httpd-app created
通過命令行方式部署apache服務(wù)
1.2 配置文件方式
cat >> nginx.yml << EOF
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 3
template:
metadata:
labels:
app: nginx
spec:
restartPolicy: Always
containers:
- name: nginx
image: nginx:latest
EOF
[root@master ~]# kubectl apply -f nginx.yml
deployment.extensions/nginx created
通過配置文件方式部署nginx服務(wù)
2. 狀態(tài)查看
2.1 查看節(jié)點(diǎn)狀態(tài)
[root@master ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master Ready master 47h v1.14.2
node01 Ready <none> 22h v1.14.2
node02 Ready <none> 6h55m v1.14.2
2.2 查看pod狀態(tài)
[root@master ~]# kubectl get pod --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default httpd-app-6df58645c6-42zmc 1/1 Running 0 176m
default httpd-app-6df58645c6-g6bkc 1/1 Running 0 176m
default httpd-app-6df58645c6-kp8tl 1/1 Running 0 176m
default nginx-9d4cf4f77-dft2f 1/1 Running 0 18m
default nginx-9d4cf4f77-dztxq 1/1 Running 0 18m
default nginx-9d4cf4f77-l9gdh 1/1 Running 0 18m
kube-system coredns-fb8b8dccf-bxvrz 1/1 Running 1 47h
kube-system coredns-fb8b8dccf-mqvd8 1/1 Running 1 47h
kube-system etcd-master 1/1 Running 3 47h
kube-system kube-apiserver-master 1/1 Running 3 47h
kube-system kube-controller-manager-master 1/1 Running 4 47h
kube-system kube-flannel-ds-amd64-lkh5n 1/1 Running 0 6h55m
kube-system kube-flannel-ds-amd64-pv5ll 1/1 Running 1 24h
kube-system kube-flannel-ds-amd64-wnn5g 1/1 Running 1 22h
kube-system kube-proxy-42vb5 1/1 Running 3 47h
kube-system kube-proxy-7nrfk 1/1 Running 0 6h55m
kube-system kube-proxy-x7dmk 1/1 Running 1 22h
kube-system kube-scheduler-master 1/1 Running 4 47h
2.3 查看副本數(shù)
[root@master ~]# kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
httpd-app 3/3 3 3 178m
nginx 3/3 3 3 19m
[root@master ~]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
httpd-app-6df58645c6-42zmc 1/1 Running 0 179m 10.244.0.6 master <none> <none>
httpd-app-6df58645c6-g6bkc 1/1 Running 0 179m 10.244.1.2 node01 <none> <none>
httpd-app-6df58645c6-kp8tl 1/1 Running 0 179m 10.244.2.2 node02 <none> <none>
nginx-9d4cf4f77-dft2f 1/1 Running 0 20m 10.244.0.7 master <none> <none>
nginx-9d4cf4f77-dztxq 1/1 Running 0 20m 10.244.2.3 node02 <none> <none>
nginx-9d4cf4f77-l9gdh 1/1 Running 0 20m 10.244.1.3 node01 <none> <none>
[root@master ~]#
可以看到nginx和httpd的3個(gè)副本pod均勻分布在3個(gè)節(jié)點(diǎn)上
2.4 查看deployment詳細(xì)信息
[root@master ~]# kubectl describe deployments
2.5 查看集群基本組件狀態(tài)
[root@master ~]# kubectl get cs
NAME STATUS MESSAGE ERROR
scheduler Healthy ok
controller-manager Healthy ok
etcd-0 Healthy {"health":"true"}