kubeadm是官方社區(qū)推出的一個(gè)用于快速部署kubernetes集群的工具。
這個(gè)工具能通過兩條指令完成一個(gè)kubernetes集群的部署：
# 創(chuàng)建一個(gè) Master 節(jié)點(diǎn)
$ kubeadm init
# 將一個(gè) Node 節(jié)點(diǎn)加入到當(dāng)前集群中
$ kubeadm join <Master節(jié)點(diǎn)的IP和端口 >
1. 安裝要求
在開始之前，部署Kubernetes集群機(jī)器需要滿足以下幾個(gè)條件：
- 一臺(tái)或多臺(tái)機(jī)器，操作系統(tǒng) CentOS7.x-86_x64
- 硬件配置：2GB或更多RAM，2個(gè)CPU或更多CPU，硬盤30GB或更多
- 可以訪問外網(wǎng)，需要拉取鏡像，如果服務(wù)器不能上網(wǎng)，需要提前下載鏡像并導(dǎo)入節(jié)點(diǎn)
- 禁止swap分區(qū)
2. 準(zhǔn)備環(huán)境
| 角色 | IP |
|---|---|
| master1 | 192.168.44.155 |
| master2 | 192.168.44.156 |
| node1 | 192.168.44.157 |
| VIP（虛擬ip） | 192.168.44.158 |
# 關(guān)閉防火墻
systemctl stop firewalld
systemctl disable firewalld
# 關(guān)閉selinux
sed -i 's/enforcing/disabled/' /etc/selinux/config # 永久
setenforce 0 # 臨時(shí)
# 關(guān)閉swap
swapoff -a # 臨時(shí)
sed -ri 's/.*swap.*/#&/' /etc/fstab # 永久
# 根據(jù)規(guī)劃設(shè)置主機(jī)名
hostnamectl set-hostname <hostname>
# 在master添加hosts
cat >> /etc/hosts << EOF
192.168.44.158 master.k8s.io k8s-vip
192.168.44.155 master01.k8s.io master1
192.168.44.156 master02.k8s.io master2
192.168.44.157 node01.k8s.io node1
EOF
# 將橋接的IPv4流量傳遞到iptables的鏈
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system # 生效
# 時(shí)間同步
yum install ntpdate -y
ntpdate time.windows.com
3. 所有master節(jié)點(diǎn)部署keepalived
3.1 安裝相關(guān)包和keepalived
yum install -y conntrack-tools libseccomp libtool-ltdl
yum install -y keepalived
3.2配置master節(jié)點(diǎn)
master1節(jié)點(diǎn)配置
cat > /etc/keepalived/keepalived.conf <<EOF
! Configuration File for keepalived
global_defs {
router_id k8s
}
vrrp_script check_haproxy {
script "killall -0 haproxy"
interval 3
weight -2
fall 10
rise 2
}
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 51
priority 250
advert_int 1
authentication {
auth_type PASS
auth_pass ceb1b3ec013d66163d6ab
}
virtual_ipaddress {
192.168.44.158
}
track_script {
check_haproxy
}
}
EOF
master2節(jié)點(diǎn)配置
cat > /etc/keepalived/keepalived.conf <<EOF
! Configuration File for keepalived
global_defs {
router_id k8s
}
vrrp_script check_haproxy {
script "killall -0 haproxy"
interval 3
weight -2
fall 10
rise 2
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
virtual_router_id 51
priority 200
advert_int 1
authentication {
auth_type PASS
auth_pass ceb1b3ec013d66163d6ab
}
virtual_ipaddress {
192.168.44.158
}
track_script {
check_haproxy
}
}
EOF
3.3 啟動(dòng)和檢查
在兩臺(tái)master節(jié)點(diǎn)都執(zhí)行
# 啟動(dòng)keepalived
$ systemctl start keepalived.service
設(shè)置開機(jī)啟動(dòng)
$ systemctl enable keepalived.service
# 查看啟動(dòng)狀態(tài)
$ systemctl status keepalived.service
啟動(dòng)后查看master1的網(wǎng)卡信息
ip a s ens33
4. 部署haproxy
4.1 安裝
yum install -y haproxy
4.2 配置
兩臺(tái)master節(jié)點(diǎn)的配置均相同，配置中聲明了后端代理的兩個(gè)master節(jié)點(diǎn)服務(wù)器，指定了haproxy運(yùn)行的端口為16443等，因此16443端口為集群的入口
cat > /etc/haproxy/haproxy.cfg << EOF
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
# to have these messages end up in /var/log/haproxy.log you will
# need to:
# 1) configure syslog to accept network log events. This is done
# by adding the '-r' option to the SYSLOGD_OPTIONS in
# /etc/sysconfig/syslog
# 2) configure local2 events to go to the /var/log/haproxy.log
# file. A line like the following can be added to
# /etc/sysconfig/syslog
#
# local2.* /var/log/haproxy.log
#
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
# turn on stats unix socket
stats socket /var/lib/haproxy/stats
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode http
log global
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000
#---------------------------------------------------------------------
# kubernetes apiserver frontend which proxys to the backends
#---------------------------------------------------------------------
frontend kubernetes-apiserver
mode tcp
bind *:16443
option tcplog
default_backend kubernetes-apiserver
#---------------------------------------------------------------------
# round robin balancing between the various backends
#---------------------------------------------------------------------
backend kubernetes-apiserver
mode tcp
balance roundrobin
server master01.k8s.io 192.168.44.155:6443 check
server master02.k8s.io 192.168.44.156:6443 check
#---------------------------------------------------------------------
# collection haproxy statistics message
#---------------------------------------------------------------------
listen stats
bind *:1080
stats auth admin:awesomePassword
stats refresh 5s
stats realm HAProxy\ Statistics
stats uri /admin?stats
EOF
4.3 啟動(dòng)和檢查
兩臺(tái)master都啟動(dòng)
# 設(shè)置開機(jī)啟動(dòng)
$ systemctl enable haproxy
# 開啟haproxy
$ systemctl start haproxy
# 查看啟動(dòng)狀態(tài)
$ systemctl status haproxy
檢查端口
netstat -lntup|grep haproxy
5. 所有節(jié)點(diǎn)安裝Docker/kubeadm/kubelet
Kubernetes默認(rèn)CRI（容器運(yùn)行時(shí)）為Docker，因此先安裝Docker。
5.1 安裝Docker
$ wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo
$ yum -y install docker-ce-18.06.1.ce-3.el7
$ systemctl enable docker && systemctl start docker
$ docker --version
Docker version 18.06.1-ce, build e68fc7a
$ cat > /etc/docker/daemon.json << EOF
{
"registry-mirrors": ["https://b9pmyelo.mirror.aliyuncs.com"]
}
EOF
5.2 添加阿里云YUM軟件源
$ cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
5.3 安裝kubeadm，kubelet和kubectl
由于版本更新頻繁，這里指定版本號(hào)部署：
$ yum install -y kubelet-1.16.3 kubeadm-1.16.3 kubectl-1.16.3
$ systemctl enable kubelet
6. 部署Kubernetes Master
6.1 創(chuàng)建kubeadm配置文件
在具有vip的master上操作，這里為master1
$ mkdir /usr/local/kubernetes/manifests -p
$ cd /usr/local/kubernetes/manifests/
$ vi kubeadm-config.yaml
apiServer:
certSANs:
- master1
- master2
- master.k8s.io
- 192.168.44.158
- 192.168.44.155
- 192.168.44.156
- 127.0.0.1
extraArgs:
authorization-mode: Node,RBAC
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta1
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: "master.k8s.io:16443"
controllerManager: {}
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.16.3
networking:
dnsDomain: cluster.local
podSubnet: 10.244.0.0/16
serviceSubnet: 10.1.0.0/16
scheduler: {}
6.2 在master1節(jié)點(diǎn)執(zhí)行
$ kubeadm init --config kubeadm-config.yaml
按照提示配置環(huán)境變量，使用kubectl工具：
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
$ kubectl get nodes
$ kubectl get pods -n kube-system
按照提示保存以下內(nèi)容，一會(huì)要使用：
kubeadm join master.k8s.io:16443 --token jv5z7n.3y1zi95p952y9p65 \
--discovery-token-ca-cert-hash sha256:403bca185c2f3a4791685013499e7ce58f9848e2213e27194b75a2e3293d8812 \
--control-plane
查看集群狀態(tài)
kubectl get cs
kubectl get pods -n kube-system
7.安裝集群網(wǎng)絡(luò)
從官方地址獲取到flannel的yaml，在master1上執(zhí)行
mkdir flannel
cd flannel
wget -c https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
安裝flannel網(wǎng)絡(luò)
kubectl apply -f kube-flannel.yml
檢查
kubectl get pods -n kube-system
8、master2節(jié)點(diǎn)加入集群
8.1 復(fù)制密鑰及相關(guān)文件
從master1復(fù)制密鑰及相關(guān)文件到master2
# ssh root@192.168.44.156 mkdir -p /etc/kubernetes/pki/etcd
# scp /etc/kubernetes/admin.conf root@192.168.44.156:/etc/kubernetes
# scp /etc/kubernetes/pki/{ca.*,sa.*,front-proxy-ca.*} root@192.168.44.156:/etc/kubernetes/pki
# scp /etc/kubernetes/pki/etcd/ca.* root@192.168.44.156:/etc/kubernetes/pki/etcd
8.2 master2加入集群
執(zhí)行在master1上init后輸出的join命令,需要帶上參數(shù)--control-plane表示把master控制節(jié)點(diǎn)加入集群
kubeadm join master.k8s.io:16443 --token ckf7bs.30576l0okocepg8b --discovery-token-ca-cert-hash sha256:19afac8b11182f61073e254fb57b9f19ab4d798b70501036fc69ebef46094aba --control-plane
檢查狀態(tài)
kubectl get node
kubectl get pods --all-namespaces
5. 加入Kubernetes Node
在node1上執(zhí)行
向集群添加新節(jié)點(diǎn)，執(zhí)行在kubeadm init輸出的kubeadm join命令：
kubeadm join master.k8s.io:16443 --token ckf7bs.30576l0okocepg8b --discovery-token-ca-cert-hash sha256:19afac8b11182f61073e254fb57b9f19ab4d798b70501036fc69ebef46094aba
集群網(wǎng)絡(luò)重新安裝，因?yàn)樘砑恿诵碌膎ode節(jié)點(diǎn)
檢查狀態(tài)
kubectl get node
kubectl get pods --all-namespaces
7. 測(cè)試kubernetes集群
在Kubernetes集群中創(chuàng)建一個(gè)pod，驗(yàn)證是否正常運(yùn)行：
$ kubectl create deployment nginx --image=nginx
$ kubectl expose deployment nginx --port=80 --type=NodePort
$ kubectl get pod,svc
訪問地址：http://NodeIP:Port
