在Kubernetes中使用CRI-O運(yùn)行時(shí)

容器實(shí)踐線路圖中介紹了容器技術(shù)選型,關(guān)于容器運(yùn)行時(shí),提到了CRI規(guī)范與CRI-O實(shí)現(xiàn),使用CRI-O可以在運(yùn)行時(shí)完全替代docker。CRI-O提供了crictl工具,類似docker client,可以pull鏡像、ps容器進(jìn)程、attach到容器進(jìn)程內(nèi)等等,除了build與tag/push鏡像沒提供之外,其他都有了。至于為何不提供鏡像build/tag/push操作,官方解釋是crictl不是替代docker,呵呵。
本文介紹如何使用CRI-O運(yùn)行時(shí)替換docker運(yùn)行時(shí),基于CRI-O對接Kubernetes編排。本文所有環(huán)境都基于 CentOS7.6 操作系統(tǒng),內(nèi)核版本為 3.10.0-957.21.3.el7.x86_64 。

1. 安裝CRI-O

  1. 啟用內(nèi)核模塊
modprobe overlay
modprobe br_netfilter
  1. 設(shè)置內(nèi)核參數(shù)
# Setup required sysctl params, these persist across reboots.
cat > /etc/sysctl.d/99-kubernetes-cri.conf <<EOF
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

sysctl --system
  1. 安裝CRI-O
# Install prerequisites
yum install yum-utils
yum-config-manager --add-repo=https://cbs.centos.org/repos/paas7-crio-311-candidate/x86_64/os/

# Install CRI-O
yum install --nogpgcheck cri-o
  1. 配置 CRI-O,設(shè)置 crio pause 鏡像下載地址為阿里云鏡像倉庫

默認(rèn)配置在 /etc/crio/crio.conf

sed -i "s/pause_image = .*/pause_image = \"registry.cn-hangzhou.aliyuncs.com\/google_containers\/pause:3.1\"/g" /etc/crio/crio.conf

  1. 啟動 CRI-O

systemctl start crio

  1. 驗(yàn)證 CRI-O 部署
curl -v --unix-socket /var/run/crio/crio.sock http://localhost/info
{"storage_driver":"overlay","storage_root":"/var/lib/containers/storage","cgroup_driver":"systemd","default_id_mappings":{"uids":[{"container_id":0,"host_id":0,"size":4294967295}],"gids":[{"container_id":0,"host_id":0,"size":4294967295}]}}s

2. 安裝kubeadm/kubelet/kubectl

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kube*
EOF

# 將 SELinux 設(shè)置為 permissive 模式(將其禁用)
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes

systemctl enable kubelet && systemctl start kubelet

不過谷歌的包倉庫大陸無法訪問,使用阿里云鏡像站替換:

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
        http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

# 將 SELinux 設(shè)置為 permissive 模式(將其禁用)
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes

CRI-O 運(yùn)行時(shí)使用的 cgroup driversystemd ,因此需要設(shè)置 kubelet 參數(shù)保持一致:

echo "KUBELET_CGROUP_ARGS=--cgroup-driver=systemd" >> /etc/sysconfig/kubelet

# !!在 kubelet 啟動文件 /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf 增加 KUBELET_CGROUP_ARGS 參數(shù) !!
# ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS $KUBELET_CGROUP_ARGS

systemctl daemon-reload
systemctl enable kubelet
systemctl restart kubelet

確保 kubelet 啟動參數(shù)中有 KUBELET_CGROUP_ARGS 設(shè)置的值:

# systemctl status -l kubelet
kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: active (running) since ...
     Docs: https://kubernetes.io/docs/
 Main PID: 15971 (kubelet)
   CGroup: /system.slice/kubelet.service
           └─15971 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=/var/run/crio/crio.sock --cgroup-driver=systemd

否則后續(xù)會因 cgroup driver 不匹配(crio使用的cgroup driver是systemd, kubelet使用的是cgroupfs)報(bào)錯(cuò):

kuberuntime_sandbox.go:68] CreatePodSandbox for pod "kube-scheduler-k8s-master-01_kube-system(...)" failed: rpc error: code = Unknown desc = cri-o configured with systemd cgroup manager, but did not receive slice as parent: /kubepods/burstable/...

3. 使用 kubeadm 安裝 Kubernetes

  1. 確定使用哪種網(wǎng)絡(luò)插件實(shí)現(xiàn),kubeadm 只支持 CNI 網(wǎng)絡(luò)插件;這里使用 Flannel 插件

  2. 修改 kubeadm 的默認(rèn)倉庫為阿里云倉庫鏡像

由于 Kubernetes 鏡像倉庫在 k8s.gcr.io 上,大陸無法訪問,需要使用阿里云的鏡像站,因此需要修改 kubeadm 默認(rèn)配置。

# 導(dǎo)出默認(rèn)配置
kubeadm config print init-defaults > kubeadm-init-config.yaml

#修改倉庫鏡像地址與kubernetes版本號
sed -i "s/imageRepository: .*/imageRepository: registry.cn-hangzhou.aliyuncs.com\/google_containers/g" kubeadm-init-config.yaml
sed -i "s/kubernetesVersion: .*/kubernetesVersion: v1.15.1/g" kubeadm-init-config.yaml
sed -i "s/criSocket: .*/criSocket: \/var\/run\/crio\/crio.sock/g" kubeadm-init-config.yaml
address=`ifconfig eth0 | egrep "inet\s" | awk '{print $2}'` && sed -i "s/advertiseAddress: .*/advertiseAddress: ${address}/g" kubeadm-init-config.yaml

# 在 kubeadm-init-config.yaml 增加 podSubnet: 參數(shù),如
...
kind: ClusterConfiguration
networking:
  podSubnet: 10.244.0.0/16
...

修改后的 kubeadm-init-config.yaml :


apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.128.165
  bindPort: 6443
nodeRegistration:
  criSocket: /var/run/crio/crio.sock
  name: k8s-master-01
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.15.1
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12
  podSubnet: 10.244.0.0/16
scheduler: {}

  1. kubeadm init
kubeadm init  --config ~/kubeadm-init-config.yaml

如果 kubeadm init 命令出錯(cuò)了,修復(fù)之后重試,需要先執(zhí)行 kubeadm reset

  1. kubectl 連接到 Kubernetes 集群
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

如果使用 root 用戶,則直接使用:

export KUECONFIG=/etc/kubernetes/admin.conf
  1. 安裝網(wǎng)絡(luò)插件 Flannel
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/62e44c867a2846fefb68bd5f178daf4da3095ccb/Documentation/kube-flannel.yml

4. 增加 node 節(jié)點(diǎn)到集群

  1. 在 node 上執(zhí)行上面的大步驟1/2,安裝 CRI-O 與 kubeadm/kubelet/crio-tools

  2. 登錄到node節(jié)點(diǎn)上,執(zhí)行 join 的命令

kubeadm join <master-ip>:<master-port> --token <token> --discovery-token-ca-cert-hash sha256:<hash>

如果忘記了token,在Master節(jié)點(diǎn)上通過如下命令查看:

kubeadm token list

如果token過期了,在Master節(jié)點(diǎn)上通過如下命令重新生成:

kubeadm token create

--discovery-token-ca-cert-hash 參數(shù)的值,在管理節(jié)點(diǎn)上通過如下命令獲取:

openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'

5. 部署一個(gè) nginx 應(yīng)用

部署一個(gè) nginx 應(yīng)用

Reference

install-kubeadm

使用kubeadm 部署 Kubernetes(國內(nèi)環(huán)境)

too hard to install k8s in china

更多云最佳實(shí)踐 https://best.practices.cloud

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時(shí)請結(jié)合常識與多方信息審慎甄別。
平臺聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡書系信息發(fā)布平臺,僅提供信息存儲服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

  • 這篇zhe pian首先接觸這個(gè)概念是k8s官方文檔中https://kubernetes.io/zh/docs/...
    PENG先森_曉宇閱讀 3,311評論 0 0
  • 前 言 作為后臺支撐,Kubernetes優(yōu)勢明顯,具有自動化部署、服務(wù)伸縮、故障自我修復(fù)、負(fù)載均衡等特性。咪付的...
    陳sir的知識圖譜閱讀 2,111評論 0 5
  • 導(dǎo)讀 接觸kubernetes的時(shí)候, 搞不懂OCI,CRI,runC,containerd,shim 之間的區(qū)別...
    allenhaozi閱讀 7,403評論 2 7
  • 初始化 kubeadm安裝 root用戶執(zhí)行以下命令 執(zhí)行完后,可以看到已經(jīng)安裝成功 切換到非root用戶 此時(shí)k...
    SparkOnly閱讀 550評論 0 0
  • 16宿命:用概率思維提高你的勝算 以前的我是風(fēng)險(xiǎn)厭惡者,不喜歡去冒險(xiǎn),但是人生放棄了冒險(xiǎn),也就放棄了無數(shù)的可能。 ...
    yichen大刀閱讀 8,162評論 0 4

友情鏈接更多精彩內(nèi)容