給kubernetes社區(qū)貢獻(xiàn)代碼時，經(jīng)常需要寫e2e測試，本地測試通過后再提PR效率更高，也專業(yè)一點(diǎn)。
前提條件
- 本地完成kubernetes代碼編譯（可以參看文末refs）
- 8GB以上內(nèi)存
本地搭建kubernetes開發(fā)測試環(huán)境
社區(qū)推薦minikube和kind，個人覺得kind目前最適合kubernetes開發(fā)測試，很方便在自己的開發(fā)機(jī)快速創(chuàng)建kubernetes集群。
- 安裝kind
brew install kind
- 填寫kind配置文件, kind使用kubeadm創(chuàng)建k8s集群，如果有定制化參數(shù)可以按照kubeadmConfig。
cat kubernetes-v1.29.2.yaml
# this config file contains all config fields with comments
# NOTE: this is not a particularly useful config file
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
## patch the generated kubeadm config with some extra settings
#kubeadmConfigPatches:
#- |
# apiVersion: kubelet.config.k8s.io/v1beta1
# kind: KubeletConfiguration
# evictionHard:dd
# nodefs.available: "0%"
## patch it further using a JSON 6902 patch
#kubeadmConfigPatchesJSON6902:
#- group: kubeadm.k8s.io
# version: v1beta3
# kind: ClusterConfiguration
# patch: |
# - op: add
# path: /apiServer/certSANs/-
# value: my-hostname
# 1 control plane node and 3 workers
networking:
ipFamily: ipv4
nodes:
# the control plane node config
- role: control-plane
image: kindest/node:v1.29.2@sha256:51a1434a5397193442f0be2a297b488b6c919ce8a3931be0ce822606ea5ca245
# the three workers
- role: worker
image: kindest/node:v1.29.2@sha256:51a1434a5397193442f0be2a297b488b6c919ce8a3931be0ce822606ea5ca245
- role: worker
image: kindest/node:v1.29.2@sha256:51a1434a5397193442f0be2a297b488b6c919ce8a3931be0ce822606ea5ca245
可以通過nodes配置控制節(jié)點(diǎn)、woker節(jié)點(diǎn)副本數(shù)量，以及使用的鏡像?？捎玫溺R像從releases-node 查找
- 創(chuàng)建集群
kind create cluster --config kubernetes-v1.29.2.yaml --name kubernetes-v1.29.2
- 查看集群
$ kind get clusters
$ kubernetes-customize-v1.29
kubernetes-v1.29.2 # 剛才創(chuàng)建的集群
- 集群切換
# 切換到 kubernetes-v1.29.2
kind get clusters # 查看所有集群
kind export kubeconfig --name kubernetes-v1.29.2 # 導(dǎo)出 kubernetes-v1.29.2 的 kubeconfig
kubectl cluster-info # 查看 kubernetes-v1.29.2 的集群信息
# 切換到 kubernetes-customize-v1.29
kind export kubeconfig --name kubernetes-customize-v1.29 # 導(dǎo)出 kubernetes-customize-v1.29 的 kubeconfig
kubectl cluster-info # 查看 kubernetes-customize-v1.29 的集群信息
使用kind運(yùn)行自己編譯過的kubernetes集群
- 使用kind構(gòu)建kubernetes鏡像
修復(fù)一個bug后，使用kind構(gòu)建自定義鏡像，搭建測試集群
$ kind build node-image --image="kindest/node-custom-customize:v1.29" /Users/zhangzhifei/work/go/src/github.com/kubernetes
- 修改鏡像配置，然后通過kind創(chuàng)建即可
% cat kubernetes-customize-v1.29.yaml
# this config file contains all config fields with comments
# NOTE: this is not a particularly useful config file
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
## patch the generated kubeadm config with some extra settings
#kubeadmConfigPatches:
#- |
# apiVersion: kubelet.config.k8s.io/v1beta1
# kind: KubeletConfiguration
# evictionHard:dd
# nodefs.available: "0%"
## patch it further using a JSON 6902 patch
#kubeadmConfigPatchesJSON6902:
#- group: kubeadm.k8s.io
# version: v1beta3
# kind: ClusterConfiguration
# patch: |
# - op: add
# path: /apiServer/certSANs/-
# value: my-hostname
# 1 control plane node and 3 workers
networking:
ipFamily: ipv4
nodes:
# the control plane node config
- role: control-plane
image: kindest/node-custom-customize:v1.29
# the three workers
- role: worker
image: kindest/node-custom-customize:v1.29
- 創(chuàng)建集群
% kind create cluster --config kubernetes-customize-v1.29.yaml --name kubernetes-customize-v1.29
% kind get clusters
kubernetes-customize-v1.29
kubernetes-v1.29.2 # 剛才創(chuàng)建的集群
詳細(xì)配置請看官網(wǎng)：https://kind.sigs.k8s.io/docs/user/quick-start/#installation
Kubernetes End-to-End (e2e) 測試
- 編譯e2e.test
make WHAT="test/e2e/e2e.test"
- 運(yùn)行e2e測試用例
~/work/go/src/github.com/kubernetes/_output/bin/e2e.test -context kind-kubernetes-customize-v1.29-fore2e -ginkgo.focus="\[sig-network\].*Conformance" -num-nodes 2
-
返回如下，說明流程跑通
image.png
refs:
- https://github.com/kubernetes-sigs/kind/issues/1181
- https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md#running-conformance-tests
- https://medium.com/programming-kubernetes/testing-kubernetes-controllers-with-the-e2e-framework-fac232843dc6
- https://github.com/kubernetes/community/blob/master/contributors/devel/development.md#building-kubernetes-on-a-local-osshell-environment
- https://kind.sigs.k8s.io/docs/user/quick-start/
