1.知識(shí)準(zhǔn)備

1.Webhook 是一種用于接收準(zhǔn)入請(qǐng)求并對(duì)其進(jìn)行處理的 HTTP 回調(diào)機(jī)制
2.Webhook 接收來(lái)自apiserver的回調(diào)，對(duì)回調(diào)資源做一些校驗(yàn)、注入、修改元數(shù)據(jù)等工作
3.來(lái)一張圖片

webhook.png

2.環(huán)境準(zhǔn)備

3.部署

下載代碼

3.1 創(chuàng)建相關(guān)證書

|># sh webhook-create-signed-cert.sh
creating certs in tmpdir ./webhook-certs
Generating RSA private key, 2048 bit long modulus
..................................................+++
.................+++
e is 65537 (0x10001)
certificatesigningrequest.certificates.k8s.io/wilsonchai-webhook-svc.default created
NAME                             AGE   REQUESTOR          CONDITION
wilsonchai-webhook-svc.default   0s    kubernetes-admin   Pending
certificatesigningrequest.certificates.k8s.io/wilsonchai-webhook-svc.default approved
secret/wilsonchai-webhook-certs created

3.2 創(chuàng)建權(quán)限

|># kubectl apply -f yaml/rbac.yaml
serviceaccount/wilsonchai-webhook-sa changed
clusterrole.rbac.authorization.k8s.io/wilsonchai-webhook-cr changed
clusterrolebinding.rbac.authorization.k8s.io/wilsonchai-webhook-crb changed

3.3 創(chuàng)建 mutetingwebhookconfiguration

|># cat yaml/mutatingwebhookconfiguration.yaml | sh webhook-patch-ca-bundle.sh | kubectl apply -f -
mutatingwebhookconfiguration.admissionregistration.k8s.io/wilsonchai-webhook-example-cfg created
|># kubectl get mutatingwebhookconfiguration
NAME                             CREATED AT
wilsonchai-webhook-example-cfg   2021-11-08T03:34:39Z

3.4 namespace 打 label

kubectl label namespace default wilsonchai-webhook=enabled

3.5編譯代碼并且上傳鏡像

|># sh build.sh 0.0.1
go: downloading github.com/golang/glog v1.0.0
go: downloading k8s.io/apimachinery v0.22.3
go: downloading github.com/unrolled/secure v1.0.9
go: downloading k8s.io/api v0.22.3
go: downloading github.com/gin-gonic/gin v1.7.4
go: downloading golang.org/x/sys v0.0.0-20210616094352-59db8d763f22
go: downloading github.com/golang/protobuf v1.5.2
go: downloading github.com/go-playground/validator/v10 v10.4.1
go: downloading google.golang.org/protobuf v1.26.0
go: downloading github.com/gogo/protobuf v1.3.2
go: downloading k8s.io/klog/v2 v2.9.0
go: downloading github.com/google/gofuzz v1.1.0
go: downloading github.com/go-logr/logr v0.4.0
go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.1.2
go: downloading github.com/json-iterator/go v1.1.11
go: downloading golang.org/x/net v0.0.0-20210520170846-37e1c6afe023
go: downloading github.com/google/go-cmp v0.5.5
Sending build context to Docker daemon  22.04MB
Step 1/3 : FROM alpine:latest
 ---> 389fef711851
Step 2/3 : Add wilsonchai-webhook /wilsonchai-webhook
 ---> 05381b24d25c
Step 3/3 : ENTRYPOINT ["./wilsonchai-webhook"]
 ---> Running in c36e1e7a0bfb
Removing intermediate container c36e1e7a0bfb
 ---> c4a4c7042625
Successfully built c4a4c7042625
Successfully tagged registry.cn-beijing.aliyuncs.com/wilsonchai/mutating-webhook:0.0.1
The push refers to repository [registry.cn-beijing.aliyuncs.com/wilsonchai/mutating-webhook]
85e0257311fe: Pushed
777b2c648970: Pushed
0.0.1: digest: sha256:6750fe64823810caa0ad7551a61ba4b4e6ee0ccd40ab93a76b6a1fb8bcdc5bee size: 740

3.6部署鏡像

|># kubectl apply -f yaml/deploy.yaml
deployment.apps/wilsonchai-webhook-deployment created
service/wilsonchai-webhook-svc created
|># kubectl get pod | grep wilsonchai-webhook
wilsonchai-webhook-deployment-8444c7d8dd-89qwp   1/1     Running     0          19s

至此，整個(gè)部署完成，是不是非常簡(jiǎn)單，現(xiàn)在我們來(lái)測(cè)試一下是否能夠正常工作

4.測(cè)試

4.1 先打開一個(gè)shell 1，監(jiān)控wilsonchai-webhook的日志輸出

|># kubectl logs -f wilsonchai-webhook-deployment-8444c7d8dd-89qwp
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)

[GIN-debug] POST   /mutate                   --> main.WebhookCallback (4 handlers)
[GIN-debug] Listening and serving HTTPS on :443

4.2 打開另一個(gè)shell 2，部署一個(gè)測(cè)試的deployment，busybox-test

|># kubectl apply -f yaml/busybox.yaml
deployment.apps/busybox-test created

4.3 回到shell 1，wilsonchai-webhook成功接收到來(lái)自api的回調(diào)

[GIN] 2021/11/08 - 03:53:59 | 200 |     176.172μs |      10.244.0.0 | POST     "/mutate?timeout=30s"

4.4 回到shell 2，查看busybox-test是否部署成功

|># kubectl get pod | grep busybox
busybox-test-59c6487468-h4zx9                    1/1     Running     0          110s

至此，部署成功

5.調(diào)試模式

由于我們的開發(fā)環(huán)境在遠(yuǎn)端（相對(duì)于k8s集群），我們想要配置可以直接回調(diào)到遠(yuǎn)端的開發(fā)環(huán)境，方便我們調(diào)試代碼，here we go!!

5.1 k8s master上操作

5.1.1 刪除 k8s 集群中的 wilsonchai-webhook

|># kubectl delete -f yaml/deploy.yaml
deployment.apps "wilsonchai-webhook-deployment" deleted
service "wilsonchai-webhook-svc" deleted

5.1.2 部署調(diào)試模式

|># kubectl apply -f yaml/debug.yaml
service/wilsonchai-webhook-svc created
endpoints/wilsonchai-webhook-svc created

<font color='red'>注意： 這里的10.248.33.220，是我的k8s master ip，請(qǐng)大家自行替換成k8s master ip</font>

...
apiVersion: v1
kind: Endpoints
metadata:
  name: wilsonchai-webhook-svc
subsets:
- addresses:
  - ip: 10.248.33.220
  ports:
    - port: 443

5.1.3 修改k8s master 的ssh配置（如果沒有，請(qǐng)?zhí)砑樱?，然后重?/p>

|># grep GatewayPorts /etc/ssh/sshd_config
GatewayPorts yes
|># systemctl restart sshd

5.2 回到開發(fā)機(jī)器操作

5.2.1 打開一條ssh隧道

? ssh -N -R 10.248.33.220:443:127.0.0.1:443 root@10.248.33.220
root@10.248.33.220's password:

5.2.2 運(yùn)行wilsonchai-webhook，這里需要手工指向證書位置（證書就在“創(chuàng)建相關(guān)證書”小節(jié)中創(chuàng)建出來(lái)的webhook-certs中）

? go run main.go webhook.go -tlsCertFile webhook-certs/server-cert.pem -tlsKeyFile webhook-certs/server-key.pem
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)

[GIN-debug] POST   /mutate                   --> main.WebhookCallback (4 handlers)
[GIN-debug] Listening and serving HTTPS on :443

至此，開發(fā)環(huán)境搭建完成，我們來(lái)測(cè)試一下

5.3 重建busybox-test

|># kubectl delete -f yaml/busybox.yaml
deployment.apps "busybox-test" deleted
|># kubectl apply -f yaml/busybox.yaml
deployment.apps/busybox-test created

5.4 查看開發(fā)環(huán)境

[GIN] 2021/11/08 - 12:08:59 | 200 |    3.812455ms |       127.0.0.1 | POST     "/mutate?timeout=30s"

沒錯(cuò)，apiserver已經(jīng)回調(diào)到我們的開發(fā)環(huán)境了

6.原理淺析

6.1 mutate webhook

● 通過mutatingwebhookconfiguration，告訴apiserver哪一種資源需要回調(diào)
● 用一個(gè)wilsonchai-webhook來(lái)接收回調(diào)，并且進(jìn)行適當(dāng)?shù)奶幚恚ū疚闹皇谴蛲鞒?，并沒有處理回調(diào)內(nèi)容）
● wilsonchai-webhook最后將回調(diào)內(nèi)容經(jīng)過處理之后，回寫到apiserver去
● mutate webhook有30s的超時(shí)時(shí)間，超時(shí)之后apiserver將進(jìn)入自己的流程，并在日志打印一行錯(cuò)誤

6.2 調(diào)試

● 第一步，修改service 的endpoint，把請(qǐng)求流出k8s集群
● 第二步，修改k8s master 的sshd_config，這樣做的目的是讓隧道監(jiān)聽0.0.0.0，否則只能監(jiān)聽127.0.0.1
● 第三步，建立一條ssh隧道，將訪問到k8s master的請(qǐng)求導(dǎo)入到開發(fā)環(huán)境來(lái)
● 總的來(lái)說：apiserver --> wilsonchai-webhook-svc --> k8s master ip --> 開發(fā)環(huán)境

<font color='red'>注意：如果你的k8s集群能夠直接訪問開發(fā)環(huán)境，那就更加簡(jiǎn)單，只需要把endpoint的address指向你的開發(fā)環(huán)境ip即可</font>

7.小結(jié)

● 需要注意的是，本文的k8s是1.15.2版本的，如果是高于 v1.16，mutatingwebhookconfiguration的apiversion有變化，具體請(qǐng)參考官網(wǎng)
● 本文只是打通了流程，并沒有演示webhook的具體作用
● 本文中的代碼：代碼

8.參考

https://kubernetes.io/zh/docs/reference/access-authn-authz/extensible-admission-controllers/
https://www.qikqiak.com/post/k8s-admission-webhook/

至此，本文結(jié)束
在下才疏學(xué)淺，有撒湯漏水的，請(qǐng)各位不吝賜教...