Kuberntes部署MetalLB負載均衡器
MetalLB簡介
MetalLB是一個為基礎 Kubernetes集群提供負載均衡實現的工具，使用標準路由協(xié)議。
Kubernetes在基礎集群中不提供網絡負載均衡器（類型為LoadBalancer的服務）的實現。Kubernetes提供的網絡負載平衡器實現都是調用各種IaaS平臺（如GCP、AWS、Azure等）的接口代碼。如果您沒有運行在受支持的IaaS平臺上（如GCP、AWS、Azure等），則創(chuàng)建時LoadBalancers將無限期處于“掛起”狀態(tài)。
在基礎集群中，操作員只有兩個接口來將用戶流量引入他們的集群，“NodePort”和“externalIPs”服務。
這兩個選項在生產使用中都有顯著的缺點，這使得基礎集群成為 Kubernetes 生態(tài)系統(tǒng)中的二等公民。
MetalLB旨在通過提供與標準網絡設備集成的網絡負載均衡器實現來解決這種不平衡，以便基礎群集上的external services盡可能“正常工作”。
root@master:~# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k8s-master01 Ready control-plane 18h v1.30.2 192.168.1.31 <none> Ubuntu 24.04 LTS 6.8.0-35-generic containerd://1.7.18
k8s-master02 Ready control-plane 18h v1.30.2 192.168.1.32 <none> Ubuntu 24.04 LTS 6.8.0-35-generic containerd://1.7.18
k8s-master03 Ready control-plane 18h v1.30.2 192.168.1.33 <none> Ubuntu 24.04 LTS 6.8.0-35-generic containerd://1.7.18
k8s-node01 Ready <none> 18h v1.30.2 192.168.1.34 <none> Ubuntu 24.04 LTS 6.8.0-35-generic containerd://1.7.18
k8s-node02 Ready <none> 18h v1.30.2 192.168.1.35 <none> Ubuntu 24.04 LTS 6.8.0-35-generic containerd://1.7.18
修改kube-system
如果您正在使用IPVS模式下的kube-proxy，則自Kubernetes v1.14.2起，您必須啟用嚴格的ARP模式。
請注意，如果您使用kube-router作為服務代理，則不需要此操作，因為它默認啟用了 strict ARP。
您可以通過編輯當前集群中的kube-proxy配置來實現此操作：
# kubeadm 部署方式 修改kube-system
kubectl get configmap kube-proxy -n kube-system -o yaml | \
sed -e "s/strictARP: false/strictARP: true/" | \
kubectl apply -f - -n kube-system
# 二進制 部署方式 修改kube-system
cat > /etc/kubernetes/kube-proxy.yaml << EOF
apiVersion: kubeproxy.config.k8s.io/v1alpha1
bindAddress: 0.0.0.0
clientConnection:
acceptContentTypes: ""
burst: 10
contentType: application/vnd.kubernetes.protobuf
kubeconfig: /etc/kubernetes/kube-proxy.kubeconfig
qps: 5
clusterCIDR: 172.16.0.0/12,fc00:2222::/112
configSyncPeriod: 15m0s
conntrack:
max: null
maxPerCore: 32768
min: 131072
tcpCloseWaitTimeout: 1h0m0s
tcpEstablishedTimeout: 24h0m0s
enableProfiling: false
healthzBindAddress: 0.0.0.0:10256
hostnameOverride: ""
iptables:
masqueradeAll: false
masqueradeBit: 14
minSyncPeriod: 0s
syncPeriod: 30s
ipvs:
strictARP: true
masqueradeAll: true
minSyncPeriod: 5s
scheduler: "rr"
syncPeriod: 30s
kind: KubeProxyConfiguration
metricsBindAddress: 127.0.0.1:10249
mode: "ipvs"
nodePortAddresses: null
oomScoreAdj: -999
portRange: ""
udpIdleTimeout: 250ms
EOF
systemctl restart kube-proxy
systemctl status kube-proxy
部署metallb
# 下載應用包
wget https://mirrors.chenby.cn/https://github.com/metallb/metallb/archive/refs/tags/v0.14.5.tar.gz
tar -zxvf v0.14.5.tar.gz
cd metallb-0.14.5/config/manifests
# 修改鏡像地址
sed -i "s#quay.io#quay.chenby.cn#g" metallb-native.yaml
cat metallb-native.yaml | grep image
image: quay.chenby.cn/metallb/controller:v0.14.5
image: quay.chenby.cn/metallb/speaker:v0.14.5
# 執(zhí)行部署
kubectl apply -f metallb-native.yaml
root@k8s-master01:~# kubectl -n metallb-system get all
NAME READY STATUS RESTARTS AGE
pod/controller-6975f6bf7b-nm2d6 1/1 Running 0 23m
pod/speaker-4jtb4 1/1 Running 0 23m
pod/speaker-fpd6q 1/1 Running 0 23m
pod/speaker-mmfxq 1/1 Running 0 23m
pod/speaker-rxs2b 1/1 Running 0 23m
pod/speaker-sfxvb 1/1 Running 0 23m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/metallb-webhook-service ClusterIP 10.96.95.84 <none> 443/TCP 23m
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/speaker 5 5 5 5 5 kubernetes.io/os=linux 23m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/controller 1/1 1 1 23m
NAME DESIRED CURRENT READY AGE
replicaset.apps/controller-6975f6bf7b 1 1 1 23m
root@k8s-master01:~#
創(chuàng)建池
# 新版本metallb使用了CR（Custom Resources），這里我們通過IPAddressPool的CR，進行地址池的定義。
# 如果實例中不設置IPAddressPool選擇器L2Advertisement；那么L2Advertisement默認為該實例所有的IPAddressPool相關聯。
cat > metallb-config-ipaddresspool.yaml << EOF
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: first-pool
namespace: metallb-system
spec:
addresses:
- 192.168.1.70-192.168.1.79
EOF
# 進行L2關聯地址池的綁定。
cat > metallb-config-L2Advertisement.yaml << EOF
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: example
namespace: metallb-system
spec:
ipAddressPools:
- first-pool
EOF
# 執(zhí)行部署
kubectl apply -f metallb-config-ipaddresspool.yaml
kubectl apply -f metallb-config-L2Advertisement.yaml
測試
# 測試
cat > metallb-nginx.yaml << EOF
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
namespace: default
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1
ports:
- name: http
containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx
namespace: default
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
EOF
# 執(zhí)行部署
kubectl apply -f metallb-nginx.yaml
查看
# 查看
# EXTERNAL-IP已經獲取到IP地址
root@k8s-master01:~# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nginx-648c475cfb-5pvvv 1/1 Running 0 4m24s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/nginx LoadBalancer 10.96.197.147 192.168.1.51 80:30752/TCP 4m24s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx 1/1 1 1 4m24s
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-648c475cfb 1 1 1 4m24s
root@k8s-master01:~#
訪問
# 訪問
root@k8s-master01:~# curl 192.168.1.51
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a >nginx.org</a>.<br/>
Commercial support is available at
<a >nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
root@k8s-master01:~#
