k8s集群網(wǎng)絡(luò)從cilium集群遷移至calico集群后發(fā)現(xiàn)集群網(wǎng)絡(luò)中pod->service部分網(wǎng)絡(luò)不通，經(jīng)過排查發(fā)現(xiàn)是cilium ebpf程序未卸載干凈。經(jīng)過各種折騰，終于成功。以下步驟均需在每個節(jié)點(diǎn)執(zhí)行。
1 移除iptable，虛擬網(wǎng)卡
iptables -t filter -F
iptables -t filter -X
iptables -t filter -Z
iptables -t raw -F
iptables -t raw -X
iptables -t raw -Z
iptables -t nat -F
iptables -t nat -X
iptables -t nat -Z
iptables -t mangle -F
iptables -t mangle -X
iptables -t mangle -Z
for link_name in $(ip link | grep -Ei "lxc|cilium|cali|dumm|ipvs|veth" | awk -F ':' '{print $2}' | awk -F '@' '{print $1}' | xargs );do echo $link_name;ip link delete $link_name;done
2 刪除cilium tc xdp ebpf程序
# 執(zhí)行命令，將其中cilum image替換為自己的
sudo docker run -it -v /run/cilium/cgroupv2/:/run/cilium/cgroupv2/ -v /sys/fs/bpf:/sys/fs/bpf -v /sys/fs/cgroup/:/sys/fs/cgroup/ --privileged --net=host harbor.cars.rails.cn/k8s/cilium:v1.11.9 -- cilium cleanup -f --all-state
3 刪除cilium socket ebpf程序
執(zhí)行命令，進(jìn)入容器，將其中cilum image替換為自己的
docker run -it -v /run/cilium/cgroupv2/:/run/cilium/cgroupv2/ -v /sys/fs/bpf:/sys/fs/bpf -v /sys/fs/cgroup/:/sys/fs/cgroup/ --privileged --net=host harbor.cars.rails.cn/k8s/cilium:v1.11.9 -- bash
安裝依賴
sed -i "s@http://.*archive.ubuntu.com@https://mirrors.tuna.tsinghua.edu.cn@g" /etc/apt/sources.list
sed -i "s@http://.*security.ubuntu.com@https://mirrors.tuna.tsinghua.edu.cn@g" /etc/apt/sources.list
apt update && apt install -y jq vim
執(zhí)行腳本
#!/usr/bin/env bash
if [[ "${EUID}" -ne 0 ]]; then
echo "This script must be run as root"
exit 1
fi
while read -r cgroup_json; do
echo $cgroup_json
cgroup="$(jq -r .cgroup <<<"${cgroup_json}")"
# Don't want to detach systemd programs. Cilium attaches its program at the root and under docker.
while read -r id attach_type; do
bpftool cgroup detach "${cgroup}" "${attach_type}" id "${id}"
done < <(jq -r '.programs[] | [.id,.attach_type] | @tsv' <<<"${cgroup_json}")
done < <(bpftool cgroup tree -j | jq -c '.[]')
執(zhí)行命令
cilium cleanup
