在前面的三篇文章中，已經(jīng)實(shí)現(xiàn)了使用Jenkins拉取GitHub的項(xiàng)目，并進(jìn)行構(gòu)建，最后在推送到采用Harbor搭建的私有鏡像倉(cāng)庫(kù)中。CI的流程基本上可以說(shuō)已經(jīng)實(shí)現(xiàn)完了，這節(jié)將介紹如何將構(gòu)建好的服務(wù)部署到k8s集群中。
以下的操作全部在namespace是tlh的命名空間執(zhí)行，所以之前先創(chuàng)建namespace。
kubectl create namespace tlh
1、部署MySQL
1.1 創(chuàng)建pv用戶掛載初始化腳本
-
創(chuàng)建pvc.yaml文件
kind: PersistentVolumeClaim apiVersion: v1 metadata: name: mysql-init-storage namespace: tlh spec: accessModes: - ReadWriteMany resources: requests: storage: 50M -
創(chuàng)建pvc
這里也是使用前面章節(jié)創(chuàng)建的默認(rèn)的storageclass來(lái)動(dòng)態(tài)創(chuàng)建pv
kubectl apply -f pvc.yaml -n tlh -
上傳初始化腳本
-
查看創(chuàng)建好的pv
1.png -
將源碼中的tlh.sql文件上傳到對(duì)應(yīng)的文件夾中
2.png
-
1.2 創(chuàng)建部署
-
創(chuàng)建deployment.yaml文件
--- apiVersion: apps/v1 kind: Deployment metadata: name: mysql spec: selector: matchLabels: app: mysql template: metadata: labels: app: mysql spec: containers: - image: mysql:5.7 name: mysql env: - name: MYSQL_ROOT_PASSWORD # 設(shè)置root密碼 value: "123456" - name: MYSQL_DATABASE # 初始化創(chuàng)建的數(shù)據(jù)庫(kù) value: "tlh" ports: - containerPort: 3306 name: mysql volumeMounts: - name: mysql-persistent-storage # 掛載數(shù)據(jù)存儲(chǔ) mountPath: /var/lib/mysql - name: mysql-init mountPath: /docker-entrypoint-initdb.d # 掛載初始化腳本目錄 volumes: - name: mysql-persistent-storage # 聲明使用的pv persistentVolumeClaim: claimName: mysql-pv-claim - name: mysql-init persistentVolumeClaim: claimName: mysql-init-storage --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: mysql-pv-claim # 創(chuàng)建存儲(chǔ)的pvc spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi -
創(chuàng)建deployment
kubectl apply -f deployment.yaml -n tlh -
查看deployment
kubectl get deployment mysql -n tlh3.png
1.3 創(chuàng)建服務(wù)
-
創(chuàng)建service.yaml文件
apiVersion: v1 kind: Service metadata: name: mysql spec: ports: - port: 3306 selector: app: mysql -
創(chuàng)建service
kubectl apply -f service.yaml -n tlh -
查看服務(wù)
kubectl get service mysql -n tlh4.png
2、部署Tlh服務(wù)
因?yàn)樽罱K在Jenkins構(gòu)建完之后是將docker鏡像上傳到的harbor搭建的私有倉(cāng)庫(kù)中，而k8s不能直接從私有倉(cāng)庫(kù)拉取鏡像，所以先要配置密鑰。
2.1 配置k8s登陸harbor的密鑰
-
在master主機(jī)執(zhí)行docker login，這里填寫上一章中在harbor中創(chuàng)建的jenkins賬戶的信息。
docker login -
生成密鑰
kubectl create secret generic regcred \ --from-file=.dockerconfigjson=.docker/config.json \ --type=kubernetes.io/dockerconfigjson \ -n tlh # 注意命名空間 -
查看密鑰
kubectl get secret regcred --output=yaml -n tlh
2.2 創(chuàng)建configmap
在部署的服務(wù)中需要使用到MySQL服務(wù)，而其配置信息需要數(shù)據(jù)庫(kù)驅(qū)動(dòng)、用戶名和密碼，在以往docker方式部署的時(shí)候通過(guò)環(huán)境變量的方式注入到應(yīng)用中。這里先采用configmap來(lái)存儲(chǔ)應(yīng)用需要的配置信息，方便進(jìn)行管理，最后也通過(guò)環(huán)境變量的方式注入到pod中。
- 創(chuàng)建tlh-env-file.properties文件
mysql.user=root
mysql.pwd=123456
mysql.url=jdbc:mysql://mysql.tlh.svc.cluster.local:3306/tlh # 這里使用k8s的dns來(lái)填寫數(shù)據(jù)庫(kù)的url地址
-
創(chuàng)建configmap
kubectl create configmap tlh-configmap --from-env-file=tlh-env-file.properties -n tlh -
查看configmap
kubectl get configmaps tlh-configmap -o yaml -n tlh5.png
2.3 創(chuàng)建deployment
-
創(chuàng)建deploymen.yaml文件
apiVersion: apps/v1 kind: Deployment metadata: name: tlh-deployment namespace: tlh spec: selector: matchLabels: app: tlh-server template: metadata: labels: app: tlh-server spec: containers: - image: harbor.tlh.com/tlhhup/jks:1.2 name: tlh env: - name: MYSQL_USER valueFrom: configMapKeyRef: # 這里引用configmap中的key name: tlh-configmap key: mysql.user - name: MYSQL_PASSWORD valueFrom: configMapKeyRef: name: tlh-configmap key: mysql.pwd - name: MYSQL_URL valueFrom: configMapKeyRef: name: tlh-configmap key: mysql.url ports: - containerPort: 8089 name: tlh imagePullSecrets: - name: regcred -
創(chuàng)建deployment
kubectl apply -f deployment.yaml -n tlh -
查看
kubectl get deployment tlh-deployment -n tlh6.png
2.4 創(chuàng)建service
-
創(chuàng)建service.yaml
apiVersion: v1 kind: Service metadata: name: tlh spec: ports: - port: 8089 selector: app: tlh-server -
創(chuàng)建service
kubectl apply -f service.yaml -n tlh -
查看
kubectl get service tlh -n tlh7.png
3、部署Ingress
為了達(dá)到單獨(dú)的IP地址的效果，這里重新部署一個(gè)ingress-controller，最終由metallab進(jìn)行重新分配一個(gè)新得IP地址。
3.1 部署ingress-controller
-
創(chuàng)建deployement.yaml文件
kind: ConfigMap apiVersion: v1 metadata: name: nginx-configuration namespace: tlh labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: tcp-services namespace: tlh labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: udp-services namespace: tlh labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- apiVersion: v1 kind: ServiceAccount metadata: name: nginx-ingress-serviceaccount namespace: tlh labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: Role metadata: name: nginx-ingress-role namespace: tlh labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx rules: - apiGroups: - "" resources: - configmaps - pods - secrets - namespaces verbs: - get - update - list - apiGroups: - "" resources: - configmaps resourceNames: # Defaults to "<election-id>-<ingress-class>" # Here: "<ingress-controller-leader>-<nginx>" # This has to be adapted if you change either parameter # when launching the nginx-ingress-controller. - "ingress-controller-leader-nginx" verbs: - get - update - list - apiGroups: - "" resources: - configmaps verbs: - create - update - list - apiGroups: - "" resources: - endpoints verbs: - get - list --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: RoleBinding metadata: name: nginx-ingress-role-nisa-tlh-binding namespace: tlh labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: nginx-ingress-role subjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: tlh --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: nginx-ingress-clusterrole-nisa-tlh-binding labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: nginx-ingress-clusterrole subjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: tlh --- apiVersion: apps/v1 kind: Deployment metadata: name: nginx-ingress-controller namespace: tlh labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: replicas: 1 selector: matchLabels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx template: metadata: labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx annotations: prometheus.io/port: "10254" prometheus.io/scrape: "true" spec: # wait up to five minutes for the drain of connections terminationGracePeriodSeconds: 300 serviceAccountName: nginx-ingress-serviceaccount containers: - name: nginx-ingress-controller image: wistiaanders/nginx-ingress-controller:0.25.1 args: - /nginx-ingress-controller - --configmap=$(POD_NAMESPACE)/nginx-configuration - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - --publish-service=$(POD_NAMESPACE)/ingress-nginx - --annotations-prefix=nginx.ingress.kubernetes.io - --ingress-class=tlh-nginx-ingress # 注意這個(gè)值的設(shè)置，必需唯一 - --enable-ssl-passthrough securityContext: allowPrivilegeEscalation: true capabilities: drop: - ALL add: - NET_BIND_SERVICE # www-data -> 33 runAsUser: 33 env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace ports: - name: http containerPort: 80 - name: https containerPort: 443 livenessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 readinessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 lifecycle: preStop: exec: command: - /wait-shutdown --- -
創(chuàng)建deployment
kubectl apply -f deployment.yaml -n tlh -
創(chuàng)建service.yaml文件
kind: Service apiVersion: v1 metadata: name: ingress-nginx namespace: tlh labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: externalTrafficPolicy: Local type: LoadBalancer selector: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx ports: - name: http port: 80 targetPort: http - name: https port: 443 targetPort: https --- -
創(chuàng)建service
kubectl apply -f service.yaml -n tlh
3.2 部署Tlh的ingress
-
創(chuàng)建ingress.yaml文件
apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: tlh-ingress namespace: tlh annotations: kubernetes.io/ingress.class: "tlh-nginx-ingress" # 設(shè)置使用的ingress-controller spec: rules: - host: dev.tlh.com # 通過(guò)host來(lái)匹配規(guī)則 http: paths: - path: / backend: serviceName: tlh servicePort: 8089 -
創(chuàng)建ingress
kubectl apply -f ingress.yaml -n tlh -
查看ingress
kubectl describe ingress -n tlh8.png -
將查看得到的IP地址配置到宿主機(jī)的hosts文件中，打開瀏覽器http://dev.tlh.com/tlh,輸入用戶名和密碼(admin/admin)
9.png
