kubernetes部署高可用Harbor

前言

本文Harbor高可用依照Harbor官網(wǎng)部署，主要思路如下，大家可以根據(jù)具體情況選擇搭建。

一、Harbor部署前準(zhǔn)備

本文僅說(shuō)明高可用配置，其余部署請(qǐng)查看《kubernetes搭建Harbor無(wú)坑及Harbor倉(cāng)庫(kù)同步》

.安裝方式

  • helm安裝

  • 直接使用博主整理好的編排文件安裝（通過(guò)Helm生成）

1.helm安裝

安裝Helm請(qǐng)查看《kubernetes搭建Harbor無(wú)坑及Harbor倉(cāng)庫(kù)同步》，其中包含Helm安裝。

1.1.下載 harbor-helm 
git clone https://github.com/goharbor/harbor-helm.git
cd XXX/harbor-helm
1.2.修改value.yaml

database的Postgresql配置

database:
  # if external database is used, set "type" to "external"
  # and fill the connection informations in "external" section
  type: external
  internal:
    image:
      repository: goharbor/harbor-db
      tag: v1.8.2-dev
    # The initial superuser password for internal database
    password: "changeit"
    # resources:
    #  requests:
    #    memory: 256Mi
    #    cpu: 100m
    nodeSelector: {}
    tolerations: []
    affinity: {}
  external:
    host: "stolon-proxy-service" #管理postgresql的stolon的service，因?yàn)槎荚赑od中可相互訪問(wèn)
    port: "5432"
    username: "postgres"
    password: "password1"
    coreDatabase: "registry"
    clairDatabase: "clair"
    notaryServerDatabase: "notaryserver"
    notarySignerDatabase: "notarysigner"
    sslmode: "disable"
  ## Additional deployment annotations
  podAnnotations: {}

redis的配置

redis:
  # if external Redis is used, set "type" to "external"
  # and fill the connection informations in "external" section
  type: external
  internal:
    image:
      repository: goharbor/redis-photon
      tag: v1.8.2-dev
    # resources:
    #  requests:
    #    memory: 256Mi
    #    cpu: 100m
    nodeSelector: {}
    tolerations: []
    affinity: {}
  external:
    host: "10.8.4.133" #haproxy的地址通過(guò)haproxy管理redis集群
    port: "6379"
    # The "coreDatabaseIndex" must be "0" as the library Harbor
    # used doesn't support configuring it
    coreDatabaseIndex: "0"
    jobserviceDatabaseIndex: "1"
    registryDatabaseIndex: "2"
    chartmuseumDatabaseIndex: "3"
    password: ""

修改Harbor其他組件replicas(副本數(shù))

# 例如nginx的副本數(shù)更改
nginx:
  image:
    repository: goharbor/nginx-photon
    tag: v1.8.2-dev
  replicas: 3
1.3.準(zhǔn)備Harbor所需的registry、notarysigner、notaryserver、clair數(shù)據(jù)庫(kù)，Harbor會(huì)自動(dòng)在其中建表。

執(zhí)行sql語(yǔ)句腳本，供stolon-init-database-job.yaml使用

cat <<EOF > ./postgresql.sh
#!/bin/bash

host="stolon-proxy-service"
user="postgres"
db="postgres"
export PGPASSWORD="password1"

args=(
        # force postgres to not use the local unix socket (test "external" connectibility)
        --host "$host"
        --username "$user"
        --dbname "$db"
        --quiet --no-align --tuples-only
)

if select="$(echo 'SELECT 1' | psql "${args[@]}")" && [ "$select" = '1' ]; then
   echo "====notaryserver==database==creating===="
   psql -h stolon-proxy-service -p 5432 -U postgres -f "/docker-entrypoint-initdb.d/initial-notaryserver.sql"
   echo "====notarysigner==database==creating===="
   psql -h stolon-proxy-service -p 5432 -U postgres -f "/docker-entrypoint-initdb.d/initial-notarysigner.sql"
   echo "====registry==database==creating===="
   psql -h stolon-proxy-service -p 5432 -U postgres -f "/docker-entrypoint-initdb.d/initial-registry.sql"
   echo "====clair==database==creating===="
   psql -h stolon-proxy-service -p 5432 -U postgres -f "/docker-entrypoint-initdb.d/initial-clair.sql"   
   exit 0
fi
exit 1
EOF

創(chuàng)建registry數(shù)據(jù)庫(kù)

cat <<EOF > ./initial-registry.sql
CREATE DATABASE registry ENCODING 'UTF8';
\c registry;
CREATE TABLE schema_migrations(version bigint not null primary key, dirty boolean not null);
EOF

創(chuàng)建notaryserver數(shù)據(jù)庫(kù)

cat <<EOF > ./initial-notaryserver.sql
CREATE DATABASE notaryserver;
CREATE USER server;
alter user server with encrypted password 'password';
GRANT ALL PRIVILEGES ON DATABASE notaryserver TO server;
EOF

創(chuàng)建notarysigner數(shù)據(jù)庫(kù)

cat <<EOF > ./initial-notarysigner.sql
CREATE DATABASE notarysigner;
CREATE USER signer;
alter user signer with encrypted password 'password';
GRANT ALL PRIVILEGES ON DATABASE notarysigner TO signer;                                                           
EOF

創(chuàng)建clair數(shù)據(jù)庫(kù)

cat <<EOF > ./initial-clair.sql
CREATE DATABASE clair;
EOF

創(chuàng)建一個(gè)job的yaml（stolon-init-database-job.yaml），用于創(chuàng)建數(shù)據(jù)庫(kù)，注意更改腳本的掛載位置，并復(fù)制腳本到各個(gè)節(jié)點(diǎn)或?yàn)閚ode和yaml加上nodeselect標(biāo)簽，只在當(dāng)前標(biāo)簽node下復(fù)制腳本

apiVersion: batch/v1
kind: Job
metadata:
  name: stolon-init-database-job
spec:
  template:
    spec:
      containers:
      - name: stolon-proxy
        image: sorintlab/stolon:master-pg10
        command:
          - "/bin/bash"
          - "/docker-entrypoint-initdb.d/postgresql.sh"
        volumeMounts:
        - mountPath: /docker-entrypoint-initdb.d
          name: database
      restartPolicy: OnFailure     #失敗重啟
      volumes:
        - name: database
          hostPath:
            path: /root/tmp/harbor/stolon/examples/kubernetes/sql
  activeDeadlineSeconds: 600   #10分鐘沒(méi)有complete，不再重啟并移除Pod
1.3.部署Postgresql、redis
  • 按照《kubernetes下Stolon部署高可用Postgresql》部署Postgresql，注意加入stolon-init-database-job.yaml。
  • 按照《kubernetes部署高可用redis》部署redis，之后用haproxy管理redis集群（不可直接使用redis的service暴露，service會(huì)訪問(wèn)到slave節(jié)點(diǎn)，redis副本是只讀不可寫(xiě)的，在harbor中會(huì)有報(bào)錯(cuò)）
  • 部署haproxy
    1. 安裝haproxy
      yum -y install haproxy
      cp /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg-back
      vim /etc/haproxy/haproxy.cfg
    2. 加入配置
frontend ft_redis
 bind 0.0.0.0:6379 name redis
 default_backend bk_redis

backend bk_redis
 option tcp-check
 tcp-check connect
 tcp-check send PING\r\n
 tcp-check expect string +PONG
 tcp-check send info\ replication\r\n
 tcp-check expect string role:master
 tcp-check send QUIT\r\n
 tcp-check expect string +OK
 server R1 redis-0.redis-headless.default.svc.cluster.local:6379 check inter 1s
 server R2 redis-1.redis-headless.default.svc.cluster.local:6379 check inter 1s
 server R3 redis-2.redis-headless.default.svc.cluster.local:6379 check inter 1s

listen admin_stats
        stats   enable
        bind    *:9090
        mode    http
        option  httplog
        log     global
        maxconn 10
        stats   refresh 30s
        stats   uri /admin 
        stats   realm haproxy
        stats   auth admin:admin
        stats   hide-version 
        stats   admin if TRUE

systemctl start haproxy && systemctl enable haproxy && systemctl status haproxy
訪問(wèn) harbor節(jié)點(diǎn)Ip:9090/admin如圖所示，便成功

redis

??k8s master節(jié)點(diǎn)高可用可閱讀《haproxy+keepalive實(shí)現(xiàn)master集群高可用》

1.4.部署Harbor

安裝harbor并將日志寫(xiě)入文件，可編輯文件保留.yaml編排文件，以便以后使用

helm install . --debug --name hub |sed 'w harbor.yaml'

或執(zhí)行以下命令，編排chart不執(zhí)行，作用生成編排文件，刪除多余部分，進(jìn)行使用

helm install . --debug --dry-run --name hub |sed 'w harbor.yaml'

2.通過(guò)整理好的編排文件執(zhí)行

鏈接:https://pan.baidu.com/s/1cr1fnWGHc-70HAxx1YH4kg 密碼:21a8
直接使用這個(gè)編排文件可能會(huì)有問(wèn)題，最好勤勞以下使用helm跑，也可避免更改配置遺漏或錯(cuò)誤的問(wèn)題，適用用于實(shí)驗(yàn)，如若搭建請(qǐng)注意修改Volum、requestsource等Pod設(shè)置

由于國(guó)內(nèi)對(duì)k8s集群資料極少，本文為作者爬坑手碼，如果對(duì)您有所幫助希望可以留下你的喜歡

最后編輯于 ：
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡(jiǎn)書(shū)系信息發(fā)布平臺(tái)，僅提供信息存儲(chǔ)服務(wù)。

友情鏈接更多精彩內(nèi)容