背景介紹
離線開發(fā)環(huán)境需要連接阿里云 OSS 服務(wù)， 通過調(diào)研我們決定使用 oss-emulator 模擬 OSS 服務(wù)。
項(xiàng)目介紹
oss-emulator 是阿里開源的輕量級(jí) OSS 服務(wù)模擬器，提供與 OSS 服務(wù)相同的 API 接口
但該項(xiàng)目僅提供基于 linux 部署方案，不支持容器化，無法滿足容器化部署場(chǎng)景
linux-install.png
因此，我將 oss-emulator 項(xiàng)目打包成鏡像發(fā)布到 docker hub，并提供容器化部署方案，便于后續(xù)使用。
容器化過程及驗(yàn)證
需求依賴：docker hub 賬號(hào)
- fork項(xiàng)目
fork.png
- 配置 docker hub 賬號(hào)口令
用于后續(xù) github action 自動(dòng)構(gòu)建使用
new-secrect.png
票據(jù)內(nèi)容如下：
github-secret.png
- 登錄 docker hub 創(chuàng)建 oss-emulator 倉庫
create-repo.png
到此為止，我們準(zhǔn)備工作已經(jīng)完成，接下來就是將項(xiàng)目打包成鏡像
- 項(xiàng)目根新增 Dockerfile ,內(nèi)容如下
FROM ruby:alpine
MAINTAINER weiliang-ms@github
WORKDIR /work
ADD . .
RUN gem install thor builder webrick
CMD ["ruby","/work/bin/emulator", "-r", "store", "-p", "8080"]
- 項(xiàng)目根新增 github action 流水線配置
配置路徑：.github/workflows/build.yml
配置內(nèi)容如下：
name: ci
on:
push:
branches:
- 'master'
jobs:
docker:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
-
name: Set up QEMU
uses: docker/setup-qemu-action@v1
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
-
name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
-
name: Build and push
uses: docker/build-push-action@v2
with:
context: ./
file: Dockerfile
push: true
tags: xzxwl/oss-emulator
提交變更后，github action 流水線基于配置進(jìn)行鏡像構(gòu)建，最終將生成的 xzxwl/oss-emulator 鏡像推送至 docker hub
- 啟動(dòng) oss-emulator 容器
$ mkdir -p /work/oss-data
$ docker run -idt --name oss-emulator -p 8080:8080 -v /work/oss-data:/work/store xzxwl/oss-emulator
- 下載 ossutil 測(cè)試管理 oss-emulator
下載授權(quán)
$ wget https://gosspublic.alicdn.com/ossutil/1.7.14/ossutil64
$ chmod 755 ossutil64
- 測(cè)試 OSS 可用性
測(cè)試創(chuàng)建 bucket 功能
$ ./ossutil64 -e http://127.0.0.1:8080 -i AccessKeyId -k AccessKeySecret mb oss://bucket-test
0.005875(s) elapsed
測(cè)試查詢 bucket
$ ./ossutil64 -e http://127.0.0.1:8080 -i AccessKeyId -k AccessKeySecret ls oss://bucket-test
Object Number is: 0
0.006463(s) elapsed
測(cè)試上傳文件
$ touch test.file
$ ./ossutil64 -e http://127.0.0.1:8080 -i AccessKeyId -k AccessKeySecret cp test.file oss://bucket-test/
Succeed: Total num: 1, size: 0. OK num: 1(upload 1 files).
average speed 0(byte/s)
0.014518(s) elapsed
$ ./ossutil64 -e http://127.0.0.1:8080 -i AccessKeyId -k AccessKeySecret ls oss://bucket-test/
LastModifiedTime Size(B) StorageClass ETAG ObjectName
2023-01-11 21:56:51 -0500 EST 0 Standard D41D8CD98F00B204E9800998ECF8427E oss://bucket-test/test.file
Object Number is: 1
0.008415(s) elapsed
該項(xiàng)目并非完全兼容 OSS 接口，僅支持以下接口內(nèi)容，使用時(shí)注意
接口兼容性
-
oss-emulator 支持
put, get, list, copy, delete, multipart等數(shù)據(jù)操作API接口，支持部分Bucket操作接口。
Bucket相關(guān)接口
- 支持
ListBuckets(GetService),PutBucket(CreateBucket),GetBucket,DeleteBucket,
GetBucketLocation,GetBucketInfo,PutBucketACL,GetBucketACL
- 不支持
PutBucketLogging,PutBucketWebsite,PutBucketReferer,PutBucketLifecycle,
GetBucketLogging,GetBucketWebsite,GetBucketReferer,GetBucketLifecycle,
DeleteBucketLogging,DeleteBucketWebsite,DeleteBucketLifecycle
Object相關(guān)接口
- 支持
PutObject,CopyObject,AppendObject,GetObject,DeleteObject,DeleteMultipleObjects,
HeadObject,GetObjectMeta,PutObjectACL,GetObjectACL
- 不支持
PostObject,Callback,PutSymlink,GetSymlink,RestoreObject
Multipart相關(guān)接口
- 支持
InitiateMultipartUpload,UploadPart,CompleteMultipartUpload
- 不支持
UploadPartCopy,AbortMultipartUpload,ListMultipartUpload,ListParts
容器化部署方式
docker
創(chuàng)建數(shù)據(jù)持久化目錄
$ mkdir -p /oss-store
啟動(dòng)服務(wù)
$ docker run -idt --name oss-emulator -p 8080:8080 --restart=always -v /work/oss-data:/work/store xzxwl/oss-emulator
kubernetes
創(chuàng)建 pvc
$ cat <<EOF | kubectl apply -f -
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: oss-emulator-volume
namespace: test
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 50Gi
EOF
創(chuàng)建 deployment
$ cat <<EOF | kubectl apply -f -
kind: Deployment
apiVersion: apps/v1
metadata:
name: oss-emulator
namespace: test
labels:
app: oss-emulator
spec:
replicas: 1
selector:
matchLabels:
app: oss-emulator
template:
metadata:
labels:
app: oss-emulator
spec:
volumes:
- name: host-time
hostPath:
path: /etc/localtime
type: ''
- name: volume-b5cw22
persistentVolumeClaim:
claimName: oss-emulator-volume
containers:
- name: container-14y7vu
image: xzxwl/oss-emulator
ports:
- name: http-8080
containerPort: 8080
protocol: TCP
resources:
limits:
cpu: 200m
memory: 400Mi
volumeMounts:
- name: host-time
readOnly: true
mountPath: /etc/localtime
- name: volume-b5cw22
mountPath: /work/store
imagePullPolicy: IfNotPresent
restartPolicy: Always
terminationGracePeriodSeconds: 30
dnsPolicy: ClusterFirst
serviceAccountName: default
serviceAccount: default
securityContext: {}
schedulerName: default-scheduler
EOF
創(chuàng)建 service
$ cat <<EOF | kubectl apply -f -
kind: Service
apiVersion: v1
metadata:
name: oss-emulator-svc
namespace: test
labels:
app: oss-emulator-svc
spec:
ports:
- name: http-8080
protocol: TCP
port: 80
targetPort: 8080
selector:
app: oss-emulator
EOF
此時(shí)即可在 kubernetes 內(nèi)部通過以下地址使用 oss 服務(wù)
oss-emulator-svc.test:80
AK, SK 可以配置如下
AK: AccessKeyId
SK: AccessKeySecret
對(duì)于 xzxwl/oss-emulator 這個(gè)鏡像，離線環(huán)境下可以上傳至私有鏡像庫（例如harbor）進(jìn)行管理使用
