SpringBoot整合分布式文件系統(tǒng)

一.文件本地上傳

1.1 文件上傳目錄

服務(wù)端接收上傳的目的是提供文件的訪問(wèn)服務(wù),那么對(duì)于SpringBoot而言,有哪些可以提供文件訪問(wèn)的靜態(tài)資源目錄呢?

  • classpath:/META-INF/resources/ ,
  • classpath:/static/ ,
  • classpath:/public/ ,
  • `classpath:/resources/

以上目錄都在工程里面,用來(lái)存儲(chǔ)動(dòng)態(tài)上傳的文件是有很多問(wèn)題的。spring boot 為我們提供了使用spring.web.resources.static-locations配置自定義靜態(tài)文件的位置。

web:
  upload-path: E:/data/
  
spring:
  web:
    resources:
      static-locations: classpath:/META-INF/resources/,classpath:/resources/,classpath:/static/,classpath:/public/,file:${web.upload-path}
  • 配置web.upload-path為與項(xiàng)目代碼分離的靜態(tài)資源路徑,即:文件上傳保存根路徑
  • 配置spring.web.resources.static-locations,除了帶上Spring Boot默認(rèn)的靜態(tài)資源路徑之外,加上file:${web.upload-path}指向外部的文件資源上傳路徑。該路徑下的靜態(tài)資源可以直接對(duì)外提供HTTP訪問(wèn)服務(wù)。

1.2 文件上傳的Controller實(shí)現(xiàn)

@RestController
public class FileUploadController {

    @Value("${web.upload-path}")
    private String uploadPath;

    @PostMapping("/upload")
    public String upload(MultipartFile uploadFile, HttpServletRequest request) throws IOException {

        // 在 uploadPath 文件夾中通過(guò)日期對(duì)上傳的文件歸類保存
        // 比如:/2019/06/06/cf13891e-4b95-4000-81eb-b6d70ae44930.png
        SimpleDateFormat sdf = new SimpleDateFormat("yyyy/MM/dd/");
        String format = sdf.format(new Date());
        File folder = new File(uploadPath + format);
        if (!folder.isDirectory()) {
          folder.mkdirs();
        }

        // 對(duì)上傳的文件重命名,避免文件重名
        String oldName = uploadFile.getOriginalFilename();
        String newName = UUID.randomUUID().toString() + oldName.substring(oldName.lastIndexOf("."));

        // 文件保存到本地
        File destFile = new File(folder, newName);
        uploadFile.transferTo(destFile);

        // 返回上傳文件的訪問(wèn)路徑
        //https://localhost:8888/2020/10/18/a9a05df4-6615-4bb5-b859-a3f9bf4bfae0.jpg
        String filePath = request.getScheme() + "://" + request.getServerName()
                + ":" + request.getServerPort() + "/"   + format + newName;
        return filePath;
    }
}

二.分布式文件系統(tǒng)MinIO

2.1 MinIO簡(jiǎn)介

MinIO 是一個(gè)基于Apache License v2.0開源協(xié)議的對(duì)象存儲(chǔ)服務(wù)。它兼容亞馬遜S3云存儲(chǔ)服務(wù)接口,非常適合于存儲(chǔ)大容量非結(jié)構(gòu)化的數(shù)據(jù),例如圖片、視頻、日志文件、備份數(shù)據(jù)和容器/虛擬機(jī)鏡像等,而一個(gè)對(duì)象文件可以是任意大小,從幾kb到最大5T不等。
MinIO官網(wǎng):https://min.io

2.2 Docker下安裝MinIO

1.創(chuàng)建目錄
一個(gè)用來(lái)存放配置,一個(gè)用來(lái)存儲(chǔ)上傳文件的目錄。啟動(dòng)前需要先創(chuàng)建Minio外部掛載的配置文件(如:E:/minio/config),和存儲(chǔ)上傳文件的目錄(如:E:/minio/data)

2.拉取MinIO的Docker鏡像:

docker pull minio/minio

3.創(chuàng)建Minio容器并運(yùn)行

docker run  -p 9000:9000 -p 9090:9090 --name minio -d --restart=always -e MINIO_ACCESS_KEY=minioadmin -e MINIO_SECRET_KEY=minioadmin -v E:/minio/data:/data -v E:/minio/config:/root/.minio minio/minio server /data  --console-address ":9090" --address ":9000"
  • 9000: S3-API端口,用于程序訪問(wèn);
  • 9090:控制臺(tái)端口,用戶通過(guò)瀏覽器訪問(wèn);
  • MINIO_ACCESS_KEY:指定 minio 默認(rèn)的 access_key ,同時(shí)也是管理服務(wù)的登錄用戶名,這里指定的是minioadmin;
  • MINIO_SECRET_KEY:指定 minio 默認(rèn)的 secret_key ,同時(shí)也是管理服務(wù)的的登錄密碼,這里指定的是minioadmin;

4.訪問(wèn)MinIO控制臺(tái)
本機(jī):http://127.0.0.1:9090或?qū)嶋HIP地址訪問(wèn)

三.整合MinIO的JavaSDK

3.1 整合MinIO

1.pom.xml引入:

        <dependency>
            <groupId>io.minio</groupId>
            <artifactId>minio</artifactId>
            <version>8.4.5</version>
            <exclusions>
                <exclusion>
                    <artifactId>okhttp</artifactId>
                    <groupId>com.squareup.okhttp3</groupId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>com.squareup.okhttp3</groupId>
            <artifactId>okhttp</artifactId>
            <version>4.9.0</version>
        </dependency>

2.application.yml,服務(wù)信息要和我們安裝的MinIO服務(wù)一致,這樣才能正常連接測(cè)試:

# MinIo文件服務(wù)器
minio:
  endpoint: http://127.0.0.1:9000
  accessKey: minioadmin
  secretKey: minioadmin

3.MinIoProperties.java 配置實(shí)體,將上文配置文件屬性裝載到實(shí)體配置對(duì)象中:

@Data
@ConfigurationProperties(prefix = "minio")
public class MinIOProperties {
    private String endpoint;
    private String accessKey;
    private String secretKey;
}

4.MinIO提供了MinioClient對(duì)bucket、文件等進(jìn)行操作,我們這里再封裝一個(gè)工具類:

@Component
@Configuration
@EnableConfigurationProperties({MinIOProperties.class})
public class MinIOTemplate {

    private MinIOProperties minIo;

    public MinIOTemplate(MinIOProperties minIo) {
        this.minIo = minIo;
    }

    private MinioClient instance;

    //minio操作對(duì)象實(shí)例化
    @PostConstruct
    public void init() {
        instance = MinioClient.builder()
                 .endpoint(minIo.getEndpoint())
                 .credentials(minIo.getAccessKey(), minIo.getSecretKey())
                 .build();
    }

    /**
     * 判斷 bucket是否存在
     */
    public boolean bucketExists(String bucketName)
            throws IOException, InvalidKeyException, InvalidResponseException,
            InsufficientDataException, NoSuchAlgorithmException,
            ServerException, InternalException, XmlParserException, ErrorResponseException {

            return instance.bucketExists(BucketExistsArgs.builder().bucket(bucketName).build());
    }

    /**
     * 創(chuàng)建 bucket
     */
    public void makeBucket(String bucketName) throws IOException, InvalidResponseException,
            InvalidKeyException, NoSuchAlgorithmException, ServerException,
            ErrorResponseException, XmlParserException,  InsufficientDataException, InternalException {

            boolean isExist = this.bucketExists(bucketName);
            if(!isExist) {
                instance.makeBucket(MakeBucketArgs.builder().bucket(bucketName).build());
            }
    }

    /**
     * 文件上傳
     * @param bucketName  bucket名稱
     * @param objectName 對(duì)象名稱,文件名稱
     * @param filepath 文件路徑
     */
    public ObjectWriteResponse putObject(String bucketName, String objectName, String filepath)
            throws IOException, InvalidKeyException, InvalidResponseException,
            InsufficientDataException, NoSuchAlgorithmException, ServerException,
            InternalException, XmlParserException, ErrorResponseException {

            return instance.uploadObject(
                         UploadObjectArgs.builder().bucket(bucketName).object(objectName).filename(filepath).build());
    }

    /**
     * 文件上傳
     * @param bucketName bucket名稱
     * @param objectName 對(duì)象名稱,文件名稱
     * @param inputStream 文件輸入流
     */
    public ObjectWriteResponse putObject(String bucketName, String objectName, InputStream inputStream)
            throws IOException, InvalidKeyException, InvalidResponseException,
            InsufficientDataException, NoSuchAlgorithmException, ServerException,
            InternalException, XmlParserException, ErrorResponseException {

            return instance.putObject(
                         PutObjectArgs.builder()
                                 .bucket(bucketName)
                                 .object(objectName).stream(
                                 inputStream, -1, 10485760)
                                 .build());
    }

    /**
     * 刪除文件
     * @param bucketName  bucket名稱
     * @param objectName  對(duì)象名稱
     */
    public void removeObject(String bucketName, String objectName)
            throws IOException, InvalidKeyException, InvalidResponseException, InsufficientDataException,
            NoSuchAlgorithmException, ServerException, InternalException, XmlParserException, ErrorResponseException {

            instance.removeObject(RemoveObjectArgs.builder()
                    .bucket(bucketName)
                    .object(objectName)
                    .build());
    }

    public String getObjectUrl(String bucketName, String objectName)
            throws IOException, InvalidKeyException, InvalidResponseException, InsufficientDataException,
            NoSuchAlgorithmException, ServerException, InternalException, XmlParserException, ErrorResponseException {
        return instance.getPresignedObjectUrl(
                        GetPresignedObjectUrlArgs.builder()
                            .method(Method.GET)
                            .bucket(bucketName)
                            .object(objectName)
//                            .expiry(2, TimeUnit.HOURS)
                            .build());
    }
}

3.2 測(cè)試

對(duì)1.2 文件上傳的Controller實(shí)現(xiàn)進(jìn)行改造,增加minio上傳的代碼:

@RestController
public class FileUploadController {

    @Resource
    private MinIOTemplate minIOTemplate;

    @Value("${web.upload-path}")
    private String uploadPath;

    @PostMapping("/upload")
    public String upload(MultipartFile uploadFile, HttpServletRequest request) throws IOException {

        // 在 uploadPath 文件夾中通過(guò)日期對(duì)上傳的文件歸類保存
        // 比如:/2019/06/06/cf13891e-4b95-4000-81eb-b6d70ae44930.png
        SimpleDateFormat sdf = new SimpleDateFormat("yyyy/MM/dd/");
        String format = sdf.format(new Date());
        File folder = new File(uploadPath + format);
        if (!folder.isDirectory()) {
          folder.mkdirs();
        }

        // 對(duì)上傳的文件重命名,避免文件重名
        String oldName = uploadFile.getOriginalFilename();
        String newName = UUID.randomUUID().toString() + oldName.substring(oldName.lastIndexOf("."));

        // 文件保存到本地
        File destFile = new File(folder, newName);
        uploadFile.transferTo(destFile);

        //保存文件到minIO
        uploadToMinIO("test", format + newName, destFile.getAbsolutePath());

        // 返回上傳文件的訪問(wèn)路徑
        //https://localhost:8888/2020/10/18/a9a05df4-6615-4bb5-b859-a3f9bf4bfae0.jpg
        String filePath = request.getScheme() + "://" + request.getServerName()
                + ":" + request.getServerPort() + "/"   + format + newName;
        return filePath;
    }

    private void uploadToMinIO(String bucketName, String objectName, String filePath) {
        try {
            minIOTemplate.makeBucket(bucketName);
            ObjectWriteResponse response = minIOTemplate.putObject(bucketName, objectName, filePath);
            System.out.println(response.object());

            //獲取文件訪問(wèn)路徑
            String url = minIOTemplate.getObjectUrl("test", objectName);
            System.out.println(url);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

注意:MinIO默認(rèn)獲取到的文件范圍URL有效期為7天,可以通過(guò)在控制臺(tái)bucket里面進(jìn)行規(guī)制配置,實(shí)現(xiàn)永久訪問(wèn)。

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡(jiǎn)書系信息發(fā)布平臺(tái),僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容