java如何實(shí)現(xiàn)大文件加速下載

針對(duì)大文件下載,Java可以使用多線程實(shí)現(xiàn)加速下載,并且結(jié)合斷點(diǎn)續(xù)傳功能,可以提高下載的穩(wěn)定性和效率。具體實(shí)現(xiàn)步驟如下:

獲取文件總長(zhǎng)度和已下載長(zhǎng)度,如果已下載長(zhǎng)度等于文件總長(zhǎng)度,則說(shuō)明文件已下載完成,不需要進(jìn)行下載。

如果已下載長(zhǎng)度小于文件總長(zhǎng)度,則需要進(jìn)行多線程下載。根據(jù)文件總長(zhǎng)度和線程數(shù)計(jì)算出每個(gè)線程需要下載的文件塊大小,然后每個(gè)線程分別下載對(duì)應(yīng)的文件塊。

在HTTP請(qǐng)求中添加Range頭信息,指定下載的起始位置和結(jié)束位置,例如Range:bytes=0-499表示下載文件的前500個(gè)字節(jié)。每個(gè)線程下載時(shí)需要指定不同的Range范圍。

下載過(guò)程中,記錄已下載的字節(jié)數(shù)和文件塊的下載狀態(tài),并在下載中斷時(shí)保存下載狀態(tài),以便下次繼續(xù)下載。

下載完成后,將所有文件塊合并成完整的文件。

下面是一個(gè)簡(jiǎn)單的Java實(shí)現(xiàn)多線程下載的示例代碼:


import java.io.*;
import java.net.*;
import java.util.concurrent.*;

public class DownloadManager {
    private static final int THREAD_POOL_SIZE = 5;
    private final URL url;
    private final String savePath;
    private final int bufferSize;
    private final int threadCount;
    private final ExecutorService executorService;

    public DownloadManager(String url, String savePath, int bufferSize, int threadCount) throws MalformedURLException {
        this.url = new URL(url);
        this.savePath = savePath;
        this.bufferSize = bufferSize;
        this.threadCount = threadCount;
        this.executorService = Executors.newFixedThreadPool(threadCount);
    }

    public void download() throws IOException, InterruptedException {
        HttpURLConnection conn = (HttpURLConnection) url.openConnection();
        int totalSize = conn.getContentLength();
        int blockSize = totalSize / threadCount;
        int downloadedSize = 0;
        boolean isCompleted = true;

        RandomAccessFile out = new RandomAccessFile(savePath, "rw");
        out.setLength(totalSize);
        out.close();

        for (int i = 0; i < threadCount; i++) {
            int start = i * blockSize;
            int end = (i == threadCount - 1) ? totalSize - 1 : (i + 1) * blockSize - 1;
            DownloadTask task = new DownloadTask(url, savePath, start, end, bufferSize);
            executorService.execute(task);
        }

        executorService.shutdown();
        executorService.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);

        for (int i = 0; i < threadCount; i++) {
            int start = i * blockSize;
            int end = (i == threadCount - 1) ? totalSize - 1 : (i + 1) * blockSize - 1;
            File file = new File(savePath + "." + i);
            if (!file.exists() || file.length() != end - start + 1) {
                isCompleted = false;
                break;
            }
        }

        if (isCompleted) {
            FileOutputStream outStream = new FileOutputStream(savePath);
            for (int i = 0; i < threadCount; i++) {
                FileInputStream inStream = new FileInputStream(savePath + "." + i);
                byte[] buffer = new byte[bufferSize];
                int len;
                while ((len = inStream.read(buffer)) != -1) {
                    outStream.write(buffer, 0, len);
                }
                inStream.close();
                File file = new File(savePath + "." + i);
                file.delete();
            }
            outStream.close();
        }
    }

    private class DownloadTask implements Runnable {
        private final URL url;
        private final String savePath;
        private final int start;
        private final int end;
        private final int bufferSize;

        public DownloadTask(URL url, String savePath, int start, int end, int bufferSize) {
            this.url = url;
            this.savePath = savePath;
            this.start = start;
            this.end = end;
            this.bufferSize = bufferSize;
        }

        @Override
        public void run() {
            try {
                HttpURLConnection conn = (HttpURLConnection) url.openConnection();
                conn.setRequestMethod("GET");
                conn.setRequestProperty("Range", "bytes=" + start + "-" + end);

                InputStream in = conn.getInputStream();
                RandomAccessFile out = new RandomAccessFile(savePath + "." + Thread.currentThread().getId(), "rw");
                out.seek(start);

                byte[] buffer = new byte[bufferSize];
                int len;
                while ((len = in.read(buffer)) != -1) {
                    out.write(buffer, 0, len);
                }

                in.close();
                out.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
}

在上面的代碼中,我們使用了ExecutorService線程池來(lái)管理多個(gè)下載線程。每個(gè)線程下載對(duì)應(yīng)的文件塊,并將下載狀態(tài)保存在單獨(dú)的文件中。在所有線程下載完成后,我們將所有文件塊合并成完整的文件。

在使用時(shí),可以創(chuàng)建DownloadManager對(duì)象,并調(diào)用download()方法啟動(dòng)下載。例如:


DownloadManager manager = new DownloadManager("http://example.com/largefile.zip", "C:/Downloads/largefile.zip", 1024, 5);
manager.download();

其中,第一個(gè)參數(shù)是要下載的文件URL,第二個(gè)參數(shù)是保存路徑,第三個(gè)參數(shù)是緩沖區(qū)大小,第四個(gè)參數(shù)是線程數(shù)。



其他方法客供參考:


public void downLoadFile(HttpServletRequest request, HttpServletResponse response) throws IOException {
        // 設(shè)置編碼格式
        response.setCharacterEncoding(UTF_8);
        //獲取文件路徑
        String fileName = request.getParameter("fileName");
        String path = OUTPUT_PATH;
        //參數(shù)校驗(yàn)
        log.info(fileName, path);
        //完整路徑(路徑拼接待優(yōu)化-前端傳輸優(yōu)化-后端從新格式化  )
        String pathAll = path + File.separator + fileName;
        log.info("pathAll{}", pathAll);
        Optional<String> pathFlag = Optional.ofNullable(pathAll);
        File file = null;
        if (pathFlag.isPresent()) {
            //根據(jù)文件名,讀取file流
            file = new File(pathAll);
            log.info("文件路徑是{}", pathAll);
            if (!file.exists()) {
                log.warn("文件不存在");
                return;
            }
        } else {
            //請(qǐng)輸入文件名
            log.warn("請(qǐng)輸入文件名!");
            return;
        }
        try (InputStream is = new BufferedInputStream(new FileInputStream(file))) {
            //分片下載
            long fSize = file.length();//獲取長(zhǎng)度
            response.setContentType("application/x-download");
            String file_Name = URLEncoder.encode(file.getName(), "UTF-8");
            //獲取下載文件名
            String newFileName = URLEncoder.encode(request.getParameter("sourceDept"), "UTF-8") + ".zip";
            response.addHeader("Content-Disposition", "attachment;filename=" + (StringUtils.isNotEmpty(newFileName) ? newFileName : fileName));
            //根據(jù)前端傳來(lái)的Range  判斷支不支持分片下載
            response.setHeader("Accept-Ranges", "bytes");
            response.setHeader("fName", file_Name);
            //定義斷點(diǎn)
            long pos = 0, last = fSize - 1, sum = 0;
            //判斷前端需不需要分片下載
            if (null != request.getHeader("Range")) {
                response.setStatus(HttpServletResponse.SC_PARTIAL_CONTENT);
                String numRange = request.getHeader("Range").replaceAll("bytes=", "");
                String[] strRange = numRange.split("-");
                if (strRange.length == 2) {
                    pos = Long.parseLong(strRange[0].trim());
                    last = Long.parseLong(strRange[1].trim());
                    //若結(jié)束字節(jié)超出文件大小 取文件大小
                    if (last > fSize - 1) {
                        last = fSize - 1;
                    }
                } else {
                    //若只給一個(gè)長(zhǎng)度  開(kāi)始位置一直到結(jié)束
                    pos = Long.parseLong(numRange.replaceAll("-", "").trim());
                }
            }
            long rangeLength = last - pos + 1;
            String contentRange = new StringBuffer("bytes").append(pos).append("-").append(last).append("/").append(fSize).toString();
            response.setHeader("Content-Range", contentRange);
            response.setHeader("Content-Length", String.valueOf(rangeLength));
            response.setHeader("Connection", "keep-alive");
            response.setHeader("Keep-Alive", "timeout=300, max=100");

            is.skip(pos);//跳過(guò)已讀的文件(重點(diǎn),跳過(guò)之前已經(jīng)讀過(guò)的文件)
            byte[] buffer = new byte[8192];  // 8KB buffer
            int length = 0;
            try (OutputStream os = new BufferedOutputStream(response.getOutputStream())) {
                while (sum < rangeLength) {
                    length = is.read(buffer, 0, (rangeLength - sum) <= buffer.length ? (int) (rangeLength - sum) : buffer.length);
                    sum = sum + length;
                    os.write(buffer, 0, length);
                }
            } // 這里的OutputStream將在這個(gè)塊結(jié)束時(shí)關(guān)閉
            log.info("下載完成");
        } catch (IOException ex) {
            // handle exception here
            log.error("下載失敗", ex);
        }
    }

最后編輯于
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡(jiǎn)書系信息發(fā)布平臺(tái),僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容