針對(duì)大文件下載,Java可以使用多線程實(shí)現(xiàn)加速下載,并且結(jié)合斷點(diǎn)續(xù)傳功能,可以提高下載的穩(wěn)定性和效率。具體實(shí)現(xiàn)步驟如下:
獲取文件總長(zhǎng)度和已下載長(zhǎng)度,如果已下載長(zhǎng)度等于文件總長(zhǎng)度,則說(shuō)明文件已下載完成,不需要進(jìn)行下載。
如果已下載長(zhǎng)度小于文件總長(zhǎng)度,則需要進(jìn)行多線程下載。根據(jù)文件總長(zhǎng)度和線程數(shù)計(jì)算出每個(gè)線程需要下載的文件塊大小,然后每個(gè)線程分別下載對(duì)應(yīng)的文件塊。
在HTTP請(qǐng)求中添加Range頭信息,指定下載的起始位置和結(jié)束位置,例如Range:bytes=0-499表示下載文件的前500個(gè)字節(jié)。每個(gè)線程下載時(shí)需要指定不同的Range范圍。
下載過(guò)程中,記錄已下載的字節(jié)數(shù)和文件塊的下載狀態(tài),并在下載中斷時(shí)保存下載狀態(tài),以便下次繼續(xù)下載。
下載完成后,將所有文件塊合并成完整的文件。
下面是一個(gè)簡(jiǎn)單的Java實(shí)現(xiàn)多線程下載的示例代碼:
import java.io.*;
import java.net.*;
import java.util.concurrent.*;
public class DownloadManager {
private static final int THREAD_POOL_SIZE = 5;
private final URL url;
private final String savePath;
private final int bufferSize;
private final int threadCount;
private final ExecutorService executorService;
public DownloadManager(String url, String savePath, int bufferSize, int threadCount) throws MalformedURLException {
this.url = new URL(url);
this.savePath = savePath;
this.bufferSize = bufferSize;
this.threadCount = threadCount;
this.executorService = Executors.newFixedThreadPool(threadCount);
}
public void download() throws IOException, InterruptedException {
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
int totalSize = conn.getContentLength();
int blockSize = totalSize / threadCount;
int downloadedSize = 0;
boolean isCompleted = true;
RandomAccessFile out = new RandomAccessFile(savePath, "rw");
out.setLength(totalSize);
out.close();
for (int i = 0; i < threadCount; i++) {
int start = i * blockSize;
int end = (i == threadCount - 1) ? totalSize - 1 : (i + 1) * blockSize - 1;
DownloadTask task = new DownloadTask(url, savePath, start, end, bufferSize);
executorService.execute(task);
}
executorService.shutdown();
executorService.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
for (int i = 0; i < threadCount; i++) {
int start = i * blockSize;
int end = (i == threadCount - 1) ? totalSize - 1 : (i + 1) * blockSize - 1;
File file = new File(savePath + "." + i);
if (!file.exists() || file.length() != end - start + 1) {
isCompleted = false;
break;
}
}
if (isCompleted) {
FileOutputStream outStream = new FileOutputStream(savePath);
for (int i = 0; i < threadCount; i++) {
FileInputStream inStream = new FileInputStream(savePath + "." + i);
byte[] buffer = new byte[bufferSize];
int len;
while ((len = inStream.read(buffer)) != -1) {
outStream.write(buffer, 0, len);
}
inStream.close();
File file = new File(savePath + "." + i);
file.delete();
}
outStream.close();
}
}
private class DownloadTask implements Runnable {
private final URL url;
private final String savePath;
private final int start;
private final int end;
private final int bufferSize;
public DownloadTask(URL url, String savePath, int start, int end, int bufferSize) {
this.url = url;
this.savePath = savePath;
this.start = start;
this.end = end;
this.bufferSize = bufferSize;
}
@Override
public void run() {
try {
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Range", "bytes=" + start + "-" + end);
InputStream in = conn.getInputStream();
RandomAccessFile out = new RandomAccessFile(savePath + "." + Thread.currentThread().getId(), "rw");
out.seek(start);
byte[] buffer = new byte[bufferSize];
int len;
while ((len = in.read(buffer)) != -1) {
out.write(buffer, 0, len);
}
in.close();
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
在上面的代碼中,我們使用了ExecutorService線程池來(lái)管理多個(gè)下載線程。每個(gè)線程下載對(duì)應(yīng)的文件塊,并將下載狀態(tài)保存在單獨(dú)的文件中。在所有線程下載完成后,我們將所有文件塊合并成完整的文件。
在使用時(shí),可以創(chuàng)建DownloadManager對(duì)象,并調(diào)用download()方法啟動(dòng)下載。例如:
DownloadManager manager = new DownloadManager("http://example.com/largefile.zip", "C:/Downloads/largefile.zip", 1024, 5);
manager.download();
其中,第一個(gè)參數(shù)是要下載的文件URL,第二個(gè)參數(shù)是保存路徑,第三個(gè)參數(shù)是緩沖區(qū)大小,第四個(gè)參數(shù)是線程數(shù)。
其他方法客供參考:
public void downLoadFile(HttpServletRequest request, HttpServletResponse response) throws IOException {
// 設(shè)置編碼格式
response.setCharacterEncoding(UTF_8);
//獲取文件路徑
String fileName = request.getParameter("fileName");
String path = OUTPUT_PATH;
//參數(shù)校驗(yàn)
log.info(fileName, path);
//完整路徑(路徑拼接待優(yōu)化-前端傳輸優(yōu)化-后端從新格式化 )
String pathAll = path + File.separator + fileName;
log.info("pathAll{}", pathAll);
Optional<String> pathFlag = Optional.ofNullable(pathAll);
File file = null;
if (pathFlag.isPresent()) {
//根據(jù)文件名,讀取file流
file = new File(pathAll);
log.info("文件路徑是{}", pathAll);
if (!file.exists()) {
log.warn("文件不存在");
return;
}
} else {
//請(qǐng)輸入文件名
log.warn("請(qǐng)輸入文件名!");
return;
}
try (InputStream is = new BufferedInputStream(new FileInputStream(file))) {
//分片下載
long fSize = file.length();//獲取長(zhǎng)度
response.setContentType("application/x-download");
String file_Name = URLEncoder.encode(file.getName(), "UTF-8");
//獲取下載文件名
String newFileName = URLEncoder.encode(request.getParameter("sourceDept"), "UTF-8") + ".zip";
response.addHeader("Content-Disposition", "attachment;filename=" + (StringUtils.isNotEmpty(newFileName) ? newFileName : fileName));
//根據(jù)前端傳來(lái)的Range 判斷支不支持分片下載
response.setHeader("Accept-Ranges", "bytes");
response.setHeader("fName", file_Name);
//定義斷點(diǎn)
long pos = 0, last = fSize - 1, sum = 0;
//判斷前端需不需要分片下載
if (null != request.getHeader("Range")) {
response.setStatus(HttpServletResponse.SC_PARTIAL_CONTENT);
String numRange = request.getHeader("Range").replaceAll("bytes=", "");
String[] strRange = numRange.split("-");
if (strRange.length == 2) {
pos = Long.parseLong(strRange[0].trim());
last = Long.parseLong(strRange[1].trim());
//若結(jié)束字節(jié)超出文件大小 取文件大小
if (last > fSize - 1) {
last = fSize - 1;
}
} else {
//若只給一個(gè)長(zhǎng)度 開(kāi)始位置一直到結(jié)束
pos = Long.parseLong(numRange.replaceAll("-", "").trim());
}
}
long rangeLength = last - pos + 1;
String contentRange = new StringBuffer("bytes").append(pos).append("-").append(last).append("/").append(fSize).toString();
response.setHeader("Content-Range", contentRange);
response.setHeader("Content-Length", String.valueOf(rangeLength));
response.setHeader("Connection", "keep-alive");
response.setHeader("Keep-Alive", "timeout=300, max=100");
is.skip(pos);//跳過(guò)已讀的文件(重點(diǎn),跳過(guò)之前已經(jīng)讀過(guò)的文件)
byte[] buffer = new byte[8192]; // 8KB buffer
int length = 0;
try (OutputStream os = new BufferedOutputStream(response.getOutputStream())) {
while (sum < rangeLength) {
length = is.read(buffer, 0, (rangeLength - sum) <= buffer.length ? (int) (rangeLength - sum) : buffer.length);
sum = sum + length;
os.write(buffer, 0, length);
}
} // 這里的OutputStream將在這個(gè)塊結(jié)束時(shí)關(guān)閉
log.info("下載完成");
} catch (IOException ex) {
// handle exception here
log.error("下載失敗", ex);
}
}