Multiple small chunks of a large file are downloaded here. this program explores the concurrency, file input output operations and how it transferred.
Let us explore the steps.
Steps:
- First, initialize the URL.
- The content length is retrieved.
- Split the download task.
- By the use of Fixed Thread Pool, the parallel downloads are done.
- Finally, the chunks are written.
Built-in Packages used for this process:
ExecutorService ,HttpURLConnection and RandomAccessFile.
Program implementation:
import java.io.*;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class MThreadedDownloader {
private static
final int T_COUNT = 3;
private final
String fURL;
private final
String opFile;
public
MThreadedDownloader(String fURL, String opFile) {
this.fURL =
fURL;
this.opFile =
opFile;
}
public void
downloadIt() throws Exception {
URL url1 = new
URL(fURL);
HttpURLConnection con1 = (HttpURLConnection) url1.openConnection();
int cLength =
con1.getContentLength();
con1.disconnect();
int chunk_Size
= cLength / T_COUNT;
ExecutorService exe1 = Executors.newFixedThreadPool(T_COUNT);
for (int i =
0; i < T_COUNT; i++) {
int
start_Byte = i * chunk_Size;
int
end_Byte = (i == T_COUNT - 1) ? cLength - 1 : (start_Byte + chunk_Size - 1);
exe1.execute(new DThread(fURL, opFile, start_Byte, end_Byte, i));
}
exe1.shutdown();
while
(!exe1.isTerminated()) {
Thread.sleep(100);
}
System.out.println("Download complete.");
}
private static
class DThread implements Runnable {
private final
String fURL;
private final
String opFile;
private final
int start_Byte;
private final
int end_Byte;
private final
int thread_Id;
public
DThread(String fURL, String opFile, int start_Byte, int end_Byte, int
thread_Id) {
this.fURL
= fURL;
this.opFile = opFile;
this.start_Byte = start_Byte;
this.end_Byte = end_Byte;
this.thread_Id = thread_Id;
}
@Override
public void
run() {
try {
HttpURLConnection con1 = (HttpURLConnection) new
URL(fURL).openConnection();
String
bRange = "bytes=" + start_Byte + "-" + end_Byte;
con1.setRequestProperty("Range", bRange);
con1.connect();
try
(InputStream in1 = con1.getInputStream();
RandomAccessFile raf1 = new RandomAccessFile(opFile, "rw")) {
raf1.seek(start_Byte);
byte[] buffer1 = new byte[4096];
int bytes_Read;
while ((bytes_Read = in1.read(buffer1)) != -1) {
raf1.write(buffer1, 0, bytes_Read);
}
}
System.out.printf("Thread is %d completed downloading bytes %d to
%d%n", thread_Id, start_Byte, end_Byte);
} catch
(IOException e) {
System.err.printf("Thread is %d encountered an error: %s%n",
thread_Id, e.getMessage());
}
}
}
public static void
main(String[] args) throws Exception {
String url1 =
"C:\\raji\\blog\\keywords.zip";
String output1
= "largefile.zip";
new
MThreadedDownloader(url1, output1).downloadIt();
}
}
This is the sample implementation of Multithreaded File
Downloader. Hope, this code is useful to you.