Skip to content

Instantly share code, notes, and snippets.

Last active June 3, 2023 05:29
  • Star 58 You must be signed in to star a gist
  • Fork 8 You must be signed in to fork a gist
Star You must be signed in to star a gist
What would you like to do?
Download large files in rust with progress bar using reqwest, future_util and indicatif
// you need this in your cargo.toml
// reqwest = { version = "0.11.3", features = ["stream"] }
// futures-util = "0.3.14"
// indicatif = "0.15.0"
use std::cmp::min;
use std::fs::File;
use std::io::Write;
use reqwest::Client;
use indicatif::{ProgressBar, ProgressStyle};
use futures_util::StreamExt;
pub async fn download_file(client: &Client, url: &str, path: &str) -> Result<(), String> {
// Reqwest setup
let res = client
.or(Err(format!("Failed to GET from '{}'", &url)))?;
let total_size = res
.ok_or(format!("Failed to get content length from '{}'", &url))?;
// Indicatif setup
let pb = ProgressBar::new(total_size);
.template("{msg}\n{} [{elapsed_precise}] [{wide_bar:.cyan/blue}] {bytes}/{total_bytes} ({bytes_per_sec}, {eta})")
pb.set_message(&format!("Downloading {}", url));
// download chunks
let mut file = File::create(path).or(Err(format!("Failed to create file '{}'", path)))?;
let mut downloaded: u64 = 0;
let mut stream = res.bytes_stream();
while let Some(item) = {
let chunk = item.or(Err(format!("Error while downloading file")))?;
.or(Err(format!("Error while writing to file")))?;
let new = min(downloaded + (chunk.len() as u64), total_size);
downloaded = new;
pb.finish_with_message(&format!("Downloaded {} to {}", url, path));
return Ok(());
Copy link

thanks for this example, do you know if the progressbar is possible with POST request ? like

Copy link

@ozkanpakdil I'm pretty sure that if you change line 16 from .get to .post it will work.

Copy link

can you please provide a full working example?
Not sure how to call this async function from a non-async main..

Copy link

For future reference, I've pushed a full example here.

Copy link

@giuliano-oliveira I could not find a way to create progress bar for upload :) feel free to help

Copy link

@ozkanpakdil : have a look at the implementation of aim for how upload with a progress bar is handled.
I'm adding more features in the coming weeks/months.

Copy link

firasuke commented Jan 2, 2022

@giuliano-oliveira @mihaigalos can this be used to download multiple files concurrently?

Copy link

orhun commented Feb 4, 2022

You might want to consider using write_all instead of write:

Copy link

Yes, I agree. It is important to use write_all instead of write, otherwise it won't work

Copy link

Makes perfect sense, thanks @orhun @lennartkloock

Copy link

Tails commented Jun 23, 2022

Does .get().send().await not immediately retrieve the whole response body to memory?

Copy link

elimerl commented Sep 19, 2022

@Tails that just sends initial http headers and waits for the server http headers to come back, but doesn't block on the response (this is why e.g. .json on responses needs await)

Copy link

Boscop commented Dec 16, 2022

@mihaigalos Thanks for your example with resuming if the file exists. But don't you have to seek also in the response stream the same amount (file_size)? Or can you please clarify by which mechanism the response stream will be forwarded to the right byte index, because I don't see anything that advances the stream:

Copy link

mihaigalos commented Dec 17, 2022

Hi @Boscop,
You're right, you do need to forward the byte index.

That needs to happen even before the download, however, because the backend needs to know from which byte it needs to start the stream.

Long story short, see this.

Copy link

Hi @giuliano-oliveira, thank you for this solution. But May I suggest improving it? Before appending an existing file, we need to check its file size compared with the response file size. If we don't check, it will append every time when the file exists then the file size becomes larger and larger. Hope you understand my idea! Thanks.

Copy link

@lehaidangdev I like the ideia (and I miss pull requests in gists :( ), could you fork, modify it and make another comment with the fork?
Then i can comment on your gist and we can "merge" it, what do you think? I could create a repo just for this example as well, and replace this gist with a link to the repo.

If we don't check, it will append every time when the file exists then the file size becomes larger and larger. Hope you understand my idea! Thanks.

Though I'm might be misunderstanding what you are saying, but if the file already exists in line 32 it would truncate the original file (which is bad as well in my opinion, maybe we should check if the file already exists, and if does, return an error)

Copy link

I found this discussion by chance because I too wanted to download files easily using Tauri and Tauri Upload Plugin as now it also has a download function with a ProgressHandler which can download a file from a URL to a Disk

Just in Case anyone stumbles here with the same problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment