Skip to content

Instantly share code, notes, and snippets.

@henrik
Last active December 4, 2020 05:41
Show Gist options
  • Star 13 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save henrik/108e5fc23b66131fc3aa to your computer and use it in GitHub Desktop.
Save henrik/108e5fc23b66131fc3aa to your computer and use it in GitHub Desktop.
Example of progressive downloading with HTTPotion, showing percentage and downloaded bytes as you go.
defmodule ProgressiveDownloader do
def run do
url = "https://ia600308.us.archive.org/2/items/HealthYo1953/HealthYo1953_512kb.mp4"
HTTPotion.get url, stream_to: self, timeout: :infinity
receive_data(total_bytes: :unknown, data: "")
end
defp receive_data(total_bytes: total_bytes, data: data) do
receive do
%HTTPotion.AsyncHeaders{headers: h} ->
{total_bytes, _} = h[:"Content-Length"] |> Integer.parse
IO.puts "Let's download #{mb total_bytes}…"
receive_data(total_bytes: total_bytes, data: data)
%HTTPotion.AsyncChunk{chunk: new_data} ->
accumulated_data = data <> new_data
accumulated_bytes = byte_size(accumulated_data)
percent = accumulated_bytes / total_bytes * 100 |> Float.round(2)
IO.puts "#{percent}% (#{mb accumulated_bytes})"
receive_data(total_bytes: total_bytes, data: accumulated_data)
%HTTPotion.AsyncEnd{} ->
path = "/tmp/posture.mp4"
File.write!(path, data)
IO.puts "All downloaded! See: #{path}"
end
end
defp mb(bytes) do
number = bytes / 1_048_576 |> Float.round(2)
"#{number} MB"
end
end
iex -S mix  # In a project with HTTPotion in mix.exs.
iex(1)> ProgressiveDownloader.run
Let's download 43.2 MB…
2.31% (1.0 MB)
4.63% (2.0 MB)
6.94% (3.0 MB)
9.26% (4.0 MB)
11.57% (5.0 MB)
13.89% (6.0 MB)
16.2% (7.0 MB)
18.52% (8.0 MB)
20.83% (9.0 MB)
23.15% (10.0 MB)
25.46% (11.0 MB)
27.78% (12.0 MB)
30.09% (13.0 MB)
32.4% (14.0 MB)
34.72% (15.0 MB)
37.03% (16.0 MB)
39.35% (17.0 MB)
41.66% (18.0 MB)
43.98% (19.0 MB)
46.29% (20.0 MB)
48.61% (21.0 MB)
50.92% (22.0 MB)
53.24% (23.0 MB)
55.55% (24.0 MB)
57.87% (25.0 MB)
60.18% (26.0 MB)
62.49% (27.0 MB)
64.81% (28.0 MB)
67.12% (29.0 MB)
69.44% (30.0 MB)
71.75% (31.0 MB)
74.07% (32.0 MB)
76.38% (33.0 MB)
78.7% (34.0 MB)
81.01% (35.0 MB)
83.33% (36.0 MB)
85.64% (37.0 MB)
87.96% (38.0 MB)
90.27% (39.0 MB)
92.58% (40.0 MB)
94.9% (41.0 MB)
97.21% (42.0 MB)
99.53% (43.0 MB)
100.0% (43.2 MB)
All downloaded! See: /tmp/posture.mp4
@henrik
Copy link
Author

henrik commented Sep 18, 2015

Combine with https://gist.github.com/henrik/b950bfd7fa432eacd902 for a progress bar.

See it in action in https://github.com/henrik/sipper.

Copy link

ghost commented Oct 28, 2015

I tried to use this snippet, but apparently there's something wrong with HTTPotion at some point:

iex(1)⋅❯ ProgressiveDownloader.run
** (Protocol.UndefinedError) protocol List.Chars not implemented for #PID<0.139.0>
    (elixir) lib/list/chars.ex:1: List.Chars.impl_for!/1
    (elixir) lib/list/chars.ex:12: List.Chars.to_char_list/1
             lib/httpotion.ex:157: anonymous fn/1 in HTTPotion.request/5
    (elixir) lib/enum.ex:1043: anonymous fn/3 in Enum.map/2
    (elixir) lib/enum.ex:1385: Enum."-reduce/3-lists^foldl/2-0-"/3
    (elixir) lib/enum.ex:1043: Enum.map/2
             lib/httpotion.ex:157: HTTPotion.request/5
    (sploit) lib/1-progressive_downloader.ex:5: ProgressiveDownloader.run/0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment