Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Iteratively copy files in Elixir
# We use streams (open the file convert the device to a stream, using 4MB chunks)
# as a demo, using this method we copied a 2.5GB file with no significant memory usage on
# an Odroid with only 2GB of RAM. Not super fast... Do line oriented by replacing the byte
# count in IO.binstream(:line), very very slow. Process line by line or chunk by chunk
# by passing a function to Stream.into(out, fn(line) -> do_someting(line) end).
fin = File.open!("file.in", [:read, :read_ahead]) |> IO.binstream(4194304)
fout = File.open!("file.out", [:write] |> IO.binstream(4194304)
fin |> Stream.into(fout) |> Stream.run
# to close the files
fin.device |> File.close()
fout.device |> File.close()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.