Skip to content

Instantly share code, notes, and snippets.

@kai5263499
Created July 11, 2014 01:30
Show Gist options
  • Save kai5263499/766b4cbf9f2e621ecfa4 to your computer and use it in GitHub Desktop.
Save kai5263499/766b4cbf9f2e621ecfa4 to your computer and use it in GitHub Desktop.
produce a list of tuples which describes the chunks adjusted to the nearest newline
getChunkDivisions(File, 0, Chunksize, ChunkDivisions) ->
[{chunk,0,Chunksize}|ChunkDivisions];
getChunkDivisions(File, Size, Chunksize, ChunkDivisions) ->
if
Size-Chunksize=<0 ->
ComputedChunkEnd = Chunksize-(Chunksize-Size),
CorrectedChunkEnd = walkToNextLineBreak(File, ComputedChunkEnd),
getChunkDivisions(File, 0, CorrectedChunkEnd, ChunkDivisions);
true ->
ComputedChunkEnd = Size-Chunksize,
CorrectedChunkEnd = walkToNextLineBreak(File, ComputedChunkEnd),
getChunkDivisions(File, CorrectedChunkEnd, Chunksize, [{chunk,CorrectedChunkEnd,Size}|ChunkDivisions])
end.
walkToNextLineBreak(File,Start) ->
file:position(File, Start-1),
{ok, Data} = file:read(File, 1024),
Start+string:chr(Data, $\n).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment