Skip to content

Instantly share code, notes, and snippets.

Created June 16, 2019 10:04
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
What would you like to do?
Load data in parallel
private static async Task StreamDataInParallel(StreamReader fileSource, HttpClient httpClient, string path, int maxParallel)
var block = new ActionBlock<string>(
async json =>
await httpClient.PostAsync(path, new StringContent(json, Encoding.UTF8, "application/json"));
}, new ExecutionDataflowBlockOptions
// Tells the action block how many we want to run at once.
MaxDegreeOfParallelism = maxParallel,
// 'Buffer' the same number of lines as there are parallel requests.
BoundedCapacity = maxParallel
string line;
while ((line = await fileSource.ReadLineAsync()) != null)
// This will not continue until there is space in the buffer.
await block.SendAsync(GetDataJson(line));
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment