Skip to content

Instantly share code, notes, and snippets.

@RachidAZ
Created January 16, 2022 18:09
Show Gist options
  • Save RachidAZ/d1b57a078769d825e56c03454e8b8d3e to your computer and use it in GitHub Desktop.
Save RachidAZ/d1b57a078769d825e56c03454e8b8d3e to your computer and use it in GitHub Desktop.
read/parse PARQUET file in C# using ChoParquetReader library
using ChoETL;
if (fileType.ToUpper().Equals("PARQUET"))
{
memoryStream.Seek(0, SeekOrigin.Begin);
int recordsRead = 0;
// loop through the rows of the file
foreach (dynamic e in new ChoParquetReader(memoryStream))
{
var dict = (IDictionary<string, object>)e;
nbrColumnsReturned = dict.Count;
if (dict["id"] != null)
dict["id"] = dict["id"].ToString();
else continue;
//Console.WriteLine(dict["id"] + "--" + dict["id"].ToString());
string json = JsonConvert.SerializeObject(dict);
recordsRead++;
fileContentJsonSplit.Add(json); ;
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment