Skip to content

Instantly share code, notes, and snippets.

View jepma's full-sized avatar

Marcel Jepma jepma

  • Jispro
  • Amsterdam, the Netherlands
  • X @jepma
View GitHub Profile

Keybase proof

I hereby claim:

  • I am jepma on github.
  • I am jepmam (https://keybase.io/jepmam) on keybase.
  • I have a public key ASCDD0-p78lAcN8WkTaBzhm1rqJWjtKoqo3Ax-ESTrN7uAo

To claim this, I am signing this object:

@jepma
jepma / csv-to-pandas.py
Last active December 29, 2018 16:00
Pandas
import pandas as pd
# Read file
results = pd.read_csv("/tmp/data.csv", encoding='utf-8',sep=',', quoting=csv.QUOTE_ALL)
# Print rows
for row in results:
print(row)
# Iterate over rows
@jepma
jepma / read-file-replace-templates.py
Created June 14, 2017 08:50
Read a file and replace template variables
def read_query_from_file(query, *args):
"""Read query from queries/schema.table.sql"""
try:
with open(str('%s.sql' % (query)), 'r') as query_file:
sqlquery = query_file.read()
# Replace parameters
for argument in args:
argument_split = argument.split('=')
@jepma
jepma / parquet-pandas.py
Last active June 21, 2017 05:22
Read parquet-file and use data via Pandas
import numpy as np
import pandas as pd
import pyarrow as pa
import pyarrow.parquet as pq
# read parquet-file
table = pq.read_table("FILENAME_HERE")
table_pd = table.to_pandas()
# retrieving columns
@jepma
jepma / parquet-read-info.py
Last active June 14, 2017 08:44
PYTHON: Parquet example
import pyarrow as pa
import pyarrow.parquet as pq
print("DEMO TIME!")
parquet_file = pq.ParquetFile('example-01.parquet')
print(parquet_file.metadata)
print(parquet_file.schema)
print("END OF DEMO!")