Skip to content

Instantly share code, notes, and snippets.

@aialenti
Created September 17, 2020 22:09
Show Gist options
  • Save aialenti/83df27ef49ab7660c416f510daa998f5 to your computer and use it in GitHub Desktop.
Save aialenti/83df27ef49ab7660c416f510daa998f5 to your computer and use it in GitHub Desktop.
# Read the source tables in Parquet format
sales_table = spark.read.parquet("./data/sales_parquet")
'''
SELECT *
WHERE bill_raw_text LIKE 'ab%cd%'
'''
sales_table_execution_plan = sales_table.where(
col('bill_raw_text').like("ab%cd%")
)
sales_table_execution_plan.show()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment