Skip to content

Instantly share code, notes, and snippets.

@2shou
2shou / spark_log_extract.py
Last active August 8, 2019 13:30
log-extract script with Spark
# coding: utf-8
# command example:
# $ spark-submit spark_log_extract.py \
# --name test \
# --notblankkeys dn,stm,ev_ac,pg_url \
# --filterregex ".*(=ac_pl\`|=ac_dl\`).*" \
# --usegzip \
# /path/to/source \
# /path/to/atom \