Skip to content

Instantly share code, notes, and snippets.

@johntmyers
Last active June 15, 2020 15:15
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save johntmyers/a6a4a077c4671c00f46243e02aa041bb to your computer and use it in GitHub Desktop.
Save johntmyers/a6a4a077c4671c00f46243e02aa041bb to your computer and use it in GitHub Desktop.
gretel-synthetics-0.10.0-example-1
# pip install gretel-synthetics
from gretel_synthetics.batch import DataFrameBatch
# my_source_df = ...
# Same configurations parameters as using LocalConfig
# https://gretel-synthetics.readthedocs.io/en/stable/api/config.html
config_template = {
"max_lines": 0,
"max_line_len": 2048,
"epochs": 15,
"vocab_size": 20000,
# "gen_lines": 100,
"dp": True,
"field_delimiter": ",", # Must provide!
"overwrite": True,
"checkpoint_dir": str(Path.cwd() / "checkpoints")
}
batcher = DataFrameBatch(df=my_source_df, config=config_template)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment