Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save piercelamb/8647d5506b83eb4f8c74504ddcdf8907 to your computer and use it in GitHub Desktop.
Save piercelamb/8647d5506b83eb4f8c74504ddcdf8907 to your computer and use it in GitHub Desktop.
snsc.sql("create table adImpressions(times_tamp timestamp, publisher string, " +
"advertiser string, website string, geo string, bid double, cookie string) " +
"using column options ( buckets '29', persistent 'asynchronous')")
snsc.sql("CREATE SAMPLE TABLE sampledAdImpressions" +
" OPTIONS(qcs 'geo,publisher', fraction '0.02', strataReservoirSize '50', baseTable 'adImpressions')")
snsc.getSchemaDStream("adImpressionStream").foreachDataFrame( df => {
df.write.insertInto("adImpressions")
df.write.insertInto("sampledAdImpressions")
})
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment