Skip to content

Instantly share code, notes, and snippets.

@niektemme

niektemme/smarttherm_part_score_1.scala Secret

Created July 31, 2015 09:54
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save niektemme/ab2cc33fac4328eabb85 to your computer and use it in GitHub Desktop.
Save niektemme/ab2cc33fac4328eabb85 to your computer and use it in GitHub Desktop.
Download ZIP
Smart Thermostat AWS Spark - part Spark score 1
Raw
smarttherm_part_score_1.scala
//scan hsensval table to retreive information to score scenario part 1: temperature information
//this scan collects the actual inside temperature sensor data
val valstartrow : String = "40b5af01_rx000A04_" + valmaxrevepoch.toString + "_9999999"
val valstoprow : String = "40b5af01_rx000A04_" + valrevepoch.toString + "_9999999"
val conf = HBaseConfiguration.create()
val scan = new Scan()
conf.set(TableInputFormat.INPUT_TABLE, "hsensvals")
scan.setStartRow(valstartrow.getBytes())
scan.setStopRow(valstoprow.getBytes())
conf.set(TableInputFormat.SCAN, convertScanToString(scan))
val hBaseRDD = sc.newAPIHadoopRDD(conf, classOf[TableInputFormat], classOf[org.apache.hadoop.hbase.io.ImmutableBytesWritable],classOf[org.apache.hadoop.hbase.client.Result])
val valRows = hBaseRDD.map(tuple => tuple._2).map(result => (Bytes.toString(result.getRow()).substring(18,28) , Bytes.toString(result.getValue("fd".getBytes(), "cd".getBytes())).toInt ) )
valRows.cache()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment