Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Smart Thermostat AWS Spark - part Spark score 1
//scan hsensval table to retreive information to score scenario part 1: temperature information
//this scan collects the actual inside temperature sensor data
val valstartrow : String = "40b5af01_rx000A04_" + valmaxrevepoch.toString + "_9999999"
val valstoprow : String = "40b5af01_rx000A04_" + valrevepoch.toString + "_9999999"
val conf = HBaseConfiguration.create()
val scan = new Scan()
conf.set(TableInputFormat.INPUT_TABLE, "hsensvals")
scan.setStartRow(valstartrow.getBytes())
scan.setStopRow(valstoprow.getBytes())
conf.set(TableInputFormat.SCAN, convertScanToString(scan))
val hBaseRDD = sc.newAPIHadoopRDD(conf, classOf[TableInputFormat], classOf[org.apache.hadoop.hbase.io.ImmutableBytesWritable],classOf[org.apache.hadoop.hbase.client.Result])
val valRows = hBaseRDD.map(tuple => tuple._2).map(result => (Bytes.toString(result.getRow()).substring(18,28) , Bytes.toString(result.getValue("fd".getBytes(), "cd".getBytes())).toInt ) )
valRows.cache()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment