Skip to content

Instantly share code, notes, and snippets.

View bachwehbi's full-sized avatar

Bachar Wehbi bachwehbi

15 followers · 1 following
View GitHub Profile
@bachwehbi
bachwehbi / write_fast_2_s3.py
Last active August 29, 2021 15:07
Simple approach to accelerate writing to S3 from Spark.
import base64
import os
import time
"""
Writing from Spark to S3 is ridiculously slow. This is because S3 is an object
store and not a file system. Because of consistency model of S3, when writing
Parquet (or ORC) files from Spark. Data will be stored to a temporary destination
then renamed when the job is successful. As S3 is an object store, renaming files
is very expensive (complete rewrite). The Spark job will only terminate when all
@bachwehbi
bachwehbi / onoffSubscribe.py
Last active April 10, 2016 17:41
#!/usr/bin/python
# Copyright (c) 2013-2016 Beebotte <contact@beebotte.com>
# This program is published under the MIT License (http://opensource.org/licenses/MIT).
############################################################
# This code uses the Beebotte API, you must have an account.
# You can register here: http://beebotte.com/register
#############################################################
@bachwehbi
bachwehbi / subsriber.c
Last active June 18, 2021 11:54
Connect to Beebotte using Paho MQTT C client - with encryption
/**
* to compile this code
* cc -g -o subscriber subscriber.c -lpaho-mqtt3cs -lpthread
**/
#include "stdio.h"
#include "stdlib.h"
#include "string.h"
#include "MQTTClient.h"
// set protocol to ssl and port number to 8883 to use encryption
@bachwehbi
bachwehbi / build.sh
Last active January 20, 2023 23:43
Simple MQTT publish and subscribe using Paho embedded C/C++ Client
cp ../../src/MQTTClient.c .
sed -e 's/""/"MQTTLinux.h"/g' ../../src/MQTTClient.h > MQTTClient.h
gcc sub.c -I ../../src -I ../../src/linux -I ../../../MQTTPacket/src MQTTClient.c ../../src/linux/MQTTLinux.c ../../../MQTTPacket/src/MQTTFormat.c ../../../MQTTPacket/src/MQTTPacket.c ../../../MQTTPacket/src/MQTTDeserializePublish.c ../../../MQTTPacket/src/MQTTConnectClient.c ../../../MQTTPacket/src/MQTTSubscribeClient.c ../../../MQTTPacket/src/MQTTSerializePublish.c -o sub ../../../MQTTPacket/src/MQTTConnectServer.c ../../../MQTTPacket/src/MQTTSubscribeServer.c ../../../MQTTPacket/src/MQTTUnsubscribeServer.c ../../../MQTTPacket/src/MQTTUnsubscribeClient.c
gcc pub.c -I ../../src -I ../../src/linux -I ../../../MQTTPacket/src MQTTClient.c ../../src/linux/MQTTLinux.c ../../../MQTTPacket/src/MQTTFormat.c ../../../MQTTPacket/src/MQTTPacket.c ../../../MQTTPacket/src/MQTTDeserializePublish.c ../../../MQTTPacket/src/MQTTConnectClient.c ../../../MQTTPacket/src/MQTTSubscribeClient.c ../../../MQTTPacket/src/MQTTSerializeP
@bachwehbi
bachwehbi / bbt_latency.cs
Created November 27, 2014 21:03
Measures the latency of Beebotte using the C# client SDK
/////////////////////////////////////////////////////////////
// Copyright (c) 2013-2014 Beebotte <contact@beebotte.com>
// This program is published under the MIT License (http://opensource.org/licenses/MIT).
//
// This code uses the Beebotte API, you must have an account.
// You can register here: http://beebotte.com/register
//
// This program measures the latency of Beebotte real time service.
//
// Use the NuGet package manager console install dependencies:
@bachwehbi
bachwehbi / glileo_tutorial.py
Created September 25, 2014 22:22
Connecting Intel Galileo to Beebotte to report sensor data
#!/usr/bin/python
# coding: utf8
############################################################
# This code uses the Beebotte API, you must have an account.
# You can register here: http://beebotte.com/register
############################################################
import sys
import os
import time
@bachwehbi
bachwehbi / raspi_dht11.py
Last active October 21, 2023 08:25
Temperature and humidity monitoring using Raspberry Pi, DHT11 sensor and Beebotte
#!/usr/bin/env python
############################################################
# This code uses the Beebotte API, you must have an account.
# You can register here: http://beebotte.com/register
############################################################
import time
import Adafruit_DHT
from beebotte import *
@bachwehbi
bachwehbi / bbt_mqtt_sigauth.js
Created August 27, 2014 01:01
Beebotte MQTT example with signature based authentication
var mqtt = require('mqtt')
var crypto = require('crypto');
//Creates a hash mac of a string
function sign (key, algo, str){
var hmac = crypto.createHmac(algo, key);
hmac.setEncoding('base64');
hmac.write(str);
// you can't read from the stream until you call end()
hmac.end();
@bachwehbi
bachwehbi / bbt_mqtt_pub.js
Created August 15, 2014 18:49
Simple publisher: using Beebotte MQTT with secure connection, AES 256 encrypted data, QoS level 1
// Copyright (c) 2013-2014 Beebotte <contact@beebotte.com>
// This program is published under the MIT License (http://opensource.org/licenses/MIT).
/////////////////////////////////////////////////////////////
// This code uses the Beebotte API, you must have an account.
// You can register here: http://beebotte.com/register
/////////////////////////////////////////////////////////////
var mqtt = require('mqtt')
var crypto = require('crypto');
@bachwehbi
bachwehbi / bbt_mqtt_sub.js
Created August 15, 2014 18:48
Simple subscriber: using Beebotte MQTT with secure connection, AES 256 encrypted data, QoS level 1
// Copyright (c) 2013-2014 Beebotte <contact@beebotte.com>
// This program is published under the MIT License (http://opensource.org/licenses/MIT).
/////////////////////////////////////////////////////////////
// This code uses the Beebotte API, you must have an account.
// You can register here: http://beebotte.com/register
/////////////////////////////////////////////////////////////
var mqtt = require('mqtt')
var crypto = require('crypto');