Skip to content

Instantly share code, notes, and snippets.

View atharvai's full-sized avatar

Atharva Inamdar atharvai

View GitHub Profile
set -ex
docker run --rm --volume "/var/lib/go-agent/pipelines/:/clean" busybox chown -R $(id -u):$(id -g) /clean/${PIPELINE_NAME} > /dev/null 2>&1 || true
rm -rf /var/lib/go-agent/pipelines/${PIPELINE_NAME}
@atharvai
atharvai / bootstrap-s3.sh
Last active November 1, 2023 10:44
PySpark Localstack S3 read CSV example
#!/usr/bin/env bash
set -x
awslocal s3 mb s3://my-bucket
cat > requests.csv <<EOF
"email1"
"email2"
"email3"
EOF
import botocore
from botocore.exceptions import ClientError
from mock import patch
import boto3
from util import get_credentials
orig = botocore.client.BaseClient._make_api_call
def mock_get_cluster_credentials(self, operation_name, kwarg):
if operation_name == 'GetClusterCredentials':
@atharvai
atharvai / eddystone_scan.py
Last active September 4, 2020 22:09
Scan Bluetooth LE for advertised Eddystone URLs
# Reads Bluteooth LE Eddystone beacon URL from Microbit
# https://github.com/frawau/aioblescan/blob/master/aioblescan/__main__.py#L27
# enable bluetooth without sudo https://www.raspberrypi.org/forums/viewtopic.php?t=108581
import asyncio
import aioblescan as aiobs
from datetime import datetime
from aioblescan.plugins import EddyStone
@atharvai
atharvai / microbit_adc_ble.js
Created September 4, 2020 21:36
Microbit Bluetooth LE Eddystone URL to publish ADC inputs
basic.forever(function () {
control.waitMicros(5000)
bluetooth.advertiseUrl(
"adc/" + pins.analogReadPin(AnalogPin.P0) + "/" + pins.analogReadPin(AnalogPin.P1) + "/" + pins.analogReadPin(AnalogPin.P2),
7,
false
)
})
// list databases
db;
//select database to use
use test;
// find existing collections
db.getCollectionNames();
//insert 1 document into a collection that doesn't exist
@atharvai
atharvai / oplog_timestamp.py
Created February 4, 2019 07:09
BSON timestamp construct & deconstruct example
from bson import Timestamp
from datetime import datetime
ts=Timestamp(datetime.utcnow(), 1)
def ts_to_int(ts):
return (ts.time << 32) + ts.inc
def int_to_ts(t):
return Timestamp(t>>32, t & (2**32)-1)
field optional description
ts N BSON Timestamp. Often converted to 64bit INT for JSON
h Y Random hash
v N oplog protocol version. default 1.
op N Type of op. one of i, u, d, n, c
ns N BSON Namespacestring. Serialised as db.collection
o N Operation applied. object
o2 Y Object 2. Additional information, usually operand
fromMigrate Y Boolean. Indicates if this operation is part of chunk migration between shards
{
"ts": 6642324230736183304,
"h": -1695843663874728470,
"v": 2,
"op": "u",
"ns": "analysts.analysts",
"o": {
"$set": {
"r": 0
}
@atharvai
atharvai / snowplow-analytics-sdk-lambda-example-python2.py
Created January 8, 2019 14:28
AWS Lambda Snowplow Analytics SDK example functions
from __future__ import print_function
import snowplow_analytics_sdk.event_transformer
import snowplow_analytics_sdk.snowplow_event_transformation_exception
import base64
import json
print('Loading function')