Skip to content

Instantly share code, notes, and snippets.

arnesund /
Last active Feb 25, 2021
Fetch Netatmo Weather Station measurements and store in InfluxDB
#!/usr/bin/env python
import os
import sys
import json
import time
import requests
# Get your client ID and secret by creating an App at
arnesund /
Created Jul 13, 2015
Word count of Twitter hashtags using Apache Spark
# Count the number of occurrences for each hashtag,
# by first extracting the hashtag and lowercasing it,
# then do a standard word count with map and reduceByKey
countsRDD = (filteredTweetsRDD
.flatMap(lambda tweet: [hashtag['text'].lower() for hashtag in tweet['entities']['hashtags']])
.map(lambda tag: (tag, 1))
.reduceByKey(lambda a, b: a + b)
# Get the most used hashtags (order countsRDD descending by count)
arnesund /
Created Jul 13, 2015
Load tweets into Spark and filter
# Extract tweets from MongoDB
allTweets = []
for doc in db.tweets.find():
# Load tweets into Spark for analysis
allTweetsRDD = sc.parallelize(allTweets, 8)
# Set up filter to only get tweets from the last week
arnesund /
Last active Jan 30, 2017
Fetch tweets from Twitter using Python module Tweepy
# Get details about own user
me =
friends = api.friends_ids(
# Initialize data structure
tweets = {}
# Fetch lists recent tweets for each of the user IDs in the list 'friends'
for user in friends:
# Only query Twitter for data not already cached
arnesund / cloudinit.conf
Created Feb 5, 2015
Simple Cloud-init Config Example for Apache
View cloudinit.conf
- apache2
- [ a2ensite, "000-default" ]