Skip to content

Instantly share code, notes, and snippets.

/social_stats.py Secret

Created November 9, 2014 04:30
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save anonymous/51d467a6cbb4d4f4749d to your computer and use it in GitHub Desktop.
Save anonymous/51d467a6cbb4d4f4749d to your computer and use it in GitHub Desktop.
How Do I Setup 10 Processes?
import psycopg2
from socialanalytics import pinterest
from socialanalytics import facebook
from socialanalytics import twitter
from socialanalytics import google_plus
from time import strftime, sleep
def getSocialStats(url):
p = pinterest.getPins(url[2])
f = facebook.getObject(url[2])
t = twitter.getShares(url[2])
g = google_plus.getPlusOnes(url[2])
return [p, f, t, g]
def saveToDB(stas):
# Save results
try:
now = strftime("%Y-%m-%d %H:%M:%S")
cur.execute("INSERT INTO ...")
conn.commit()
except:
conn.rollback()
conn = psycopg2.connect("dbname='***' user='***' host='***' password='***'")
cur = conn.cursor()
# Select all URLs
cur.execute("SELECT * FROM urls;")
urls = cur.fetchall() # 160,000+
# 1) Span 10 processes
# 2) Each process takes a url from urls
# 3) Calls getSocialStats(url)
# 4) Calls saveToDB(data)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment