Skip to content

Instantly share code, notes, and snippets.

@scottpersinger
scottpersinger / cs_team_job_posting.md
Last active February 2, 2017 17:37
cs team job posting

##Server-side Developer with the Core Services team##

This posting is for a server-side developer position with the Heroku Core Services engineering team. Our team is responsible for the master control logic inside Heroku - the business logic that drives our platform. When anyone accesses the Heroku API (including users of the Heroku web dashboard which itself uses the API) they are talking to our systems. We are looking for a dedicated software engineer to add to this critical team.

We welcome candidates from any geography within UTC-10 to UTC-5 time zones (roughly the Americas plus Hawaii). Heroku headquarters are in San Francisco, but more than half of our team works remotely.

Most of our services are written in Ruby, but we are increasingly writing new services in Go. That said, we aren’t necessarily looking for Ruby or Go experts - just really good developers. We manage a lot of complex business logic inside a service that operates at high scale. We operate all of our services in production and everyo

tion_id=3 sync=<Mapping.2 Lead->lead POLLING_SF_CHANGES> instance_id=2 state=u'POLLING_SF_CHANGES' message=None event='>> started'
12:29:25 fastworker.1 | ERROR:mirror.lib.db_adapter connection_id=3 event='!!!!!!!!!!!!!! CREATED A DBADAPTER!!!!!!!!!!!'
12:29:26 fastworker.1 | INFO:mirror.logger mapping_id=2 instance_id=2 event='Polling Salesforce found 10 updated records' connection_id=3 mode=None
12:29:26 fastworker.1 | INFO:mirror.logger mapping_id=2 instance_id=2 event='Polling SELECT Id FROM Lead WHERE SystemModstamp > 2015-07-17T19:03:37.000Z AND SystemModstamp < 2015-07-17T19:29:30.000Z ORDER BY SystemModstamp ASC' connection_id=3 mode=None
12:29:26 fastworker.1 | INFO:mirror.logger count=10 instance_id=2 target=1 connection_id=3 mapping_id=2 record_type=u'Lead' mode=None duration=0.430449 operation=6 event='Lead, QUERY, \xe2\x86\xb7SALESFORCE, 10 rows, (0.43 secs)'
12:29:26 fastworker.1 | INFO:mirror.logger count=10 instance_id=2 target=2 connection_id=3 event='Lead, INSERT, \xe2\x86\x93DATABASE,
@scottpersinger
scottpersinger / gist:e038ddc7c094c14bde0a
Last active November 23, 2023 04:34
Node-based ETL pipeline

Node.js offers a great environment for building ETL scripts. This is because Node is very easy to program and work with, AND has interface libraries for almost everything under the sun.

We need a framework that makes writing ETL scripts easy:

  • Supports creation and re-use of components
  • Cleanly divides input/output concerns from transformation concerns
  • Works identically in both event-driven and batch scenarios
  • Supports an easy development/testing mode
  • Provides good visibility when running in production
  • Has a simple DSL for expressing transformation logic
function parse_git_branch {
git branch --no-color 2> /dev/null | sed -e '/^[^*]/d' -e "s/* \(.*\)/[\1]/"
}
export PS1='\u@\h \[\033[1;33m\]\w\[\033[0m\]$(parse_git_branch)$ '
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from requests import Session
from requests.adapters import BaseAdapter
from requests.auth import HTTPBasicAuth
from requests.models import Response
from suds.properties import Unskin
from suds.transport import Transport, TransportError, Reply
@scottpersinger
scottpersinger / rest_db.md
Last active August 29, 2015 14:00
REST DB - strawman

Simple RESTful Storage API

We need a simple, schemaless REST-based storage API for use by Javascript and native mobile apps. This "backend as a service" should provide a simple mechanism for storing state for thick-client apps that don't want to invest, at least initially, in server-side logic.

Parse is the best known example of this kind of service. A service like Parse enables this kind of simple app dev workflow:

def salesforce_oauth(s, oauth_url, username=None, password=None):
"""s should be a requests session"""
r = s.get(oauth_url)
params = urlparse.parse_qs(urlparse.urlparse(r.url).query)
data = {"un":username,
"width":2560,
"height":1440,
"hasRememberUn":True,
"startURL":params['startURL'],
@scottpersinger
scottpersinger / gist:6118327
Last active December 20, 2015 10:49
try this one

cloudconnect is an add-on that connects your Salesforce account to your Heroku app.

Cloudconnect creates a relational database containing your Salesforce data, and synchronizes data changes in both directions to that database.

By using cloudconnect, you can interact with your Salesforce data by connecting to a PostgresSQL or Mysql database using your favorite language or web framework like Rails, Django, or PHP.

Provisioning the add-on