Google I/O 2014

Auth.md

#Authentication and third party APIs

Was cancelled. Instead Go language Q&A

http://tour.golang.org/#1

Learning curve is very fast, allegedly one of the easiest languages to learn.

CI.md

#Continuous Integration with Release Pipelines

Google Apps Developer Console

Pipeline takes care of Jenkins. You don't have to know about Jenkins if you don't want to.

Eg Dev > QA > Prod

Engineers mess with QA. Every night things get sent to QA and run for you.

Unit tests are run automatically.

#Demo

Java sample web app

Outputs the word of the day

###test success

• make a change in the file and in the unit test
• commit, push origin master
• build, test, deploy done or you can configure your own workflow
• can send to prod as wanted
• Google gives you a Jenkins VM, they do upgrades, security patches etc for you
• can use different language in the test etc
• multiple devs can contribute, builds are done in order they come in, other are blocked while that one is running
• will launch v1 soon after i/o
• in v1 each dev probably won't have their own environment, there will only be one central but in future version that would be added

###test failure

• log

###Q&A

• push button deploy?
  • yes, possible. there are two ways
  • a) in the script define the push the deploy button
  • b) have just one 'stage' which is prod, and if all tests pass including integration tests then deploy to that one which is production. Then later on can add more stages.
• just maven? no. maven and cradle as well

Polymer.md

#Polymer

Polymer and web components

• can be used in prototyping, design as well as development
• from 80 lines of JS to 1 line of HTML
• don't be left behind
• TODO http://www.polymer-project.org/ Absolute must-do

1. layout
2. material
3. transitions
4. themes

PreziMeetup.md

Prezi

Django meetup

Lighning talks

Python packages you should love and use

• bycrypt
• rq
• djangorestframework
• django-sslify
• django-debug-toolbar
• django compressor
• gunicorn
• newrelic
• heroku
• stormpath

Gargoyles on Gutters

Passionate about feature-switching

• gargoyle
• gutter
• similar but not the same
• feature switches
• how to use both?
• srsly, how? ;)
• memcache, ok but still slow
• zookeper is the secret sauce + awesome signals

Experimenting with Python 2 and 3 and virtualenvwrapper

• Python 3.4 asyncio compatible with 3.3 - node.js will be out in a year ?!
• python points to Python 2, python3 will point to Python 3 (PEP-0394)
• pip install virtualnvwrapper ...
• smoke test
• Zen of Python says "explicit is better than implicit! so use py2- and py3- prefixes
• set up different virtualenv for each, activate each and don't worry any more. The key is they reside in different virtual ENVIRONMENTS that need to be activated!
• first issue PEP-0328
• second issue Tasty Pie isn't compatible with Django 1.7

6 Years with django at Prezi

@szilveszter Szilveszter Farkas employee #2

Past/Present/Future One of the most trafficked python application ~44 million monthy active users, 1.5 million new users each month

Past

• Build for scale from the start
  • Aha moment: we used to have hardware. when we switched to Amazon s3 everything became easy
• CI
  • automation, automated tests
  • Jenkins ~ 750 jobs
  • at peak times it will launch 400+ instances an hour
  • avoid Django's test client - slows down your tests. unit tests are disguised as integration tests
  • avoid ORM/db access
  • http://carljm.github.io/django-testing-slides/
  • acceptance/integration/behavioral test
  • Twist framework was introduced, QA team got dissolved, everyone has to write their own tests. Twist was too slow
  • Moving to Cucumber
  • http://prezi.com/az1jfe5n
• CD
  • continuous deployment/automated deployment
  • feature flags
  • gargoyle/gutter
  • https://github.com/disqus/gutter
  • monitoring, dashboards
  • Nagios/StatsD/Logster -> Graphite -> Dashboard (HTML5)
  • Alerting, if something reaches a certain threshold
  • PagerDuty
  • all developers are responsible for their own code, no dedicated ops team
  • http://prezi.com/

Present

• PRIO1, critical path (cp): if xyz happens to at least 3 users
  • critical path until 2 weeks ago
  • took 2 years to add login page to the cp
  • landing page
  • dynapps: project container for 3 apps
• SOA
  • and microservices
  • communication protocol btw services: Thrift (Python and Java)
  • backend services move towards Scala
  • django-thriftify*: generates Thrift interface definition based on Django models, provides CRUD interface (not identical to ORM, but similar: not as cheap as a direct DB call. It's a remote call, actually), fallback to ORM
  • generated client libraries Java & Python
  • client libraries actually used by clients
• Conway's law (1968)
  • organize teams based on SOA
  • small teams with few engineers, each are responsible for a few services
  • loosely coupled, not too many dependencies
  • their team building philosophy
  • e.g. storage team, backend-api, online marketing, website team
• Tooling is very important
  • wrote a lot of tools
  • Snakebasket: thin layer on top of pip recursively install dependencies from req files
  • Simply*: manage local development environment
  • Please*: run all the services locally using HAProxy
  • Igor* infastructure for microservices bootstrap AWS config, Chef, Jenkins
  • Sergei*
• Deprecate APIs
  • lots of old APIs in the monolith
  • lots of clients out there using them
  • currently no policy in place for these (!)
  • if you have a startup, make sure you have a policy how to deprecate your APIs
  • solution: proxy APIs to bridge the old and new
• code review
  • cathc bugs early
  • distribute knowledge
  • "quality through social accountability"

Future

• e.g. Signup component
  • break it out of monolith
  • architecture diagram here
• local vs staging/prod env
  • vagrant and docker
• future of Django/Python
  • functional and reactive programming
  • moving towards Scala
  • Netflix OSS stack, Akka
  • Save django/python?