Skip to content

Instantly share code, notes, and snippets.

@jabley
Last active March 18, 2016 21:13
Show Gist options
  • Save jabley/b37f8477d40962bde703 to your computer and use it in GitHub Desktop.
Save jabley/b37f8477d40962bde703 to your computer and use it in GitHub Desktop.
Scale Summit 2016

Fraud detection

  • People are scaling
    • Some parts of government can throw people at the problem
  • Delays inherent in the financial system can be helpful
  • batch jobs/overnight things can help fraud teams have the time to detect it
  • Things seeem based on:
    • IP address
    • new customer?
    • value of transaction
  • and scoring a transaction to see whether it should be flagged
  • Is it possible to share something like SpamAssassin but for fraud?
  • Machine learning possibilities
    • Getting a training set of data is hard?

Is the general-purpose OS holding us back?

  • Initial question of who is running a non-general-purpose OS?
  • CoreOS usage for personal things
  • If you’re using open-source CoreOS, you’re the beta tester and are sending telemetry back
    • Paying customers will get that update after the bugs are fixed
  • Some people don’t care about the OS
    • Java is their abstraction
  • Using Boxfuse to package that — https://boxfuse.com/
  • Similarly, http://runtimejs.org/ for Node apps
  • Optimising container sizes using Ubuntu, but probably want to use Alpine
  • Should people be using a different general-purpose OS (Illumos)?
    • Is it too late to switch?
  • Docker’s ease of use for Docker, and universal unit of deployment for Python/Ruby/Go etc
  • Diet Ubuntu, or Ubuntu Lite
  • JVM is almost there
    • Licensing of the JVM in containers
  • OSV — http://osv.io/
  • Pony has some interesting tooling around this – http://www.ponylang.org/
    • LLDB
    • Used in IoT and high-frequency training
  • What is the problem statement?
    • Attack surface of a full-blown OS
    • Cost to configure
      • Things you configure
      • Things you don’t
  • We aren’t being incentivised to solve this problem for good
  • Lower-power
  • efficient resource utilisation and packing
  • Can we measure the cost of running a general-purpose OS in this way?
  • systemd thingy
  • Feedback loop of having a complex system is too long
  • How do we get new people ramping up on this complexity
  • Illumos security model rocks
  • Being able to spin up Solaris Zones to create a ZooKeeper cluster, create a network partition and assert how it behaves. Very powerful.

Secrets management

  • who is storing secrets in version control
    • (quite a few hands raised)
  • who is doing that intentionally?
    • (still some hands up)
  • gpg-encrypted files in version control
  • looking at Vault from Hashicorp
  • how are other people solving this issue?
  • many passwords in a single file?
  • something else?
  • files on S3 with ACLs around this, with name-spaces on keys
  • Heavy usage of IAM
  • Starting to define IAM as CloudFormation stuff, Infrastructure as code
    • a feature can define this as an IAM role in CloudFormation
  • define permissions on roles rather than groups
  • Are peopling using Vault?
  • over the last 18 months, some experience with it
  • sealing and unsealing the Vault. If the cluster goes down, then no apps can access the secrets.

PaaS (mainly CloudFoundry)

  • why aren’t more people using it
  • x has been using it for a while. Surprised that more people haven’t heard of it, and aren’t using it
  • Docker and CF comparison
    • Docker is based on LXC
    • CF found LXC wouldn’t work at the time, so created Warden (and now Guardian)
  • what’s the right way to run containers
  • Any sufficiently complicated Paas contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Cloud Foundry.
  • API quality
    • Cloud Foundry API is lovely and well-supported, properly versioned etc
    • Bosh isn’t :(
  • People misunderstand it
    • Trying to download and run Bosh, rather than `cf push`
    • See the same thing with Kubernetes

What has changed since Scale Summit 2015

  • new things last year that have bedded in or not?
    • Faking interfaces (Google CloudSQL; RDS). Things pretending to be MySQL but giving cloud-scale-y properties.
      • Still going on. Google have said they’re doing it.
    • High integrity databases. Built on Merkel trees etc. “The word blockchain is banned in this room.”
      • We now have registers (and some people outside government care)!
    • Event streams like Kafka
      • Seem to be taking hold.
    • Time series DBs are hot. But this time for real.
      • InfluxDB now has a business model and has got usable. (but you need to pay for clustering, which is what makes it usable).
    • node build tools are still a thing :(
      • Boxen -> node -> npm -> gulp -> bower
    • related JavaScript build files :(
    • Bazel
      • Good
    • Buck
      • ?
    • Go
      • (confirmation bias from jabley) \o/
    • Rust
      • Active, good. Not as good marketing?
    • Dart is still dead
    • VirtualDOM and Shadow DOM; diffing against the DOM and so forth.
      • Not so much.
    • React all the things!
      • React is even shinier.
    • Service Workers were probably going to be a big thing.
      • Not happened yet. There was a conference from Google in London this week which talked about it.
    • HTTP2!
      • Fuck yeah. We did that at GDS for internal things (might have changed since jabley left). Fastly don’t (yet) do it. Other vendors are available.
    • TLS deprecation pushes.
      • SSLv3 is nearly dead?
    • Named/branded exploits! And the world ending every other week.
      • Still kind of a thing. Named/branded exploits as a hiring filter.
    • All JS frameworks are basically going all the time (Ember, Angular2, Bower, Grunt, )
      • This is still a thing. Although maybe Bower is declining. Will it be dead by next year?
    • ES6 is good enough now.
      • Is it going to be delivered?
    • Mozilla added support for DRM to HTML5 player.
      • We don’t really care.
    • Responsive images/.
      • Yeah, got better. Spec is a thing.
    • Malvertising. Thoughts of the day checked IP and dropped malware on defence/govt/pharma.
      • People are really ramping up on Ad-blocker usage.
      • Move from AdBlockPlus to the new thing, people!
      • Sites are detecting and trying to respond to ad-blockers.
    • Kubernetes/GContainer Engine, Rocket. Docker as a specification?
      • The lines are being drawn for the impending battles
      • Brace yourself for the Container Wars of 2016
    • Mesos?
      • MS are using it on Azure.
      • Generally containers and orchestration are still being talked about, by people, on the internet. Fact.
    • Unikernels, microkernels, *kernels.
      • They are hot shit this year.
    • Perl 6 this Christmas! Or at least one of the implementations.
      • This fucking happened, people!
    • Mongo, Express, Angular, Node (MEAN stack) is a thing.
      • Um, yeah. Hiring filter?
    • HSTS. US Gov’t going to use it now, as are GDS.
    • Free certificate authorities! Also standards around that and open sourcing.
      • Letsencrypt.org happened, Amazon have done a thing. ACME, will that get traction?
    • SIM hack. Big story, heard almost nothing.
      • We didn’t talk about this in 2016.
    • IE is dead! Spartan now!
      • Edge now
    • Servo rendering engine
      • Still developing, still experimental
    • .NET CLR open source.
      • That’s continuing to happen with MS
      • Just bought Xamarin
      • SQL Server on Linux
      • Automation on Windows, lots of interesting things coming from MS.
    • People giving a shit about people! Codes fo conduct, gender equality, diversity, burnout, mental health, etc.
      • Yay, this is still a thing
  • new stuff
    • Tensorflow (deep-learning open source framework).
      • seeing this on the front page of news
      • AlphaGo
    • self-driving cars
    • Juniper backdoor
    • Proposed iPhone backdoor
    • Volkswagon emissions
    • IP Bill
    • Safe Harbour
    • Privacy Shield
    • Ubuntu ZFS not happening, because GPL. Fight!
      • 16.04 is basically done. Ubuntu won’t want to rip that out
      • Will Oracle sue:
        • Canonical
        • Amazon
        • no-one, because they’re basically fine with it
    • Oracle will be suing more people
    • Unsafe being removed from the JVM, then it isn’t.
    • Death throes and lawsuits from various large enterprisey companies that are increasingly not relevant
    • Tax avoidance and efficiency from large corporations
    • Codes of Conduct
    • VR
    • Slack
  • Falling out of favour:
    • Python
  • Still things:
    • Michael’s hair
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment