Skip to content

Instantly share code, notes, and snippets.

@oldfartdeveloper

oldfartdeveloper/elixirconf_notes.org

Created October 23, 2015 23:14
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save oldfartdeveloper/75fbfec6bcd05d9a6462 to your computer and use it in GitHub Desktop.
Save oldfartdeveloper/75fbfec6bcd05d9a6462 to your computer and use it in GitHub Desktop.
Download ZIP
My Elixirconf 2015 Austin Notes
Raw
elixirconf_notes.org

Advanced Phoenix Traiing

Organization

  • web only has code that has no state. Changes to these files auto- restarts the server.
  • lib functionality manages state. (Note: server does not auto-reload w/ changes to these.

What is above is a property of :reloadable_paths.

External service

These may or may not succeed so:

  • Give up if the 3rd-party service fails.
  • Don’t remember about state; so mark :temporary. Supervisor strategy can be :simple_one_for_one.

Worker restart

  • :permanent - restart always
  • :transient - restart only if crashing. Don’t restart if normal termination.
  • :temporary - Never restart.

Links

  • github.com/livehelpnow/ObjCPhoenixClient

Presenter Best-Practices

  1. He constantly uses iex help
  2. Task.yield useful for tasks that you are willing to give update if they aren’t absolutely required. Task.yield is brand new.
  3. Logging to the JS console is extremely valuable; can be really difficult to see what is broken when it’s not logged.

Handle Out

handle_out

handle_out("new_message", params, socket) docchan
  if muted?(params.user) do
    push socket, "new_message", params
    {:noreply, socket}
  end
end

Could be used for client customization.

But can be slow.

Using #intercept better.

Task GenServer Agent

Task GenServer Agent

Task

Good for parallelizing a bunch of stuff to run concurrently.

GenServer

Both

Agent

Primarily carries state. If it works on one machine, it may not work the way you expect on another machine. Issues registering global process.

Volunteer?

Need people to write up channel testing for Phoenix.

2nd Day

Keynote: The Pendulum

TL;DR

A history of how the “best-practices” pendulum has switched repeatedly between functional (“batch”) and stateful and why.

Right now the pendulum is swinging towards functional because of how the next wave of desired functionality emphasizes greater linkaging between apps.

Detailed Notes

“A tendency of a situation to oscillate”

Batch

  • On one side is more stateless (“batch” or “interactive”)
  • Batch processing: “The execution of a series of programs on a computer w/o manual intervention.”
    • Sabre System (Airlines) “…handling transactions at the time the occurred”
    • Abstractions
    • Concurrency
    • Isolation

New requirement: Conversation.

Now need to be thousands/millions conversations.

Future of Phoenix - Chris McCord

TL;DR

Phoenix will be betting heavily on Facebook’s GraphQL as the primary interface to the client. i18n will be implemented using gettext very soon now.

Detailed Notes

  • Lots of production deployments

Beyond the Browser

Channel Clients:

  • Javascript
  • Swift
  • ObjC
  • C#
  • Java

Phoenix v1.1

  • I18n
  • Channel Presence

gettext

Is an existing standard for I18n

<%= translate "pages.titles.welcome" %>

<%= gettext "Welcome to I18n done right" %>

Phoenix.Presence

Will be simplified code wise

vNext

GraphQL

Facebook open-sourced GraphQL

language-agnostic syntax.

Replaces tradition client-server data requests

  • REST
  • Ad-hoc endpoints

Maps to a JSON structure.

Client shapes the expected response.

GraphQL has lots of support tools.

  • Introspectino
  • Docs
  • Query validation
  • Deprecations

Server and language agnostic

React + GraphQL
  • Declarative data arequirements
  • Single server endpoint
  • One-way reactive data flow
  • Collocated queries
    • Qeries live next to the views that rely on them
  • Composition of components
  • Caching fro free

JS SPA vs Server HTML

Phoenix + GraphQL

Both client and Phoenix.View HTML talk to GraphQL.

Longer term, can we run these processes on the server?

Like a virtual DOM on a server.

GraphQL is naturally parallelizable

slide shows a massive GraphQL JSON request

Can we do this in Ecto?

Understand releeases

TL;DR

The deployment tooling for Elixir and Phoenix appears to be remarkably capable and complete. It appears that, even when you don’t specifically hot swap, the deployment seems to act like it.

They have made deploying to disparate servers seamless by inclduing the Erlang libraries for that server in the deployment scripts.

Very impressive.

Detailed Notes

  • Conform
  • Exrm

OTP Applications

  • Well defined structure and lifecycle
  • Explicit dependencies
  • Useful metadata (name, version ,exports, included apps..)

What does better look like?

  • Self-contained apps wiht all dependencies
  • No need for production to have tooling installed
  • Application lifecycle mnemennt out of the box
  • Pathfor hotupgrades/downgrades
  • Easy cross-compillation
  • Easy deplyment
  • Easily reproduced

OTP releases

Have all of the above.

including…

  • Aplication health monitoring (heart)
  • Comes as a tar ball.
  • Easily reproduced as immutable tar ball.

So what is a release?

  • Set of versinoed OTP aps
  • ERTS (Erlang Runtime System)
  • Release metadata (app starting order, etc.)
  • Explicit configuraiton (sys.config.vm.args)
  • Scripots for managing the release
  • Packaged as a tarball.

Release Structure

showed tree of the appp structure.

What is EXRM

  • A tool for building releases
  • Extends Relx, the equiv tool for Erlang
  • Easy to integrate into your existing build process
  • Provieds: Mix tasks
  • Provides: Automatic appup generation
  • Provides: plugin system
  • Provides: Intellignet defaults, but you can still provide your own custom relx.onfig, vm.args, etc.

Mix release

  • Read configuration
  • Generates relx.config, sys.config, vm.args
  • Runs before_release hooks
  • Relx: Perform discovery (apps, previous releases)
  • Relx: Resolve dependencies
  • Relx: Build release
  • Relx: Output release to <lproject>/rel
  • Runs after_release hooks
  • Repacages release
  • Runs after_package hooks

What it looks ikes

MIX_ENV=prod mix release
# cp the tar somewhere
# go to that location
start

Showed update where the site didn’t go down. Wow!

Showed how to downgrade to previous version w/o going down. Wow!

Configuration - Conform

Presenter wrote conform

STatic configuration files are sufficient but less than ideal for sys ops.

  • Developers define configuraiton schema
  • Users configure application via init-style config
  • Schema provides config mappings, transforms, validators
  • Extend schemas provided by dependencies
  • Expose important config settings, hide advanced settings, provide documentation, sand defaults
  • Configuraiton is merged over config.exs settings, so they can co-exist

Inspired by cuttlefish

Conform Schema - Mappins

  • Define how to map user-facing configuraiton to system configuration
  • Defines datatype of the settings
  • Defines default values
  • Defines validation rules for the setting
  • Exposes other options (hidden, ommeted, etc.)
  • Provides documentation

Conform Schema - Transforms

  • Functions which receive a reference to the

configuraiton state, and return a value to be used for the given setting

  • Can execute any Elixir/Erlang code (including your own modules)
  • Can be defined in the schema itself, or in a module, by extending a behaviour (Conform.Schema.Transform)

Conform Schema - Validators

  • Functions which receive a mapped value + args, and validate some rule
  • REturn :ok, {:warn, msg}, or (:error, msg}
  • Like transforms, can be defined in their own module, by extending a behavior (Conformam.Schema.Validator)
  • Conform provides Range Validator

Demo

  • Showed configuration file.

Questions

  1. How do you manage things like database (Phoenix)?
  2. How does it track state changes? Callback available from GenServer.
  3. What happens when Erlang runtime is different on deployment machine? Answer: by copying the target Erlang runtime to release folder.

CRDTS: Data types for the Apolcalypse

TL;DR

Academia.

  • How certain computer science concepts solve many of the problems of keeping data properly synchonized across servers.
  • The presenter pleads for participants to implement these concepts in Elixir so that these problems are solved or at least improved.

Detailed Notes

Disclaimer

CRDT’s are an arcane knowledge hidden behind logic and Greek letters.

Presenter has an implemented one called “boom”.

Barriers to understanding can be high.

They aren’t complicated to implement.

They just have to be precisely implemented.

What?
  • CRDT’s are Conflict-free Replicated Data Types
  • Update operations mutate state
    • commutative
    • associstive

    -idempotent

  • CRDTs are trivial to communicate over a network.
  • Great for distributed systems.
  • If it involves a browser, you are probably implementing a distributed system.
  • Erlan/Elixir process themselves have distributed properties (message passing).

Types

  • Flags - Booleans
  • Counters - up or down
  • Sets
  • Registers
  • Maps - Can hold nested CRDTs
  • Graphs
  • Documents … kind of

Caveats

  • Certain kinds of app logic require strong coordination
  • Forging ahead w/o reading the academic literature is a mistake.
  • Need heavy test coverage
  • Presenter has been bitten by this.

Examples

G-Counter
  • It counts up
  • Implemented as a simple mapped
  • A signle “actor” 9node or process)

CRDTS un the community

  • Ther eis a problem w/ CRDTs and the wider programming community which seem solvable within the alues of the Elixir community.
  • “I know I can solve my problems using a CRDT, but I have no idea how to do that.”
  • Current CRDT adopters seem to adopt them when faced with strong external pressures.
  • This status quo can be broken w/ some quick wins.
  • But some solutions need CRDT’s in rich client contexts (SPAs)

Ecto + CRDT Map Support

Only thing you have to pass through CRDT are updates.

The Wishlist

  • Gossip Protocol
  • External Representation
  • Otehr Language Support
  • Library support (particularly Ember and React)

MOre Seriously

  • CRDT’s can provide serious value, please be tempted
  • Rich and/or native apps can always apply CRDT updates locally and defer synchronization for speed and resiliency
  • Helps your network-connected app work on the subway (or during server outage)
  • And I could use help! PR’s welcome

Interesting People /Work

Taking Phoenix Beyond The Browser with iOS and Apple Watch

TL;DR

A call to action to implement code in the “IoC”, and how Elixir makes it easier. He demo’d a physical labyrinth toy whose knobs were controlled by servos connected to arduino-type computer with Elixir loaded in it. The objective was to enable the user to control the board by tilting his iPhone to tilt the labyrinth toy the same way. Impressive demo.

Detailed Notes

The Inernet of Dumb Things

Justin Schenck

  • LiveHelpNow
  • Nerves - run Elixir on tiny cpu boards
  • Bakeware

twitter: @mobileoverlord github: mobileoverlord IRC / Slack: jschneck

Affect Your Environment

  • Arduino
  • Home Beer Brewing
  • Motorcycle - turn signals control lights on the back of the motorcycle jacket

Devices

  • iOS
  • Apple Watch
  • Embedded BeagleBone Black Raspberry Pi

Intentions

  • Problems w/ the traditiional web
  • MotMetrics (code) and [more:[code]]
Problems w/ tradiitonal web
  • Connectivity

Change of the state of change.

Now have disparity of kinds of client computers/devices

  • Including sensors, lots of them.
Whats App

Creativity

Walled Gardens

IoT’s often depend upon a proprietary system.

Proposition

Power to the People

  • Picaxe
  • Popeller
  • Stamp

Arduino

  • Easy to use IDE
  • Libraries
  • (missed it)

Raspberry Pi

  • Power
  • Higher Connectivity
  • Higher Lang

Performance and Productivity (from the connectivity point of view)

Summed by the following components

  • nerves
  • objective-C
  • Phoenix

For the motorcycle apps:

  • maps
  • start button on watch

Elm

TL;DR

Showed the Elm language that transpiles onto Javascript. It is a very functionally-oriented language that has scored some really good speed comparisons w/ other transpiled languages.

A lot of Elixirists are enthusiastic about the possibilities of Elm.

However, it is not quite functionally complete. Worth keeping an eye on.

Detailed Notes

Why Elm

  • runs anywhere JavaScript does

What does it look like?

[elm-lang.org/try][Example]

Model/update/view architecture

Elixir in the Browser

TL;DR

The presenter has been toying w/ what Elixir as a Javascript transpiled language could do and what it would mean.

At this point, he’s just trying things out and seeing what he feels.

The challenge is identifying what pieces of Erlang are important to bring along with the Elixir syntax.

Detailed Notes

Bryan Joseph

github, twitter, #elixir-lang=bryanjos

  • New Orleans
  • New Orleans Open Source Hackathon
  • Operation Spark
  • Programming in Elixir since late 2013

ElixirScript

Transpilor

Goals

  • Translate a full or subset of Elixir to Javascript
  • ES2015
  • Complement Elixir backends
  • Ieiomatic
  • Embrace the Environment

Why?

  • JavaScript is tolerable
  • Elixir is awesome!
  • Fun and interesting challenge

Learning Resources

  • Metaprogramming Elixir by Chris McCord
  • SpiderMonkey AST and ESTree
  • Mardi Gras
ESTree
  • Made a Structs for each JavaScript node
  • Builder module
  • Generator module

Transformation

  • Compiler in Elixir
  • Runtime in Javascript

Embrace the Environment

  • Embrace the environment that you’re in.

Elixir runs in the beam. Unapologetically. Should be able to call javascript functions.

3 Ways

  • escript
  • mix
  • ElixirScript code within your code itself.

Denos

Growing Elixir

TL;DR

Main sponsor of Elixirconf wants attendees to really market Elixir/Phoenix to their bosses; get companies to use.

Call to participate in Elixir-promoting activities such as:

  • Participating in meetups
  • Blog posts that can help others.

Detailed Notes

  • Get employers to hire.
  • @jessitron available for meetups. Dockyard

Jessica Kerr

TL;DR

An inspirational talk on how the same Ruby

Detailed Notes

Rationally changed from physics to programming.

Physicists stay w/ physics

The Structure of Scientific Revolutions by Thomas S. Kuhn 1962

What Wendy learned:

“Science grows towards great Truth”.

Actually a lot more complicated. Fits and starts. Complex people.

Ben Franklin pulled theory together.

  1. Ideas are shared.
  2. Paradigm: what ideas are worth considering.
  3. Ideas keep coming.
  4. Ideas are shared.

When an idea is ready, it doesn’t come to just one person.

If multiple people are talking for an idea, it must be worth discussing.

Distributed: ugly, hard, and here to stay.

Each scaling problem causes the head to explode coz it can’t handle it all anymore.

Unfortunately functional programming is not enough.

OO is not enough. Mutable state = circles.

OO organizes code well.

Alan Kay simply says: “Tell Don’t Ask”.

Elixir has the strengths of both functional and OO.

Erlang includes failure as a first-class citizen.

Failure paths are myriad. Failure is the common case. Perfectly normal for most of our code to be failure-handling code.

Erlang. PropertyTest automates code to find the outlying edge failures.

Erlang CI testing learns where the failures are and runs the failing tests first.

FP: Immutability OO: Independence of our active processes Isolation

“The sky is the limit”.

We need to consider what comes next?

What comes after agile? code -> show -. code -> retrospective -> code.

Agile development:

ideas -> discovery -> delivery.

Lean ->

  1. Learn
  2. Build
  3. Measure

rinse and repeat

What comes after no estimates?

Give a range estimate: 1/2 - 2 months

Tackle the unknowns problems first.

“How to Measure Anything” 2007

Make uncertainty a first-class citizen.

Scientific method doesn’t help our team.

“Will my team work better w/ Elixir?” Can’t know.

What comes after the scientific method?

Dr. Brene Brown listen to her 20 minute podcast.

We can still collect data.

Grounded Theory: Data -> categories -> theory -> rinse and repeat

What comes after MVC?

Presenter thinks Elm architecture.

Immutable model -> view function -> HTML -> operator -> events -> update function -> rinse and repeat.

Sends events one at a time.

Scales in your head really well.

Elm

or

react.js + redux

with

CQRS & Event sourcing

What comes after REST?

Many questions -> one request “graphql”

What comes after microservices?

Answer: better organized microservices which is Elixir is perfect for.

You can organize microservices into applications (in Elixir)

  • FP
  • OO
  • Isolation
  • People!

Experts are not acceptable to the rest of the world.

Want more like Ruby and Rails where everyone is on different steps on the stairway.

More than one stairway.

If you have figured out something, publish it.

People with separate paradigms have problems communicating w/ each other.

People with different backgrounds are valuable because they don’t have assumptions.

People who don’t know anything don’t have preconceptions. They will push the state-of-the-art to the next level.

What we should aim for is to take the staircase and, through helpfulness, smooth it out.

An idea doesn’t belong to one person.

3rd Day

Thanks You’s

  • HR Toolbox - last night’s party
  • Norris Conference Center
    • Thanks for decent pastries
  • Keynote speakers
    • Bruce Tate
    • Jessica Kerr
  • Training
    • Phoenix
    • Other than Phoenix (OTP)
  • About half the attendees are using Phoenix in production.

Keynote - Jose Valim - Where is Elixir now?

TL;DR

Details

Elixir v1.0

  • Sept/2014
  • >180 contributors
  • 3 books out
  • First Elixirconf!

Extensibility

  • Web infrastructure
  • Embedded systems
  • Financial/Video platforms
  • Graphical User Interfaces

Elixir v1.1

  • Sept/2015
  • >295 contributors
  • ~1000 packages on hex.pm
  • >6.5 million downloads

What’s new w/ v1.1

  • More functions

Enum

  • dedup - remove duplicates
  • random
  • take_random

String

  • String.jaro_distance

Task

Now has = async= and await.

Better

yield and shutdown.

yield returns :ok but returns nil if times out.

ExUnit v1.1

  • Capture Log - can suppress logging while running tests. Is a test tag to turn on or off.
  • Not implemented tag
  • Stacktrace depth is now configurable.
  • More detailed assertion error messages
  • Ability to sip tests
  • Proper line numer in doctests failures

Mix v1.1

  • mix profile.fprof - Can now profiles an Elixir file.
  • mix compile - tracks only compile-time dependencies for Elixir.
  • “recompile()” inside IEx
  • Use the safer https protocol instead of git for :github dependencies.

Deprecations

  • “\x{2661}” is soft deprecated. Because it was using unicode which accidentally.
  • No longer using Behavior. The new syntax is much cleaner.
  • Access protocol deprecated - matters only in development. opts[key] still works. Access defines a subset of the Dict behaviour
  • The bottle neck does not exist in prod.
  • Common hot-paths have been inlined

Community

  • Added code of conduct.
  • Much prettier ExDoc documentation.

Elixir v1.2

Migration to Erlang 18

  • Elixir v1.0, v1.1
  • Can now have variables in maps.
  • Official support for large months.
  • Keyword, Map, HashDict - which one do I use?
    • Unified via the Dict API
  • Too many options. Often confusing.

Elixir v1.2

  • Keyword lists - used as options, allow duplicatekeys, usdr ordered
  • Maps pattern matching, fast & scalable, …missed

Dict and Set

  • HashDict will be soft deprecated - use Mapped
  • HashSet will be soft deprecated - use MapSet
  • missed

multi-aliases

Can declare multiple aliases on the same line.

Expected delivery in dec 2015

Elixir v1.3

Appear next year or maybe never. ;-)

Collections

Collections + Laziness

This uses Stream instead of Enum

If it’s streaming, maybe we can handle in in parallel.

Pipeline parallelism

  • Each async stage is a process
  • What happens if one of them crahs?
  • How to provide back-pressure?
  • I.e. how to make streams supervised processes?
  • Pipeline parallelism isn’t even a good strategy.

Better question:

  • How to make supervised processes stream data?
  • Answer: GenRouter

GenRouter

  • Connects sources to sinks
GenRouter.start_link(
  GenRouter.SingleIn, [],
  GenRouter.BroadcastOut, []
)

Above may use supervisors.

Some Use Cases
  • A GenEvent replacement that is process-based
  • A way to load-balance jobs across a pool of processes.
  • Demand-driven - push backpressue to client which is what we want.
  • It is a message contracdt
  • It pushes back-pressure to the boundary.
  • GenRouter is one impl of this contract
  • Inspired by Akka Streams
Supervised TCP Acceptor

The path forward

  • Define the demand-driven message contract
  • Implement GenRouter and related abstractions
  • Integrate with streams
Keep Paying Attention to the Competition

Spark is a big data manager that acts like a query engine in SQL

***

Contributing to Elixir - Wendy Smoak

tl;dr

Helpful participating hints, some of which I didn’t know that surprised me.

Details

Introduction

  • Started in Java
  • Missed Ruby
  • Saw an opportunity to get started in Elixir

Thinking About How You’re Going to Go Forward

  • Ron Jeffires was looking for something new to learn.
  • Wendy is interested in the community.

Why?

You already have a day-job and come home and not get paid for.

  • Fix something that bugs use.
  • Learn something new.
  • Making new friends.
  • Good on your resume.

What to do

  • Ask questions - if someone gets you an answer, you can publish the answer so that others can benefit.
  • Experts not best people to teach new people.
  • Reproduce Issues - if someone else reports a problem and you can verify it. Maybe you can fix it.
  • Make improvements
    • Not just code
      • Documentation
    • Associated code
      • The release package I saw yesterday.
    • Don’t have to commit to Elixir core.

Asking Questions

We have now:

  • Mailing Lists
  • Google Groups
  • Slack
  • IRC
  • Stack Overflow
  • Reddit
  • Gist
  • Wiki
  • Blog
  • Official Docs

You can certainly answer.

Answers on IRC or Slack disappear; should be moved to site or wiki.

Fixing problems

Install the newest version and build a new “empty” project.

mix phoenix.new <proj name> --dev

Building bleeding edge

  1. Compile from source.
  2. Instructions seem clear enough.
  3. Modify path to point to your newly-built Elixir.
  4. elixir -v will verify that you’ve acquired the bleeding edge.

Release tags *

It’s very helpful to test release candidates w/ what you are working on.

Build the Docs

If you’ve messed w/ the documentation, you need to rebuild the docs and check your changes.

You need to checkout ExDoc. New ExDoc depends upon mix which is part of Elixir package.

mix do ...

Then make docs

Here’s your list of what will be generated:

…missed

Resources

Her blog post on building Elixir from source:

http://wsmoak.net/2015/09/08/building-elixir-from-source.html

Questions

IRC vs Slack

IRC more by core team; slack is somewhat more informal.

Erlang Engine

tl;dr

Details

Scheduler

  • Allocates hardware resources - memory, etc
  • Concurrency - Preemptively schedules; nothing can hog it.
    • Light loads get precedence.
    • Rebalances things periodically

Monitor appear

:observer.start
  • System Overview
  • Load charts
  • Application view
  • Processes
  • Table Viewer
  • Trace Overview
  • Information about a process

You can look through the “C” interface (command line instead of GUI)

Types in BEAM

  • Sgtrongly Typed
    • Small set of Types
  • Success Typing is the system telling you everything the Beam knows
    • “It will hurt your feelings”

TypEr

  • Shows Type information
  • Adds Type annotations to source code

Testing

  • Property Testing
    • QuickCheck for Erlang (by QuviQ) (for pay)
    • Proper
  • Concurrency Testing
    • Concuerror
      • What happens if I get data out of order?

Behaviors

  • Generic implementations of common tasks
  • GenServer, GenA

You can create your own.

  • “Error”-able task

Learn YOu Some Erlang

  • Don’t need to know the whole language
  • Just enough to have general idea of what Elixir does/doesn’t

Other Languages

  • Core Erlang
  • LEF (Lisp Flavoured Erlang) - Lisp
  • Joxa - Lisp
  • Erlog - Prolog on Erlang VM
  • Yhc - York Haskell Compiler
  • erlyjs - JavaScript interpreter for Erlang
  • microkanren implementation
  • luerl - Lua in Erlang

Other cool stuff

  • webmachine
  • ruby port
  • Ling - Erlang on Xen
    • Unikernels
  • Erlang on Mega Core Architecture Research
  • …and lots more

Calls to Action

  • Expand your User Groups
  • Elixir application usage across BEAM languages
    • Plug
  • Reach beyond Elixir and share what you know.

If the BEAM doesn’t grow, then Elixir won’t grow either.

Other ecosystems are generating really interesting ideas that don’t want to miss knowing about.

About Me

  • @stevnproctor
  • Functional Geekery
    • @fngeekery
  • PlanetErlang
    • @planeterlang
  • DFW Erlang User Group
    • @dfwerl

OTP Has Done It (Nick DeMonner)

TL;DR

DETAILS

OTP PRINCIPLES

  • SEPARATE GENERIC FROM SPECIFIC. GENERIC SOLUTIONS.
  • EVERY PROCESS IS EITHER A SUPERISOR OR A WORKER.
  • APPLICATIONS ARE JUST TREES OF PROCESSES.
  • HTTP://WWW.ERLANG.ORG/DOC/DESIGN_PRINCIPLES
  • ANY SUFFICIENTLY COMPLICATED ONCURRENT PROGRAM IN ANOTHER LANGUAGE CONTAINS AN AD HOC, INFORMALLY-SPECIFIED,

BUG-RIDDEN, SLOW IMPLEMENTATION OF HALF OF ERLANG - VINDING’S 1ST RULE

GENERIC VS. SPECIFIC

  • THOUGH YOUR DOMAIN PROBLEM MIGHT BE UNIQUE, MUCH OF YOUR TECHNICAL PROBLEM IS PROBABLY NOT.
  • WE NEED TO GET OVER OUR REINVENTION ADDICTION
    • IT’S A CHALLENGE TO REINVENT.
    • IT’S FUN TO REINVENT.
    • IT’S MOSTLY A WASTE TO REINVENT.
  • MAKING A GAME ENGINE VS. MAKING A GAME.
  • IF THE ABSTRACTION STOPS BEING USEFUL, STOP USING IT.

THE COMMONALITY BETWEEN OUR PROBLEM AND THE OTHER PROBLEMS HAVE A LOT OF OVERLAP.

ASK YOURSELF: HAS OTP DONE THIS? ANSWER: PROBABLY.

OTP APPS

  • FUNCTIONALITY THAT CAN BE STARTED AND STOPPED AS A UNIT.
  • OPTIONALLY REUSABLE (LIBRARY)
  • DON’T WORRY ABOUT IT IN MOST ELIXIR PROJECTS – MIX TAKES CARE OF IT FOR YOUR WITH THE –SUP FLAG
  • FOR MORE COMPLEX PROJECTS WHERE CONFIG.EXS ISN’T ENOUGH, SEE CONFORM.

SUPERVISORS

  • RESPONSIBLE FOR STARTING, STOPPING, AND MONITORING PROCESSES (ETIHER WORKERS OR MORE SUPERVISORS).
  • WHO WILL WATCH THE WATCHMEN? OTHER SUPERVISORS, AD INFINITUM OR UNTIL YOU’RE SATISFIED.
  • STICK WITH ONE_FOR_ONE OR ONE_FOR_ALL RESTART STRATEGIES.
  • TERMINATION TRICKLES DOWN FROM PARENT SUPERVISORS. CHILDREN ARE STOPPED IN REVERSED START ORDER.
  • CHILD SPECS (KEYWORD LIST) DETERMINE HOW CHILDREN BEHAVE DURING THEIR LIFECYCLE (START, RESTART, SHUTDOWN, TYPE).

GEN_*

  • GENERIC IMPLEMENTATIONS OF COMMON DESIGN PATTERNS.
  • ROCK-SOLID AND BATTLE-TESTED AT SCALES YOU PROBABLY WON’T HIT (AND IF YOU DO, YOU CAN REST EASY).
  • USE THEM VIA BEHAVIOURS, WHICH ARE IMPLEMENTED IN CALLBACK MODULES.

GEN_SERVER

  • 90% OF ERLANG CODE.
  • ENCAPSULATES THE REQUEST/RESPONSE CYCLE INHERENT TO MOST FORMS OF INTERACTION.
  • CLIENT <-> SERVER SYNCHRONOUS INTERACTIONS SUPPORTED WITH GENSERVER.CALL AND HANDLE_CALL.
  • CLIENT -> SERVER ONE-WAY INTERACTIONS, SUPPORTED USING GENSERVER.CAST AND HANDLE_CAST.

GEN_FSM

  • SPECIALIZED VERSION OF GEN_SERVER
  • FINITE STATE MANAGEMENT AS A PROCESS.
  • GET FROM STATE TO STATE USING EVENTS.
  • ASYNC AND SYNC EVENTS
  • DEFINE CALLBACKS AS STATE FUNCTIONS THAT PATTERN MATCH ON THE EVENT
DEF PENDING(:APPROVAL, DATA)...

GEN_EVENT

  • EVENT HANDLING AS A (MANAGED) PROCESS.
  • YOU DON’T MANAGE HANDLERS DIRECTLY – GEN_EVENT DOES IT FOR YOU USING GENEVENT.ADD_HANDLER.
  • YOU JUST CALL GENEVENT.START_LINK DURING STARTUP, ADD A BUNCH OF HANDLERS FOR EVENTS, AND CALL GENEVENT.ACK_NOTIFY WHEN SOMETHING HAPPENS.
  • THERE ARE OTHERS BUT DON’T WORRY ABOUT THEM.

ETS (ERLANG TERM STORAGE)

EPHERMERAL STORAGE (CACHES)

  • DEFAULTS TO :SET THYPE, BUT ALSO SUPPORTS :ORDERED_SET, :BAG, AND :DUPLICATE_BAG.
  • MAKE SURE TO LOOK AT :READ_CONCURRENCY AND :WRITE_CONCURRENCY DEPENDING ON USE CASE.
  • USUALLY STICK WITH :PROTECTED ACCESS.
  • TABLE IS DESTROYED WHEN PROCESS DIES.
  • CHECK OUT MNESIA AS WELL. FULL RELATIONAL DATABASE.

RELEASES

  • REFER TO PAUL SCHOENFELDER’S EXCELLENT TALK GIVEN YESTERDAY.
  • COMPLETELY SELF-CONTAINED BY DEFAULT.
  • PRODUCE A SINGLE ARTIFACT FOR EASY DEPLOYMENT.
  • HIGHLY CONFIGURABLE.
  • HOT UPGRADES/DOWNGRADES
  • GEN_* FLAVORS ALL SUPPORT. CODE_CHANGE

DISTRIBUTED APPS

  • BUILT-IN FAILOVER AND TAKEOVER ACROSS NODES.
  • CONFIGURED AS PART OF THE RELEASE.
  • FAILOVER LOKS FOR TEH NEXT AAILABLE NODES TO JUMP TOGETHER
  • …MISSED

MY DREAM

  • SYSTEM THAT USES OTP TO MANAGE ALL MY SERVICES, INCLUDING “THIRD-PARTY” NATIVE PROCESSES LIKE RUBY, PYTHON, OR POSTGRES AS A SINGLE UMBRELLA APP APP THE ROOT.
  • PORTS CAN BE USED TO GET A LARGE AMOUNT OF THE SUPERVISOR FUNCTIONALITY.
  • WHAT HAPPENS IF AN UPGRADE AFFECTS NATIVE PROCESSES? LET’S PASS THE BUCK TO JAMES SMITH – HE CAN FIGURE IT OUT.

RECAP

  • AWESOME IMPLEMENTATIONS OF COMMON DESIGN PATTERNS
  • ROBUST EPHEMERAL STORAGE OPTIONS
    • CHECK OUT DETS TOO
  • POWERFUL DEPLOYMENT STORY
    • EVEN BETTER THAN GO.
  • WHEN YOU’RE ROLLING WITH OTP, YOU’LL FIND THAT OTHER PLATFORMS IN YOUR STACK END UP AS THE POINTS OF FAILURE.

QUESTIONS?

ALSO, THEY ARE HIRING.

  1. WHEN DO I USE ETS OVER JUST USING THE GEMSERVER ITSELF. ANSWER: WHEN THE STATE IS DISTRIBUTED.

STREAMS, EXTERNAL SERVICES, AND OTP

SOMEHOW I LOST MY NOTES ON THIS. IT IS AN INTERESTING EFFORT ON HOW TO INTERACT W/ 3RD PARTY SERVICES THAT LEND THEMSELVES TO STREAMING.

I’LL SEE IF I CAN RESURRECT MY NOTES; THEY WEREN’T THAT GOOD ANYWAY.

OTHERWISE WILL WAIT FOR THE SCREENCAST.

Interoperability In Elixir: Dealing with the World Outside the Beam

TL;DR

Details

Really enjoying himself w/ Elixir

Showed the comparison table.

Have to roll your own solution to interoperability to other systems that have useful.

Interoperability

Wikipedia Telcom Definition:

The ability of systems, units or forces to prove services to and accept services from other systems, units or forces and to use the services exchanged to enable them to operate effectively together.

Erlang is a Big Place

  • Lots of power.
  • Docs are good but spread out.
  • Have solve a wide range of problems.

IUneroperability Tools

  • Ports
  • Port Drivers
  • JInterface
  • ?

Ports

  • Covers about 80% of your use case.
  • Leverages BEAM.
  • Error handling

Bind to external foreign process.

byte-oriented protocol. full bytes are used for control codes.

Initializes with bytes list including binary list.

If the port owner terminates, so does the port.

open_port(PortName, PortSettings)

# Example:

port = Port.open({:spawn, "pwd"}, [:binary])

receive do
 {^port, {:data, path}} ->
    IO.puts("Path: #{path}")
end

Shows ruby example as calling ruby from bash.

Looks similar to above example.

newlines indicate when the message content is complete.

Can use ETF (External Term Format)

Libraries

  • Ruby: erlang-etf gem.
  • Clojure: clojure-erlastic
  • Python: python-erlastic
  • Node: node_erlastic
  • C, Java, C#

Talk is going to deal w/ Ruby (yay!)

More complex problems

Shows how elixir can call Ruby.

Caution: used eval for Ruby

Reliability

In the real world, use a supervised GenServer.

KNow that the Ruby system is up.

Handle crashing and recover it.

Resources

Should look at “erlport” on GitHub Ruby can send requests to Erlang.

nifs faster but more vulnerable.

Composability Queries with Ecto

TL;DR

It is easy to build composable database query expressions similar to scopes in ActiveRecord. It appears that Ecto is even more powerful than ActiveRecord *scopes; you can even exclude “scope” fragments.

Also, since this can get complex and sometimes not work, you can also drop down closer to the database’s native SQL syntax when needed.

Details

Separate construction and query

Extract the query string, then use it later. Pretty powerful. INdeed, the query string doesn’t have to be directly associated w/ the database.

Developing more complicated queries

  1. Now joining tables.

Query Expressions

  • Feels almost like SQL while still being high-level.
  • Almost looks like AREL or scopes in ActiveRecord


Composition


How do I share semantic meanings?


    
  
  • Both query styles are coposable.
    • 
  
  • Queries can be the “subject” of other queries
    • 



The name of a model is actually a query itself.


Hence,


query1 = MyApp.Comment

query2 = from c in query1,
         where: c.votes > 5
MyApp.Repo.all(query2)



(He had a 3rd clause but this gets the point across.)


We can now extract reuable components, name them, and compose them.


Comment
| > comment.popular
| > Comment.for_post(1) |
|                       |




His examples’s typespecs’ names are made up.


    
  
  • A source is the starting point for a query.
    • 



MyApp.Post
MyApp.Post.owned_by(user)


    
  
  • A transformation expands or constains an existing query.
    • 



MyApp.Post.published(query)
MyApp.Comment.for_post(query, posts)


    
  
  • A sink executes a query and results a results
    • 



MyApp.Repo.all(query)
MyApp.Repo.one(query)
MyApp.Repo.paginate(query) # this would be nice sink


Presenter created a package named scribner


Importnan: Piplines are fractal


#### Pagination


    
  
  • Acts as a sink, but is really several smaller pipelines.
    • 



defmodule MyApp.repo do
  def paginate(query, page_number \\ 1) do
    (entries(query, page_number), total_entries(query)}
  end
end


There is an “exclude” clause where you can reject a query phrase in a pre-existing


    
  
  • blog.drewolson.org
    • 
  
  • hex.pm/packages/scrivener
    • 



@drewolson


Q/A


    
  
  • Elm solves the client-side problem the same way as Elixir solves
    server-side programming.
    • 




  


  





      
      

        

          

            









          


          

            

  
    Sign up for free
    to join this conversation on GitHub.
    Already have an account?
    Sign in to comment