pavlos

This runs a build for a small elixir (phoenix) project in about 40 seconds by caching as much of the compiled files as possible.

We've been using this for months in multiple projects without any issues. Please ping be if there is any issues with this script and I'll update it.

It should be generic enough to work on any elixir app using mix.

If you have a elixir_buildpack.config, then enable that section in the build script to keep versions in sync!

2016-08-09: Updated to newer Erlang and Elixir and fixed curl command.

pavlos

#!/usr/bin/env python
#
# Shows GOP structure of video file. Useful for checking suitability for HLS and DASH packaging.
# Example:
#
# $ iframe-probe.py myvideo.mp4
# GOP: IPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP 60 CLOSED
# GOP: IPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP 60 CLOSED
# GOP: IPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP 60 CLOSED
# GOP: IPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP 60 CLOSED

pavlos

#!/bin/sh

# sh gifenc.sh input.mp4 output.gif
# Optionally accepts width / height (one or both).

palette="/tmp/palette.png"
filters="fps=15"
if [ ! -z $3 ]; then
  if [ ! -z $4 ]; then
    filters="$filters,scale=$3:$4"

pavlos

Erlang and Named Pipes

The intention of this post is to provide a solution (with examples) to a somewhat uncommon issue in Erlang. I hate searching for answers to the same problems over and over, and I had a hard time finding answers to this particular problem, so I wrote it all down once I figured it out. If one day you decide to read and write data through fifo's (named pipes) and then decide you want to read or write the other end of the pipe from Erlang, this post is for you.

The Problem

I wanted to read and write to a fifo from a C/C++ app and have an Erlang app communicate over the other end of that fifo. Put simply, Erlang doesn't really support what I was trying to do. You cannot just file:open/2 a fifo, or any special device for that matter, in Erlang and expect it to work. This is documented in Erlang's FAQ.

The Solution! ... ???

pavlos

Demonstrate Erlang's 'busy wait' effect on CPU utilization

How?

Just make Erlang to do some light-load task in a bunch of relatively short leaving processes. Calculating PI to 80th digit in batches per 10 procs with short, 8 ms, sleep in-between, to give schedulers a breath space, will do.

Running

Run erl with 8 schedulers and no busy waiting on schedulers at all.

pavlos

In the case you do need to work with a priority in your messages and can't use such a catch-all clause, a smarter way to do it would be to implement a min-heap or use the gb_trees module and dump every received message in it (make sure to put the priority number first in the key so it gets used for sorting the messages). Then you can just search for the smallest or largest element in the data structure according to your needs.

In most cases, this technique should let you receive messages with a priority more efficiently than selective receives. However, it could slow you down if most messages you receive have the highest priority possible. As usual, the trick is to profile and measure before optimizing.

http://jlouisramblings.blogspot.com/2013/01/how-erlang-does-scheduling.html

Toward the operating system, Erlang usually has a thread per core you have in the machine. Each of these threads runs what is known as a scheduler. This is to make sure all cores of the machine can potentially do work for the Erlang

pavlos

defimpl Inspect, for: PID do def inspect(pid, _opts) do
  "~P" <> :erlang.list_to_binary(:erlang.pid_to_list(pid)) end
end


defmodule SigilP do
  defmacro sigil_P(term, modifiers)
  defmacro sigil_P({:<<>>, _line, [string]}, []) when is_binary(string) do
    "<#{string}>" |> String.to_char_list |> :erlang.list_to_pid
  end

pavlos

defmodule Extension do

  defmacro extends(module) do
    # As above...
  end

  defmacro implements(module, protocol: protocol) do
    quote do
      defimpl unquote(protocol), for: unquote(module) do
        import Extension

pavlos

Original tweet size in JSON 2624 Msgpack = 1817 Gzip + Json = 1058 Gzip + Msgpack = 1116 LZ4 + Json = 1628 LZ4 + Msgpack = 1361
Original tweet size in JSON 1863 Msgpack = 1443 Gzip + Json = 0783 Gzip + Msgpack = 0835 LZ4 + Json = 1153 LZ4 + Msgpack = 1040
Original tweet size in JSON 2074 Msgpack = 1670 Gzip + Json = 0842 Gzip + Msgpack = 0894 LZ4 + Json = 1229 LZ4 + Msgpack = 1139
Original tweet size in JSON 2025 Msgpack = 1617 Gzip + Json = 0845 Gzip + Msgpack = 0895 LZ4 + Json = 1238 LZ4 + Msgpack = 1143
Original tweet size in JSON 2069 Msgpack = 1663 Gzip + Json = 0846 Gzip + Msgpack = 0901 LZ4 + Json = 1243 LZ4 + Msgpack = 1164
Original tweet size in JSON 2035 Msgpack = 1634 Gzip + Json = 0852 Gzip + Msgpack = 0907 LZ4 + Json = 1247 LZ4 + Msgpack = 1167
Original tweet size in JSON 1988 Msgpack = 1464 Gzip + Json = 0804 Gzip + Msgpack = 0862 LZ4 + Json = 1220 LZ4 + Msgpack = 1061
Original tweet size in JSON 1910 Msgpack = 1502 Gzip + Json = 0775 Gzip + Msgpack = 0832 LZ4 + Json = 1154 LZ4 + Msgpack = 1060

pavlos

Code.require_file "test_helper.exs", __DIR__

defmodule ProcessTest do
  use ExUnit.Case, async: true

  doctest Process

  test "dictionary" do
    assert Process.put(:foo, :bar) == nil
    assert Process.put(:foo, :baz) == :bar