ityonemo

A half-hour to learn Zig

This is inspired by https://fasterthanli.me/blog/2020/a-half-hour-to-learn-rust/

Basics

the command zig run my_code.zig will compile and immediately run your Zig program. Each of these cells contains a zig program that you can try to run (some of them contain compile-time errors that you can comment out to play with)

ityonemo

We're currently using ziglang in-prod at https://positron.ai.

Uses the zigler package to interface with elixir for robust and resilient layer on top of unreliable and pre-alpha grade hardware (talk coming soon!).

We heavily use AVX-512 instructions for token picking. http://antirez.com/news/142 and we use a highly optimized sorter in place of what you would see in that code (np.argsort).

The plan is to move away from shipping fp32 values at the interface to shipping bf16 values, so bf16 support at the type-level in zig would be amazing. It seems like the ML industry has settled on

ityonemo

const std = @import("std");
const posix = std.posix;
const numaif = @cImport({
  @cInclude("numaif.h");
  @cInclude("errno.h");
});

extern threadlocal var errno: c_int;

fn numa_alloc(size: usize, node: ?u1) ![]align(std.mem.page_size) u8 {

ityonemo

Mix.install([
  {:zigler, "~> 0.13"}
])

Logger.configure(level: :warning)

defmodule Main do
  use Zig,
    otp_app: :zigler,
    c: [link_lib: {:system, "curl"}],

ityonemo

:net_kernel.start([:"main@127.0.0.1"])
:erlang.set_cookie(:cook)
{:ok, peer, peername} =
   :peer.start(%{connection: 0, name: :peer, host: ~C'127.0.0.1'})
:peer.call(peer, :erlang, :set_cookie, [:cook])
Node.connect(peername)

# add code paths
:rpc.call(peername, :code, :add_paths, [:code.get_path()])

ityonemo

defmodule RockE.DynamicImprover do
  alias OpenAI

  require EEx
  EEx.function_from_string(:defp, :prompt, """
  <% assigns = Map.from_struct(round) %>
  Hi ChatGPT!  I'm trying to prompt ChatGPT 3.5 to perform a function call.
  However, the function call often makes mistakes in the passed argument:

  The current accuracy is <%= @accuracy %> I'd love to see the accuracy go up to 90%.

ityonemo

defmodule NxExt do
  import Nx.Defn

  @bitshift Nx.tensor([[1], [16]], type: {:u, 8})

  @doc """
  Takes an N-vector of floats (arbitrarily typed) and converts it into 4-bit gptq, which has
  a range of -8..-7.  Should be compacted into two "floats" per byte, with the lower indexed
  value in the less significant nybble 
  

ityonemo

Elixir Guide

This guide is a quick set of guidelines for reading or writing elixir code. It's designed to get you through stuff that is different from Elixir relative to other programming languages you might be used to, and which you might miss while learning elixir.

Up and running

Linux

ityonemo

defmodule Aaa do
  # terminal condition.  If we've gotten through the whole pattern we're done.
  def matches(<<>>, <<>>), do: true

  # If the first character matches and has a question mark we must make an
  # epsilon transition.  We have to try both "universes".  In one "universe"
  # we proceed as if we consumed the optional match, and the other we proceed
  # as if we didn't.  The success of either means the match succeeds.  This
  # means for this branch we don't get TCO =( but this should be relatively rare.
  def matches(<<char, ??, rest1 :: binary>>, string = <<char, rest2 :: binary>>) do

ityonemo

#include <stdio.h>
#include <string.h>
#include <stdint.h>

typedef uint8_t minifloat;

/*  
    minifloat variable: sign bit, then three bits of exponent, and four bits of 
    mantissa.  Like the 1:4:3 minifloat, but instead with the exponent as the
    least significant bits.