Skip to content

Instantly share code, notes, and snippets.

View imranismail's full-sized avatar

Imran Ismail imranismail

View GitHub Profile
@imranismail
imranismail / README.md
Last active December 15, 2025 23:16
Dynamic Elixir Node Discovery on Kubernetes

Dynamic Elixir Node Discovery on Kubernetes

One way one would do it i to connect nodes together by having the sys.config/vm.args as suggested by chrismccord here

However, when deploying to platform such as Kubernetes or AWS ElasticBeanstalk whereby the cluster is elastic and the IPs that are attached to the nodes are ephemeral, you probably don't want to update sys.config/vm.args everytime you do a deployment now would you.

Now, to solve this one would need to use a Discovery Service such as Consul/etcd/Zookeeper, I opt to use Kubernetes as it's one of the best PaaS for containerized apps available right now and also has a RESTful API for querying resources such as node IPs.

How it's done is that the Iris.Kubernetes process will poll the API endpoint every 5 seconds for new IPs that are available in the cluster and attempt a connection.

@imranismail
imranismail / README.md
Created February 23, 2025 09:20
README.md

Batch No More, Stream Galore: Zero ETL Pipelines from Day 0

In the early stages, many startups rely on traditional batch-processing systems that perform adequately when data volumes are low. However, as organizations grow, these batch pipelines often become inefficient - resulting in slow processing times and complex troubleshooting when issues arise. In many cases, organizations face debates over selecting the optimal batch processing tool, which can consume valuable resources without resolving the underlying performance challenges. Moreover, when failures occur, the process of tracking down issues within a complex network of scheduled jobs can be arduous, sometimes necessitating the reprocessing of entire data batches and causing operational delays.

Recent advancements in the field of Extract, Transform, Load (ETL) have led to the emergence of Zero-ETL strategies. With these developments, streaming data processing is increasingly recognized as the preferred approach from the outset, rather than as a lat

@imranismail
imranismail / README.md
Last active August 1, 2024 23:38
An AWK script that supresses terraform helm_release metadata attributes

Motivation

The terraform helm provider is rather noisy printing unnecessary metadata in the plan output making it hard to review changes. This awk script redacts unnecessary field from the output to make it easier to review.

Usage

terraform plan -out plan.tfplan | awk -f supress_helm_release.awk
terraform show -no-color plan.tfplan | awk -f supress_helm_release.awk
@imranismail
imranismail / autocomplete.php
Last active April 4, 2024 13:15
Laravel And JqueryUI's Autocomplete Plugin
//SearchController.php
public function autocomplete(){
$term = Input::get('term');
$results = array();
$queries = DB::table('users')
->where('first_name', 'LIKE', '%'.$term.'%')
->orWhere('last_name', 'LIKE', '%'.$term.'%')
->take(5)->get();
@imranismail
imranismail / formatDistance.js
Created February 6, 2018 15:23
formatDistance for luxon extracted from date-fns
const FORMAT_DISTANCE_LOCALE = {
lessThanXSeconds: {
one: "'less than a second'",
other: "'less than 's' seconds'"
},
xSeconds: {
one: "s' second'",
other: "s' seconds'"
},
@imranismail
imranismail / README.md
Last active September 19, 2023 01:21
README.md
flowchart BT
    Jedan[Jedan\nChristmas] -->|KomentarA| Dva[Neštonešto]
    Dva --> Tri{Let me think}
    Tri -->|Broj4| Četiri[LeKomentar]
    Tri -->|PremaDva| Pet[Tekst5]
    Tri -->|Tri| Šest[fa:fa-car Car]

Pet[<img src='https://iconscout.com/ms-icon-310x310.png' width='40' /> \nasd]
@imranismail
imranismail / deployment.yaml
Last active August 24, 2023 11:12
Using Kustomize with Terraform
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
spec:
selector:
matchLabels:
app: myapp
template:
metadata:
@imranismail
imranismail / queue.ex
Created April 10, 2017 08:24
Elixir Wrapper for Erlang Queue module
defmodule Queue do
def insert(queue, item), do: :queue.in(item, queue)
def insert_last(queue, item), do: :queue.in_r(item, queue)
def member?(queue, item), do: :queue.member(item, queue)
def filter(queue, fun), do: :queue.filter(fun, queue)
def split(queue, n) do
@imranismail
imranismail / mg.sh
Created March 31, 2023 08:24
Add argocd manifest generation path to all applications
#! /usr/bin/env bash
# This script is used to patch all argocd apps in a cluster
# the patch will add a new annotation to all apps
# the annotation will be used to prevent the app from being refreshed unnecessarily
# the following annotation will be added to all apps
# argocd.argoproj.io/manifest-generate-paths: .
set -euo pipefail
@imranismail
imranismail / params.ex
Last active April 14, 2022 11:08
Ecto as Parameter Validator
defmodule Web.InvalidParamsError do
defexception [:changeset, plug_status: 422]
def message(value) do
"""
Invalid parameters
#{inspect(error_messages(value.changeset))}
"""
end