Skip to content

Instantly share code, notes, and snippets.

@a1mzone
Created January 29, 2024 19:51
Show Gist options
  • Save a1mzone/2da6a8b9c19c1a0debfc68a9d1c48844 to your computer and use it in GitHub Desktop.
Save a1mzone/2da6a8b9c19c1a0debfc68a9d1c48844 to your computer and use it in GitHub Desktop.
Tech Stack

Develop

Java - Spring Framework (1st release was in 2002)
Relational DB - PostgreSQL (Started in 1986, most populat open source DB with several TB deployments, Yahoo in fact had a 2PB deployment in 2008)
Mobile - React / Flutter / Cordova - maybe Xamarin

Big Data

Hadoop (2002, based on Google File System paper, in 2007, Yahoo successfully tests Hadoop on a 1000 node cluster, a year later Apache Software Foundation successfully tested a 4000 node cluster)
Accumullo (Based on Google Bigtable, created in 2008 by US National Security Agency)
Spark (Research Project in 2009, 80% of Fortune 500 use Spark with over 2,000 contributors from industry to academia)
Spark Libraries - GeoMesa + Sedona

ETL

NiFi (Niagra Files, 2006 developed by US NSA, now also commercially available from Hortonworks/Cloudera)

Streaming

Kafka (2011 LinkedIn, processes over 7 trillion messages a day in 2019)
Flink ()
Beam (Unified Model for Flink & Spark)

ML / AI / Statistical Models

Images - YOLO (other options available)
Keywords - Spark
Human Insights / Profiles

Bot

LLM - LLaMA, BLOOM or BERT

Reporting

Druid (Metamarkets, 2011 to power their analytics product, adopted by many eBay, Netflix, Yahoo, Cisco, Twiiter, Airbnb etc)
Presto / Trino (Facebook, 2012 to run queries against 300PB of Hadoop data)

Routing & Maps

Openstreetmap or Commercial Solution
pgRouting
OSRM (Openstreetmap Routing Machine)

Billing

Lago (Open Source w/ commercial option metering and usage-based billing engine)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment