Skip to content

Instantly share code, notes, and snippets.

@wout
Last active February 19, 2022 14:56
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save wout/82895bb1cceb105588edb951b3c22750 to your computer and use it in GitHub Desktop.
Save wout/82895bb1cceb105588edb951b3c22750 to your computer and use it in GitHub Desktop.

DendroRithms App Structure

Introduction

The app is written in Crystal using Lucky Framework (https://luckyframework.org/). It's a batteries-included web framework that makes heavy use of compile-time checks. As a result, it's very helpful in tracking errors and catching bugs even before the app reaches runtime.

Lucky is also extremely fast compared to similar frameworks written in other programming languages. The following benchmark illustrates it well:

https://www.techempower.com/benchmarks/#section=data-r20&hw=ph&test=plaintext&l=zdk2rj-sf&f=zik06v-zik073-zik0zj-zik0zj-zik0zj-zik0zj-zik0zj-zik0zj-zik0zj-zik0zj-yelngf-cn3

Structure

The app itself consists of six major components.

1. Web App

This is the user-facing application containing both frontend and backend code. The app is mainly server-side code with a thin layer of JavaScript and CSS. The JavaScript part is written in Stimulus.js (https://stimulus.hotwired.dev/), an HTML-first approach to building frontends. The CSS part is written using the principles of Every Layout (https://every-layout.dev/).

2. Background jobs

Eight background jobs are running periodically, most of them every 10 seconds. They are classified into four categories.

2.1. bookkeeper

This category has four jobs running in the background, all in the same process:

  • fetch blockchain load: pulls the block sizes from Blockfrost and caches the calculated load in a database table
  • fetch payments: queries the blockchain to pull in individual UTXOs with their outputs and translates transaction ids to buyer addresses using the Blockfrost API
  • assign payments: assigns payments to nfts based on the amount
  • evaluate timeouts: checks if the timeout margin for an NFT has been reached and marks payments as refundable if their respective NFTs have reached that threshold

As those jobs are all relatively lightweight, there's only one process running all of them.

2.2. generator

Generating a Dendro takes about 15 seconds, and while the job is running, all other jobs would be held up. That's why this job uses two dedicated processes. This job comes into action when an NFT is marked as paid.

2.3. pinner

When the generator is ready, the pinner takes over. But before pinning, the PNG files are optimised. This process takes up to 45 seconds but reduces the file size by about 40%. When done optimising, the files are pinned to Infura's IPFS. Similar to the generator, this job has two dedicated processes.

2.4. minter

When the pinner is ready, the minter will generate the metadata and mint the NFT.

Locking mechanism

The resource and time-intensive jobs all have a simple locking mechanism using timestamps in the database. For example, the generator has two timestamps called started_generating_at and finished_generating_at. That way, multiple instances of the same job can be invoked, even on different machines, without risking double output.

Job management

Systemd service files represent all job roles. So all those jobs are booted from the same compiled binary and assigned their roles using environment variables.

3. PostgreSQL database

The app is backed by a Postgres database, allowing both relational and document database structures (JSON).

4. Redis database

The Redis backend is used by the background job runner to time and track the background jobs.

5. Cardano Node

Used to fetch UTXOs and mint the NFTs.

6. Blockfrost API

The Blockfrost API is mainly used to fetch data missing from the cardano-cli.

Deployment

The whole app, including the web part and the background jobs, is contained in one Github repo. Because all individual parts of the app are managed and configured by systemd, deploying the app is done with one command.

Missing parts

Initially, I planned to implement automatic refunds and withdrawals. In the end, I did not because of time constraints. And in hindsight, it wouldn't have been necessary because I only did one refund since the second launch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment