🤯 zART Microservices: With ML, Blockchain, and IoT
This brief aims to create a product with the following requirements:
- Web Client
- Mobile Client
- Recommendation Engine
- Crypto Based Payment Processing
- Managed IoT Service
- Selfhosted Infrastructure
This post also proposes the tRPC gateway pattern. It leverages trpc's seamless communication with expo and nextjs. It also acts as gateway for microservices communicating via gRPC. Figure 1 below is the generalized high level overview
Figure 1 - The zART Gateway Pattern
For this example we will be building a store called Ethos. This store caters to "ethical tech". This is defined as concern for privacy, the environment and/or openness. In addition to selling smart home devices, customers will be given the option to control devices via a dashboard. The store only accepts payments via cryptocurrencies. A recommendation engine will weigh customer purchases using the aforementioned factors of privacy, openness and environment. Lastly, an in-house presence detection device will be developed.
Note: The following does not include a guest checkout flow, which would be created for any ecommerce product.
When user visits ethos.cx or downloads app
And creates an account
Then an partial database entry is created for the user
When the user clicks the link in the confirmation email
Then the full entry is completed with a confirmation flag set to true
When user logs in
And buys a device using bitcoin
Then the database creates a relation between user and device
When user logs with the same store credentials into iot.ethos.cx/app
Then user manages purchased device
When user goes back to ethos.cx/app
Then user gets recommendations based on previous purchasesFrom the specification above we can see that we will need an identity provider (Cognito, Auth0), and email server (sendgrid, mailgun), payment processor (stripe, plaid) and an MQTT broker (AWS IoT, Hive). This post aims to be vendor-agnostic therefore we will be using opensource selfhosted infrastructure. We will be using the following infrastructure:
- Identity Provider: keycloak
- Mail Server: postal
- Payment processor: btcpay
- MQTT Broker: emqx
- Search Engine: meilisearch
Figure 2 below is a lower level diagram that renames for components for our domain, as well as adds necessary infrastructure.
Figure 2 - Lower Level Domain Diagram
- Code first instead of schema first
- No boilerplate. Always strive to use opensource libraries, infrastructure, and templates.
- Store in JSON for infrequently changing data. Store in relational database for frequently changing data.
- Authorizer
- Execute/CanExecute
Will will be using the Chicago (classical) School of software testing. More specifically, we will adhere to the principles, practices and patterns in the book Unit Testing: Principles, Practices Patterns by Vladimir Khorikov. While this also applies to UI testing, we will also adhere to the complimentary principles in the testing library docs. There will be an e2e flag throughout all the codebases. Once flipped, all the integration tests become e2e test and use the actual out-of-process dependencies. This will only be done nightly on the development branch.
Development order will be treated like a dependency graph. Where the nodes are in the following order:
- Set up the infrastructure
- Expo UI
- NextJS UI
- Mobile
- Desktop
- tRPC Gateway
- Crypto Manager (From the story above after login, the user would buy something)
- Device Manager
- Recommendation Engine
- Presence Detector
A passing integration test of the Main User Story is required for every step. Also, every step will mock out its unimplemented dependencies during the integration test.
- Expo UI: Mocks out the tRPC gateway and keycloak
- NextJS UI: Mock out the tRPC gateway and keycloak
- tRPC Gateway: Mocks out gRPC microservices
- Crypto Manager: Mocks out tRPC gateway and btcpay
- Recommendation Engine: Mocks out tRPC gateway
- Device Manager: Mocks out tRPC and emqx
- Presence Detector: Mocks out emqx