abasu0713

Deploy BentoML services as server-less functions on Kubernetes using Knative

In this gist we are going to deploy a containerized BentoML service to Kubernetes as a server-less function using Knative.

Prerequisites

1. A BentoML service that you have already locally tested. Refer to this gist for more information on how to create one as an example
2. Containerized the BentoML Service. Refer to this gist for more information on how to containerize existing BentoML services.
3. A Virtual Machine/Bare-metal server with Ubuntu/Debian based OS and NVIDIA CUDA enabled GPU that you can use to deploy Kubernetes and test this in.

I'm doing this on a small dekstop I have at home. This one has a old GTX 1660 with 6GB VRAM. Since the model we are loading is only 600 MB. This system is enough to run our Prompt Engineering service (detailed in step

abasu0713

Containerize BentoML applications for deployment in K8s

In this gist we are simply going to containerize a simple Bento Service with pre-packaged models.

Prerequisites

1. You have already created a Bento Service locally and have tested it. Refer to this Gist if you need to get started on creating Bento Services
2. You have docker installed locally

Step 1: Containerize BentoML service

From the root of the directory containing your Bento File (bentofile.yaml) and the Bento Service (in most cases service.py) run the following command: Refer to official docs for more information.

abasu0713

Using BentoML as an Inference Platform for Machine Learning Models

In this gist we are going to use BentoML to locally serve two Machine Learning models as Services for developmental testing.

Prerequisites

• Any Laptop/Desktop/Cloud VM that is an Ubuntu Jammy or a Debian Bookworm based system with a GPU that is atleast 6GB VRAM in size.

• Nvidia drivers installed. Follow Ubuntu Installation docs or the official Nvidia CUDA installation docs for Debian or other Linux based OS.

• (Optional but recommended) Conda Installed. Refer quick 1 line installation of Miniconda.

  

abasu0713

MongoDB Community Kubernetes Operator on ARM64

This gist aims to provide the shortest path to deploying MongoDB Community Kubernetes Operator on ARM64 machines with some clarifications in response to the following Open Issues about ARM64 support on the official repository:

1. #1514
2. #1420

Prerequisite

• Any ARM64 machine which can run Kubernetes with a Linux Operating System that supports snap daemon. I will be doing this on, you guessed it right, an Orange Pi 5B

You are free to use any Kubernetes Installer of your choice. I am using Microk8s since it's zero-ops and the light

abasu0713

Staggered Continuous Integration using Github Actions

In this gist we are going to look at the simple steps required to build a staggered CI process using Github Actions that generate multi-platform images/packages/artifacts for any application. .

Some typical assumptions

• We have an application that is deployed across 2 environments:

  1. live
  2. preview

• We want to generate artifacts targeted for each environment that we can later use for manual and/or Continuous Deployment. Below represents a directory structure of a sample application

abasu0713

Create S3 access credentials for use on Ceph Object Gateway

Ceph Object Storage user management involves managing users who access the Ceph Object Storage service, not the Ceph Object Gateway itself. To allow end users to interact with Ceph Object Gateway services, create a user along with an access key and secret key. Users can also be organized into Accounts for easier management.

rgw-admin-ops-user with access to all S3 APIs.

# Required to run alias command only 1 time
alias radosgw-admin="mircoceph.radosgw-admin"

abasu0713

Enable S3 Compatible Ceph Object Gateway

You can run this from any node that is part of the Ceph Cluster deployed using Microceph. The node you run the following commands from becomes the host for the Object/Rados Gateway service.

If you are going to use this cluster as a storage layer for your different Kubernetes applications, then I recommend doing this from your Active leader.

You might see an error like the following on a ceph cluster that you had previously tore down. It's just a false negative: Error: failed placing service rgw: failed to add DB record for rgw: failed to record role: This "services" entry already exists

abasu0713

Minimal High Availability Ceph cluster using Microceph in less than 10 min

This gist provides you with the steps to deploy a minimally viable High Availability (HA) Ceph Cluster. It follows the Microceph Multi-node install guide but adds a little more detail to make the deployment simpler.

Architecture

*Minimal HA Ceph cluster

Prerequisites

3 Bare-metal nodes.