dougbtv

Mistral-Small-4-119B Howto

Build Info

Field	Value
Model	mistralai/Mistral-Small-4-119B-2603
Image	quay.io/vllm/rhaiis-early-access:mistral-4-small
Build Run	24369571413
nm-cicd branch	doug/mistral-4-small

dougbtv

You had ONE JOB -- A Kubernetes job.

Let's take a look at how Kubernetes jobs are crafted. I had been jamming some kind of work-around shell scripts in the entrypoint* for some containers in the vnf-asterisk project that Leif and I have been working on. And that's not perfect when we can use Kubernetes jobs, or in their new parlance, "run to completion finite workloads" (I'll stick to calling them "jobs"). They're one-shot containers that do one thing, and then end (sort of like a "oneshot" of systemd units, at least how we'll use them today). I like the idea of using them to complete some service discovery for me when other pods are coming up. Today we'll fire up a pod, and spin up a job to discover that pod (by querying the API for info about it), and put info into etcd. Let's get the job done.

This post also exists as a [gist on github](https

dougbtv

Clearing IP allocations manually with Whereabouts

This outlines a process for clearing IP address allocations with Whereabouts manually. This clears all allocations, you could be more surgical about it, however, this is efficient if it's possible.

NOTE I have another procedure somewhere which has fancy bash commands to make this easier, and is fully tested, however, in theory this "should just work" (you've heard that before)

Overview

1. Stop all pods which use Multus + Whereabouts (if possible)
2. Clear IP allocations

dougbtv

# Set ip_forward to 1
/sbin/sysctl -w net.ipv4.ip_forward=1

# Install docker (plus a handy wget)
yum install -y docker wget

# Setup docker to allow an "insecure" registry.
sed -i -e "s|\# INSECURE_REGISTRY='--insecure-registry'|INSECURE_REGISTRY='--insecure-registry 172.30.0.0/16'|" /etc/sysconfig/docker

# Start and enable docker.

dougbtv

Install Helm with a specific service account for tiller...

curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get > get_helm.sh
chmod 0700 get_helm.sh 
./get_helm.sh 
kubectl create serviceaccount --namespace kube-system tiller
kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'      
helm init --service-account tiller --upgrade

dougbtv

Running LTX-2 Video Generation with vLLM-Omni and ComfyUI

This guide shows you how to run LTX-2 video generation (text-to-video and image-to-video) using vLLM-Omni as the inference backend and ComfyUI as the frontend.

Background

LTX-2 is a powerful video generation model from Lightricks that supports both text-to-video (T2V) and image-to-video (I2V) generation with audio synthesis.

Resources:

• LTX-2 GitHub: https://github.com/Lightricks/LTX-2 - Python stack for inference and LoRA training, model links

dougbtv

RHAII Preview: Voxtral Realtime

This guide covers running and trying out the Red Hat AI Inference Server to serve Mistral Voxtral-Mini-4B-Realtime-2602 model, powered by vLLM.

You can find the Voxtral Mini model card @ https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602

From the model card:

Voxtral Mini 4B Realtime 2602 is a multilingual, realtime speech-transcription model and among the first open-source solutions to achieve accuracy comparable to offline systems with a delay of <500ms. It supports 13 languages and outperforms existing open-source baselines across a range of tasks, making it ideal for applications like voice assistants and live subtitling.

dougbtv

RHAIIS Preview: NVIDIA Nemotron v3 (Nano 30B-A3B) on Red Hat AI Inference Server

This is a technical quick-start gist for the latest Red Hat AI Inference Server (RHAIIS) preview image, featuring NVIDIA Nemotron v3 Nano 30B-A3B models on vLLM.

Preview image tag (this release):

• registry.redhat.io/rhaiis-preview/vllm-cuda-rhel9:nvidia-nemotron-v3

Upstream model family (Hugging Face):

• nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16
• nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16

dougbtv

Using this dummy CNI script...

Pay attention to the cniresult() routine, which returns... two interfaces.

#!/usr/bin/env bash

# DEBUG=true
# LOGFILE=/tmp/seamless.log

dougbtv

Enable the reconciler...

oc edit networks.operator.openshift.io cluster and add the additionalNetworks section like:

  additionalNetworks:
  - name: whereabouts-shim
    namespace: openshift-multus
    rawCNIConfig: |-
      {