Skip to content

Instantly share code, notes, and snippets.

View williamcaban's full-sized avatar

William Caban Babilonia williamcaban

View GitHub Profile
from datasets import load_dataset
# Combine 'question' and 'answer' into a single 'text' field
def combine_qa(local_dataset):
local_dataset['text'] = f"User: {local_dataset['Question']}\nAssistant: {local_dataset['Answer']}"
return local_dataset
####################################################################################
# main
####################################################################################
@williamcaban
williamcaban / dataset_to_ilab.py
Last active July 7, 2024 18:53
Convert a Q&A custom dataset to InstructLab format
#
import sys, json
from pathlib import Path
from datetime import datetime
import pandas as pd
TSTAMP = datetime.now().replace(microsecond=0).isoformat().replace(":", '_')
ILABGEN = "granite-7b-lab-7b-Q4_K_M"+f"_{TSTAMP}"
DEBUG = True
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@williamcaban
williamcaban / Simulating Bare Metal with VirtualBMC.md
Last active May 30, 2024 11:39
Bare Metal IPMI for VMs with Virtual BMC

Simulate Bare-Metal IPMI for VMs in RHEL8 Libvirt

  • Install the Virtual BMC package in the machine to use as the vBMC server. Note1: When using OpenStack repos the python3-virtualbmc RPM might be available. These instrucctions do not use that package.
    pip3 install virtualbmc
    
    • The vBMC server can be running in any machine. When using a remote libvirt server it is recommended to have passwordless authentication from the vBMC server to the libvirt machines.
    • For this document the vBMC Server export vBMCServerIP=192.168.1.13

Using Sushy Tools with Libvirt

  • Add linux user to libvirt group
sudo usermod -a -G libvirt <your-user>
sudo -y install libvirt-devel gcc python3-devel
  • Login and validate user can execute virsh commands
@williamcaban
williamcaban / api-access-from-jumphost.yaml
Last active April 5, 2024 01:17
Example of limiting the access to OpenShift control-plane, Kube API and SSH to control plane nodes using the Ingress Node Firewall Operator
---
apiVersion: ingressnodefirewall.openshift.io/v1alpha1
kind: IngressNodeFirewallConfig
metadata:
name: ingressnodefirewallconfig
namespace: openshift-ingress-node-firewall
spec:
nodeSelector:
node-role.kubernetes.io/control-plane: ""
---
@williamcaban
williamcaban / Export RHV to QCOW2.md
Last active January 11, 2024 11:06
Customizing and Expanding QCOW2 Images

Export RHV/Ovirt VM to qcow2 format

  1. Configure export domain in RHV or Ovirt

  2. Export the VM to the export domain. A new directory with the vdisk uuid will be created and the VM will be exported to it.

    • A sample export /exportfs/export_domain/<vdisk_uuid>/images/<disk_id>
  3. Mount the export domain ito a Linux mcahine and go into the <vdisk_uuid>/images

  4. In the folder there iwll be a file with a uuid representing the <disk_id> and a meta file

  5. Use qemu-img convert to convert to qcow2 format

#!/bin/bash
set -euoE pipefail
# Redfish commands related to Virtual Media.
# Redfish doc reference: https://www.supermicro.com/manuals/other/RedfishRefGuide.pdf
export BMC_ADDRESS=''
export ISO_IMAGE=http://192.168.117.9:8080/ocp4-rwn-1-small.iso
export username_password='Administrator:superuser'

Troubleshooting CPU isolation

  • Identifying cores assigned to a particular container
ID=$(crictl ps --label io.kubernetes.container.name=<name-of-container> -o json | jq -r '.containers[0].id')
PID=$(crictl inspect $ID | jq -r '.info.pid')
taskset -pc $PID 

Troubleshooting

@williamcaban
williamcaban / check-api-latency-and-speed.sh
Created September 17, 2021 12:34
Script to test latency and speed towards OpenShift Kubernetes API Server
# If $USERNAME and $PASSWORD are not defined
# in the environment prompt for them
if [ -z "${USERNAME+x}" ]; then
read -p 'Username: ' USERNAME
fi
if [ -z "${PASSWORD+x}" ]; then
read -sp 'Password: ' PASSWORD
fi
echo -e "\nAttempting to find the K8s API Server and token"