andrewssobral

andrewssobral

# This is a modified version of TRL's `SFTTrainer` example (https://github.com/huggingface/trl/blob/main/examples/scripts/sft_trainer.py), 
# adapted to run with DeepSpeed ZeRO-3 and Mistral-7B-V1.0. The settings below were run on 1 node of 8 x A100 (80GB) GPUs.
#
# Usage:
#   - Install the latest transformers & accelerate versions: `pip install -U transformers accelerate`
#   - Install deepspeed: `pip install deepspeed==0.9.5`
#   - Install TRL from main: pip install git+https://github.com/huggingface/trl.git
#   - Clone the repo: git clone github.com/huggingface/trl.git
#   - Copy this Gist into trl/examples/scripts
#   - Run from root of trl repo with: accelerate launch --config_file=examples/accelerate_configs/deepspeed_zero3.yaml --gradient_accumulation_steps 8 examples/scripts/sft_trainer.py  

andrewssobral

docker run \
--rm \
-it \
-e ONNXRUNTIME_REPO=https://github.com/microsoft/onnxruntime \
-e ONNXRUNTIME_COMMIT=v1.15.1 \
-e BUILD_CONFIG=Release \
-e CMAKE_VERSION=3.26.4 \
-e CPU_ARCHITECTURE=$(uname -m) \
-e CUDA_ARCHITECTURES="70;75;80;86" \
-v /usr/lib/aarch64-linux-gnu/tegra:/usr/lib/aarch64-linux-gnu/tegra:ro \

andrewssobral

# Clone llama.cpp
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp

# Build it
LLAMA_METAL=1 make

# Download model
export MODEL=llama-2-13b-chat.ggmlv3.q4_0.bin
wget "https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML/resolve/main/${MODEL}"

andrewssobral

tmux Cheat Sheet

Table of Contents

• General Usage
• Shortcuts
• Commands
• Scripting tmux
• Configuring tmux

andrewssobral

import openai
import asyncio
from typing import Any

async def dispatch_openai_requests(
    messages_list: list[list[dict[str,Any]]],
    model: str,
    temperature: float,
    max_tokens: int,
    top_p: float,

andrewssobral

X11 forwarding on macOS and docker

A quick guide on how to setup X11 forwarding on macOS when using docker containers requiring a DISPLAY. Works on both Intel and M1 macs!

This guide was tested on:

• macOS Catalina 10.15.4
• docker desktop 2.2.0.5 (43884) - stable release
• XQuartz 2.7.11 (xorg-server 1.18.4)
• Macbook Pro (Intel)

andrewssobral

1. Copy m3u8 link

2. Run command

echo "Enter m3u8 link:";read link;echo "Enter output filename:";read filename;ffmpeg -i "$link" -bsf:a aac_adtstoasc -vcodec copy -c copy -crf 50 $filename.mp4

andrewssobral

Tips and Tricks for Jetson AGX Xavier

This document contains useful software and links to documentation that we have found to be useful when working with Jetson AGX Xavier.

Copied from: https://git.its.aau.dk/WW82ZE/docs_xavier/src/branch/master

Links

Nvidia Developer forum Jetson AGX Xavier topic

• https://forums.developer.nvidia.com/c/agx-autonomous-machines/jetson-embedded-systems/jetson-agx-xavier/75

Jetson Community Projects

andrewssobral

# https://doc.ubuntu-fr.org/wakeonlan

# check if your target pc is wake-on-lan enabled:
sudo ethtool enp8s0  | egrep "^[[:blank:]]*Wake-on: (g|d)"
# you should have:
# Wake-on: g

# get target pc mac address
ifconfig enp8s0