jawond

Prompt Chaining with QwQ, Qwen, o1-mini, Ollama, and LLM

Here we explore prompt chaining with local reasoning models in combination with base models. With shockingly powerful local models like QwQ and Qwen, we can build some powerful prompt chains that let us tap into their capabilities in a immediately useful, local, private, AND free way.

Explore the idea of building prompt chains where the first is a powerful reasoning model that generates a response, and then use a base model to extract the response.

Play with the prompts and models to see what works best for your use cases. Use the o1 series to see how qwq compares.

Setup

• Bun (to run bun run chain.ts ...)

jawond

import argparse
import json
import logging
import os
import re
import shutil
from concurrent.futures import ProcessPoolExecutor, as_completed
from dataclasses import dataclass
from datetime import datetime
from typing import Any, Callable, Dict, List, Literal, Optional, Tuple

jawond

Extracting financial disclosure reports and police blotter narratives using OpenAI's Structured Output

tl;dr this demo shows how to call OpenAI's gpt-4o-mini model, provide it with URL of a screenshot of a document, and extract data that follows a schema you define. The results are pretty solid even with little effort in defining the data — and no effort doing data prep. OpenAI's API could be a cost-efficient tool for large scale data gathering projects involving public documents.

OpenAI announced Structured Outputs for its API, a feature that allows users to specify the fields and schema of extracted data, and guarantees that the JSON output will follow that specification.

For example, given a Congressional financial disclosure report, with assets defined in a table like this:

jawond

# This script will transcribe an audio file (mp3, wav, etc.) to text and then clean the text using a local LLM model.

# GETTING STARTED:
# 1. Install openai
# 2. Git clone a copy of ggerganov/whisper (https://github.com/ggerganov/whisper.cpp)
# 3. Build the whisper binary
# 4. Download the whisper model (largev2 is the most accurate for all languages, though the base model works pretty well for English).
# 5. Install ffmpeg
# 6. Install ollama (https://ollama.com/download)
# 7. Download an LLM model (https://ollama.com/library)

jawond

There are two prompts, that chain together. The first prompt does most of the work, and the second prompt organizes the sections. I found because of the nature of how LLMs write, I couldn't get just one prompt to never jump back and forth in topics.

Prompt 1, which takes as input a raw transcript and generates a structured-text version...

"""# Instructions
A transcript is provided below of a voice memo I recorded as a "note to self". please extract all the points made or thoughts described, and put them in bullet-point form. use nested bullet points to indicate structure, e.g. a top-level bullet for each topic area and sub-bullets underneath. use multi-level nesting as appropriate to organize the thinking logically. use markdown formatting with `*` instead of `-` for bullet points.

DO NOT OMIT ANY POINTS MADE. This is not a summarization task — your only goal is to structure the thoughts there so they are logically organized and easy to read. Be concise because the reader is busy, but again DO NOT omit any

jawond

from typing import Any, Dict, List, Type

import instructor
from anthropic import Anthropic
from config.settings import get_settings
from openai import OpenAI
from pydantic import BaseModel, Field


class LLMFactory:

jawond

Clone the model

This can take a little while:

# Make sure you have git-lfs installed (https://git-lfs.com)
git lfs install

git clone https://huggingface.co/black-forest-labs/FLUX.1-schnell

jawond

"""
A minimal, fast example generating text with Llama 3.1 in MLX.

To run, install the requirements:

    pip install -U mlx transformers fire

Then generate text with:

    python l3min.py "How tall is K2?"

jawond

# --------------------------------------------------------------
# Customer Support Ticket Classification System
# --------------------------------------------------------------

import instructor
from pydantic import BaseModel, Field
from openai import OpenAI
from enum import Enum
from typing import List

jawond

MLX LM with the OpenAI Python Package

1. Install

Install MLX LM and openai:

pip install mlx-lm openai