Let's make some Wavess

finetuning-llms.md

Wavess Guide to Fine-tuning LLMs

Wavess is a play to explore funding opportunities with Astrik in the marketing co-pilot AI B2B service space.

Ahead AI specializes in Machine Learning & AI research and is read by tens of thousands of researchers and practitioners who want to stay ahead in the ever-evolving field.

General Fine-tuning

Resources

Fine-tuning large language models (LLMs) in 2024 Fine-tuning open source large language models (LLMs) Fine tuning pipeline for open-source LLMs [D] Have you tried fine-tuning an open source LLM?

Code

Hackable implementation of state-of-the-art open-source LLMs based on nanoGPT. Supports flash attention, 4-bit and 8-bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training.

Great starter kit to experiment with LLMs and fine-tuning.

Fine tuning Microsoft's Phi-2

Instruction Fine-tuning

Resources

• Difference between Instruction Tuning vs Non Instruction Tuning Large Language Models
• 🧑‍🏫 Instruction Tuning Vol. 1

Papers

• Training language models to follow instructions with human feedback

Reinforcement Learning Fine-Tuning (RLFT)

Research

-OpenChat: Advancing Open-source Language Models with Mixed-Quality Data

Metrics

Popularity and Engagement

• Expecting to be HIP: Hawkes Intensity Processes for Social Media Popularity
• Online Popularity under Promotion: Viral Potential, Forecasting, and the Economics of Time

llms-field-guide.md

The Ultimate LLM Field Guide

Logging & Analytics

DuckDB

• Using Web Server Logs to Answer Product and Business Questions
• Equal Code: DuckDB for logs analysis
• Ad-hoc structured log analysis with SQLite and DuckDB
• KalDB: Log analytics with DuckDB – DuckCon #3 (San Francisco)

Application/Serving

Tools

• THE BACKBONE FOR VERSATILE AI
• AGI for Work

Talks and Sessions

Hosting LLM Apps @ Scale

• Recording

Bending LLMs to Your Will…(and your will is to create structured data) - Haystack (deepset)

• Tuana Celik | Slides | Colab

Challenges in serving LLM Applications at enterprise scale - Titan ML

• Fergus Finn | Slides

From Lab to Life: The Evolution of Jina Embeddings V2 - Jina AI

Isabelle Mohr | Slides

Prompt-Engineering for Open-Source LLMs

• Recording

Technical Product Sales Call (wavess.io) with Astrik + Kirstin

• Recording

wavess-gen-ai.md

Wavess

A play to explore funding opportunities with Astrik in the marketing co-pilot B2B service space.

Ahead AI specializes in Machine Learning & AI research and is read by tens of thousands of researchers and practitioners who want to stay ahead in the ever-evolving field.

Competitors

Blabigo

Reading

Practical Data Privacy

Copy Writer AI

A product to explore re-writing marketing copy based on user input (in order to drive higher engagement) or create full-text marketing copy based on a minimal prompt.

Papers

A Machine Learning Approach for Automated Filling of Categorical Fields in Data Entry Forms

LMCanvas: Object-Oriented Interaction to Personalize Large Language Model-Powered Writing Environments

Leveraging Pre-trained Checkpoints for Sequence Generation Tasks

BERTScore: Evaluating Text Generation with BERT

Distilling the Knowledge of BERT for Text Generation

Learning Neural Templates for Text Generation

Constitutional AI: Harmlessness from AI Feedback

xLSTM: Extended Long Short-Term Memory

Printed

• Attract me to Buy: Advertisement Copywriting Generation with Multimodal Multi-structured Information
• CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities
• RewriteLM: An Instruction-Tuned Large Language Model for Text Rewriting

Resources

• 8 Top Open-Source LLMs for 2024 and Their Uses

• How to rewrite text with ChatGPT/Llama

• What’s new in Llama 2 & how to run it locally

• Jot uses AI to turn product descriptions into high-converting ad copy fast.

• What's the Difference Between LLMs?

• CauseWriter vs Jasper vs Copy AI

• How LLMs are Transforming the Field of Copywriting and Code Generation

• How we use large language models (LLM) to personalise customer communications

• What is Humanloop?

• Understanding Azure Cognitive Services

• LaMini is here: a little giant LLM on your CPU

Code

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

Get started with open source LLMs on a GPU

An implementation of transformer-based language model for sentence rewriting tasks such as summarization, simplification, and grammatical error correction.

AirLLM optimizes inference memory usage, allowing 70B large language models to run inference on a single 4GB GPU card.

Embeddings and Vectors

• Embeddings in Machine Learning: Types, Models, and Best Practices

• Easy to use, Embeddings API for developers.

• Multilingual Information Retrieval Across a Continuum of Languages

• OpenAI vs Open Source LLM Comparison for Document Q&A

• Is openai text-embedding-ada-002 the best embeddings model?

• MTEB: Massive Text Embedding Benchmark

• 📚The Current Best of Universal Word Embeddings and Sentence Embeddings

• Fast, Accurate, Lightweight Python library to make State of the Art Embedding

Lightweight embeddings

• Compact word vectors with Bloom embeddings

• Getting Started With Embeddings

• LadaBERT - Lightweight Adaptation of BERT through Hybrid Model Compression

• Lightweight Composite Re-Ranking for Efficient Keyword Search with BERT

• Learning Neural Templates for Text Generation

• SentenceTransformers Documentation

Vector Databases

• Vespa: Big Data + AI, online.

App UI

Flask-based

• What's the easiest and fastest way to get a nice UI in a Flask app?

• Dash Python User Guide

Streamlit

• Streamlit Extras in Python

• Streamlit-option-menu

Prompting

A guide to prompting Llama 2 Understanding prompts and how to make them work explained like I'm 5