Skip to content

Instantly share code, notes, and snippets.

@TACIXAT

TACIXAT/dl-quickstart.md

Created November 19, 2023 16:38
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save TACIXAT/9d1b305d134bccf57f2ab059dea6a66d to your computer and use it in GitHub Desktop.
Save TACIXAT/9d1b305d134bccf57f2ab059dea6a66d to your computer and use it in GitHub Desktop.
Download ZIP
Deep Learning "Quick" Start
Raw
dl-quickstart.md

Deep Learning "Quick" Start

Positioning - The industry is extremely young. Not a lot of companies are hiring for these things right now. If you can't find work right now in these areas, network and position yourself as the person who does the thing. When someone needs the thing, they will come talk to you.

Innovation - The industry is so young and there is a ton of opportunity. Training a model is not particularly difficult. Apply these tools seriously in your domain.

Intro Courses

I would recommend trying both of these and choosing one. Two different approaches that work for different people.

Deeplearning.ai

https://www.deeplearning.ai/courses/deep-learning-specialization/

This course is a bottom up approach, starts with fundamentals (matrix math) and builds up from there.

Fast.ai

https://www.fast.ai/

Top down course that gets you working quickly but leaves some questions unanswered at the start.

Intermediate Course

Karpathy - Neural Networks: Zero to Hero

https://karpathy.ai/zero-to-hero.html

There are a number of courses out there that are just someone really good livecoding from scratch on an advanced topic. This is in that spirit. It is a course by a really good educator taking you from the basics to some modern architectures.

Applications

The point here is to do some projects. Get working on stuff then apply it to a domain you are familiar with. The previous courses will teach you how stuff works, but you need to grab the tools and understand how to use them. You don't need to do all of this, just the areas you're interested working in.

Language Models

  • Use ChatGPT
  • Use OpenAI's API
  • Create a GPT (OpenAI)
    • Track your prompt changes
    • What affects your prompts?
  • Run local models
    • Whisper
    • text-generation-webui
    • HuggingFace
    • Llama.cpp
    • How big of a model can your GPU handle?
    • How fast are the different models on your GPU?
  • Try different modalities
    • Untuned model
    • Instruct model
    • Chat modality
    • Code modality
  • Quantize a model
  • Implement a system for retrieval augmented generation
  • Finetune a model
    • Base
    • LoRA
    • QLoRA

Vision

  • Diffusions models
    • Use MidJourney
    • Use Dalle 3
  • Run stable diffusion locally
    • Run a LoRA
  • Run YOLO on some video
  • Build a photo search using OpenCLIP
  • Use segment anything for segmentation
  • Finetune YOLO on a new class
  • Finetune stable diffusion
  • Setup Frigate on your cameras

General

  • Implement a vector store
  • Use pgvector
  • Run a cloud model
    • Understand cloud costs
  • Build a dataset
  • Build a LAION search (text / image)
    • Embeddings!
  • Reinforcement learning
    • RL?F
    • What can you get feedback from? (besides humans)

Datasets

  • Case studies
  • How would you build a dataset?
  • Label formats
  • Labeling tools
  • Hire some labelers
    • Mechanical Turk
    • Craigslist
  • Automate labeling
  • Evol-instruct
  • Back translation
  • Hosting / Distribution
    • S3 costs
    • R2 costs
    • Torrents
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment