Skip to content

Instantly share code, notes, and snippets.

@konijn

konijn/llm_journey.txt

Last active November 26, 2023 16:40
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save konijn/da37cf189c776997657c8da94c8a6733 to your computer and use it in GitHub Desktop.
Save konijn/da37cf189c776997657c8da94c8a6733 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
llm_journey.txt
25 Nov 2023
-----------
Realized I lost track of my last model??
Realized I need to take notes ;)
Downloaded and installed https://github.com/RahulSChand/gpu_poor
Name RTX-2060
bandwidth 336 GB
compute 13 TFlops/s
memory 6 GB
Open LLM leaderboard
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
Ollama, easy quick installer for models
https://ollama.ai
26 Nov 2023
-----------
https://github.com/fastai/fastbook
Todo
----
Check out https://ollama.ai/fakezeta/neural-chat-7b-v3-1
Colab
-----
data_table.enable_dataframe_formatter()
Notes to self
-------------
DROP -> Deep reading over paraphrases
GSM8K -> Grade School Math 8K
Winogrand -> a large-scale dataset of pronoun resolution problems
TruthfulQA -> a benchmark dataset that measures the truthfulness of language models
MMLU -? 10,000 multiple-choice questions that test the ability to understand the meaning of words & sentences in context.
HellaSwag -> This tests common-sense reasoning capabilities. 2500 multiple-choice questions that require inferences about the world based on common-sense knowledge.
ARC -> Evaluates question-answering abilities. 7,787 multiple-choice questions that test questions based on a given context
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment