Odkazy z přednášky na FF 2023 - Jazykové modely - svět vs. ChatGPT

odkazy-ff-2023-jazykove-modely.md

Prezentace na google drive

https://docs.google.com/presentation/d/1EF85-EMGZEo9CetreshR7SdARJjC4M-c/edit?usp=sharing&ouid=102632222370728552977&rtpof=true&sd=true

ne-akademické články

• turing test

  • https://bigthink.com/the-future/turing-test-imitation-game/
  • https://en.wikipedia.org/wiki/Turing_test

• google lamda

  • https://www.wired.com/story/lamda-sentient-ai-bias-google-blake-lemoine/
  • https://www.washingtonpost.com/technology/2022/06/17/google-ai-lamda-turing-test/
  • https://futurism.com/fired-google-engineer-ai-sentience
  • https://www.washingtonpost.com/technology/2022/06/11/google-ai-lamda-blake-lemoine/
  • https://www.scientificamerican.com/article/google-engineer-claims-ai-chatbot-is-sentient-why-that-matters/
  • https://www.theguardian.com/technology/2022/aug/14/can-artificial-intelligence-ever-be-sentient-googles-new-ai-program-is-raising-questions

• počet vrstev deep NN

  • víc než 3 http://wiki.pathmind.com/neural-network#:~:text=More%20than%20three%20layers%20(including,qualifies%20as%20%E2%80%9Cdeep%E2%80%9D%20learning.
  • 3 a víc https://stats.stackexchange.com/questions/229619/minimum-number-of-layers-in-a-deep-neural-network ale včetně vstupní vrstvy je to taky víc než 3

• awesome llms https://github.com/Hannibal046/Awesome-LLM

• word2vec

  • https://en.wikipedia.org/wiki/Word2vec
  • http://www.mlguru.com/cs/2015/03/
  • http://www.fit.vutbr.cz/~imikolov/rnnlm/thesis.pdf

• rlhf

  • https://huyenchip.com/2023/05/02/rlhf.html
  • https://www.reddit.com/r/reinforcementlearning/comments/zqfw7r/why_cant_we_do_supervised_learning_in_step_3_of/
  • http://new-savanna.blogspot.com/2023/02/chatgpt-goes-wild-is-solidgoldmagikarp.html

• transformery https://github.com/gaussalgo/L2L_MLPrague23/blob/main/presentation.pdf

• tokenizace

  • https://blog.quickchat.ai/post/tokens-entropy-question/
  • https://platform.openai.com/tokenizer
  • https://github.com/openai/tiktoken
  • https://openaipublic.blob.core.windows.net/encodings/cl100k_base.tiktoken

• bugy u tokenizeru

  • solidgoldmagikarp https://www.lesswrong.com/posts/aPeJE8bSo6rAFoLqg/solidgoldmagikarp-plus-prompt-generation
  • pokračování https://www.lesswrong.com/posts/Ya9LzwEbfaAMY8ABo/solidgoldmagikarp-ii-technical-details-and-more-recent
  • https://www.reddit.com/r/countingcirclejerk/comments/11ng6oa/usolidgoldmagikarp_will_be_forever_immortalized/
  • https://www.reddit.com/r/artbn_bots/wiki/rc_hoc_500/
  • https://github.com/artbn/RC/blob/master/hoc.txt

• vysvětlení transformerů

  • http://jalammar.github.io/illustrated-transformer/
  • https://towardsdatascience.com/transformers-explained-visually-not-just-how-but-why-they-work-so-well-d840bd61a9d3

• red pajamas 7B

  • online https://api.together.xyz/open-chat
  • blogpost https://www.together.xyz/blog/redpajama-7b

• superHOT

  • https://kaiokendev.github.io/context
  • https://kaiokendev.github.io/til

• openai modely https://platform.openai.com/docs/models/gpt-4

• prompt hacking https://learnprompting.org/docs/category/-prompt-hacking

  • prompt injection bingu https://greshake.github.io/

• prompt leaking

  • https://matt-rickard.com/a-list-of-leaked-system-prompts
  • https://arstechnica.com/information-technology/2023/02/ai-powered-bing-chat-spills-its-secrets-via-prompt-injection-attack/

• capture the flag hry

  • https://gandalf.lakera.ai/
  • https://pihack.stratosphereips.org/

• hádání filmů z emoji

  • https://www.reddit.com/r/WebGames/comments/136iqpn/puzzlemoji_i_made_a_single_player_emoji/
  • https://flowgpt.com/prompt/qj3XLSqxRq7dLWGpSYJCY

• falcon

  • https://falconllm.tii.ae/index.php
  • https://huggingface.co/tiiuae/falcon-40b
  • online demo: https://huggingface.co/blog/falcon

• google no moat https://www.economist.com/leaders/2023/05/11/what-does-a-leaked-google-memo-reveal-about-the-future-of-ai

• gpt3 http://jalammar.github.io/how-gpt3-works-visualizations-animations/

• gpt4 parameters https://the-decoder.com/gpt-4-has-a-trillion-parameters/

• deepmind chinchilla https://www.deepmind.com/publications/an-empirical-analysis-of-compute-optimal-large-language-model-training

• llama

  • https://the-decoder.com/metas-llama-language-model-shows-that-parameters-are-not-everything/
  • llama stable diffusion moment https://simonwillison.net/2023/Mar/11/llama/
  • gpu requirements https://www.reddit.com/r/LocalLLaMA/comments/11o6o3f/how_to_install_llama_8bit_and_4bit/

• openllama https://github.com/openlm-research/open_llama

• huggingface open llm leaderboard https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard

• hellaswag

  • paper https://arxiv.org/pdf/1905.07830.pdf
  • validační data https://raw.githubusercontent.com/rowanz/hellaswag/master/data/hellaswag_val.jsonl

• GPT4Chan

  • video s vysvěltením https://www.youtube.com/watch?v=efPrtcLdcdM&ab_channel=YannicKilcher
  • huggingface https://huggingface.co/ykilcher/gpt-4chan
  • web modelu https://gpt-4chan.com/
  • zdroják https://github.com/yk/gpt-4chan-public
  • 1bit i 32bit modely dostupné přes torrent
    • https://archive.org/details/gpt4chan_model_float16
    • https://archive.org/download/gpt4chan_model_float16
    • https://archive.org/details/gpt4chan_model
    • https://archive.org/download/gpt4chan_model
  • dataset raiders of the lost kek https://zenodo.org/record/3606810#.YpjGgexByDU

• 4bit operace https://astojanov.github.io/projects/clover/

• autogpt https://docs.agpt.co/

  • zdroják https://github.com/Significant-Gravitas/Auto-GPT

• agentgpt https://agentgpt.reworkd.ai/

• langchain https://python.langchain.com/docs/get_started/introduction.html

  • meetup s ukázkami langchainu https://www.youtube.com/watch?v=fl1jluGawzQ&ab_channel=Applifting

• promptsource https://github.com/bigscience-workshop/promptsource

• deepmind chinchilla https://www.deepmind.com/publications/an-empirical-analysis-of-compute-optimal-large-language-model-training

• právník chatgpt https://www.cbsnews.com/news/chatgpt-judge-fines-lawyers-who-used-ai/

• žaloba proti microsoftu https://www.theverge.com/2023/1/28/23575919/microsoft-openai-github-dismiss-copilot-ai-copyright-lawsuit

• character.ai https://beta.character.ai/

• alpaca https://crfm.stanford.edu/2023/03/13/alpaca.html

  • zdroják https://github.com/tatsu-lab/stanford_alpaca

• vicuna https://lmsys.org/blog/2023-03-30-vicuna/

• guanaco https://guanaco-model.github.io/

• export z character.ai na on-prem

  • https://np.reddit.com/r/PygmalionAI/comments/10h37u4/my_basic_guide_to_pygmalion_for_begginers/
  • https://www.reddit.com/r/CharacterAi_NSFW/comments/112d104/converting_characterai_to_tavern_ai/
  • https://github.com/0x000011b/characterai-dumper

papery

• chain of thought https://arxiv.org/pdf/2201.11903.pdf
• position interpolation https://arxiv.org/pdf/2306.15595.pdf
• lora https://arxiv.org/pdf/2106.09685.pdf
  • zroják https://github.com/microsoft/LoRA
• tree of thoughts https://arxiv.org/pdf/2305.10601.pdf

PygmalionAI

• hlavní stránka https://pygmalion.chat/
• discord https://discord.gg/pygmalionai
• dokumentace https://docs.pygmalion.chat/
• neplést s http://pygmalion.ai/, to je něco úplně jiného
• přehled potřebné RAM podle velikosti modelu https://rentry.co/pygmalion-ai
• jeden z návodů na zprovoznění: https://rentry.org/pygmalion-local
  • není úplně aktuální, je třeba nainstalovat u KoboldAI všechny závislosti pomocí install_requirements.bat puštěného jako admin.
• rozcestník na některé guides https://rentry.org/wAIfu_Pygmalion_Guides

web UI

• Tavern AI https://github.com/TavernAI/TavernAI
• oobabooga web ui
  • github https://github.com/oobabooga/text-generation-webui/
  • reddit https://www.reddit.com/r/oobaboogazz/
  • discord https://discord.gg/jwZCF2dPQN
• KoboldAI pro 4bit modely https://github.com/0cc4m/KoboldAI
• doporučený fork KoboldAI https://github.com/henk717/KoboldAI/releases
• silly tavern https://docs.sillytavern.app/
  • zdroják https://github.com/SillyTavern/SillyTavern
• llama cpp https://github.com/ggerganov/llama.cpp
• kobold cpp https://github.com/LostRuins/koboldcpp

ostatní

• galerie promptů:
  • https://prompts.aidg.club/
  • https://aetherroom.club/
  • https://github.com/cr4bl1fe/AIDSPrompts
• seznam rolí/person https://rentry.org/wAIfu_Bot_List_Final#generation-2
• 4chan rozcestník na vtuber chatboty https://boards.4channel.org/vt/thread/52434778
• rozcestník na hodně věcí https://rentry.org/diy-virtualwaifu
• aids wiki https://aids.miraheze.org/wiki/Main_Page
• tips https://rentry.org/pygtips
• koboldai discord https://discord.gg/UCyXV7NssH
• diskuze o instalaci na pygmalionAI discordu https://discord.com/channels/1066323170866495609/1066342105837998120/1125408856336506900
• další tutoriál na zprovoznění https://rentry.org/ipfub
• jailbreaky pro tavern ai https://rentry.org/GPTJailbreakPrompting