talk_padova.md

Visualization of self-attention maps in vision

https://epfml.github.io/attention-cnn

BertViz: Visualization of attention in NLP models

https://github.com/jessevig/bertviz

Visualization of RNNs

https://distill.pub/2019/memorization-in-rnns

Peter Bloem blog on Transformers

https://peterbloem.nl/blog/transformers

Harvard NLP on Transformers

https://nlp.seas.harvard.edu/2018/04/03/attention.html

Explained Transformers

https://e2eml.school/transformers.html

Loss landscapes artistic visualization

https://losslandscape.com/

Dylan Patek blog on semiconductors industry and AI

https://semianalysis.com

ArsTechnica on the latest interpretability attempts for LLMs

https://arstechnica.com/science/2023/07/a-jargon-free-explanation-of-how-ai-large-language-models-work/

Related sources and news

• https://www.reuters.com/legal/litigation/judge-pares-down-artists-ai-copyright-lawsuit-against-midjourney-stability-ai-2023-10-30
• https://analyticsindiamag.com/is-ai-fast-becoming-a-technology-built-on-worker-exploitation-from-global-south/
• https://www.businessinsider.com/openai-kenyan-contract-workers-label-toxic-content-chatgpt-training-report-2023-1
• https://www.artisana.ai/articles/gpt-4-outperforms-elite-crowdworkers-saving-researchers-usd500-000-and-20

Papers

• Vision is worth 16x16 words
• GraphCast: Learning skillfull medium range global weather forecasting
• UNetR
• DallE-Hidden language
• Word2Vec
• Language models represent space and time
• Interpretability in the wild
• Scaling laws
• Generative models as complex systems
• Formal algorithms for Transformers