bhavv04
/ grokking.py
Created
May 27, 2026 01:18
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """ | |
| Reproduces the "grokking" phenomenon from: | |
| "Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets" | |
| Power et al., 2022 (Nanda et al. mechanistic interpretability follow-up) | |
| The task: learn modular addition (a + b) mod P | |
| - Input: two integers a, b in [0, P) | |
| - Output: (a + b) mod P (P-way classification) | |
| What you'll see: |