This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """ | |
| 33-parameter hand-coded adder (1L decoder, d=3, 4h, ff=2). | |
| Causal self-attention with RoPE-style digit routing, fixed analytic queries, | |
| selector-basis routing for k/v/up/o projections, SiLU carry detection via MLP, | |
| and tied embedding decode. Only learned params: embed (10x3) + final norm (3). | |
| """ | |
| from __future__ import annotations |