LLM Wiki + Embeddings: theoretical improvements for deduplication, granularity and scale Introduction In 2026 Andrej Karpathy published LLM Wiki, a framework for building a personal knowledge base where each concept has its own page that progressively enriches as new documents are added. It is a significant evolution over classic RAG: instead of feeding raw text chunks to the LLM on every query, knowledge is distilled once and consulted in structured form.
However, the system has several practical open problems that limit its reliability and scale. This document proposes theoretical extensions based on the use of vector embeddings on distilled concepts.
What is LLM Wiki A personal Wikipedia where each concept has its own page. When a new document is added, the pages of the concepts it contains are enriched with new information — not compressed. The result is a knowledge base that grows and deepens over time.
Proposed technical stack
-
Two-phase distillation Phase 1 — extract all concepts from all documents before writing any page Phase 2 — generate pages already knowing the full picture This eliminates order-dependence and allows duplicate concepts to be merged before creating separate pages.
-
Embeddings on distilled concepts In classic RAG, each text chunk may contain multiple concepts mixed together, or just one concept but incomplete because it was cut arbitrarily. The resulting vector represents an undefined mixture, making the vector space noisy and search imprecise.
With LLM Wiki, each page contains exactly one concept, complete and isolated. Every vector represents one single thing — the space is cleaner, clusters are more coherent, and search is more precise.
Advantages:
Deduplication on write — LLM Wiki already searches for existing concepts when integrating a new document, to update them rather than create duplicates. But this search is keyword-based: concepts with ambiguous names, synonyms or variants easily slip through, creating duplicates that are hard to find and fix. With vector search, the lookup is performed by a script, without consuming LLM tokens, and is more accurate precisely on the ambiguous cases where text search fails. Duplicate maintenance — in vector space, two vectors that are very close are almost certainly the same concept expressed differently. Identifying them is trivial: a periodic automated scan of the closest pairs is enough. Without embeddings, finding duplicates would require scanning the entire wiki manually or with an LLM, consuming tokens and time. Efficient retrieval on read — finds the relevant pages for a query without making the LLM read everything. 3. Granularity controlled by a numeric threshold The vector distance between two concepts defines their degree of overlap. By setting a threshold — the values below are purely indicative and should be calibrated on the specific domain — the granularity of information is decided once and for all, becoming constant and uniform across the entire wiki:
distance < X → duplicate, merge automatically distance X–Y → distinct but related concepts, no action distance > Y → page too generic, consider splitting it The threshold is calibrated once at the start on a representative sample of concepts and kept fixed. This single parameter determines how fine or coarse the granularity of the entire knowledge base is — a system decision, not an editorial one.
- Hierarchical clustering Concepts are grouped by distance into clusters — each cluster becomes a separate wiki with its own local index. The number of hierarchical levels is not fixed: it is determined by the size of the wiki. A small wiki can work with a single level, a large one may require three or more.
Search happens top-down: the relevant cluster is identified first, then only that cluster's index is consulted — not the global index of all concepts in the database. This keeps the context passed to the LLM minimal and allows the system to scale to enormous amounts of data without losing speed or precision.
This approach is already known in the literature as hierarchical RAG and is implemented in LlamaIndex, LangChain and the RAPTOR paper (Stanford 2024). The novelty here is applying it to distilled concepts instead of raw text chunks, where vectors are cleaner and clustering more reliable.
Future research: structured semantic space The problem with current embeddings The vector space emerges statistically from training — no one decided what the dimensions represent. It works but is opaque, not interpretable, not optimizable in a principled way.
Explicit semantic axes Define interpretable axes per domain — for example in finance: liquidity, risk, temporality, scale. Concepts are positioned relative to these axes.
Advantages:
You know why two concepts are close Deduplication and clustering become deterministic Spaces from different organizations become comparable and interoperable if they use the same axes Calibration with anchor concepts There is no need to annotate everything. A set of 50–100 reference concepts are manually positioned and well distributed across the space — the model generalizes from those. It is the same principle as instrumental calibration.
Those anchor concepts become a shared standard — the missing piece in embeddings so far, which have remained emergent effects observed after the fact rather than designed structures.
Number of axes derived from required precision It is not chosen arbitrarily. You start from the precision required by the application and add axes until the target is reached — measurable and optimizable.
Empty spaces as hypothesis generation With a structured and interpretable space, an empty region is not noise — it is a concept that does not yet exist. Models that convert vectors to text can verbalize that point and generate a new hypothesis, coherent with the defined axes.
Immediate application: finding gaps in domain knowledge in a systematic, not random, way.
State of the art Hierarchical RAG — already implemented in LlamaIndex, LangChain, and the RAPTOR paper (Stanford 2024) LLM Wiki — nascent ecosystem with implementations like llm-atomic-wiki and llm-wiki-compiler addressing two-phase distillation Structured semantic space with explicit axes — open field, not yet standardized Embedding models specific to distilled concepts — real gap, no one has done this yet Open problems None of the proposed solutions addresses the most serious problem of the system: the cost of distillation. Every new document requires an LLM to read it, identify all the concepts it contains, search for existing ones and update the relevant pages. It is a costly process in terms of tokens and time, and does not scale easily to large volumes of documents.
No specialized models for this task exist yet, nor optimized methodologies. It is probably the most important problem to address in order to make LLM Wiki viable at scale.