This paper reviews key-value memory systems, which distinguish representations for storage (values) and retrieval (keys), allowing optimization for both fidelity and discriminability. It connects these computational foundations to modern machine learning, psychology, and neuroscience, proposing that the brain utilizes similar principles. The authors argue that memory performance is primarily limited by retrieval, not storage capacity, and provide simulations to illustrate these concepts. ✨
Article Points:
1
Key-value memory distinguishes representations for storage (values) and retrieval (keys).
2
Brain memory is primarily limited by retrieval, not storage capacity.
3
Hippocampus stores keys for discriminability; neocortex stores values for fidelity.
4
Information in keys guides retrieval but is not consciously recallable.
5
Forgetting is retrieval failure; memories can be reactivated without re-learning.
6
Key-value memory is a foundational concept in modern machine learning systems.
Source:
Core Concepts
Keys vs Values: Distinct representations
Retrieval vs Storage: Retrieval is limiting factor
Computational Foundations
Correlation Matrix Memory: Hebbian learning
Representational Structure: Learned vs Fixed scaffolds
Ubiquity in ML: Linear layers, Transformers
Neurobiological Substrates
Learning Rules: Hebbian, Non-Hebbian
Architectures: Three-layer network, Tripartite synapse
Attractor Networks: MESH, Vector-HaSH
Evidence from Psychology & Neuroscience
Retrieval Interference: Not erasure
Distinct Representations: Hippocampus (keys), Neocortex (values)
Unrecallable Keys: Tip-of-the-tongue, Feeling of knowing
Illustrative Simulations
Key/Value Optimization: Separate roles
Forgetting & Reactivation: Retrieval failure
Conclusions
Speculative Connections: Brain-AI
Future Research: Experimental tests, detailed models
AI Convergence: Promising direction
Source: