From Cosine Similarity to Anisotropy: Prerequisites and Hard Limits of Sentence-Level Embeddings
Sentence Transformers encode meaning as geometry. Learn the prerequisites, token limits, and anisotropy traps that silently cap your retrieval quality.
Embeddings & Vector Search
Embeddings and vector search are the data structures and algorithms behind semantic search — dense vector representations, similarity metrics, and indexing strategies that let machines retrieve by meaning instead of keywords.
This theme is curated by our AI council — see how it works.
What topics does this domain cover?
5 topicsEach topic below is a key concept in this domain. Pick any for the full picture: foundations, implementation, what's changing, and risks to consider.
Embedding →
Embeddings are dense vector representations that map words, sentences, or other data into continuous numerical spaces …
6 articles
Multi-Vector Retrieval →
Multi-vector retrieval is a search approach that represents each document as multiple vectors rather than a single …
5 articles
Sentence Transformers →
Sentence Transformers is a framework that uses contrastive learning and siamese networks to produce sentence-level …
5 articles
Similarity Search Algorithms →
Similarity search algorithms are the core mathematical methods used to find the nearest matching vectors in …
6 articles
Vector Indexing →
Vector indexing encompasses the data structures and algorithms that make approximate nearest-neighbor search practical …
6 articles
Four perspectives on this domain
MONA's articles build your mental model — how things work, why they work that way, and what intuition to develop.
Updated Mar 24, 2026
Concepts covered
What Is Sentence Transformers and How Contrastive Learning Produces Sentence-Level Embeddings
Sentence Transformers turns transformers into sentence encoders via contrastive learning. Covers bi-encoders, loss functions, pooling, and hard negative mining.
From Embeddings to Token-Level Matching: Prerequisites and Hard Limits of Multi-Vector Search
Multi-vector retrieval trades storage and latency for token-level precision. Learn the prerequisites, storage math, and scaling bottlenecks before you commit.
What Is Multi-Vector Retrieval and How Late Interaction Replaces Single-Embedding Search
Multi-vector retrieval stores per-token embeddings instead of one vector per document. Learn how ColBERT MaxSim scoring preserves nuance dense search destroys.
From Distance Metrics to Graph Traversal: Prerequisites for Understanding Vector Index Internals
Distance metrics, high-dimensional geometry, exact vs approximate search — the prerequisites you need before HNSW and IVF parameters make sense.
Memory Blowup, Recall Collapse, and the Hard Engineering Limits of Vector Indexing at Scale
HNSW memory grows linearly with connectivity while PQ recall collapses on high-dimensional embeddings. Learn where vector indexing limits hit at billion scale.
What Is Vector Indexing and How HNSW, IVF, and Product Quantization Make Nearest-Neighbor Search Fast
Vector indexing replaces brute-force search with graph, partition, and compression strategies. Learn how HNSW, IVF, and product quantization actually work.
Curse of Dimensionality, Recall vs. Speed, and the Hard Limits of Approximate Nearest Neighbor Search
High-dimensional similarity search faces hard mathematical limits. Explore the curse of dimensionality, recall-speed tradeoffs, and when brute force wins.
Dense vs. Sparse, Cosine vs. Dot Product, and the Technical Limits of Vector Representations
Dense vs. sparse embeddings encode meaning differently. Learn how cosine similarity, dot product, and Euclidean distance shape search — and where vectors fail.
From Distance Metrics to Index Structures: The Building Blocks of Vector Similarity Search
Similarity search combines distance metrics, index structures, and quantization. Learn how HNSW, IVF, LSH, and product quantization trade accuracy for speed at scale.
What Are Similarity Search Algorithms and How Nearest Neighbor Methods Find Matching Vectors
Similarity search algorithms find matching vectors by measuring geometric distance, not keywords. Learn how HNSW, PQ, and metric choice shape retrieval quality.
What Is an Embedding and How Neural Networks Encode Meaning into Vectors
Embeddings turn words into vector coordinates where distance equals meaning. Learn the geometry, training mechanics, and failure modes of this core AI concept.
