Tag: linear-algebra

Blog Post·2026-06-20·9 min read

SVD: The Geometry of Every Matrix

Every matrix, no matter how ugly, factors into a rotation, a scaling, and another rotation. That single fact — the singular value decomposition — is the engine behind PCA, image compression, latent semantic analysis, and the low-rank tricks that let us fine-tune giant models cheaply.

svd linear-algebra pca low-rank dimensionality-reduction

Blog Post·2024-06-19·6 min read

Linear Algebra for LLMs: Vectors, Matrices, and What They Do

Every forward pass is a sequence of matrix multiplications. Understanding what those matrices do — rotate, scale, project — is the foundation for understanding why transformers work.

math linear-algebra transformers

Blog Post·2024-06-19·8 min read

SVD and Low-Rank Approximation: The Math Behind LoRA

LoRA works because weight matrices are empirically low-rank. SVD explains why, and the math of low-rank approximation explains how LoRA exploits it to fine-tune billion-parameter models on a laptop.

math svd lora fine-tuning linear-algebra