S. Roy

Blog Post·2026-06-20·9 min read

SVD: The Geometry of Every Matrix

Every matrix, no matter how ugly, factors into a rotation, a scaling, and another rotation. That single fact — the singular value decomposition — is the engine behind PCA, image compression, latent semantic analysis, and the low-rank tricks that let us fine-tune giant models cheaply.

svd linear-algebra pca low-rank dimensionality-reduction

Blog Post·2024-06-19·8 min read

SVD and Low-Rank Approximation: The Math Behind LoRA

LoRA works because weight matrices are empirically low-rank. SVD explains why, and the math of low-rank approximation explains how LoRA exploits it to fine-tune billion-parameter models on a laptop.

math svd lora fine-tuning linear-algebra

Tag: svd

SVD: The Geometry of Every Matrix

SVD and Low-Rank Approximation: The Math Behind LoRA