Blog Post··13 min read
Why Your Norms Are Screaming at You
Activation norms, gradient norms, weight norms — what they tell you, why they explode or vanish, and every practical cause from hyperparameters to data bugs.
Activation norms, gradient norms, weight norms — what they tell you, why they explode or vanish, and every practical cause from hyperparameters to data bugs.