Blog Post··12 min read
Circuits: How Transformers Implement Algorithms
How to identify the minimal subgraph of attention heads and MLP layers that implements a specific behavior — and what we've learned from the indirect object identification circuit in GPT-2.