Autoregressive transformers face 3 structural boundaries: reliance on static training distributions that struggle with out-of-distribution structural extrapolations, absence of built-in symbolic ground-truth verification, and quadratic attention compute costs over long contexts.