Generative AI: How It Works, Why It Fails, and Why It Matters

Abstract

An explainer on how large language models work and why they fail. They are trained to predict the next token, not to check the truth, so fluent output can still be wrong. The paper defines hallucination through NIST AI 600-1’s “confabulation” and walks through documented failures, from fabricated citations to the sanctions in Mata v. Avianca. It covers how training data shapes errors and lays out a four-step routine for verifying model output. Then it turns to my own systems. The multi-agent and council designs I build supply three working theories: adjudication beats synthesis, the writer should not be the checker, and agreement is not proof. BEMA, the model reproduction I architected and ran by directing AI coding agents, supplies a measured result: two adjacent-character typos cut a classifier’s accuracy from 81.9% to 51.2% (n = 3,080) while its mean confidence fell only from 0.79 to 0.61. All eight references are real and checkable.