It FEELS like it works because that's the target of its optimization function. The term "bullshit generator" is both colloquially and precisely true. It's a big expensive gradient-descent optimizer and *the metric it is optimizing* is PLAUSIBILITY. It is fooling you because it is DESIGNED to fool you, and that is the one thing it's good at. It's an infohazard. Thinking that your critical thinking skills are so good that you are too smart to be tricked *is a risk factor for letting it trick you*.