@NanoRaptor You've been lucky, not right. Even if you notice nothing, the glass is weakened by such extreme thermal stress, and more likely to fail catastrophically, even without warning at some random moment.
Experts usually know what they're talking about. Your neighbour might not be one, but plenty of qualified experts do say the same thing, and are probably correct.
Also, being older than someone isn't equivalent to knowing better than them.