You may not know this, but I spend a lot of time around Reddit. One of my favorite subreddits is /r/AskScienceFiction, where people ask questions about sci-fi, fantasy, and occasionally other fictional universes, and the answers are provided from an in-universe perspective. Some of the time the answers come from an obscure bit of canon, but a lot of the time it's just a platform for who can come up with the most interesting speculative answer.
Recently, someone asked what defensive fortifications look like in a high-magic fantasy world. After all, he reasoned, real castles evolved to combat the siege tactics that were being used historically, but a world with magic-users, dragons, and other fantasy elements would present different challenges to a lord looking to protect his holdings.
One common answer we use is that these things are so rare that it seldom presents an issue. But that doesn't quite jive with the assumptions most versions of D&D (really, any versions that I can think of) present about the world by default. But there's another answer that
does. To put it simply, the keep as we know it is obsolete as a form of defense in D&D. It is replaced, interestingly enough, by the humble dungeon. A thick layer of earth provides a barrier to many forms of divination, and hinders many magical forms of ingress (teleportation becomes a risky proposition if you can't see where you're going, even with otherwise relatively safe short-range methods like
Dimension Door), The narrow, twisting passages of a dungeon reduce visibility, create defensible chokepoints to halt invaders from, and discourage the use of area-of-effect spells by an attacking force, and can be constructed to be a difficult or impossible fit for larger enemies.
More traditional fortifications do retain some use-- after all, moats, palisades and curtain walls will still serve as obstacles for an approaching army, and towers offer a vantage point for spotting any approach, as well as high ground for archers and mages to rain death from above on. But the construction of these will differ. I would expect to see, rather than the more conventional open roofs with parapets, a wide room or machicolation with a sturdy roof and many arrow slits, the better to protect the defenders from being snatched up or dive-bombed by flying monsters.
Most people, and most domesticated animals, will still prefer to live aboveground, so there will likely be buildings inside the bailey. Generally these will be made from cheaply-replaceable wood, as the auxiliary buildings of many real-world castles are, but even the main hall stands a good chance of being made of wood. Any strategically-important business must be conducted underground where prying magical eyes and ears will have a harder time spying on it.
In short, the reason your campaign setting is peppered with underground complexes is because the dungeon is the natural evolution of a castle to defend against the kind of threats unique to a fantasy world. Much as abandoned or ruined castles are scattered across the countrysides of Europe today, so will the remains of an old or conquered fortress be a common sight for travelers and explorers. The advantages of underground fortifications against magical or monstrous assault will have been realized early, probably even before the advent of masonry, so the dungeon will remain even when the wooden structures and fortifications of more ancient castles are gone.
As always, questions, critiques, or further insights are welcome.
