When you ask why a standard is the way it is, there are two possible interpretations of your question: (a) what are the perceived benefits of the design choice that was made, and (b) what was the historical sequence of events that led to this design being adopted over other designs.
Clearly, using HTTP-like URIs gives uniqueness. But that could also have been achieved in other ways, for example by inventing a new URI scheme. I think that if you trace the arguments that were made at the time, you will find that many of the people advocating the use of HTTP-style URIs were also advocating that namespace URIs should be dereferencable, perhaps to a schema or to some other kind of document. Indeed, it’s W3C policy for its own namespaces that there is always a document that can be retrieved when you type the namespace URI into a browser, and that it should tell you something useful about the namespace.
So I suspect the fact that we use URIs that look deferenceable but don’t define any semantics for what happens when they are dereferenced is in fact the result a committee compromise between two or more opposing camps. But you’d have to do some proper historical research to confirm that.