It’s was a design decision when they created the CPU. More addressing modes you have and more instructions, larger (in terms of number of bits required to represent instruction) the instructions get. And larger instruction take longer to fetch from memory.
In other words, x86’s instruction set is not orthogonal. It is not necessarily bad, at least not now, that only very limited amount of assembly code is written.
Some other CPUs (for example MC680xx designed in same era) can do that.