It's not entirely "random", but it is
contingent. I.e., yes, we would be fine if we could see other wavelengths. Indeed, there are creatures than
can see other wavelengths. Lots of insects and birds can see UV, and lots of snakes (specifically pit vipers) can see IR.
Seeing IR difficult because it is blocked by water and our own bodies radiate it, so it would be useless for our retinas to be able to detect it. You need completely different kinds of eyes (hence pit vipers' eponymous pit organs).
IR aside, the visual spectrum is centered where it is because that's where the sunlight that can make it through our atmosphere is brightest. We're adapted to see the light that's actually there, in nature. The range of light that's available actually does include some UV, and the ancestral tetrapod presumably could see that, which is why birds still can; mammals, however, went through a period in our evolution where our primarily-nocturnal ancestors had no need for four-color vision (because there wasn't enough light to support it) and not much UV was available to them, so we lost the UV sensitivity entirely and went down to only 2 kinds of receptors. This is why, e.g., dogs (and most other mammals) are colorblind. Apes eventually independently re-evolved a third color receptor because that was advantageous in things like identifying edible fruit which were important to their lifestyle, but there was never a strong evolutionary pressure to regain UV sensitivity.