The web of the late 90s immediately followed Windows 95 and the gaudy design of interactive CDs trend. The aesthetic of the time lent towards the skeumorphic and into the vividness and richness of colourful graphics, animations and non-grid layouts. There was a novelty-effect driving this trend. GUIs and colour displays were still quite new. And tools like Macromedia Director were only just unlocking those kinds of bespoke GUIs. The web was trying to recreate that feel with a more limited toolset.
We’ve now pivoted back towards simplicity and boxy designs being perceived as more professional. But that doesn’t necessarily mean that simple designs are inherently better. It’s more true that the current trend was a reaction to early web design. And that pivot became reinforced by the technology changes like CSS features and the death of Flash.