i think increased screen resolution (and smaller sized devices) led to a pixel art identity crisis. as you mentioned, everything today is expected to
scale and fit on any sized screen. that wasn't the case when everything was explicitly designed for monitors with a resolution between 800x600 and 1024x768. the nice pixel art of windows 98 wouldn't really translate well to both a phone, a desktop computer, and a big screen TV at the same time.
except i doubt that's the entire story, because even though pixel art doesn't scale well, you could make very nice, detailed, inspired, responsive vector graphics using SVG, which would scale up or down to any size. but that probably would cost way too much money to produce, so we're now in the era where companies have brazenly embraced 'phoning it in' as an aesthetic (e.g. material icons, bootstrap, font awesome, etc.). so far, users have either put up with it, or have been tricked into embracing it too