Most importantly, arcade games by their nature are finite. The limits of whatever storage medium used as well as the inability to rewrite or update them easily means that they exist in a vacuum. Once you've 'explored' the levels or found the 'secrets' they cease to be new. There are no more secrets. The strategy to beat single-player games never changes.
However the person you're playing against in a fighting game is always going to be different. Even if they play the same character as another person they will play them differently. They will have different skill levels and unique weak points to attack. No round of a fighting game is ever exactly the same. This is why fighting games are beautiful. They turn a static game into something timeless. There are still weekly Street Fighter 2 tournaments all over the world. When was the last time you competed with tens of other friends in say, Qbert 25+ years later?
I do enjoy some games of this type like Robotron that have enough variation in the random patterns of the enemies to still feel fresh. I also love games like Pole Position or Outrun that have well-defined endings and force you to play 'perfectly' like you describe. However most games of this earlier time period are unfortunately just not designed that way.
And it's not as simple as just 'memorizing' the combos if you never play well enough to get yourself in a position to land them. There's still a journey there both outside the game to learn and inside the game to break down the patterns in your opponent's play that will never be present in a single-player game. That's what keeps fighting games interesting and new 20+ years later for me.
The skill ceiling rising was mostly due to consoles, and I'd put it during the dreamcast/psone era when fighters started becoming really intricate.