The reason you feel RPi4B aren’t performing is because it’s running software written using 2010s and 2020s software standards, languages and runtimes.
Before 2000, many many commercial games and software were written using “classical” C/C++ and had any performance critical areas written in assembly language. People would actually profile their apps and optimize their code and loops. Most any language that had large runtimes or included a garbage collector was not considered an option. Only the most complex, data oriented or GUI oriented parts of the code would be written in high level language. A good example of this mixed language code base is the Allegro 4.x Game Programming Library for DOS, Linux, and Windows.
Going back further into the 80s and 90s, many many commercial games and software were written in all assembly language. Look at the old MS DOS code bases on GitHub for a reference.
All that to say that modern software does a ton things in periphery that old software considered superfluous. We know some of it makes easier to write software, or that the software is more secure, or that it can take advantage of the features of the hardware old machines didn’t have. However, that is still extra code that needs run, often in (event) loops which multiples the amount of time dedicated to that code.
Look at codegolf or other minimal code challenges to see what could be accomplished with a different programming process.