Even if that computer had up-to-date certificates, basically nothing about it would be compatible with the modern internet. It's not just a question of of "expiry dates", it's that you cannot make any forward progress if you have to maintain backwards compatibility forever.
Internet protocols, file formats, hardware standards... all these things need to be able to change over time. Hell, with the old phone example, your biggest problem with a sufficiently old phone is whether it can even connect to the network - I recently tried using a retro Nokia as a backup phone, but realised that the SIM card I'd put in it was for a network which didn't offer 2G coverage. Phone couldn't even make calls or send texts because it wasn't 3G compatible.