You're not wrong: networks in general aren't super reliable, and partitions are a real problem! Peter Bailis and I wrote a paper on this in ACM Queue. I've spent much of my professional life exploring the consequences of network failure on distributed systems.

That said, I don't think it's reasonable to infer that because failures occur more often than we'd like, systems based on consensus are always inappropriate for latency and performance sensitive systems. While there are minimum bounds on latency in consensus systems, there are also plenty of human-scale problems for which that bound isn't a dealbreaker. Moreover, some types of computation (e.g. updates in sequentially consistent or snapshot isolated systems) simply can't be performed without paying the same costs as consensus. Consensus can be an appropriate, and in some cases the only possible solution, to some problems.