No. See for example the Grover search algorithm. You can use to find whether an item is inside a list in O(sqrt(n)).
It halves the exponent on the number of operations. 2^128 - reasonably secure by todays standards! 2^64 - horribly insecure.
(CAVEAT: where the speedup is applicable, which is often hard.)
Besides,in most places where that is an issue, its trivial to switch to 256bit algorithms
> (CAVEAT: where the speedup is applicable, which is often hard.)
Grover's algorithm has pretty wide applicability. Its the exponential speed ups like shor's algorithm that have super limited applicability.