It gets even better when people start switching between percentages and "percentage points" referring to a measure that's in percentages originally.
Unfortunately, most of those things are easier communicated and harder to get wrong if you try speaking in a more natural way. This is now "twice as fast" or "2.1x faster" is much clearer and can't go past zero :)
Similarly, I think it'd help to switch back from percentages to actual factors (119% = 1.19), and saying "we reduced the time for the computation by 1.19 of original time" would clearly show what's wrong (and saying "by 1.19x" would signal how it's a small reduction, so it's wrong as well).
Finally, I am 94.8% certain people will keep using percentages even where inappropriate, and with too much precision too!