People are not going to feel the difference between a page rendered in 50ms and one rendered in 100ms.
I don't think that every 50ms difference is unimportant, but the difference between a 50ms render time and 100ms render time may be unimportant. Anyone have a feel for what the maximum render time is for a page to feel fast?