That is possible, but if swapping happens due to memory pressure it can still cause slowdowns in unrelated parts of the same or other processes.