Yes, but even if the block time is longer, the probability that I will "win" the block goes up. Someone has to mine the block and if half of the competition goes away my chances go up. This incentive will counter the temptation to stop mining, dampening the effect you mention.