Reading tends to fill my head with cognitive dissonance, and especially when it becomes a habit. It is no secret that information is just as addictive as any other vice and as with all vices, there will always be the urban legend of the person who finds it hard to over indulge (in this case, a bibliophiles). Ignore them and cut down on your reading when you can.
Unless you must increase your IQ at all costs because your life depends on it, or that everyone around you is getting as smart as you, so you have to up your game!