The article touches on that a bit in the "Avoid being blocked" section. Sometimes user-agent isn't enough, I've run into other headers that can trigger a block or a change in behavior. The last one I ran into was gating on accept-language to not control the language, but serve up a honey-pot type page for automated crawlers.