You just need the word "Bot" in your user agent. Requires for fetching Twittercards for link previews too. Changed earlier this year.