> How do you make sure the same user who was on A variant yesterday doesn't get the B variant today?
You can't. You make the point that the stats generated by Birdy are not 100% accurate, but they're the best we've got given the limitations of the Twitter API :D
I make the point that the data is good enough, and statistically significant enough (especially if you let the test run enough) to help you find the better profile version.
I agree that this subset of visitors that may come twice or more can skew the data.
Birdy's stats are not to be taken as the absolute truth, but as solid clues as to which profile version it is alternating between is providing the best results.
I'm looking forward to improving Birdy as Twitter adds more capabilities to their API!