Hacker News Reader

Comment by cturhan | Hacker News Reader

It’s so interesting that YouTube doesn’t support A/B test for title and graphics.

External paid tools like TubeBuddy and VidIQ allow you to do that. Most large channels use them for constant testing.

How can an external tool do this? As far as I know, there is no way to show half of your audience a title/thumbnail and the other half another title/thumbnail. If they are not testing different variants at the same time, it limits what can be learnt from the experiment.

TubeBuddy's A/B Testing works by having you create a variation of a video's metadata (this could include the Thumbnail, Title, Tags or Description). We then alternate your video's metadata every 24 hours at Midnight PST (to line up with YouTube Analytics statistics). The test completes either based on a certain number of days that you picked or based on statistical significance being achieved.

In an ideal world, we'd be able to have each impression throughout the day alternate between the Original and Variation which would be a 'true' A/B test. Since YouTube analytics are only provided in 24 hours blocks, our current system is basically the best that can be done taking into account YouTube's limitations.

https://tubebuddy.freshdesk.com/support/solutions/articles/5...

If you think about it, every video listed as a search result is a title/thumbnail test, except instead of an A/B test it's A/B/C/D/E...

I think GP meant A/B for your own thumbnails, so you'd be able to randomly show two different thumbnail/titles to viewers and then see stats for each one. They can come clarify if that's incorrect.

Reply on news.ycombinator.com