May I ask how this is Bayesian in anyway? I understand that using the term Bayesian is good for directing clicks to a site, but this seems like good old fashion frequentist math. None of the hallmarks of a Bayesian approach to the problem are here: having a distribution over hypothesis, having an explicit prior, computing the posterior of the distributions.

I have some experience with the medical trial literature and specifically bandit algorithms and using cumulative regret verses other statistical measures like PAC frameworks. And regret is most certainly not a Bayesian idea. Instead you are explicitly modeling the cost of each action (providing an A or B test to a user) instead of assuming all costs are equal.

Yes, this is a better approach because it explicitly models the costs associated with the exploration/exploitation dilemma. But, it is not Bayesian.