I don't believe that is correct, re: UCB1

There are probably a few ways to prove the result now, but one (nicely general) way to prove that online learning result would be the method of multiplicative weights: https://www.cs.princeton.edu/~arora/pubs/MWsurvey.pdf