Wikipedia article has photos
There's a photo of an actual B battery here: https://en.wikipedia.org/wiki/File:B-AA-battery.jpg
Linked from: https://en.m.wikipedia.org/wiki/List_of_battery_sizes
The B battery you linked, is an carbon-zinc/alkaline battery and against that the comment says -
> Most commonly found within a European 4.5-volt lantern battery. Not to be confused with the vacuum tube B battery
https://www.google.com/search?q=European+4.5-volt+lantern+ba....