It's for a better sounding liaison depending on the preceding number

That's just consonant mutation[0][1], like how english speakers say "a pencil" but "an umbrella"[2]. (Ie, "hon", "pon", and "bon" are all the same word, just pronounced differently due to environment.) The fact that ほ ぼ ぽ (ho bo po) are all the same underlying letter, just with different diacritics, kind of hints at this.

0: https://en.wikipedia.org/wiki/Consonant_mutation

1: https://en.wikipedia.org/wiki/Rendaku

2: Of course, English pretty much only does the conspicous verson of this for that one word, because English.