Have you tried an off-the-shelf solution like Tesseract? It works quite well if you do the recommended preprocessing.
The preprocessing to increase DPI to 300 did not help when I tried Tesseract, unfortunately. It’s hard to achieve a good contrast between the numbers and the backdrop
If that doesn't work you'll have to add a text localization model to your pipeline.