1) AFIAK there are no hard and set rules for this. I think this would have to be the manufacturer's judgement call. Good point though, with enough time you may end up just over-fitting to the test set.

2) The FDA's guidance on ML models is still in flux. Please see https://www.fda.gov/medical-devices/software-medical-device-...