Without the training data and open sourcing the training code and protocol you cant reproduce the model from scratch, thats true, but because the weights and structure are free to download you can modify, inspect and make derivative works off it. Calling it an inscrutable binary blob is misleading.
I do agree that its not reproducible and its often talked about like it is.
In some ways its like most scientific research with a "methodology section", which in todays academic climate often falls quite a bit short of a detailed step-by-step to reproduce results with all the ingredients, but in principle could be used by other experts to do this again. Even without publishing the exact data, a catalogue or description of what data and how it was cleaned/processed would go a long way towards a reproducible methodology i agree.