It would not go unnoticed and would pretty much guarantee invasion.
Granted, the US president who was pushing the most for inspections of Israel's Dimona nuclear facility was JFK, who ended up no longer being a problem for them (how very convenient).