They actually often are in the short term (see the "significant loss of productivity from which it took the agencies years to recover" quote in the article about the similar relocation from Trump's first term), and a gutted department of agriculture can remain incompetent longer than you can avoid supply chain disruptions and food poisoning.
What you do is open small distributed offices led by a driven person eager to live in that area, and let the small offices grow over the years as the DC offices shrink. Careers aren't that long on the timescales governments work on, you just have to be patient and be ok with slow, incremental progress in your own career instead of big splashy doge headlines followed by desperately trying to rehire and hire new expertise when you realize what you've actually done.