This is nonsense. The bureaucracy is created by Congressional statute. The Constitutional order obligates the president to
take Care that the Laws be faithfully executed and the president takes an oath to that effect. This president is acting deliberately contrary to these statutes, a violation of the oath of office, and reasonably makes the president's motives questionable if not illegitimate.
A president can veto a bill passed by Congress. It doesn't become law. The Congress can override the veto and it does become law, that any law obligates the president. Not even in the most extreme case of the unitary executive hypothesis can a president legitimately ignore the law.
The bureaucracy is a creation of Congress. The president can veto budgets passed by Congress until there is a compromise.
It's not legitimate to call only the president elected government as if the Congress has no say.