Agree with the non-intrusive part. However Defender does slow down programs that do a lot of filesystem IO. I tend to disable it when I update bigger programs or run other IO heavy tasks.
As a dev house, I agree on the io, but our solution is to add our dev folders to the exclusion list. One of our build times went from 10min to start a debug to less than 1min...