While it’s not necessarily a “bad idea” depending upon your application, you may be able to do “better” in the side lobes using masking functions (windows) other than rectangles, which is what zeroing is. See hanning, hamming, etc.
In other words it will cause ringing (oscillations, Gibbs phenomenon) in the time domain (for signals, or spatial domain for images). If instead you want a smoother result, you will need to use oscillation in the FFT domain when zeroing out bins. As that is hard to do perfectly, it's easier to specify a window function (or a classically derived FIR or IIR filter kernel) and convolve it with your input signal / image. It is also more efficient to do online when the data is streaming.
Brickwall filters have the greatest possible frequency accuracy. I think you might be getting filters mixed up with windows - a boxcar window does not have the best frequency accuracy.