If the IRS is in charge of doing people’s taxes, there is a non-zero chance that the IRS will not be completely trustworthy.
The real conversation should be about moving to a consumption tax and eliminating the political power that comes from income taxes. The current system is intentionally complex — designed to give politicians power to favor and disfavor behaviors and industries.