The IRS can say "this is not a valid business expense." You can say "it clearly is a valid business expense and should be deducted!"
The IRS can say "this is contractor income." You can say "it clearly is an down payment for subcontractors, you can see the money leaving here!"
These are two obvious problems you could have, and it doesn't even require dishonesty on the IRS' part. Maybe one of your subcontractors is trying to cheat the system and is jamming you up?