No, the IRS doesn't pay anyone interest. You will owe them interest if you underpay, however.
Meh it's rare, but the IRS does occasionally pay people interest. Usually it's of the "we were reviewing taxes and you overpaid by X amount so here is (1.03)^(years)×x." Still nonzero. Happened to me. https://www.irs.gov/payments/interest#pay