You can't cheat thermodynamics! If it's being absorbed by the panels (rather than reflected) then it's absolutely becoming waste heat. Solar panels look dark blue/black because they're absorbing most of the light that hits them. This is why them overheating is a problem that can affect performance. If they were mirrors they wouldn't overheat at all, but they also wouldn't be able to produce any electricity.
Thermodyamics says that a solar panel is cooler than an equivalent black panel. Some solar energy gets converted to electricity. All of that converted energy does not end up as waste heat. A 100% efficient solar panel would not get hot. Only the inefficiency ends up as waste heat.
Maybe I'm missing something obvious, but by conservation of energy if all of the energy of the sun was converted to heat then no electricity would be produced? Conversely, that fact that panels do produce electricity means that not all of the sun's energy is heating the panel?
solar panels are only partially efficient. if the solar -> electric conversion efficiency is 21%, then 79% of the incoming solar radiation is converted to heat.
The portion of incoming light that becomes electricity is dissipated as heat primarily where it is used (e.g. in an appliance attached to the grid fed by the solar panels), not in the cells themselves. So it's true that a cell connected to an external load will be cooler at equilibrium than one that is disconnected.