We're actually heating the earth in two ways... one is by increasing the greenhouse gases in the atmosphere. The other is simply by using energy; any use of energy means moving something to higher entropy and thus producing heat, even if that energy comes from 100% clean or renewable sources. However, nobody talks about this second way of heating the earth because it's about two orders of magnitude behind the first, i.e. for now pretty insignificant.
As for making the earth less reflective with PV panels... yes, you are right and most of that absorbed solar energy directly contributes to heating the earth because PV panels aren't anywhere near 100% efficient in converting it to electricity, but even the part that is converted to electricity will become heat again in the end, like when it powers AI data centers. (If PV was 100% efficient, then the lack of reflection wouldn't directly do any heating, the heating would all happen when the electricity is used.)
For now that isn't a big concern because the greenhouse gases our civilization is releasing into the atmosphere do so much more heating than all our use of energy, but if we do become 100% renewable and survive that particular danger, then by Jevons' paradox we'll soon be using 10x to 100x more "clean" energy and be heating the earth all the same again.