I don't doubt that you're right, but to play devil's advocate Earth's surface to volume ratio is also much higher than the sun's so it cools quicker too.
v = (4/3) * pi * r^3
s = 4 * pi * r^2
s / v => 3/r
This makes a kind of sense in terms of heat transfer: the sun's got a whole lot more insides for radiation to get through before it can even reach the surface to escape.
EDIT: line breaks unbreak formulas.
Thanks.
>For a given shape, SA:V is inversely proportional to size.
When an object grows in size its volume grows faster than its surface area.