You'd still need something to handle the nights around new moon, so you'd have to overbuild enough to be sufficient when all you have is starlight.

A bit of Googling suggests that the combined energy per second that reaches Earth from all the visible stars other than the Sun is around 0.0000002% of the amount from the Sun.

I recall reading that the area needed with current solar panel technology to power the entire US would be 10000 square miles. With starlight being 0.0000002% of sunlight, that suggests we'd need 5 trillion square miles of panels to get the same amount of energy at night.

The surface area of the earth is a little under 200 million square miles.

That suggests that it isn't possible to overbuild enough to work off of starlight.

It would be even worse on cloudy nights. On cloudy days you still get a significant amount of sunlight coming through, because the Sun is giving us so much more than we need. Not so with stars.