I have had an all sky camera out for about two years now without issue, also never heard of anyone having an issue. The first I've heard is the comment above about using a spotting scope, which would have a much larger objective than a typical all sky camera, and sounds like the Bayer array (the filter for color imaging) was damaged. I imagine most sensor manufacturers know their sensors will get pointed at the Sun and some point, and are designed to account for it. Especially since pictures of Sunsets/Sunrises are pretty popular.

The Sun is actually pretty small (1/2 a degree), and the image of the Sun will only occupy a given pixel for about 2 minutes. The next day, the declination of the Sun will change, there may be some overlap in the pixels it falls on. But over the course of a year, the amount of time a given pixel will see the actual image of the sun is pretty short, likely less than 10 minutes for the worst case near the solstice. And the pixels getting it near the equinox will probably only get hit for 4 minutes a year, and some not at all. So they're not really taking the beating it may seem.

I've had a pi 1 camera looking out at the horizon for many years and it didn't really get damaged.

Don't forget the default lens is tiny. It doesn't really capture a ton of light.

Mine has been going on 4 years now with zero issues with the camera (ZWO 178MC). The acrylic domes however are a hot mess. Frequently have to replace them. Really need to find high quality glass domes preferably with anti-reflective coating.

I helped build a similar setup a "long" time ago to build a sky HDR database. We had over 40k images across hundreds of days. The longer exposures definitely did not help (even with a ND filter) and by the end the camera had some broken pixels along the sun trail. Nothing noticeable visually, but likely not good for computer vision purpose. If we were to redo this it would be definitely a valid concern to address.