I assume this is similar to delta-sigma ADC/DAC, where you shift all the noise to a higher spectrum, so you can easily cut it all off?
I'd just average the lowest (N-1) pixels. N-1 because shot noise is a very bright pixel - if I have a bright pixel not including it is best for the avg. If the pixel is not surging, not including it doesn't affect my average much.