Wouldn't noise be a huge issue?
For example, dropout, which is almost ubiquitous for deep learning, basically makes activations 'wrong' 50% of the time during training.
Dropout is not the same as random noise. By using dropout you eliminate some neurons from making contribution. As a result, you effectively train many smaller nets, each one adjusting its available weights to perform the same task. During testing, there's no noise - all neurons are back in business and contributing.
What would change with temperature that would require retraining? Are you saying the output of an op could depend sensitively on temperature, or that higher temperatures would increase things like thermal or shot noise? Why would the latter require retraining?
I was speaking loosely -- dropout is multiplicative Bernoulli noise on the hidden layers.
> That's not true. NNs don't like noise, there have been a lot of research done about effect of noise on NNs in the 90s. Random noise over a certain threshold will progressively degrade the performance of NNs, and below the threshold will have no effect.
I'd argue that dropout (and its predecessor in denoising autoencoders) are perfectly valid to see as noise, albeit multiplicative.