uhhh, 100ms isn't particularly stringent for a local system... nor should be the tolerance for the impact of control in a system that's literally at rest.
unless we're playing soccer / free-kicks, in which case, yeah, overengineer the hell out of it, sport.
ps, chapter 5 precisely specifies tolerances and minimums for things like latency of specific controls, for a control system for heavy machinery intended to be controlled by a human... and thanks for your input.