"Try to pan or zoom the function to a different region. The plotting algorithm detected one of the following: - Too many asymptotes - Too many transitions of the function from defined to undefined regions - Too many points on the graph that might not represent the function value currently due to high volatility"
It looks like can actually detect when their graphing algo starts to break down.
One way is to set a threshold for whether or not to join the dots, if it's above the threshold perform a search between those points to see if they are between the dots or if they diverge, this search can be complicated or simple, google seems to use a fair bit of cpu power when zooming out on tan(x) so I'm guessing they're using a search here rather than some alternative 'clever' induction from the nature of the curve itself.
This would produce a list of X values for which the overall function is not defined. When your pixel plotting passes one or more of those X values, don't draw a line. If you pass several in a single pixel, give up.