I deal with this, but they do let you know far ahead of time, and they don't seem to stop existing dynos from running even after a stack is retired. So I now just get a nice, clear message on one long running application I haven't taken the time to update (or more likely, wind down.)

"This app is still using the cedar-14 stack, for which the end-of life window has closed as of November 2nd, 2020. The cedar-14 stack will no longer receive security updates and builds will be disabled for apps running on the cedar-14 stack."