If you feel Sentry, PagerDuty and full observability is overkill you might be interested in Bugsink[0].

I specifically wrote about the "overkill" bit (see the blog) and slack notifications were (finally) just introduced.

Disclosure: that's me.

[0] https://www.bugsink.com/