At this point it should just die so people would be motivated to replace it with something better.
Not quite yet, but it was promised that the next release will be Matrix 2.0
Copy-paste the link, otherwise the Easter egg might be offensive to some :-D
The "for some reason", at least for my group, is because it's rock solid for us. We have no reason to want to change, so of course we are using it.
Knowing which XEPs you actually want is a bit of a pain, but has gotten easier. Now I usually just periodically look at the current prosody example[1] which lists extensions by "essential" "recommended" "nice to have" and "other" and it's easy enough to find the matching modules for ejabberd.
One of my favourite small features we added in the latest release is a 'prosodyctl check features' command which will validate that your configuration is up to date with current best practices in various ways.
Although we like to curate the default configuration with care, people tend to keep their existing configuration file when they upgrade (we generally recommend this, as we aim for backwards compatibility). The new command makes it easier for folk who upgrade to ensure they are still getting the expected experience as the recommendations evolve.
Obviously that has to be somewhat opinionated. One thing many people appreciate about Prosody is that you can customize it however you want to, even if it's not following the mainstream. E.g. if you really don't care about synchronizing to multiple devices, you can totally disable that message cache. Don't need audio/video calls? Fine!