I'd argue this is a good thing. In Android UX guidelines, the "back" button's "promise" is not that you go back to a specific item, it's that you unwind your most recent action (notably, not unwind state, it's not an "undo" button, just that it unwinds the most recent action).

Apps get to decide what metaphor makes sense for an "action" (maybe it's a page, a screen, a dialog, etc). But users can be confident that the "back" button always takes them "backwards", even if the metaphor various apps use may differ.

Users can feel reasonably safe that the "back" button will take them "back" one 'step' or one 'action' worth, even if the app is using panels instead of pages, or modal dialogs instead of inline alerts