This looks pretty nice, I like the idea and the abstraction seems nice. Super annoying problem though: you're stealing my Mac's swipe to go back gesture on your entire page even where you have no swipe actions.
that's funny because i was always annoyed about the back functionality deleting all my carefully entered form data when unintentionally activating this gesture.
Anyway you are right. we need to make this configurable. Apart from that i find it very intuitive to use the touchpad swipe for in app navigation as it feels very much as on mobile devices.