I'd just like to chip in with my (opposite) experience:
I semi-recently developed an app using Flutter and used an iPhone 6, a 6s, and my old Redmi Note 5 as a baseline for performance testing. While I'll admit I wasn't doing anything particularly graphics heavy (at most: sliding modals and some animations), I wasn't able to get things to dip under 60fps on either platform.
As for the GPay app, the only performance issue I could notice from testing just now was a dropped frame while quickly scrolling through the "explore" page. Otherwise it works perfectly on my Z Flip 3.
On the other hand - have you considered upgrading devices? I always hate whipping out the iPhone 6s (same A9 CPU as your SE) because it runs like hot garbage in most cases... a recurring theme in the iOS space.