This can very well be attributed to a multitude of things, but from the outset it looks like a product/market fit problem.
Perhaps continuously iterating on the 'product' will not lead to any desirable results if I'm addressing the wrong market.
Any similar reflections or ways to navigate it?
I took a large chunk of money, sold coffee. We made good money. But in those weeks, I got calls from former colleagues asking me to freelance, and they were paying way better than the coffee business. This was PMF - there was plenty of demand for Android devs in 2013.
After freelancing and failing projects for reasons out of mt control, I quit upon realizing that there were no good entrepreneurs. The market had a huge hole for startup leaders. I founded a startup. We got in a lot of programs because we fulfilled the requirements of having enough potential. Also customers actually bought our stuff, meaning we met some markets. But we didn't fit the VC market - the VCs wanted something sexier. But our startup had an actual value to someone and we sold it for good money.
So, I suppose you can keep chasing PMF. You might have to switch fields a lot - an app developer for a hospital can bring more value than an app developer on a web-first company.
Generally the best places are the ones that are making lots of money, but need something to spend it on. It can be enterprise. It can be a consulting gig for a failing company, or a rapidly growing startup.
From the perspective of designing how the business or organisation structures work, it may be the case that many of the individual roles are structured so that the organisation as a whole will succeed provided each person in each role performs the role to at least a mediocre level of ability. If someone does a really high quality or high efficiency job at their role, it may have a negligible impact on the organisation as a whole, as how well that role is performed is not a bottleneck.
What does this mean?
There's probably a bunch of different roles you could fill with your skills and ability, but any given organisation may have no demand for those roles.
For many roles, there's a limit to how much the org is willing to spend on someone to do it, if someone doing the role really well doesn't have a large impact on the org.
In many orgs it may be difficult for you to argue that you could bring a lot of value by doing different role Y that you are really good at, when there is no demand or no perceived demand for Y and you are currently doing role X which they need filled to at least a mediocre level of ability. Or it might be possible that someone in management can be shown the value of what a Y role might bring, but there might not be enough ongoing Y work, so it might make more sense for the org to hire a temporary Y consultant instead of promoting/switching you into the Y role.
The world is large, there are many opportunities with different organisations. Interview and create alternative options for yourself outside the system of the current org where you are employed.