You solve a lot of the same problems that other tech companies solve, just with different constraints. An example I like to use is: "Facebook can afford to lose a 'like' on a post -- whether that is through data loss or just eventual consistency -- while we cannot afford to lose a penny (exaggerating a bit here) since there are people making live decisions on that penny"

Those sort of problems are more on the product side, along with the regular slew of product development tech problems. On the infrastructure/platform side (where I work) it is remarkably close to "regular tech". Same types of distributed systems problems, database issues, CI/CD pipelines, engineering efficiency, etc. The "new" part here is working with the finance specific areas such as physical data centers and networking to clients, or domain specific items such as data retention for compliance.

I generally think of "FinTech" as startups/scale-ups working on finance related products, and they're not too different.

There's quite a difference with what I think of as pure-finance, companies that aren't producing a tech product, but are using tech to make money with financial services, or betting on the markets in various ways.

As for the problems, I was once in an interview given a coding task of moving money around a network of companies based on tax rules. Fun.