What do fintech startups use on their back end?
What tech stack would you recommend for something to constantly monitor share prices from an API feed & then action realtime alerts?
For example, is this software aimed at the end user or will the startup use it themselves? If the end user will use it (like alerts for their portfolio/watch list), then the definition of "real-time" is going to be lax because you need human interaction. If it's used by an institution for trading, then you will need to choose highly performant language, architecture, and hardware so that your algorithm can place it's orders before other institutions. You would also likely need to integrate with Swift (financial).
So, tech doesn’t really matter, pick something simple and mainstream so you can get help as needed.
Ruby/Rails, Python/Django, Nodejs, Java, whatever. Postgres for a database.
You likely won’t need any exotic cool tech, novel databases etc.
Just if you use JS make sure to google about how to do money calculations accurately.
I worked at a place that used doubles and there was no fallout. Also when dealing with very small quantities, fixed point might not work so well either.
Personally, as a general rule I use integer's and convert for display only (as mentioned in the above SO post). This removes the accuracy issue and works in every programming language that is in common use today (at least that I know of). I even do this in Postgres (and other DB's) which keeps everything consistent and removes the chance a driver between the DB and client jacks up the decimal conversion.
If your language and your database both have decimal, why not use them?
Secondly (and why I'd still use integers regardless), every language supports integers and division, not every language can support decimals properly or consistently. So for guaranteed accuracy it is best to use integers. Web being so javascript driven is a primary consideration here.
IMO this is something similar to dealing with time in distributed/complex systems. Sure there are types in postgres (and most DB's) that handles timezone, but it still is best to use UTC for everything stored in data and only do timezone conversions at the display layer when necessary.
Timezones change, and far more frequently than you'd think.
So storing in UTC is only a good idea for dates in the past.
You'd be in for a world of hurt when the EU or UK finally decide to get rid of moving their clocks forwards/back. They've both been debating it for years. Or Turkey decides to move the start of summer time again with a couple of weeks notice.
But to point out an example, Postgres (and most major DB's today) store all timestamptz as UTC internally with the region code they use for offset calculations. At least for postgres the region code and offset are taken from the IANA DB.
Essentially what I have done, many times, on complex systems is the same thing Postgres is doing just under our applications control and not relying on a specific databases implementation of conversions. This made it easier for us to ship data around between systems without having to care whether those systems had good timezone support. We then had libraries in each language that would handle all the proper conversions/lookups. Meaning (and using your example) if we needed to update future dates for Turkey, we could issue a query to do so and be done. This eliminates having to wait for Postgres to get updates from IANA, update, test and release and then we do the same. Not to mention you need to multiply that by X number of other systems which have their own TZ conversions at that point if you aren't doing it. And then what happens if one gets it wrong or is late to update.
There are many times this just isn't necessary of course, small systems, systems that are only regional, systems where time isn't a critical component etc. But for systems where time crosses many regions and is critical, or is a legal record or triggers an action it becomes more and more important.
FWIW too: A number of US states have passed laws to go to DST only, so no more switching in those either. They are just waiting on the federal government to pass the resolution allowing it (apparently states can opt for standard time but not DST without approval). When that happens a number of America/XXX regions will need to be split up in IANA. For example, America/New_York currently covers Florida but that would have to change.
But that being said, anything really - rails/node etc most would work just fine for your use-case
End result was a lot of wasted effort. It worked for one team who didn't have those requirements and even then they spent more time than they should have and then ripped out debezium and went with standard mysql replication as it doesn't require the entire kafka stack.
My experience with it reaffirmed my standard technique of data migration of write to both and move reads over when ready.
What language you use is up to you/your team. If you like python, I'd look into Airflow or Celery as frameworks to build your workflows.
For java/scala I'd take a look at Google Dataflow or Flink or Kafka Streams, depending on how heavy the "monitor share prices" part is.
I think the major issue you're going to encounter is not building a system that works, but building a process that allows your team to upgrade the system, test new features, debug, etc.
Dataflow, for example, lets you run more or less the same code both on a stream of data and on batches of historical data.
- The tech stack that is best is the one you/team are most comfortable with using.
Specific Answer:
- If you were using Elixir, you could have 1 process (GenServer) per stock ticker that calls the API for just that stock. GenServers are lightweight processes that run concurrently in the BEAM. That process could then have a set of subscribing user processes to notify which would then do an alert to a user. This is assuming you can call an API for one stock vs. having to parse a feed of data. If I were doing this I wouldn't store any of the stock data in any way, its ephemeral and by the time a DB commit happens the price changed.
Caveats:
You mention API so I assume that means you are calling an API, parsing data, (dealing with rate limits) then alerting. I'd caution the use of the term "real-time" as you are farther away from trading data than big name brokers. My intent isn't to downplay the idea, just thinking it through based on the info in the one sentence question.
You can start thinking about an efficient implementation when you've got a defined feature set and are seriously thinking about delivering an actual product. And when that happens you can just use which tech works best for which type of service anyway.
It was pretty cool for something that took a couple of days and I believe it still works that way.