I am aware of these tools like cassandra/flink/spark/kafka etc. But I am more curios about the best tools and architectural patterns that work well with each other.
Well you can go with:
Fancy: Hdfs(distributed file system) as storage - oozie as workflow scheduler for your load(python/hive/scala/spark) - Tableau for visualization (your business ussers will love it.
Mid range: SQL Server as storage - Informatica for your workload - power BI /SSRS for visuals
Open/low budget: PostgreSQL / Cassandra for storage - make your own scheduler/ there is a post for ETL open score yesterday that might help - for visuals you can make it from scratch but hire a good designer!
This is based on my experience on industries like Gambling, Banking and Telecom
I'd imagine you could use Segment (or source(s) that plugs into it) to accomplish much of this.