What would HN commenters define as a truly modern data stack right now? Mostly asking what this chart https://a16z.com/emerging-architectures-for-modern-data-infr... would look like in 2024.
- Postgres and Kafka
- Airflow and Flink
- Kudu+Impala or Clickhouse
- Iceberg/Parquet or Delta
- Ozone(HDFS) or Ceph
- Spark Rapids and/or Ray+Metaflow
- Open Metadata or Atlas+RangerJust don’t spend too much time debating what it is, or the analysts who need numbers yesterday will develop their own solutions before you’ve even published your A3 architecture diagram.
A bird in the hand, as they say.
This is extremely low maintenance at the actual requirement levels of most businesses with < 50 employees.
- Snowflake/BigQuery/Redshift
- dbt
- Looker/Metabase/Tableau