Sounds like the equivalent of a 4xl snowflake warehouse, which for such queries would take 30 seconds, with the added benefit of the data being cold stored in s3. Thus you only pay by the minute.
No, that would be equivalent to 64 4xl snowflake warehouses (though the rest of your point still stands).
Cost-wise, 64 4xl Snowflake clusters would cost: 64 x $384/hr - for a total of: $24,576/hr (I believe)
What was the cost of the duck implementation?
Apologize for getting it wrong a few orders of magnitude, but thats even more ghastly if its so overpowered and yet takes this long.
Challenge accepted - I'll try it on a 4XL Snowflake to get actual perf/cost
At that scale it cannot be cheaper than just running the same workload on BigQuery or Snowflake or?
A Standard E64pds v6 costs: $3.744 / hr on demand. At 63 nodes - the cost is: $235.872 / hr - still cheaper than a Snowflake 4XL cluster - costing: 128 credits / hr at $3/credit = $384 / hr.
At 5 seconds - the query technically cost: $0.3276
That's like calculating a trip cost based on gas cost without accounting for car rental, gas station food, and especially mandatory bathroom fee after said food.
If I used "spot" instances - it would have been 63 x $0.732/hr for a total of: $45.99 / hr.
Just noting that 4000 vCPUs usually means 2000 cores, 4000 threads
It doesn't mean that here. Epdsv6 is 1 core = 1 vCPU.