Thanks for the wealth of resources. After reading through, the only big concern I have with BigQuery is pricing. I like the model of pay-per-query, but it seems like costs can get out of hand easily.
These two issues particularly worry me:
- No non-date partitioning, which means you'll often end up (in my particular use case) scanning 10-100x as much data as you actually want to.
- Pricing is based on uncompressed data size, which is at least several times more expensive than the alternative (e.g. Athena).
Any thoughts on these? We'll have several hundreds of TB of data, so cost matters.