BigQuery has more footguns than GKE in my experience, but that’s perhaps because I have a lot more experience with GKE and know how to avoid those footguns. To me at least it’s understandable enough to say More Nodes is More Money but completely non-straightforward to say that this query I wrote is going to scan the data in a new and expensive way. Am I doing it wrong?
> To me at least it’s [...] non-straightforward to say that this query I wrote is going to scan the data in a new and expensive way. Am I doing it wrong?
When you put a query in the BigQuery console, it'll tell you "This query will process ??? MB when run" at the top right.
So if you code all your queries interactively in production (which is what everyone else is doing anyway) it's not too hard to keep an eye on.
Are you using slots (https://cloud.google.com/bigquery/docs/slots)? If you aren't, I'd highly recommend you switch. My guess is that it would make your costs much more predictable (it did for us).
