Jgrubb

Define “consistent billing” please, I don’t even know what that means and I work in this space.

It's performance optimization - you're well positioned to make decisions about execution times and app responsiveness but too often app devs are gatekept away from the cost data. Tools like this try to fix that. This is net good.

Yeah ok buddy, every job site everywhere is just anarchy. We have no idea how bridges and skyscrapers get built, really.

Yes but have you ever asked a dev if they actually need the 8 year old logs in some bucket?

That last line kind of makes the point. Is any of that actually inspiring to a young child?

Neato, and I personally appreciate your finops.sql example query :)

There's Metabase.

I’m trying to imagine the size and complexity of the indemnity clause in Ticketmaster’s contract with Snowflake, and the number of hours and redlines and delays, .. all the lawyers. All the delays for the analytics org, all the pain, all the suffering and now … it’s game time.

I think if it’s one customer you could maybe blame the customer and get away with it. If it’s multiple at once, all those customers very obviously are just pointers back.

They’ve definitely gotten the repeatedly-named Snowflake employee in a lot of hot water.

I admit I'm not crazy deep in this space but I'm _pretty_ into this space and I've never heard of Carbon.

Parquet is kind of winning the OSS columnar format race right now.

"Shift left" is a relatively hot, relatively new catchall bizterm that typically makes one visualize pushing a workload "upstream" from its current place in the flow. In this context, I think TDD is close to what it means.

Which truth are you referring to?

> I share my frustrations because I care about the platform and want it to get better.

Caring about someone else’s platform - especially when you care about it more than the people who have control over it - is not where you want to be.

It's not gonna be them, it's gonna be Gen X that gets the shaft on this one and probably millennials that raise the capital gains taxes on the old people when they finally age into leadership roles and the old people by that point will be Gen X again.

I'm relatively new to both and love them both for what they do well. Duck is "in process" like SQLite. Clickhouse is a client/server model like most other databases.

Duck is currently more optimized for the analyst or whomever working locally, importing and cleaning data - it's incredibly generous and well optimized for importing many different formats and handling all manner of CSV varieties etc. It's fast and fun to work with but being in process it's not optimized for, say, setting up remotely as the backend for a BI thing like Metabase and then running typical ETL processes to bring in new data. Only 1 process is allowed to connect at a time, so Metabase would sit on that and then you can't import new data in another process. (There is a RO connect mode where multiple processes can connect to read, but nothing is allowed to write if something else is connected.)

Clickhouse feels more like a traditional data warehouse solution, but wicked fast. It's a lot more fussy with getting data in - it's not nearly as generous with importing CSV for example - but once it's in and in the CH native format it is jaw-droppingly fast. Its compression scheme(s) is super efficient and it's also quite fun to play with.

I'm on a BigQuery backed data stack at my job but dream about usecases where we might employ Clickhouse for something more specialized. I use Duck when I'm building an analysis - rather than constantly connecting to BQ I'll just bring the base data into DuckDB locally and then work from there.

I also use them together on a side project of mine - I eat questionable CSV with Duck, then save it to Parquet which Clickhouse will easily handle.

What are his shit movies?, I’m not aware of any but need to check them out.

My piece is a little paradise. Education is top notch, roads are taken care of. Even our average RWNJs aren't as RWNJ-y as they are most other places, probably because of point A. Were you hanging out in the left lane or something?

Not nearly long enough, but I'll take what I can get.

First summary I’ve bothered to click on so mission accomplished, and I learned a lot.

The correct answer to question 3 is "it depends".

Any process that I'm aware of that uses yeast for something feeds the yeast, multiplying it and making it stronger. "Waste" brewer's yeast is more accurate, right?

Red Garland fan club roll call.

Or that usage and billing is a difficult space and this particular thread has been dogpiled by a bunch of folks that have never actually worked in it.

I’m interested if you felt like elaborating on that.

BQ lesson #1 - don't select * on a gigundous dataset.

Lesson #2 - if you select * on a gigundous dataset, make sure it's on your employers bill.

No, by amount of data scanned to answer the query. This would be about 2.2PB worth of data.

Partitioning matters y'all!

Well sure, but how do you query the data they're hosting for free without using google services?

My work laptop recently started having issues with the keyboard so I got a loaner, an unopened M1 Pro. It was still on an older version of the Mac OS, and I am so bummed that they want it back.

The UI that you're referring to is malpractice.

Yes, Evidence was my intro to ECharts. I’m more on the building datasets side of things lately, but on the occasions that I do need to produce some analyses Evidence is the model I want to use. Thank you.