Hacker News Reader

Comment by vgt | Hacker News Reader

Google BigQuery is a paid service, Dremel under the hood, used internally at Google by ~80% of Googlers every week. It's simply not going away because Google relies on it heavily.

Externally, companies like Spotify and Kabam Games rely heavily on BigQuery as well:

https://www.youtube.com/watch?v=LTVFg6YOjWo&list=PLIivdWyY5s...

https://www.youtube.com/watch?v=6Nv18xmJirs&list=PLIivdWyY5s...

So the story of Google Cloud products, which are paid, supported, and Enterprise-grade, and which are covered by a clear 1 year deprecation policy, are very different from the story of free services.

Public Datasets is just a function of BigQuery's unique ACL functionality.

View on HN

>> Externally, companies like Spotify

I hate people bringing them up - its just being a sucker for marketing. Spotify and Netflix are clearly brands recruited to promote their respective platforms. The only question is how large a discount they are getting among other special treatment in return for their name. I wouldn't be surprised if its conditional on a gag criticising the platform too. These are the sort of situations that are revealed years later to have required a totally isolated and custom solution that bore no resemblance to the service it was intended to promote.

Special partnerships prove nothing because the economics and service you get won't be remotely similar.

You can listen and read Spotify engineers talking about their experiences with Google, including running 2,000,000 qps load tests without ever telling anyone at Google.

If you read some of their blogs, you can see how brutally honest they are about the pros and cons of our services:

https://labs.spotify.com/2016/03/10/spotifys-event-delivery-...

https://www.youtube.com/watch?v=LTVFg6YOjWo&index=7&list=PLI...

https://www.youtube.com/watch?v=xT6tQAIywFQ&index=1&list=PLI...

Finally, does this sound like an engineer who got paid to speak? https://twitter.com/mpjme/status/702167231623516160

> It's simply not going away because Google relies on it heavily.

Still doesn't mean Google will maintain the public access to this database. They might just as well shut it down and limit its access to Googlers. Like they already did a few years ago.

When you use these datasets you pay for how much the query has to scan. For example, if you do a select on a single 8 byte column with 1 billion rows in the table you are paying for 8GB.

Every month the first TB is free but if you go over that you have to pay $5 per TB. They are not exactly doing this for free.

Google employees using something internally doesn't mean that it won't go away for everyone else... Also, they could decide to honor existing contracts and stop issuing new ones...

BigQuery is a truly fully-managed service. This means that SREs are on call 24/7. It's not software that one downloads. Hence it really doesn't make a lot of sense to have it available to just a few folks, incremental overhead is low.

I'd love to hear industry standards around deprecation of paid services. This is really a way of doing business in the entire world, and I'm not sure why Google is being held to an unreasonable and impossible standard of "never deprecate anything ever!". For example, here is Microsoft's:

https://en.wikipedia.org/wiki/Category:Discontinued_Microsof...

I was not blaming anybody, it's possible that a service would be more valuable to a company if it's kept internally.

Ah yes, point well taken. I think we've seen this historically at Google, but things have changed in the past few years.

Primarily as a reaction to AWS and Azure?

Google announced Colossus in 2011, is that going to make it to GCP?

You can use Colossus today! It's a very low level service, leveraged by other GCP services.

For example, Google Cloud Storage and BigQuery Storage both sit on Colossus. Read more on BQ here:

https://cloud.google.com/blog/big-data/2016/01/bigquery-unde...

BigQuery is cover by the Google Cloud deprecation policy: https://cloud.google.com/terms/deprecation

Google is required to give at least a 1 year warning on shutting down released Cloud products.

Reply on news.ycombinator.com