Hacker News Reader

Comment by nik736 | Hacker News Reader

Interesting that Dropbox is saving a lot of money chopping off AWS while Apple relies on Google for exactly the same thing. I would think that Apple would host and build an inhouse solution, really curious what the reasoning is to go with GCP.

View on HN

That would be because Dropbox is a SaaS software business where the TCO of its infrastructure is absolutely critical to its P&L. Every dollar shaved off its infrastructure per user is a huge deal for Dropbox.

Apple uses software / cloud services as a way of selling expensive hardware, and the margin (if any) made on its cloud services is a rounding error on the overall P&L.

EDIT: Generally, the 'Apple insources everything' meme is a massive oversimplification. Apple insources when there is a competitive advantage to doing so. Apple insources its chip development because it is able to get better thread performance on its phones when the software is aligned tightly to the chipset. Apple insources Cobalt because you need Cobalt to make smartphones and if others find it difficult to source Cobalt, they're going to find it hard to make smartphones, driving up their unit costs. It gets no such advantage from insourcing compute bar a slightly lower TCO, which isn't going to be a huge deal to them anyway.

HN dogma says "start with doing unscalable things". Fix what becomes a real problem, in the order that they do. With Apple's money I imagine paying a bit for GCP remains a minor hassle for a long time.

The difference is that data storage is Dropbox's business, while Apple's business is selling devices. Yes, they do have a variety of income sources, but they are all built around and integrated into selling iThings and MacThings. They go with what they are expert in. Reliable and available around the world storage requires unique and difficult to learn skills that don't map well to OS design and pc hardware architecture.

On the note of data storage, it's frustrating that I can have photos and videos -- that I intend to keep for the next 50+ years -- spread out between 2 to 4 services (iCloud, Dropbox, OneDrive, Google Photos) just by owning a couple phones and a computer or two, which I imagine is pretty typical. I know there's a start-up idea in here somewhere, because I personally would pay premium to consolidate all those into one organized location, but I don't exactly know what that is. I know Dropbox has an option to import data from other services, but in my experience it hasn't been as "smart", automated, and thorough as I'd like it to be.

Once the baby videos started rolling in all the cloud solutions started getting expensive real fast. I had to stoop down to running my own file server. Buy a pair of drives every now and then to expand, and nightly rsync to keep things backed up.

The product I would like is a photo album with an indexing and web interface components, and plugins to make sense of different phone platforms. Nannies love to text videos and getting those off the phone has been a pain; I end up saving the whole iOS backup and scraping any media looking files from it.

Running my own server and writing a script to scrape files or even doing it manually is too much work for me, I just don't have the time to dedicate to something like that. That's why I'd pay premium for a service that automates this for me.

How much baby video do you record?

I have to imagine you're usage is an outlier here.

Up to two TB now at 3yro kid. Part of the problem is the video resolutions grew a lot, and having to snapshot the whole phone whenever it fills up. (Though I did some deduping and the redundancy is not as high as I thought...)

I doubt my usage is an outlier. But I suspect that I keep more video than most people just because I can.

I do the same thing, although my data growth seems to be a bit higher. My kid is less than a year old and I'm on pace to generate about 2-2.5 TiB for the first year. (Mostly 4k video.)

At some point I might start thinking of re-encoding the originals, but so far storage is cheap to have locally, and I'm going to see how long I can keep this up.

I'm also the same way. I try to keep everything, because I can, and I'm also now storing baby videos.

I have a 2TB iCloud plan for my iPhone X. The phone is set to "Optimize Storage", so it will de-scale old "local" pictures once I hit ~256GB.

In the background, I also have the free and unlimited version of Google Photos running. It automatically uploads everything, including videos. Videos will be scaled to 16 MP / 1080p, but I'm not complaining since it's free.

I haven't hit 2TB yet, but so far it works. I also have a 20TB Synology NAS setup, but have little desire to run my own photo storage/hosting. The Photos app on my MacBook Pro also is set to download everything via iCloud Photo Library, so that's also backed up to the NAS via Time Machine.

Personally, I'd prefer to have things a little spread out. If you end up with account problems at one provider you still have other copies. (I'm thinking of scenarios like "mistaken DMCA takedown" or "account mistakenly flagged as fraudulent" or weird things like that.)

Why did owning multiple devices result in your photos being spread across multiple services? Other than iCloud they're all cross platform. They all offer easy ways to download your entire library, is there a problem with downloading everything and reuploading to a different service? Also if you really want to keep them for 50+ years I think you'd be crazy not to have at least one on-site copy and one off-site copy.

>They all offer easy ways to download your entire library,

Frequently there are proprietary things that aren't included in the library. For example, if you become dependent on Google Photos ability to recognize faces and objects, you tend to stop bothering to tag photos manually so when you export everything all you have is a big pile of unorganized photos.

>I personally would pay premium to consolidate all those into one organized location

Why would you want to increase risk of losing them all at once?

GCP charges pennies for storage; they auto-dedupe on the backend too. Although I imagine the bandwidth is a high cost.

Speaking of cost, Apples war-chest is not small- if anyone can afford a premium, it'd be them.

I am not sure what you mean with pennies, they charge the same as AWS, and Dropbox just showed that doing everything inhouse saves them millions of $$.

When you are one of the largest companies in the world, millions really isn't all that much (for your infrastructure).

What is the measure you are using for Dropbox being one of the largest companies in the world?

In the beginning of 2017 Dropbox claimed a $1 Billion dollar run rate, however at the end of 2017 Dropbox had a net loss of $110 Million. When your entire year is a loss of $110 Million, saving $75 Million is very important to the companies survival.

I was talking about Apple.

Ah. That makes much more sense.

Thanks!

The dedupe would be ineffective here unless there's key re-use since Apple AES-128s every chunk before it goes to 3rd party storage.

It appears they derive the chunk encryption key from the chunk content, so dedupe will be possible if they don't randomize the IV or nonce ([0], page 53). The guide also mention convergent encryption, which would confirm this. Personally, I find it sketchy privacy wise.

[0] https://www.apple.com/business/docs/iOS_Security_Guide.pdf

These types of busy-work machinations are often done by cloud providers around encryption. The purpose is apparently to say they use ciphers but also keep the keys in order for decrypting data.

Cloudkit and iCloudDrive use Account Keys, but it isn't clear if those are key encryption keys or data encryption keys. It also isn't clear if those are protected from information only from the client, or if the cloud is able to freely read them. The differences are massive in regards to privacy, and this document really doesn't have the needed technical information to make an informed decision.

Does GCP GCS really autodedup? I haven't seen anything in their documentation that suggests they do, and generally deduplication is known to have unavoidable performance characteristics, especially if you're in a very distributed environment.

We can't compare companies with each other since we have no idea what kind of deals they have negotiated with the cloud vendors. On Apple scale, you most certainly are not paying the list prices. Quite certainly the same thing for Dropbox, but Dropbox might have had worse negotiation position. Initially, they started small and then were quite locked into AWS. Maybe AWS did not believe they would actually be able to move away.

Since Google is working to challenge AWS, it probably makes sense for them to strike deals with companies like Apple with even small profit margins if this increases their efficiency and allows them to then make more profit from other customers.

And then we also have the option that Apple simply can't build own data centers fast enough to accommodate all their needs. Encrypted storage blocks are a safe thing to distribute, so rather distribute those than some Siri voice recognition stuff.

Dropbox and Apple are Oranges and Apples.

Dropbox and Apple are completely different businesses with different business modules. If Dropbox suffers downtime, they lose revenue. If Apple suffers downtime, they can point at a cloud provider, while they keep selling their products.

You outsource what is not you core competency. Apple has never excelled at creating services.

(In case they someday want to drop the business. They don't have to worry about shutting off their own capital investment - all they have to do is tell Google to turn it off).

It's not an all-or-nothing proposition. Apple probably uses DCs for the majority of their storage, with the option to "burst" to AWS and/or Google, while they continue to build additional in-house capacity.

Wasn't Apple a huge EMC customer not that long ago?

Reply on news.ycombinator.com