Backblaze Announces B2 Compute Partnerships | Hacker News Reader

Backblaze Announces B2 Compute Partnerships

backblaze.com

209 pointswillcodeforfoo109 comments

I see people from Blackblaze here, so let me ask: any chance we could get B2 integration in Synology's Hyper Backup ?

I know you're in Cloud Sync, but that's really not the same thing (a one/two way sync is not a backup service).

I'm not sure if you need to give Synology a nudge or something, and I would even pay a fee for it if need be, but until then I can't use B2 as one my main backup destination for all those businesses NAS I have at various small sized companies I administrate.

Yev from Backblaze here -> We'd LOVE for that to happen (and there's ways to make Cloud Sync work as a one-way street) but it looks like they are pushing their own C2 service for Hyper Backup - so it's looking unlikely that they'll add us in. There's a thread about it on their forums (https://forum.synology.com/enu/viewtopic.php?f=3&t=120647) so our hope is that if they see value in adding 3rd party platforms to the Hyper Backup service we'd be high on the list!

> and there's ways to make Cloud Sync work as a one-way street

Sadly it still means if the source gets corrupted for whatever reason the backup is done, for business files sync is really not backup (I don't think I'm teaching you anything there, but really this sadly doesn't fit my needs at all). Also Hyper Vault is all about the intelligent versionning.

While it's possible to trick it by doing the backup somewhere and then syncing THAT to B2 (and I do it for my personnal NAS), it's too much of a hassle and "complex" setup that I can't see myself pushing that to my customers.

OVH managed to get them to add HubiC in there so clearly there is a way. Hope you will keep pushing them until they move on it ! And thanks for making such a great product :)

> OVH managed to get them to add HubiC in there so clearly there is a way. Hope you will keep pushing them until they move on it ! And thanks for making such a great product :)

They know we'd like it in there, think that OVH was there before C2 was ready so they are "grandfathered in". Best way to let them know you want it is to add your voice to the thread!

I'm using Cloud Sync to backup a Synology to B2. Have you considered btrfs snapshots? I use them because I'm backing up a lot of sparsebundles and I had the same thought as you: if they get corrupted and then automatically back up... then I'm in trouble.

Anyway, snapshots give me what I'm looking for re: versioning.

>for business files sync is really not backup

With versioning it is.

It really isn't. Maybe I'm mistaken and then I'm listening to how you do it ?

The purpose of file syncing and backuping is very different, one is ensuring that a change on the source is replicated to the destination and is kept up to date on changes, the other is taking a snapshot frozen in time no matter what happens on the source.

If you have a file version, it's frozen in time. So if you have syncing with file versions you can rollback files individually.

I don't think I was clear enough: with a two way sync, if a disgruntled employee / script failure / any other thing destroys the file version 1.0, it gets uploaded and replace the old file version 1.0

Backup is not just about having a copy, it's having a copy that stops being kept in sync with the original after it is made.

The version (in S3) is a hash of the data so if you remove your local version and replace it then you won't be tricked into thinking version 1.0 is the wrong thing (unless someone brute forces a new file with the same hash).

There is also a concept of Write Once Read Many in storage which is used for compliance. So you can use sync with WORM and it's basically append only.

If Synology is moving up the chain into Cloud Backup and Storage. Will Blackblaze move down the Chain and make a simple NAS for Home user? ( With B2 services Integrated of coz )

Same question but for QNAP. It's hard to figure out the right setup to make sure my QNAP is 100% backed up, safe from user error, etc. especially without dieing from sync fees.

Yev from Backblaze -> Let 'em hear it! We'd love to be integrated, but as the integrator to B2 they would have to prioritize the project! QNAP is integrated with B2 in some ways already, you can take a look here -> https://blog.qnap.com/backup-data-backblaze-b2/.

We're thrilled about this. In short, we've directly connected our storage and their compute to ensure fast connections, and made the data transfers free. We've heard some ways people want to use this (disaster recovery, transcoding, rendering) - but would love to hear ways that you might. Thank you!

Gleb from Backblaze

Hey Gleb! This is awesome!

Are there plans to accelerate this as an offering for other providers? I think there's huge benefit in partnering with other providers where you're enabling an agnostic storage layer (any provider that needs a durable object storage system without the complexity of operationalizing one themselves). Congrats, huge Backblaze fan!

Thanks for the kudos. We're open to supporting other providers as well, including other types of services beyond compute (e.g. databases and specific verticals.)

Do you have some specific providers you were interested in having connected?

> Do you have some specific providers you were interested in having connected?

Not off hand, but lots of colleagues in hosting, exchange fabrics, SaaS products, etc and I want to be able to recommend a partnership to them whenever a good fit appears.

Excellent. We just created this Google Form to add/recommend partners: https://docs.google.com/forms/d/e/1FAIpQLSdObw8HKH4uDexOg-fl...

I've been using b2 and linode and I would benefit from having those two services connected :)

Great - we'll try to connect with them.

I can second this. We run significant bandwidth from you guys to Linode, and this would help immensely in a few of our applications.

We also use Digital Ocean, so they'd be our second choice; however, I doubt they'll be willing to integrate with a direct competitor to their Spaces product.

I guess we can explore with DO, but certainly will with Linode.

When your European region exists: I'd love to see Hetzner(.com) connected.

Looks like they're in Germany/Finland. We'll try.

Just to second that. Hetzner has a great product in Europe for compute but they lack simple storage options. Connecting both would be great.

Be still my beating heart. Seriously? I would drop my Hetzner storage boxes in a heartbeat if I could connect to you instead on a fast link. I have, through the years, have used their auctions to snatch quite a few SSD only boxes and so I need external mass storage their network storage offer is ... ahem ... not the fastest! I would welcome something better :)

I'll second what the others have said. Especially their new Hetzner Cloud would be a perfect fit. It's one of the cheapest (if not the cheapest) cloud option I could find. Combining that with your B2 storage would be a perfect match.

And the best thing: They don't only focus on price but the quality is great as well.

And for us being a german/european provider is a bonus (GDPR).

Yeah Hetzner would be awesome. We already run some stuff over there so I could foresee spinning up some transcoders if you have connectivity there.

Vultr would be great!

Doesn't Backblaze just have a single datacenter? If so, I think they're pretty limited in who they can partner up with like this. That is, they could only partner with companies who are in the same datacenter as them (or very close, network wise, due to latency) that they can connect directly to.

Backblaze has two facilities, one in California and one in Arizona. They're limited to anyone who can reach their network edge through a network port in their datacenters, but theoretically, you could ride fiber to their network ports from other facilities.

I would expect that as this grows (and it will, cheap, reliable storage and all), their network footprint would extend into more facilities (scale is required for this to be cost effective though).

Latency is not an issue until you're pushing GB/sec and crossing an ocean, and even that is feasible (lots of resources at https://es.net/ for that use case).

Are there any plans to expand this to a European location?

The CEO mentioned in an another comment that a .eu location by the end of 2018 is a goal.

This is awesome. I personally only use B2 for personal backups but I know my company is looking for storage of 4k media footage.

This makes it more compelling for them :)

Also this opens up the possibility for someone to make a SaaS on video storage and transcoding for great value

Would love to help your company with their storage needs. Feel free to email me: gleb.budman at backblaze

BTW, this person is figuring out how to do transcoding of files their files in B2 as well: https://twitter.com/orlandoferrer/status/981205299532124163

IBM's GPFS+LTFS (tape-based) is also a very low cost way to go on this and a commonly used solution in M&E as well as Geospatial Imagery space (I work in the latter). Assuming you have the facility and staff to run it, of course.

Ray

Backblaze B2 is awesome and the future looks very promising. Their team is also very open to new ideas and projects.

We have few PB of data there and never had any problem.

I honestly don't see any reason for anyone to use AWS or Google Cloud for object storage, except for the outbound network transfer issue from these providers.

Redundancy? BackBlaze still only operates one (or two but in the same region) datacenters.

Meh, for true redundancy you need separate vendors with different hardware and software infrastructure. Erasure coding and bitrot guard hit the 5 9s I need.

Yev from Backblaze here -> we actually have datacenters in California and Arizona so they aren't too close together! That said, we're also working on multi-region support so that you could move data between regions (no ETA on that yet). Hoping to get more datacenters online in the coming year!

But you're correct - we do recommend diversification, having data in multiple locations is always a best practice, and if you can have it in different vedors, all the better!

Will my data in BackBlaze get replicated to two separate buildings automatically? Or do I have to pay to store it in both California and Arizona?

S3 always stores your data in at least 3 availability zones, for instance.

We're still working on the spec so hard to say, though even if we did make you pay for it you could store it in 4 regions before hitting S3 pricing :) That said, we try to be fair in most things we do.

If Backblaze added simple, cheap functions-as-a-service, I'd switch everything away from the Big Three cloud providers and Auth0 except for fallback microservices. The biggest reason is bandwidth costs, which are disproportionately huge for small and medium businesses if there are more than a couple kilobytes of transfer per invocation.

That said, I am not super impressed by these first two compute providers, and probably would not trust them to give me enough nines even if they started doing FaaS. I can spin up something similar with Kubernetes on Hetzner for considerably less money...

Which functions are you using?

Wow nice! They figured out a way to successfully compete with Amazon by slowly backing into little corners of their space. Love it!

Yev from Backblaze -> We're just over here doing our own little thing hoping folks join in on the fun :)

Can the people from Backblaze answer when they are going to accept payement in different currencies? My Backblaze bill is around $0.50 but my bank charges fees for conversion etc so the total paid is often around £2.50 or so. Would love if they’d accept bitcoin or Paypal/Amazon or charge British pounds.

Have you looked into TransferWise? I've used their debit card a few times to work around foreign transaction fees.

Their Borderless account debit Mastercard is in beta and hard to get unfortunately.

I got within weeks from them after being a Transferwise.com client for a month.

Did you have to specifically request it in their interface?

Disclaimer: I work at Backblaze.

> currency conversion is expensive

Backblaze probably won't accept other currencies in 2018. However, we definitely have heard about this issue, and we have a feature in the works to allow you to "load up" your account with an arbitrary amount of pre-paid cash so that you pay ONE transaction fee to deposit (as an example) £250 pounds at once (incurring the £2.50 conversion fee exactly once) and then as Backblaze deducts from this account balance instead of a credit card there are zero transaction fees.

Would that help?

The OTHER reason this is a requested feature is some customers want a "reserve fund" to be used if their Credit Card payment fails, as a hedge against Backblaze deleting their data for lack of payment.

> However, we definitely have heard about this issue, and we have a feature in the works to allow you to "load up" your account with an arbitrary amount of pre-paid cash

At last. I actually did abandon B2 over DO's Spaces due to the lack of this feature, despite the far more competitive pricing on your end. That, and the lack of AWS API support, but that was mostly related to my particular use case, which was the need to mount cloud storage as disk partitions using s3ql which mainly supports AWS, OpenStack and compatible services. It does have preliminary B2 support [0] but it was really hacky and I ended up losing data because of it.

Having a prepaid credit option and the option to use PayPal for such payments like DO does would have tipped me towards Backblaze's favor to the point of trying to fix s3ql's B2 support, though. ;)

[0] https://hub.docker.com/r/zdce/s3ql-b2/

Have you looked at using a s3-compatible proxy like minio? They support b2, but I haven't tested it myself.

JuiceFS [1] could be a replacement for S3QL, which has better support for B2, can also be mounted on multiple machines in the same time.

If you don't have more than 1TiB, JuiceFS is totally free.

Disclaimer: Founder of JuiceFS here.

Nilay from Backblaze here. This is awesome! Can you reach out to us via this form? I'd love to learn more about your software and list it on our website. Just mention I sent you here: https://www.backblaze.com/b2/contact-sales.html

Get a Monzo or a Revolut card and pay with that, top up in GBP, and transfer for free at the MasterCard rate.

"$0.50 but my bank charges fees for conversion etc so the total paid is often around £2.50"

Which UK bank charges £2.15 in fees or exchange rate loading, on a £0.35 transaction?

(In my experience the most a UK bank will charge for a foreign currency card transaction is 2.75%.)

All of them seem to in my experience. They charge a "Non Sterling Transaction Fee"

I bank with First Direct. I just googled 'non-sterling transaction fee first direct', and found this text:

You will normally need to enter your PIN at checkouts. For non-Sterling (foreign currency) transactions we will charge a fee of 2.75% of the Sterling amount of the transaction. This fee will be shown as a separate line on your statement as a 'Non-Sterling Transaction Fee'.

http://www1.firstdirect.com/1/2/travel-and-international/tra...

And in my experience there's no minimum on this fee, i.e. the effective percentage isn't any higher for small amounts.

I agree with other people's comments about using Revolut and Transferwise cards for $ purchases in general. But if the commenter to whom I replied is spending $0.50 per month in USD, then the non-sterling transaction fee should be ~2 pence per month. So not worth the extra (albeit minor) hassle of maintaining another card.

Santander charge a minimum on £1.50 and I think Natwest and Halifax have.

Monzo also has 0 fees for international card payments.

Wow. I doubted what you said, but Santander does indeed have exhorbitant fees for debit card purchases:

"You will be charged a non-sterling transaction fee of 2.75% of the value plus a non-sterling purchase fee of £1.25 for each transaction"

Are you paying via debit card? Might be better to get a good credit card. Plenty offer decent exchange rates for foreign currency transactions https://www.moneysavingexpert.com/credit-cards/travel-credit...

B2 is interesting, and the price for object storage is pretty competitive. I wrote my stuff against S3's API, particularly for DO Spaces, but DO's fairly epic pants-crapping this week has me nervous.

My use case is download-heavy, though, and a little bit bursty (but with a long tail, a CDN doesn't really address my needs); what sort of bandwidth speeds can I expect from downloading B2 when a decent number of clients, let's say 100-500, are hitting a single object concurrently?

What happened with DO?

They blew up a Ceph cluster yesterday that was serving block storage (and they use Ceph for object storage, too, hence my concern). It was down for over thirty hours.

https://status.digitalocean.com/incidents/8sk3mbgp6jgl

I’d argue that Backblaze has significantly more experience dealing with large amounts of data, given their large backup service. Digital Ocean has been pretty amatureish for .. basically ever.

I agree. I'm not worried about their reliability. Just asking about bandwidth.

Sounds like a question you might be better off asking their sales team.

You realize their team is in this thread, right?

Yep, and they didn't answer. It's a question that sales would be able to answer. No need to downvote me for a legitimate response, right?

> they didn't answer

That was a strange oversight, sorry! Here is an explanation of our architecture regarding downloads:

Backblaze files are served up from a set of load balanced "download servers". Think of the download servers as "caches". The FIRST time you request a URL, the download server has to go fetch the file from the vault. The file must be reconstructed since it is striped across 20 separate pods (as soon as any 17 of these pods respond, it is enough to reconstruct the file).

The download servers have 10 Gbit networking and a fast SSD. The file is cached on the local download server's fast SSD. So the SECOND time you fetch the URL from one download server it can go really fast.

Ok, so the download servers don't talk to each other and there isn't any IP address affinity. So take the example where we have 20 download servers serving files from the area your one file comes from. If there is a lot of "hot" activity on one file (like 500 threads fetching the same one file in the same minute) then each download server fetches it ONCE from the vault, which means out of the 500 threads, about 20 early threads will get "lower performance" as the vault reconstructs the file, so maybe 2 - 5 Mbits/sec for the first file fetch on each download server? Then 480 threads will be fetching the file directly from the 10 Gbit/sec fast SSD cache servers. If the threads are spread evenly across the 20 download servers, so each download server serves up around 24 threads, which is easy.

Now some disclaimers: Backblaze has a TON of experience storing and writing files into our datacenter. However, we are still learning about serving files OUT of our datacenter. Recently we realized the SSDs in the download servers were a little small and files weren't getting cache hits enough. In other words, we could deploy fewer download servers if we gave them more local SSD space. So we're fixing that.

We also like to listen to customers and work with them. So if a knowledgeable customer experiences slow downs or they see an anomaly we can take a look and figure out how to improve.

Yev from Backblaze -> We've found that the majority of folks hit bottlenecks on their ends or latency issues before they hit any performance issues on our end!

Sysadmin at Backblaze here. Unless it's closer to 500+ and they all have 100Mbps+ connections, the clients will definitely be the bottleneck. Even in that case with smallish objects, it's unlikely that the TCP window will ramp up enough to put a dent in things. That number will of course continue going up with time as we expand and improve the infrastructure.

That's pretty awesome. Depending on how my DO Spaces tests go, I very much might try it out; it's a bummer that you guys don't support S3API, but I can deal with that. (I know Minio has a layer for B2, but I'd rather not go that way.)

Anyway, thanks a lot!

Is there any plan for a European presence? (At least, for the storage part)

Yev from Backblaze -> Yes! We do hope to have a location in Europe and are on the hunt for one! That said, it's a bit of a long process, so there's no ETA for now - but it's something we're actively pursuing!

Just to add a bit more color - we're actively aiming to have a European region by end of year.

Is it possible that it will be hosted in a similar fashion to how Azure does their German datacenters?

By 'similar' do you mean having a German trustee or something else?

Yea, I mean having a German (or other European) trustee.

Frankfurt has the best peering overall, if you also consider Eastern Europe in the equation.

The CLOUD Act thing being discussed presently (re: Microsoft) sounds like it could make things tricky. :/

Does it matter where in Europe for you? And curious which of the various reasons does a European presence matter to you most?

For me it would be speed.

Speed and latency for me as well.

Perhaps somewhere close to AMS-IX? For me, that would be network latency to non-US customers.

Thanks - definitely looking near there.

Gleb's the CEO so he'd know these things :D

Was anyone able to figure out what bandwidth/data is included with the packet.net "tiny"? It's roughly 50 usd/month and lists dual 1gps uplink - but I couldn't see anything about bandwidth. Eg hetzner will include ~30tb/month with a gbps uplink - or roughly 45 gb/hour or 100 mbps sustained.

Given the other prices packet.net lists for bandwidth (Starting at 0.05/gb) i assume it's not unmetered (that's about 300 tb for 1 gbps for a month).

Hmmm, I've found their Slack channel to be a good spot to ask Qs and get answers relatively quickly.

https://slack.packet.net/

None of the servers on Packet.net include any bandwidth - it's all usage based if it leaves the datacenter. Internal bandwidth is free from what I can tell.

Packet wants me to contact them for a quote? I like BackBlaze's transparent pricing but won't be pairing Packet's compute service with it. Sorry.

Where'd you see that?

I'd never heard of Packet before today but I was able to see their pricing without any issues:

https://www.packet.net/bare-metal/servers/compute/

They have "Pricing" right in the top menu which shows the cost for all of their products: https://www.packet.net/bare-metal/

Yev from Backblaze -> that's very interesting! What was your workflow to have them request a contact for a quote? One of the things I really liked about them was that their website is similar to ours in that it's easy to see what everything is - so very interested to hear!

Wow ... mea culpa and apologies!

I appear to have taken some weird path through the top-level "Product" menu - I'd love to say that the "Pricing" link wasn't immediately to the left of the "Product" link but at this point I'm going to have to simply claim insanity. Now that you've all steered the raving lunatic to the very transparent pricing, I'll shut up and try to find a project to justify bare-metal.

Why single out these two providers? Shared parent company? Are they paying out of some marketing budget? Cheaper peering for some reason?

Disclaimer: I work at Backblaze.

> why single out these two providers?

They are the first two. We are very open to more!

> Shared parent company?

Backblaze is owned by employees. 100% of the board of directors work at the company every day. We have no parent company. :-)

> Cheaper peering for some reason?

Yes. We made sure any candidate partner has a direct cross connect with our locations and the data goes over pipes we own so our costs are "fixed". After you own/lease the fiber line the bandwidth is "mostly" free inside of it. I say "mostly" because we still have to pay a tiny amount for each network port on each end, some electricity, etc. But it's negligible compared to paying for bandwidth at market rates.

This is great news, looks very interesting for data warehouse scenarios now with something like Spark or Drill. Are there any specs on throughput available to compute on Packet?

Side note: never heard of SlicingDice before but their website doesn't seem to be working, getting a cloudflare site offline page.

Agree - would love to connect some data warehouse services. (Seems Spark & Drill are not hosted, but a customer could run those in a compute provider.)

SlicingDice worked for me. Are you still getting an offline page?

Still doesnt work: https://i.imgur.com/g6HMbkA.png

Still can't beat a place in colo with few own transcode servers with FPGAs or transcoding asics.

What kind of performance are people seeing against B2?

I wonder how realistic running analytics over stored data is.

B2 is object storage. Similar to any object storage you generally want to use it for storing the data, not running heavy analytics directly on it. Usually you'd want to move the data to the directly attached block storage (which our compute partners offer) to run the actual analysis.

Yes! Bare metal power and Backblaze affordability! I so need this!

Hey Byran - what do you hope to do with it?

I've been a big fan of Backblaze for several years now. I'll make sure to stop by your booth at NAB and express my nerdy appreciation for your service.

Yev from Backblaze -> please stop by! :D

Reply on news.ycombinator.com