Supabase Storage v2: Image Resizing and Smart CDN | Hacker News Reader

Supabase Storage v2: Image Resizing and Smart CDN

supabase.com

109 pointssamwillis30 comments

hey hn, supabase ceo here

hopefully it's obvious from the title what we're shipping today. "Supabase Storage" is for larger assets - it's a wrapper around s3, where the buckets/folders are mapped into your postgres database so that you can write access rules using Row Level Security.

This release adds image resizing. It looks like this:

    supabase.storage.from('bucket').getPublicUrl('image.jpg', {
      transform: {
        width: 500,
        height: 600,
      },
    })

In tandem added some webhooks/events to the storage server to enable some smarter cache-busting. We're using Cloudflare for caching on our hosted platform, but the addition of webhooks should provide the primitives to use any CDN/service.

The Storage API is 100% open source and can be hosted as a standalone server (with Postgres+PostgREST): https://github.com/supabase/storage-api

> it's a wrapper around s3

> We're using Cloudflare for caching

It sounds like Supabase is fairly explicitly targeting being the glue between a bunch of lower level infrastructure, that may be in different cloud services, and providing one platform with a better UX. Is this fair to say?

If so this is an interesting strategy to be so open about this. Other platforms-as-a-service don't seem to like to acknowledge the services they use under the hood. I can see this being both a positive (brand recognition, trust), and a negative ("we could just build this ourselves and have more choice").

Is this an intentional strategy? Can you talk any more about how you're thinking about it?

> explicitly targeting being the glue between a bunch of lower level infrastructure

It might be more precise to say that we provide the primitives that a developer needs to host Supabase on any cloud provider. Becoming the "glue" appears to be a byproduct of this approach

For example: we use s3 right now, but we've built the storage server so that it can be extended with other storage providers (in fact, it has a "local disk" provider for developing locally).

We use Cloudflare right now for the CDN, but as mentioned in the blog post the Webhooks we built into this release are so that we could migrate to any CDN service without locking ourselves into any CDN-specific cache invalidation.

Portability is something we think about a lot (although harder to solve this problem at a CDN-level).

> Other platforms-as-a-service don't seem to like to acknowledge the services they use under the hood Is this an intentional strategy? Can you talk any more about how you're thinking about it?

There's no strategy behind it. We are trying to build a great developer platform, and at our current stage these platforms provide a better service for some features than we could ourselves. There's no shame in that from our POV, and these services deserve the credit for delivering something useful. I doubt we'll ever get to a "build our own datacenter" stage - we'll probably just continue to focus on the application level (with a Database-centric view of the world)

I’m perhaps overly bullish on your platform so I say never say never!

Transparently, I hope you guys are hiring developers again soon, I had look into becoming a technical engineer on the sales side but we couldn't make the numbers work.

I really can't wait to see your guys next round of innovations!

Great point. Maybe 2y ago I asked Vercel salespeople about their infr and learned it was virtually all AWS. (Technically "multi-cloud" bc they used GCP in some minor way.)

Any plans to switch to R2? It’s cheaper than S3 and would run in network with Cloudflare, allowing Supabase to take advantage of the CDN more optimally over time

It's one of the storage providers we'll investigate, now that R2 is GA.

Our Storage server is hosted on AWS, so it wouldn't immediately save money (at least for egress) but we'll also investigate moving the servers to another cloud provider.

Could this eventually be pluggable with bring-your-own s3-compatible provider? Would love this in front of my DigitalOcean object storage

yes definitely. It should be easy to add s3-compatible providers (which are numerous, since s3 seems to be the defacto file storage API now).

We haven't added "BYO" functionality to our hosted Platform because, frankly, we already spend a lot of time debugging various quirks and network issues - even for infra which we provide. Adding an extra level of opaqueness at our current team size would be very stressful from a support-perspective. That said, we're designing/developing so that it's possible and we'll continue to assess as the team grows.

Hi, I know this is off topic but I would appreciate if there was a C++ library for the https://supabase.com/realtime product. I would like to use it for sensors but no C++ library means I can't use it on my Arduino or ESP32s.

Great work. Congrats!

I am looking to build a file management app using Supabase + NextJS (e.g. Dropbox clone). Can you please point me the right way. Thanks!

I'm not sure if I could describe the full implementation of a Dropbox clone in a HN comment, but our Storage Docs[0] should give you an idea of most of the details that you need.

If you prefer videos, I see there are a few tutorials on YouTube when you search "Supabase Storage"

[0] Storage Docs: https://supabase.com/docs/guides/storage

When will the storage dashboard be open source?

Just caught up with the FE team on this one - it's pending in this PR: https://github.com/supabase/supabase/pull/10810

One of the engineers is testing this now and it should get merged within the hour. The build for amd64 takes about 6mins, and about 4 hours for arm64. I'll update here when it's merged (edit: it was actually merged earlier today, I originally linked to the wrong PR)

For more context, the Storage UI was merged a couple of weeks ago but we simultaneously rolled out a new login screen for our Dashboard (for SSO) which then got rolled out to the Docker image. We've added a feature flag. Sorry for the delays - the lead up to launch week has everyone a bit stretched.

What a great comment. I applaud the transparency and depth. Nice.

Supabase Engineer here

There were a lot of primitives we built to ship image resizing - Storage events exposed via webhooks, rate limiting, a queue on top of Postgres, a smart CDN cache, object metadata endpoints, etc. These are already available if you are self hosting Supabase Storage, so that you can integrate your own CDN, listen to storage events, etc. Over the next few months, we will be working on exposing these on the Supabase platform too.

In nearly two decades developing for the web, I've never found a good reason to enlarge an image server-side, so thank you for anticipating that footgun. So many of your competitors get it wrong.

However, it's unclear from the docs [0] if one of the most common use cases is supported: requesting an image at a specific aspect ratio while not exceeding a maximum size.

For instance, I want a 1600×900px hero image at the top of my blog posts, but my source image is only 1200px wide. I request:

  transform: {
    width: 1600,
    height: 900,
    resize: 'cover'
  }

Will the returned image be 675px tall, maintaining the 16×9 aspect ratio I want?

A few more notes:

The blog post states "default mode maintains aspect ratio and the resize mode is fill", but the docs say that cover is the default mode.

The docs say, "If only one [width or height] parameter is specified, the image will be resized and cropped, maintaining the aspect ratio." I believe that with only one parameter, you could either resize or crop, but not both. I hope it's resize.

For cropping modes, is it always from the center? Are there plans to support other "gravity" modes?

[0] https://supabase.com/docs/guides/storage/image-transformatio...

> The blog post states "default mode maintains aspect ratio and the resize mode is fill", but the docs say that cover is the default mode.

The default resize mode is cover and the blog post had a typo. Fixed now, thanks!

> resize or crop

We crop by default since the default mode is cover.

> gravity

we will be adding crop with gravity soon.. currently the crop is always from the center

> requesting an image at a specific aspect ratio while not exceeding a maximum size.

I'll get back to you on this one - Inian had to get some sleep, it's very late (early) for him

Thank you for following up. IMHO, object-fit is not the right model for server-side image manipulation, since its behavior is only specified when both width and height are supplied. I've also never seen a real world use case for stretching, squeezing, padding, or enlarging images on the server. That should be done on the client. I mentioned some of the same issues on the Cloudflare forum when they announced an image resizing product based on object-fit.

I believe all resizing operations a user might realistically want can be achieved with three parameters: width, height, and crop. Additional x and y parameters (floats between 0.0 and 1.0) could be used for setting the focal point of the crop box.

Examples:

  source.jpg?width=400

Iff source width is greater than 400px, scale down to 400px wide.

  source.jpg?height=300

Iff source height is greater than 300px, scale down to 300px high.

  source.jpg?width=400&height=300

Iff source width is greater than 400px or source height is greater than 300px, scale down to fit within a 400px by 300px box.

  source.jpg?width=400&height=300&crop=true

Crop source to a 4x3 aspect ratio. Iff result is larger than 400px by 300px, scale down.

actually, image upsizing is pretty much required for all of the AI Art generators. was even a big feature in SD 2.0. i'd love to have it as part of this API but itd probably cost too much

nginx already comes with: http://nginx.org/en/docs/http/ngx_http_image_filter_module.h...

We assessed a few resizing tools - imgproxy, Imaginary, Sharp, etc. We went with imgproxy mostly because of it's feature-completeness (for example, it supports AVIF and it looks like nginx does not)

Is image resizing built on top of Cloudflare too? https://developers.cloudflare.com/images/image-resizing/

We use imgproxy[1] for doing the resizing. We utilize open source services for the entire stack, so that you can self-host Supabase.

[1] https://github.com/imgproxy/imgproxy

Great job! What are the implications on resizing regarding storage egress & download size? Can I expect both to be smaller when resizing an image?

Yes! If you resize your images correctly, your users use lesser data to load assets from Storage. A side benefit is you will be paying us lesser in egress fees.

Are images really this expensive to resize?: "it is $5 per additional 1000 images transformed.". Perhaps then we should ship a WebAssembly module for image resizing to the client and only validate image sizes server-side.

I love the product and this update. I wondered if there are any plans for handling audio (or even video) in a similar way?

Thank you! We will be working on video after we implement more image transformations. What is your usecase for audio and what transformations would you like to see?

My use case is in education, with users uploading short audio clips. Often these are in a variety of formats. It's nice to be able to re-encode them all to the same format and save as much bandwidth as possible as well as make sure they are supported in the weirdest variety of devices.

Reply on news.ycombinator.com