Supabase Local Dev: migrations, branching, and observability
supabase.com
supabase.com
a few months ago one of customers migrated away from supabase and they wrote a blog post about it. That blog post appeared here[0] on hacker news. many of the issues they encountered were related to local development. we made several promises to improve based on that feedback and the various comments in the HN thread
today’s launch delivers on many of those promises. We’ve added better support for database migrations, seeding, backups, debugging, and documentation.
we have a lot of work ahead, this is just the first step. our next major step forward is “branching”, which we’re rolling out today for development partners and alpha testers.
we’ve coupled the branching functionality to GitHub for now. whenever you create a new PR we launch a new instance, run the database migrations in your version control, and seed the database for reproducible test environments. we’re using Firecracker[1] for every preview environment. This environment automatically pauses when it’s not in use. we’re seeing some very impressive startup times, even though we’re stuffing a lot of services inside the VM. We looked at making full-production clones but decided against that for now until we have a robust strategy for anonymizing production data and mocking out calls to external services. Ultimately we want to offer both options, it’s just easier and safer to start with seed data.
since supabase offers a few services beyond the Postgres database, we still have a few questions to work through with our alpha testers. for example, we also store images/videos/files for our customers. Do these need to be anonymized in preview environments? we don’t have all the answers yet, but we’re moving in the right direction. As hard as it was to have a customer migrate away so publicly, I’m proud of the work the team have done to improve on feedback
[0] https://news.ycombinator.com/item?id=36006018
[1] Firecracker: https://firecracker-microvm.github.io/
we receive feedback from a lot of channels so it's often hard to figure out what to build. In the early days it was about reaching feature-parity with other tools. now we have a bit more breathing-room to focus on "day 2" problems. I think our team is excited about this phase since it means we get an opportunity to build something new/innovative
I would love to hear what solutions you're choosing.
one of our goals is to provide only the tools/tech/features that you'd choose yourself or need to build to get started. If you're skeptical based on our technology choices then it's useful to receive that feedback
Unrelated to DBs, I've been thinking about trying to roll my own system for magic links recently. Even though Supabase has some of the lowest costs for MAUs, they're still too high if you're only using magic links, especially considering the related email rate limits [1]. I don't even know if I'm reading that right. Is it 4 auth related emails per hour by default?
I can run a Cloudflare Worker for $0.0000005 vs $.00325 for a Supabase MAU. Assuming it would normally take 2 Worker runs to generate and auth a magic link, a user that signs up and never comes back would cost me 3250x more if I use Supabase.
Not all users are equal and, for low value users that probably never convert to paid users, I don't need to give them a full blown user account with MFA, etc.. Magic link based auth is adequate for what I need and I don't want to pay between 300,000% (for Supabase) and 15,000,000% (for Auth0) markup above the raw compute costs for someone that signs up and never comes back. For a user that converts to a paying customer, I don't really care about the cost as long as I don't have to eat it for every free user I have.
I know there are other costs, and that the requirements for magic links are more complicated than at first glance, but those costs are relatively fixed in the context of magic links, right? If the only major ongoing cost is for email, where I'm basically expected to bring my own provider, the MAU cost for a user that only uses magic links feels like a bad deal.
This isn't just a Supabase issue either. The entire auth industry is similar. I need the simplest part of the existing solution, but I'm forced to pay, in both cost and complexity, for the complicated, expensive part of the solution that I don't need or want to use. Does that make sense?
1. https://supabase.com/docs/guides/platform/going-into-prod#au...
It's unlimited emails per hour, as long as you BYO SMTP provider. The default email service is only for testing, and not recommended for production. I usually recommen AWS SES or Resend[0] for unlimited emails
> This isn't just a Supabase issue either. The entire auth industry is similar.
Agreed - the industry prices on MAU, which isn't a great heuristic. for social websites, 1M users might be a low number. For B2B SaaS even 1,000 MAU could be high. For Supabase, we simply try to be fair and transparent (and we're an order of magnitude cheaper than other Auth providers). There are a lot of other things that you're _not_ paying for which we have to price in - regular security audits, zero-day support, etc.
Yeah. I was hesitant to toss examples of actual costs in there because I knew it wasn't really a fair comparison, but I wanted to try to make my point even though I don't have the ability to calculate the real costs.
If I had to sum it up in a way that translates into a good strategy for building mutually beneficial relationships, I'd say "don't profit off my losses". I want a partner like relationship, but the only thing anyone is currently offering is for me to be a customer. I have to make all the predictions on conversion rates, etc., so I'm taking all the risk while the platform owner (ie: you) makes a (high-margin) profit off every user I have, regardless of whether or not I'm generating revenue from that user.
> for social websites, 1M users might be a low number. For B2B SaaS even 1,000 MAU could be high
I would say it makes sense to bucket users into categories and split the feature set accordingly. I think that's what Firebase does [1A]. I think basic login types (magic links, password, social) are free and you only pay for users that need their identity platform.
However, there are a few problems with Firebase IMO. First, I don't like "free". It means my costs aren't realistic and I need to assess the risk of that offering disappearing. IMHO that's just another layer of complexity and risk and I'd rather pay fair value from the start. The second problem with Firebase is that it's going to take a decade of culture change at Google for me to trust any of their products, especially something that's "free".
Back to Supabase, what do you do if you want to add a significant feature that makes the current auth pricing unsustainable? Do you increase the price a tiny bit? What if I have a million free users and don't need that feature for them?
> and we're an order of magnitude cheaper than other Auth providers
This is a little unfair on my part because I don't know the true costs, but if that means you're only charging me 100x the underlying costs vs everyone else charging 1000x, that doesn't make it good value for me, does it?
I'd rather categorize my users and pay accordingly. I know this may not be realistic in terms of creating too many SKUs, but just to make the point (from my perspective)...
1. Free users get magic links. Pricing should be tied to real costs and be commodity like. Minimal cost (to me) is important. Long term, stable, predictable pricing is important. This comes out of my pocket, so I don't want you having a large margin on it and I don't want it fluctuating because small changes can have a large impact on me if I have a lot of free users.
2. Convertible users get passwords, social logins, TOTP, security keys, etc.. Basically they get anything that doesn't have external costs (to you). I'd be willing to subsidize these a bit, but not anything crazy.
3. Paying users get SMS, etc.. Basically they get things that have external costs (to you). I'd pay a large premium for these users and I'd be willing to take on all the external costs in addition to that premium (ex: I pay for all SMS costs).
4. B2B users get any B2B features and my (inexperienced) opinion is they fall into a category where the cost (to me) doesn't matter much.
The other thing that I don't like about having a uniform cost per user is that I know the cost per user isn't uniform and, if my costs aren't a function of your costs, that means you're taking on some risk in the prices you've set. What if your overall prices are too low even though my specific usage is already profitable? Do I have to endure a price increase?
Again, this is uninformed because I don't have a decent knowledge of the true costs, but for my use case (magic links only, bring your own email) the prices feel 100x too expensive, but for a B2B use case they feel 100x too cheap.
I'm sure it's difficult to accommodate all use cases in a way that makes everyone happy, so hopefully my perspective is useful feedback.
1A. https://github.com/255kb/stack-on-a-budget/blob/master/pages...
Are there plans to expand self-hosting support? The migrations are a big step forward.
Are you intending to fill the roll of being a framework akin to a "super django" type of deal? Again the migrations help a ton, but I've been hesitant to use Supabase for random projects because I don't want to rely on the platform, and I don't want random people on github who want to try or contribute to require a supabase account.
I'd love to use it more as a modular ORM for miscellaneous projects instead of the current "hosted platform", currently none of the tutorials (or github projects) seem to explain this route at all.
I think you actually do work for this purpose, and I think the docs mostly cover the bare minimum for self hosting, and I understand your business kinda relies on the hosted platform, but I'd love to see further tutorials and thorough explinations of all the features - currently some like the AI features aren't really explained if they work in the self-hosted or not, or if you have to do anything special for that.
EDIT: Also I do appreciate your business being opensource, and contributing to postgres so much! Sorry for the rambling, and I apologize if I'm blind and missed some obvious docs.
We don't plan to replace any specific framework. We simply want to make Postgres easier to use. You can use Django (or any other framework) and Supabase together. We provide some additional tooling on top, but we aim to make this tooling 100% compatible with other tools. As an example, here[1] is a change we made recently so that our Storage service works better with Clerk (a popular Auth service, which is a good alternative to our own Auth service). We plan to document this better - it was one of the promises I made in the OP.
> Are there plans to expand self-hosting support?
we made a few updates[0] this week to improve self-hosting based on the common feedback. if there is anything missing just let me know and i will do another round of improvements.
> currently some like the AI features aren't really explained if they work in the self-hosted or not, or if you have to do anything special for that.
nothing is feature-gated so everything works on self-hosted. That said, I agree that we can be better at explaining the self-hosting. We are putting a lot more effort into improving our docs in general. For self-hosting we can certainly be clearer about the boundaries where you are responsible (eg, you need to take care of your own backups, the AI features will require external acconts). we're working on this, but feel free to open issues where it's unclear we can address anything specific
Anyone have perspectives on pros and cons of local dev vs cloud dev environments with Supabase?
I've heard about testcontainers [1] before, which can be used for Postgres. I've used it a bit, but the Elixir library for it is still under development [2] so I haven't been able to try it at work. Elixir's Ecto library is pretty good at wrapping Postgres for tests though [3].
[1] https://testcontainers.com
[2] https://elixirforum.com/t/excontainers-throwaway-containers-...
This won't be going away. Branching will just be another option
fwiw, I've also heard from a few enterprise companies that the git-based branching model isn't as suitable for them, because every other tool in their stack works in a prod/stage/dev type model, and there is no simple way to make it work with (for example) ~30 different environments
The branching model is really an "all-in" solution. It works particularly well if you're using something like Vercel/Netlify for your frontend. If you're using a serverside framework (Django, Rails, Phoenix, etc) then it's not as simple. That said, I think it's the way the world is moving, and with the advent of cheap VMs then it makes it very plausible even for these serverside frameworks
For context, my local dev process is now as follows:
1. supabase db reset with seed.sql empty 2. run a preseed script that disables any triggers and removes default data that has been previously seeded in migrations 3. seed data 4. reenable triggers 5. execute any working migration files that I keep in a separate file
I've written a script that handles all this, so I have mostly solved this for myself - but this was mostly due to running into a bunch of challenges setting up my local env to work well. Very open to general comments on approach too - perhaps there is a simpler way
Would you mind sharing this in a gist? We can add this functionality to the CLI
We have added supabase migration up [0] command that runs only pending migrations (ie. those that don't exist in local db's migration history table). You can use that to test destructive migration locally with data from seed.sql.
After testing, you want to update your seed.sql with a data-only dump [1] from your local db. That would make CI happy with both the new migration and the new seed file.
> 2. run a preseed script that disables any triggers and removes default data that has been previously seeded in migrations
It sounds like the default data is no longer relevant for your local development. If so, I would suggest running supabase migration squash [2] to remove the default data.
To disable triggers before seeding data, you can add the following line to seed.sql [3]
SET session_replication_role = replica;
[0] https://supabase.com/docs/reference/cli/supabase-migration-u...
[1] https://supabase.com/docs/reference/cli/supabase-db-dump
[2] https://supabase.com/docs/reference/cli/supabase-migration-s...
[3] https://stackoverflow.com/questions/3942258/how-do-i-tempora...
I like being able to call supabase db dump (data only) and not touch code in the file at all - I get that adding SET session_replication_role = replica; is one line, but still my preference is to avoid. But like I said I already disable triggers ahead of the seed script running.
I currently use supabase db reset quite frequently as I make changes in development. Using supabase migration up would mean moving the latest migration out of the migrations folder, running supabase db reset, moving the file back in and then calling supabase migration up. Which is not the worst idea, I'd still be looking to automate those steps with my own script atm tho.
Re: squash I have been a little cautious to use it since I first noticed it in the CLI docs as I wasn't really sure what the actual outcome would look like If I have something like this in a migration script:
--set initial permissions INSERT INTO rbac.permissions(name) SELECT unnest(enum_range(NULL::rbac.permission_name)) except SELECT name FROM rbac.permissions;
what would squash do to handle this data?
> Using supabase migration up would mean moving the latest migration out of the migrations folder, running supabase db reset, moving the file back in and then calling supabase migration up.
We can definitely do a better job here. I'm adding support for db reset --version flag [0]. This should allow you run migration up without moving files around directories.
> I wasn't really sure what the actual outcome would look like If I have something like this in a migration script
Agree that we can do a better job with the documentation for squash command. I will add more examples.
The current implementation does a schema only dump from the local database, created by running local migration files. Any insert statements will be excluded from the dump. I believe this is not the correct behaviour so I've filed a bug [1] to fix in the next stable release.
if you haven't seen the metrics endpoint we do have an endpoint you can scrape for all your Supabase metrics, and we just improved the example repo quite a bit on how to ship those somewhere: https://github.com/supabase/grafana-agent-fly-example/
I commented on a HN post almost a year ago about how hard is to do custom Auth with Supabase. I still haven't find a good solution about it. For example, LDAP Auth is quite crucial in most enterprise settings, yet I have no idea how to do it with Supabase. I can find a workaround for PostgREST by putting a secondary API written in some other language and fiddling with reverse proxies. But how to do with Supabase, such that all other services (realtime,...) works nicely? Is it so hard to provide a function that accept a custom strategy given the HTTP request data?
I created an issue[0] almost a year ago on Supabase, which was transferred to Gotrue. I even provided some code examples from Laravel. Even if it is not specifically for LDAP, make some API available to do so, please.
Based on the issue (title and comment), it seems that you have asked specifically for LDAP support rather than a generic API.
Feel free to share some more details in the github issue, so the Auth team can figure out how best to support you. It looks like they have followed up asking for the use-case and they are just waiting for some clarifying details.
This allows for a frictionless onboarding process for startups.
Are there plans for Supabase to support that?
Is there an example project that demos the best way to setup these features?
edit: I see TFA links to the docs where most things are covered (https://supabase.com/docs/guides/cli)
You'll see there is a "supabase" folder inside it so you can clone it and run "supabase start" to get started immediately.
You can also hit the blue deploy button and you'll have a full application deployed in ~1 minute (frontend on Vercel, backend on Supabase)
from your edit above, perhaps you were just looking for the docs but let us know if there is anything else you need
[0] Migrations - https://github.com/supabase-community/vercel-ai-chatbot/tree...
[1] Logs - https://supabase.com/blog/supabase-local-dev#local-logging-a...
[2] Backup - https://supabase.com/blog/supabase-local-dev#easier-backups
Or go fully into supabase ecosystems and also use their types and their built in migrations and types
Our clients will work with the drizzle-generated tables (since they use PostgREST).
Make the platform SDKs to feature parity, make RLS easier to use, improve auth (for ex add anonymous users and native login), polish file uploads, etc.
Supabase has a ton of potential, so hopefully this is taken as constructive feedback!
Happy to provide more details on our experience.
The mentioned migrations for example, don't work if your database is not very simple (e.g. triggers typically won't work because the dependency order is wrong, custom types are not supported, etc.), and this has been the case for some time.
The documentation for the most basic functionality is also quite poor and requires digging through the TS source for detail. For example, here's the JS lib auth signInWithPassword function:
Log in an existing user with an email and password or phone and password.
Requires either an email and password or a phone number and password.
Parameters:
credentials (required) SignInWithPasswordCredentials [no link to what this is]
const { data, error } = await supabase.auth.signInWithPassword({
email: 'example@email.com',
password: 'example-password',
})
To be clear, I think it's a great product, and the open source aspect and great communication from the team is a big plus, but I do think more time could be spent getting the basic product right before chasing 100s of new features.
- features/ui missing from local development
- more secure triggering of edge functions from database (currently have to hardcode key in SQL)
- template URL support for edge function
- source maps are broken in edge functions
Can you explain what do you mean by template URLs? Do you mean route params like `/v1/functions/users/:id`? If so, you can use a framework like Oak[1] to handle them. Edge Functions will make the full path including querystring available to the router.
Also, you can use URLPattern API to match paths. Here's an example of RESTful API built using that: https://github.com/supabase/supabase/blob/master/examples/ed...
Source maps, is it broken during local dev or when you deploy the function? Also, by broken you mean in a stack trace the file / line numbers aren't accurate?
Excited to hear more! Being able to trigger functions from DB & cron triggers without having to hardcode a secret (which causes them to end up in migrations files) will be a huge improvement.
> Can you explain what do you mean by template URLs?
Oh, I had no idea that the functions routes were "wild card"! I don't think that's mentioned anywhere in the documentation, btw.
> Source maps, is it broken during local dev or when you deploy the function?
Sorry, I actually mean "import-maps": https://github.com/supabase/cli/issues/1338
"Source maps" (does deno actually use source maps?) ARE broken as well. EG: The line numbers in runtime don't line up with the function code in the IDE or even that is in the docker volume.
Good point! will update the docs.
> Sorry, I actually mean "import-maps": https://github.com/supabase/cli/issues/1338
I'll take a look at this issue. We've improved import map resolution in the last couple of CLI releases. But it looks like there are more edge cases.
fwiw, this post is exactly that. Everything in this release is an improvement to existing functionality within the CLI, as response to similar feedback we received here[0].
There are a lot of "behind the scenes" improvements which don't get visibility on HN - only new features tend to get upvoted so I think we have a reputation which isn't representative of our day-to-day focus.
That said, we know there are a lot of shortcomings remaining (as you point out, and the comments below). Please do continue to share details on your experience, in the github issues preferably, so that we can focus on the most important tasks first.
Prisma is already well-supported in supabase (since it’s just Postgres and prisma already works with Postgres)
edit: here is the prisma guide -https://supabase.com/partners/integrations/prisma