Hacker News Reader

Comment by yingw787 | Hacker News Reader

What I really liked about Ruby on Rails when I took an online course on it were the schema.rb files and `rake db:migrate` command. I don't know if there's an analog in Python, but honestly I think clean, highly structured, maintainable data is way more important than the backend you use. Having the ability to quickly iterate your SQL schemas and roll them back as necessary is essential to shipping high quality features with fast changing requirements.

View on HN

Just wait til you actually work on some production code... For the most part you don't actually want to roll back schemas as you would lose data. And iterating on SQL schemas in a live app involves all kinds of tricky edge cases and backfills, knowing when to make your changes in multiple steps, being careful not to take down the whole thing because you didn't realize a certain change required a full table lock or a certain index would take so long to create, etc.

As is so often the case in Rails, it's set up to be nice and simple for a hello world example but little more. It doesn't actually make things as simple as they seem at first, and you're on your own to figure out which of its niceties are actually helpful and which are dangerous to use that you need to build your own workarounds for.

Or you inherit a DB that’s a complete disaster scheme wise and all the nice assumptions go out the window.

I’ve mostly found that migrations work best as a structure way to just run normal SQL and know a) what ran b) when.

For that they are handy.

I’ve even considered how to structure a stand alone migration running tool with a nice API since the good ones for each language tend to be fairly different.

It annoys me that the tooling around RDBMS in 2019 is still so shit.

Prime example, try to find a halfway reasonable way to debug a badly written MySQL sproc you inherited.

I’ve used Flywheel (https://flywaydb.org/) for this exact problem on personal projects before and been fairly happy with it. It’s intended for use in JVM-based projects, but ships with a CLI that gets you a long way.

Sqitch by David Wheeler is a great way to manage database schemas. I haven’t uses it in production, because I mostly use Rails and haven’t felt enough pain to justify the switch.

https://sqitch.org

> As is so often the case in Rails, it's set up to be nice and simple for a hello world example but little more.

This is just wrong. Rails works very well for 90% of the situations. When it stops working that’s great news:

You’ve won!

You’re Twitter, Facebook or the next big thing. You’ll find talent and money to replace your rails app.

I'm having trouble understanding how one could run into a bottleneck only after you reach Facebook or Google or levels. Surely if you have performance issues when you are running on 10,000 machines, you would also have a bottleneck when you start to outgrow your single machine.

Why would you need more than a single machine to serve a request? You need multiple machines to serve many requests concurrently.

> For the most part you don't actually want to roll back schemas as you would lose data

Can you name a framework where you roll back schema changes and don't lose data?

> being careful not to take down the whole thing

You still need this caution if you don't use rails

> a certain change required a full table lock or a certain index would take so long to create, etc.

I feel like you're just describing working with a relational database in a production setting. This isn't rails specific

> Rails, it's set up to be nice and simple for a hello world example

Sure. I'm also pretty sure it's used in production by some great companies whose applications/APIs are a pleasure to use - SendGrid, Github, Stripe, and last time I checked Hulu to name just a few.

Just because it's not the framework of choice for the majority of companies doesn't make it incapable of being used at scale.

Migrations are rad! Django (the Python equivalent of Rails) has had built-in migrations for several years now: https://docs.djangoproject.com/en/2.1/topics/migrations/

Also laravel and CodeIgniter for PHP.

And long before that, it had South: https://south.readthedocs.io/en/latest/about.html

Granted my rails experience is minimal at best, but I really favor Django's migrations. They're pretty fantastic.

I agree, and most other commenters still miss the point about what makes Rails so great :

it's not the technical part (you can point slowness, quirks in how the language/framework is used) but the human and business part. It works. It makes adapting to change easy. Yes it's a monolith, yes it encourages fat models, etc. But it works so damn well.

`rake db:migrate` is not so great as a feature (other frameworks in other languages have it), the greatness comes from how well it's integrated with the Rails workflow.

> other frameworks in other languages have it.

Yep, Laravel (PHP) lifted it en masse (so much so that when I had to get up to speed with laravel years ago I used the rails docs to fill in the blanks) and Symfony with Doctrine is in some ways better.

Every time I try to write some "super simple web app" that I think is much too small for Rails I always end up in spots like this: migrations, then asset fingerprinting, then some quick sanity tests, oh, now I need this other thing and I've cobbled together a creaky messed up version of Rails.

Python SQLAlchemy/Alembic and Django has db/schema migrations.

.NET has db/schema migrations in Entity Framework.

Node/javascript has sequelize/umzug, node-db-migrate and others for db/schema migrations.

Migrations are pretty common now for ORMs, or any code first schemas, but are definitely a killer feature and needed with ORMs to see what is being changed/iterated on.

Another option is Sqitch. It's database and framework agnostic.

https://sqitch.org/

Postgres Tutorial: https://metacpan.org/pod/sqitchtutorial

Came to this thread to suggest squitch. Glad it's mentioned here. I really enjoy it's three generic concepts - deploy, verify, revert - and not having to tie my migrations to any particular language or ORM. In additon, it's supported on every operating system I've used with it and has got a docker image in case that's not good enough.

Analog in python: https://docs.djangoproject.com/en/2.2/topics/migrations/

I suggest checking out https://flywaydb.org/ It doesn't have the ORM-ness mapping rails migrate does, but it makes it really simple to organize sql scripts to move/update schema configs.

I sometimes suspect a folder full of sql scripts would be better than black box migrations (which are a folder full of sql scripts or similar anyway).

Migrations are great, until they aren't.

Sometimes if you actually manually do the thing you know how it works better and can adjust. I suppose the drawback to that is everyone else doesn't.

Yeah I've thought the exact same thing before and I've actually tried both.

The clear difference is that migrations are easily reversible. Good migration libraries can also reverse data migrations.

A folder of SQL scripts was very refreshing, but no way could I use that system in a production environment. There's just too many edge cases to keep in your head.

Manually migrating has the issue of testability and hit by the bus. I don't believe every migration needs a roll back, sometimes the better strategy is restore from a snapshot/backup.

Another option is a schema management tool that doesn't rely on migrations at all. I'm the author of one, Skeema [1], for MySQL/MariaDB. Others include migra [2] for Postgres, and sqldef [3] for MySQL or Postgres.

These all use a declarative approach [4], where the user just specifies the desired new state, by adding or changing the set of CREATE statements. The tool knows how to run the correct DDL to reach that new state.

[1] https://github.com/skeema/skeema

[2] https://github.com/djrobstep/migra

[3] https://github.com/k0kubun/sqldef

[4] https://www.skeema.io/blog/2019/01/18/declarative/

How Interesting. Thanks!

I personally tend to prefer migrate[1] as it doesn't require java.

[1]: https://github.com/golang-migrate/migrate

Migrate is awesome! My only issue: Since it uses bare SQL for the migrations, you have to do some lifting if you need to support multiple RDBMS with different DDLs. I have some regexes in my project to translate Postgres DDL into SQLite DDL for my test suite.

Why would you not test your code with the same RDBMS you will have in production? Especially if it’s something as easy to install as Postgres? It seems like begging for trouble...

If you're using an ORM, it can be an easy way to ensure that no vendor specific stuff sneaks in. Also, until recently it was difficult to run, e.g., SQL Server in Docker, so Postgres might be easier to test with. Agreed SQLite is a weird choice, but not all integration tests are robust enough to test characteristics of the underlying db, i.e. focus on business logic not performance.

I have several dozen tests that run in completely separate DBs. With SQLite, the test suite runs through in 1 second:

  $ time go test github.com/sapcc/limes/... | grep -v 'no test files'
  ok      github.com/sapcc/limes  0.004s
  ok      github.com/sapcc/limes/pkg/api  0.515s
  ok      github.com/sapcc/limes/pkg/collector    0.449s
  ok      github.com/sapcc/limes/pkg/core 0.006s

I doubt that you can setup and tear down several dozen Postgreses in that time.

Flyway doesn't require Java. The CLI ships with a private JRE as well as drivers for most databases out of the box. Simply untar and run.

Java seems ideal as it will be completely cross platform and you can add / update the database drivers after the fact.

Migrate has a different binary for each platform and from what I can see, the support for each database is compiled in, so just to update a driver you are going to be maintaining that binary all over the place.

The activerecord-migrations gem allows you to use Rails migrations with other languages too—I’ve used it with node, clojure, and others where the community migration libs aren’t as good.

https://github.com/diesel-rs/diesel

This type checks all ORM operations, and type checks against the actual schema at compile time. It's pretty cool and made by one of the long-time developers of ActiveRecord.

Reply on news.ycombinator.com