Shmig: Database migration tool written in bash | Hacker News Reader

Shmig: Database migration tool written in bash

github.com

58 pointstodsacerdoti25 comments

I would never trust a DB migration tool written in Bash.

Agreed. This looks like a monstrosity.

I dont think this type of negativity towards a shipped project do any good for any of us, why dont you start reading it's source code and telling him how can this be more secure?

I think we should keep comments in perspective. All large bash scripts look like a monstrosity. As far as bash scripts go, this one looks well done.

Counterpoint: I've seen enough tools written in C/Python/Java that are just issuing hard-coded SQL statements on the backend, so, at least with the Bash script I can at least inspect what it's doing before running it.

After looking at this repo id wager python that did the same thing would be much shorter and easier to read

Not to mention there is much better development tooling for Python: test suite runners, static type checkers, and at least one powerful IDE.

It's not like BASH is special and doesn't have any tooling. You can test BASH just as well as anyting else, use shellcheck for linting/checking, it has editor support, etc.

Genuine ignorance on my part: why is that?

For one, No checks are done to ensure dependencies exist before running a script unless done explicitly.

From Google Style Guide

> If you are writing a script that is more than 100 lines long, or that uses non-straightforward control flow logic, you should rewrite it in a more structured language now.

https://google.github.io/styleguide/shellguide.html

Sorry I hate to write snarky comments. The page says other tools have too many dependencies where this doesn't. That makes it a potentially valuable tool. If it works that's great.

Some of their style guides are a bit... particular to their situation, but this one I absolutely agree with.

There are so many pitfalls, so many weird interactions with pipes and error conditions in subprocesses, signals(!), etc. etc. that it can be hard to write even "bail as soon as ANYTHING goes wrong" logic.

(And that's not even accounting for things like "rm -rf $FOO/" going wrong. The shell fully embraced Stringly Typed Programming and this is the inevitable result.)

I have recently learned of linters and such for bash/sh scripts, but that's a bandaid at best.

If it's well written, it can do a better job than a badly written tool in any other language.

This could be quite a bit shorter and more readable in half as many lines of Python.

But bash is likely available wherever you'd like to run this.

So is Python.

Which version though?

Yeah, this is like manual vs. an electric can opener. Sometimes it's nice to have a posix script you can just drop in. I use a 200+ line bash script to set up/manage/and use my python environments and projects instead of python.

I would be incredibly nervous to use anything written in bash on a production database. It’s one primary advantage is portability but I don’t see how that’s useful for this case.

If you have giant migration scripts, for whatever reason, this tool is the only migration engine that would not choke on them.

Really? There are lots of tools out there which just rely on the db to run migrations.

I tried flywaydb, and some others. There are not a lot of migration tools that are not part of a bigger framework.

Can you link some examples that you think would gladly process data-migrations that are 1-2GiB large?

Anything which doesn't try to process the sql file but instead just passes it to the db to deal with, e.g.

    psql -d mydb -f migration.sql

I use a home-built solution that does this (160 lines of go), doesn't seem very exotic to me and did not require lots of work as there are only a few requirements for a working solution: Unique ordered names for migration sql files, use the db to store metadata about migrations run, use the db tools to run migrations.

Cool idea, honestly. I've often wondered if this were possible.

Schema looks similar to MyBatis-migrations.

Reply on news.ycombinator.com