Show HN: I open-sourced the in-memory PostgreSQL I built at work for E2E tests
github.com
github.com
The cool thing about it is that you don't need any external processes or proxies. If your platform can run WASM (Node.js, browser, etc.), it can probably run pgmock. Creating a new database with mock data is as simple as creating a JavaScript object.
It's a bit different from the amazing pglite [1] (which inspired me to open-source pgmock in the first place). pgmock runs an x86 emulator with the original Postgres inside, while pglite compiles a Postgres fork to native WASM directly and is hence much faster and more lightweight. However, it only supports single-user mode and a select few extensions, so you can't connect to it with normal Postgres clients (which is quite crucial for E2E testing).
Theoretically, it could be modified to run any Docker image on WebAssembly platforms. Anything specific you'd like to see?
Happy hacking!
There’s also https://github.com/IPS-LMU/wasmsox
Lots of cool software that people hack together for fun. Just site:github and search!
Correct on PGlite only being single user at the moment, and that certainly is a problem for using it for integration tests in some environments. But I'm hopeful we can bring a multi-connection mode to it, I have a few ideas how, but it will be a while before we do.
There are a few other limitations with PGlite at the moment (related to it being single user mode), such as lacking support for pg_notify (have plans to fix this too). Whereas with this it should "just work" as it's much closer to a real Postgres.
I think there is a big future for these in-memory Postgres projects for testing, it's looks like test run times can be brought down to less than a 1/4 with them.
(I work on PGlite)
select foo();
Error.captureStackTrace is not a function
This can be worked around by just constructing an Error and taking it's stack property, captureStackTrace is just a convenience function, so hopefully they can fix that.
[1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
https://github.com/stackframe-projects/pgmock/commit/f80d9fa...
It seems to have beendeployed out to the demo already as it's working now. :)
For unit testing (also mentioned in the tagline of the project on GitHub) I could see you wanting something snappier than a real Postgres in Docker, so then... maybe? Purists of some following will tell you to use a mocking framework instead, but I think that something closer to the real thing would be better in all cases. This might be it, just be careful not to lull yourself into a false sense of security about how close to the real thing this is (isn't).
Why can't Postgres compile to WASM instead of x86?
And then lancedb released their embedded client for rust, so I went towards that. But it's still lacking FTS. So I fell back to sqlite. have some notes here https://shelbyjenkins.github.io/blog/retrieval-is-all-you-ne...
Update: this can apparently run in a browser/Node environment so can be created/updated/destroyed by the tests. I guess I'm too much of a backend dev to understand the advantage over a more typical dev setup. Can someone elaborate on where/when/how this is better?
Why not use something like https://testcontainers.com/? Is a container engine as an external dependency that bad?
I am aware of only a few settings that make a container nestle, or not, whether it is a vm, lxc/lxd type container, etc.
As soon as you have external processes that your tests depend on, your tests need some sort of wrapper or orchestrator to set everything up before starting tests, and ideally tear it down after.
In 90% of cases I see, that orchestration is done in an extremely non-portable way (like leveraging tools built in to your CI system) which can make reproducing test failures a huge pain in the ass.
Because the emulator lets us boot an "already launched" state directly, it's also faster to boot up the emulated database than spinning up a real one (or Docker container), but this was more of a happy accident than a design goal.
What's great is that your code still just depends on "postgres", so you can test against this in-memory version most of the time then occasionally (such as in CI) run that same suite but either a "real" postgres as a way to make SURE you're not missing anything.
The moment that you shove a mock in there, your unit testing. Effective but not the same. One of the critical points of E2E is that without mocks you know that your tests are accurate. Because this isnt Postgres I'm testing it every time and not that system.
>> Can someone elaborate on where/when/how this is better?
If your building PG for an embedded, light weight, or under powered system then this would make sense for verification testing before real E2E testing that would be much slower. (a use case I have)
Other than that its just a cool project and if you ever need a PG shim it's there.
If this is actually just Postgres running in an x86 emulator (*edit: originally this said "compiled to wasm"), then how could this be faster than Postgres in any given environment? I don't understand — if it were faster, wouldn't you just want to deploy this in prod in your weird environment rather than Postgres? Why limit this to mocking?
I've done this across multiple jobs, and it's amazing to be able to run your "mostly-E2E" tests in 1-2 seconds while developing and the same suite in the full E2E env in CI. It makes developing with confidence so fast and mostly stress free (diverging behavior is admittedly annoying, but usually rare).
I highly recommend using these if feasible.
Also nothing stops you from using a mock for some tests and a real database for others. It just comes down to trust.
It's faster, can persist data to fs, though less stable under heavy use than the full x86 emu e2e test server. I found pglite-server uses only 150MB ram compared to 830MB for pgmock-server. You can then use dotenv to checkout a new .env.local with updated DATABASE_URL for all your nextjs/prisma package.json run scripts
DATABASE_URL="postgresql://postgres@localhost:5432/awesomeproject"
"db:pushlocal": "dotenv -e .env.local -- pnpm prisma db push"
For trivial applications maybe it’d work, but with more complexity like anything that has risk of deadlocking or depends on the database shape and such solution subtracts from value as even small shift in behavior can snowball into critical problems.
Today I lean towards resource constrained E2E environment so that local test runners have opportunity to break if someone write anything grossly underperforming.
Not to mention that snapshotting DB after second and distributing this snapshot to test partitions is super fast and many times shaved multiple minutes from test suites.
It’s an interesting idea and definitely great learning experience but I think that target audience is limited.
* What was the inspiration for developing this project at work? Was running Postgres in a Docker container too slow?
* What did your CI setup for E2E tests look like before and after integrating pgmock into the flow?
* Was migrating over to this solution difficult?
Thanks!
Trying out some in-memory ideas, there was not too much difference to a fast SSD.
"version": "PostgreSQL 14.5 on i686-buildroot-linux-musl, compiled by i686-buildroot-linux-musl-gcc.br_real (Buildroot 2022.08) 12.1.0, 32-bit"> because performance is not usually a concern in tests
I hate to be a downer, especially because it sounds like a lot of complex work has gone into this, but I can't disagree more with test performance. I think for many projects test performance is more important than runtime performance – a 100ms request time is unlikely to cause issues for most applications, but a 10 minute test run can significantly hamper engineering productivity.
I don't understand why, as others have suggested, running Postgres in a RAM disk isn't an option. We did this on my previous team, tuned Postgres for better performance in tests, and ran all unit tests against it, the performance was excellent. Setup was trivial, still containerised like this project aims for.
Why run Postgres on an X86 emulator, in a WASM VM, in a Docker container... when one could just run Postgres in a Docker container and have all the same advantages? Someone commented that you could run it in a browser... but Postgres doesn't run in the browser normally so why would you need that for tests?
I've heard many people talk about "interesting offline scenarios", but I've yet to see an application do them properly. Some embed SQLite to good effect, but I've not yet seen anything running something like an embedded Postgres, and other than it being a clever trick that might be fun to implement, I'm struggling to think of a use-case.
The road to a slow test suite is reached in very small increments over time, so you have to be fastidious about all of it in order to stave that off as long as possible. A simple optimization that I've seen double some test suite speeds is to swap out slow password hash algorithms on login with a no-op, but just for the test environment.
If running your DB slower significantly slows down your test suite then it's arguable that you are touching the DB too much already and repeatedly testing the same things over and over again (like creating users in the database for each test instead of using something in-memory)
Personally, I think having a database and then tuning it for performance is the best option, followed by not having a database and being faster. Having the database but then running it in an emulator on a VM on your infra (which is probably itself a VM), as in this case, seems like a bad call.