Comment by LordHeini | Hacker News Reader

Comment by LordHeini | Hacker News Reader

Parent Full thread

Because Rails is a good technology for a lot of use cases.

If you want to develop a simple crud style app, with user login, mail sending and all that stuff it is highly productive.

Performance is okay and certainly good enough for the average web project.

Just get rid of the weird parts like turbolinks (unnecessary and messes up a lot of js) and don't write APIs using it (can't even rename fields easily).

There are similar Frameworks in other languages and they all have pros and cons.

Imho when it comes to web frameworks there is way too much emphasis on the startup style company and the ones with huge traffic.

Companies doing grunt work style web development don't waste their time on flavor-of-the-month.js.

If you build a website for a niche company to sell odd machinery on, you have to get shit done and you will never need to scale.

That means reliable, batteries included, highly standardized and easy to use. Rails ticks those boxes really well.

> Imho when it comes to web frameworks there is way too much emphasis on the startup style company and the ones with huge traffic.

That's exactly what is making web development decreasingly enjoyable as the years go on.

~99.5% of the web doesn't need the hyperscalable cool-tech du jour one typically reads about on HN. As long as developers don't fall for footguns, there's a lot that can be accomplished even with a language runtime like MRI. If scale becomes an issue, those things can be solved through horizontal scaling, optimizing database queries, not doing stupid shit that's memory-hogging, and moving expensive algorithms into another language. Ruby is perfectly adequate for serving HTTP requests. Whenever I've worked on a Rails app that everyone was frustrated with, the problems were almost always a compound of a bunch of dumbass shit that various developers piled on without much thought (otherwise I'd have seen them discussed in GitHub or Pivotal stories).

edit: to down voters: please educate me and not just down vote because you disagree with my question.

---

Every tiny rails app that uses rspec ends up with +1hr test times due to tight coupling between activemodel and the db connections.

Do you know of any good blog posts the describe how to do rails testing at scale?

> Companies doing grunt work style web development don't waste their time on flavor-of-the-month.js.

Rails devs have to deal with DHH's-flavor-of-the-month.coffee (coffeescript, asset pipeline, webpacker, import_maps)

When you hit scale. Rails is not the concern. It's the datastore.

All of the major performance problems at scale I've dealt with came down to a super giant table that keeps getting locked.

In another comment, it said Gitlab moved off ActiveRecord.

The fundamental problem with rspec + ActiveRecord is for each test: you create db state (tons of inserts), run the test (more db reads and writes), and then tear down the state. This is very expensive when you have 100,000s of tests each taking 500ms.

Rails/rspec does not make it easy to stub db state with ActiveRecord.

http://underpop.online.fr/r/ruby/rails/tutorial/ruby-on-rail...

Use transactional fixtures as much as possible. Minimal db usage between tests.

yes, that improves clean up time, but doesn't help with all of the db writes you have to make to create db state for each test case.

Initial state is loaded via fixtures or whatever. (slow and expensive)

Transaction is started. Run Test 1 Transaction rolled back. (quick)

Test 2 can be run without the expensive fixtures.

Repeat for all tests.

Does this mean if you only want to run the tests for `users_controller` during development, you still have to load the fixtures for all other tests with each run?

In the normal case. When you start a test it loads from fixtures into its own database.

Any test can make use of what is in that database. So you tend to put a bunch of stuff that is useful for most tests.

If a specific test needs an unusual setup, then it can do whatever is needed to load data. Generate data, load extra fixtures, etc. You can do this on a per test basis, or a group of tests, etc.

An option is to reload everything between each test. Not recommended.

Normal usage is to rollback any changes a test did, so that each test has a clean slate. This tends to be plenty fast. Parallelism can usually be done.

If speed is still a concern, you can get fancy and load up multiple databases with fixtures and divide tests between them. Maybe have 2-8 sets of tests running at same time.

It's been a few years, but I've inherited test suites that take 14 hours and been able to get them down to a few minutes with above techniques.

Something like Guard can be setup to automatically run tests relevant to what you're working on. So you don't have to run full suite each time.

> Every tiny rails app that uses rspec ends up with +1hr test times due to tight coupling between activemodel and the db connections.

Wat? I mean test times aren't great but I've got several thousand specs for what I would call a medium-sized application that run in <1m using TurboTests or <3m using rspec alone. I can't imagine a "tiny" rails app that takes an hour to test. Maybe you're doing a lot of browser rendering or something? My app is API only.

I think the fundamental design problem for rspec is for each test you create db state (tons of writes), run the test (reads and writes), and then tear-down (truncate). Each test is at least adding an authorized user (and then removing it) at the simplest and at the worst creating complicated db state to support relational models that must be added and torn down with each test (database_cleaner).

Gems like Fabricator or FactoryBot also make the "create db state" even more excessive, because developers would be lazy and use factory methods that create more than they actually need for the test.

For example, one project I stored a tree-structure that could be modified by api calls. Each test required re-creating the 15 node tree + each node's relational data (think node = user and all users must have a profile, authorization scheme, etc.).

I never figured out how to avoid resetting the db state for each test case.

This isn't an rspec problem, it's a Factories-for-ActiveRecord problem.

The two main options to mitigate are:

- just work with in-memory objects (FactoryBot using build or build_stubbed, or just MyModel.new) and stub/mock finders as needed,

- or there's this thing people like to hate that was introduced in Rails 1 called test fixtures, which allow you to prepopulate the database with some initial data and then your test just runs in a transaction and rolls back.

The best suggestion I've heard recently (though I can't recall where to give credit) for managing complexity w/ fixtures is to write an extremely minimal fixture set aiming for 2 instances per model with minimal data required to be valid, and then customize it as part of your test. For a blog example, that means you have two Posts in your fixtures and would write a test around drafts by updating one of them to draft status (be it state, toggle, or publication date) and then test your query, or controller, or whatever. This keeps the data churn inside each test minimal, while keeping the out-of-scope data minimal as well -- if there's only 2 blog posts, all your tests can assume there will be one you care about and one you don't, which is useful for testing filtering/search/visibility/authorization/etc while remaining pretty consistent.

> - just work with in-memory objects (FactoryBot using build or build_stubbed, or just MyModel.new) and stub/mock finders as needed,

Mocking the finders is obnoxious, since ActiveRecord returns a relation class, not an array. Plus stubbing out relational coupling isn't easy (e.g. the user model might have a `company_name` method that delegates to the company table.).

> - or there's this thing people like to hate that was introduced in Rails 1 called test fixtures, which allow you to prepopulate the database with some initial data and then your test just runs in a transaction and rolls back.

Fixtures and Factories have the same issue are vulnerable to inserting unnecessary data for the test. Models may have validation requirements that must be satisfied to be stored, but completely unnecessary for the test. For example, all users need a `company_id` with a foreign key constraint, so to test _anything_ on user, you have to insert a valid company as well.

Maybe I need to re-evaluate fixtures though, since they would be simpler to run than a factory.

The idea is that you write one set of fixtures for the whole app that you then use in all tests. You'd have a valid company and a valid user in your fixtures, but that's fine because you probably want to test both the company and user models. In the user model tests, you can ignore the existence of the company fixtures, and vice versa.

Since the inserts only happen once for the whole test suite, the marginal cost of adding more fixtures is minimal, so it makes sense to just make the fixtures as complete as possible.

interesting. I haven't used fixtures before. Wouldn't this make individual tests slow (like in development?) since all fixtures must be inserted all the time?

It doesn't add as much overhead as you might expect because the fixture data is inserted into the database without instantiating any ActiveRecord models. Unless you're loading a truly crazy amount of fixtures (gigabytes?), the database can ingest all the data in single-digit milliseconds.

I don’t think anyone has to use coffee script.

Our company has a large code base. Tests run in about 15 minutes in CI.

What testing tools are you using? rspec+database_cleaner+FactorBot/Fabricator?

How many tests do you run?

I worked on GitHub's Rails app. The main way this was solved was by running tests in parallel. It's now available in default Rails: https://edgeguides.rubyonrails.org/testing.html#parallel-tes...

In addition to that, the test suite performance was regularly monitored. And like any performance issue, we'd instrument and then fix issues as they came up.

(duplicating another comment I wrote to hear your thoughts)

The fundamental design problem for rspec is for each test you create db state (tons of writes), run the test (reads and writes), and then tear-down (truncate). Each test is at least adding an authorized user (and then removing it) at the simplest and at the worst creating complicated db state to support relational models that must be added and torn down with each test (database_cleaner).

Gems like Fabricator or FactoryBot also make the "create db state" even more excessive, because developers would be lazy and use factory methods that create more than they actually need for the test.

GitHub uses minitest, but same situation.

I don't exactly agree that it's a design problem. I think there is a lot of value in testing at a high level (controller tests) and having them touch the full stack (down to the DB). Gives a lot of confidence that the code will work in production. Throwing money at it (running the tests in parallel) pretty much solves it.

I have seen FactoryBot use get out of control (creating 8x as many records as you'd expect). That's a really easy way to slow down a test suite :). One way I've found to fix that is by adding tests for the factories, and asserting they are only creating what you want them to.

On model tests, another thing GitHub did well was encourage using `.build` when possible to avoid writing the the DB.

> I think there is a lot of value in testing at a high level (controller tests) and having them touch the full stack (down to the DB).

There is a lot of wasted work. Each (similar) test would insert and then delete the same rows, but make a little tweak.

Other languages and frameworks (java, python or golang) don't need to test the db and run super quick. Most db systems do not actually need to be tested. e2e testing can be done manually via curl or qa.

> Throwing money at it (running the tests in parallel) pretty much solves it.

Tests still are slow locally.

> One way I've found to fix that is by adding tests for the factories

That is interesting. I had a meta-test that would limit the number of sql queries a test could trigger.

> Most db systems do not actually need to be tested

Is this like when people say you don't need to test functional code?

I've never heard that. I mean, you can trust postgres to correctly execute the sql you give it, as you can trust active record to work as documented.

Reply on news.ycombinator.com