Hacker News Reader

Comment by indrekju | Hacker News Reader

I hate when people are using it in the unit tests. It just makes testing very convenient, so they start using it everywhere. In the end, the test suite gets very slow. Unit tests should not hit the database.

First example there: test_new_customer_defaults. Really, it needs 3 saved models to test this? FactoryGirl isn't solution here. It's just hiding the problem.

View on HN

If I could upvote the parent any harder, it would hit orbit.

If you need a complex network of collaborators to test a method, the answer is not to obscure the creation of those collaborators, it's to refactor so that each object talks only to its immediate neighbours, and then stub the heck out of them. If it absolutely has to be something that acts like an ActiveModel, then use mock_model().

And I'd upvote it to Mars. FactoryGirl is completely an anti-pattern, unless maybe you're doing acceptance testing, but you know even then I wouldn't want FactoryGirl in my codebase, people would be too damn tempted to use it everywhere.

That's where we find ourselves currently: constructing 3-4 levels out due to Law of Demeter violations, and it becomes incredibly brittle when seemingly unrelated classes change.

IMO, DHH (and others) argue too hard against core OO principles (like SRP and LoD) that have the possibility of helping out when the domain model becomes more complex. "Just use Rails" is fine when spinning up, but as the app grows, it becomes increasingly less pleasant. Coupling increases in the name of less code.

Ideas like hexagonal architecture look nice, but I'm left wondering how to implement them when the framework insists on infecting core domain classes (AR::Base).

So can someone recommend a Ruby (edit: ideally Sinatra/Sequel, not Rails) app one could look at with a more principled test suite? I'm coming from a more traditional OO background and jumping into Ruby. It's good to see I'm not alone in finding this militant pragmatism somewhat off-putting, but I don't want to reinvent the wheel.

As an FYI to your last point - in Rails master (which will be Rails 4), you can active-record-ify a class by including ActiveRecord::Model, rather than inheriting from ActiveRecord::Base. In fact, that is all ActiveRecord::Base does itself - https://github.com/rails/rails/blob/master/activerecord/lib/....

But your core domain classes are still coupled to their persistence mechanism in the name of 'pragmatism.'

Who says they have to be? That's a design decision. There is literally nothing (besides maybe inertia?) that enforces that "rule".

If you want to be sure to keep FactoryGirl out of unit tests but still have the benefit of using it in higher-level tests, you can use a different test/spec helper. I tend to do "spec_helper_lite", which sets up the bare minimum for unit tests (no FactoryGirl), and "spec_helper", which includes FactoryGirl for higher-level tests.

I say "higher-level" tests to refer to integration, system, and acceptance tests, because I don't have an opinion of where to draw the line for FactoryGirl. Is there a reason you would consider FactoryGirl for acceptance tests, but not integration or system tests? Just curious, because I might learn something.

Integration tests just mean you are testing the integration of multiple components, say view and controller, model and database, controller and model. Acceptance tests are tests from the end-user's perspective and touch the entire stack like a real user would. Acceptance tests are integration tests. It's alright for these tests to hit the database because part of what you are testing is the integration with the database.

Yep, I'm trying to dig myself out from this problem now.

The Rails Tutorial (railstutorial.org) uses Factory Girl and integration tests for just about everything, which is a fast enough approach for its toy application, but I've found that when your application gets more complex your test suite takes forever to run, and setting up the tests gets more and more painful (especially when you start dealing with external APIs).

Does anyone have any opinions on the best resource for teaching effective unit testing in Rails? I'd love to speed up my test suite while not losing coverage.

It's not really focused on unit testing (though it does cover it) so much as it's focused on good design, but the two are related, so I found Avdi's "Objects on Rails" [1] helpful.

[1]: http://objectsonrails.com/

I'm not sure if it's the best resource, though. I'd love to hear other opinions.

You can use the build method to instantiate the model without saving to the database. Very useful because you only need use create when you need persistence in your test - such as when testing a controller.

Remember to ensure that your build strategy is set on all associations if you want this to work, e.g.

  FactoryGirl.define do
    factory :user do
      association :account, :strategy => :build
    end
  end

Otherwise when you build a user, the account association will still get persisted to the database.

Even if you do this, though, you're still spinning up a whole network of real objects in what is supposed to be a unit test.

I'll go one recommendation further. Just don't couple user and account. When making a factory named after a model, only set model attributes and never set relations.

    factory :user    # sets only user fields, no relations
    factory :account # sets only account fields, no relations
    
    factory :user_account {creates both}

When you have a large # of tests/factories, it's nearly impossible to keep track over time what the :user factory creates. Today it's one relation. Years from now, it could be 10 other tables relations. Someone later on decides they want to reuse the :user factory, but they don't want the account to be create, so then they do :user_only factory. Yada yada. Shenanigans ensue.

If you stick to the above strategy, things are much faster, but more importantly, there are less unintended surprises, like associations doing a create instead of a build, etc.

That sounds like pretty good advice. I do not have a very large base of factories and tests yet but I can see how unwieldy it could get without really sane naming conventions.

My preference is to have the base factories set everything that is needed for the generated record to be valid, and nothing more. If your User model validates the presence of the account, then the account association should be set - if not, it shouldn't. Otherwise, FactoryGirl.create(:user) is going to provide you with an unsaved, invalid record, which would certainly be surprising to me if I were using that factory for the first time. If all you want is a consistent set of initial attributes, then define a constant USER_ATTRIBUTES somewhere with a hash of values, and pass that to #new - using factories for this purpose is a bit sledgehammer/walnut.

I agree that excessive association building can get out of hand (and the latest version of FG even allows you to subscribe to factory events just so you can work out what the hell your test suite is building - a sure sign that you've lost the ability to reason about your tests if ever there was one), but completely abstaining from building associations seems to remove more or less all of the attraction of using factories as a fixture replacement in the first place.

All of the above applies only to integration/acceptance tests, though, where factories-as-fixture-replacements are at least arguably justifiable; my recommendation for unit tests is "don't use factories at all."

One compromise (edit: on the performance problem) would be to use FactoryGirl to generate some type of (yaml?) cache pre-test-run, automatically refreshing only when the model changes.

I think you're missing the point of what makes FactoryGirl an anti-pattern in unit tests. FactoryGirl reduces the feedback that you can get from unit tests by hiding complexity (of creating objects), which your unit tests are supposed to make apparent.

Edited to add: Using FactoryGirl to generate a cache only addresses the performance issues without addressing the complexity-hiding problem.

I don't disagree; I was only trying to address the 'test suite gets very slow' part rather than the 'unit tests should not hit the database' part.

If performance wasn't an issue, the convenience of auto-created objects would mean more 'unit' tests (edit: usually closer to integration tests) that wouldn't exist otherwise. Sure it's best to de-couple things, but there's room for something between no [unit] tests at all and perfect unit tests.

I could also argue that's exactly WHY to use factory girl in unit tests. You're not using FG to create the things under test, only the things to pass to the thing under test.

Complexity is a moot point for those extraneous objects. You want that complexity out of the tests since it's not the focus of the test itself, and makes it more readable.

But I'll take this one step further. To each their own. There is no one exact correct way. Do what's right for your code, domain, and tests. I wouldn't test an API codebase the same I would a site codebase or a gem codebase anyways.

I would argue that those aren't unit tests, because they depend on the collaborators that you're passing into the unit under test. A change to those objects could break your unit test. That makes it an integration test.

If you're using your unit tests to inform your design, then the complexity in the extraneous objects is relevant, because they tell you how coupled your unit under test is to its dependencies. This is what FactoryGirl hides from you.

I tend to use fake objects for the collaborators. Usually just a vanilla test double will do, but sometimes I use RSpec's stub_model if the collaborator has to behave more "model-like".

I'd recommend http://pyvideo.org/video/631/fast-test-slow-test and http://www.confreaks.com/videos/641-gogaruco2011-fast-rails-... if you haven't seen them already.

And I would argue that arguing bout what a unit test is and isn't has missed the topic of this parent post entirely. I have seen those links and I know both people as they worked where I work now. Arguing over speed and/or what a pure unit test is has never been my cup of tea because every single codebase is different for a variety of reasons.

Tests and code don't always fall into these neat cult buckets we've drawn up for ourselves. Good doesn't mean just fast. It can also mean readable, to wit FG usage addresses quite nicely. Just another tool in the toolbox.

Just like ruby, I can write crap tests with or without FG and great tests with or without FG. Same goes for stubs, doubles, and collaborators.

> And I would argue that arguing bout what a unit test is and isn't has missed the topic of this parent post entirely.

You do raise a valid point: we're arguing about a definition. I don't think that misses the point though, because this thread is specifically discussing the use of FactoryGirl in unit tests. Where I disagree with you is on the importance of definitions. I think it's important to understand the definition of a unit test vs an integration test vs a system test, because without agreeing on those definitions, we don't stand a chance of having a reasonable discussion. If we have different definitions for a word, then our arguments will be talking past each other.

> Tests and code don't always fall into these neat cult buckets we've drawn up for ourselves.

(I'll ignore the loaded language, because that's not conducive to rational discussion.) From this point on, you use the word "tests" instead of "unit tests," and your points are valid because you're talking about the broader category of tests. I'm not. I'm specifically discussing unit tests, which has been the topic of discussion for this entire thread, going all the way up to the top-level comment by indrekju.

> because every single codebase is different for a variety of reasons

I agree with you here. Every codebase is different, and so are their testing requirements. For small gems, I tend to write all integration tests with few unit tests. For larger Rails projects, I write mostly unit tests with some integration tests to catch the cases where an interface changed but I forgot to update a fake object. If you're saying that different codebases differ in their relative requirements of different types of tests (unit vs integration, etc), then I agree. In fact, I would take that a step further, and say the testing requirements also depend on the team. But if you're arguing that the definition of a unit test changes depending on the codebase's testing requirements, then I can't agree.

I'm not refuting the usefulness of FactoryGirl in integration, system, or acceptance tests. Just unit tests.

Having definitions for jargon gives us tools to learn and discuss things at a more abstract level. That doesn't necessitate applying those ideas dogmatically. If anything, I think it helps you understand the trade-offs so that you can be pragmatic, because pragmatism depends on understanding.

There's a saying in nursing: "If it's wet, dry it. It it's dry, wet it."

I think we need the same thing in programming.

If it's slow, make it faster. If it's unreadable, make it readable. If it's coupled, decouple it.

Then just stop worrying if we fall into official definitions of unit, integration, collaborator, mock, stub, double, etc. I swear we spend more time as an industry arguing about that stuff than actually writing code, myself included.

Reply on news.ycombinator.com