Rails Testing with Factory Girl | Hacker News Reader

Rails Testing with Factory Girl

hiringthing.com

38 pointseatenbyagrue35 comments

I hate when people are using it in the unit tests. It just makes testing very convenient, so they start using it everywhere. In the end, the test suite gets very slow. Unit tests should not hit the database.

First example there: test_new_customer_defaults. Really, it needs 3 saved models to test this? FactoryGirl isn't solution here. It's just hiding the problem.

If I could upvote the parent any harder, it would hit orbit.

If you need a complex network of collaborators to test a method, the answer is not to obscure the creation of those collaborators, it's to refactor so that each object talks only to its immediate neighbours, and then stub the heck out of them. If it absolutely has to be something that acts like an ActiveModel, then use mock_model().

And I'd upvote it to Mars. FactoryGirl is completely an anti-pattern, unless maybe you're doing acceptance testing, but you know even then I wouldn't want FactoryGirl in my codebase, people would be too damn tempted to use it everywhere.

That's where we find ourselves currently: constructing 3-4 levels out due to Law of Demeter violations, and it becomes incredibly brittle when seemingly unrelated classes change.

IMO, DHH (and others) argue too hard against core OO principles (like SRP and LoD) that have the possibility of helping out when the domain model becomes more complex. "Just use Rails" is fine when spinning up, but as the app grows, it becomes increasingly less pleasant. Coupling increases in the name of less code.

Ideas like hexagonal architecture look nice, but I'm left wondering how to implement them when the framework insists on infecting core domain classes (AR::Base).

So can someone recommend a Ruby (edit: ideally Sinatra/Sequel, not Rails) app one could look at with a more principled test suite? I'm coming from a more traditional OO background and jumping into Ruby. It's good to see I'm not alone in finding this militant pragmatism somewhat off-putting, but I don't want to reinvent the wheel.

As an FYI to your last point - in Rails master (which will be Rails 4), you can active-record-ify a class by including ActiveRecord::Model, rather than inheriting from ActiveRecord::Base. In fact, that is all ActiveRecord::Base does itself - https://github.com/rails/rails/blob/master/activerecord/lib/....

But your core domain classes are still coupled to their persistence mechanism in the name of 'pragmatism.'

Who says they have to be? That's a design decision. There is literally nothing (besides maybe inertia?) that enforces that "rule".

If you want to be sure to keep FactoryGirl out of unit tests but still have the benefit of using it in higher-level tests, you can use a different test/spec helper. I tend to do "spec_helper_lite", which sets up the bare minimum for unit tests (no FactoryGirl), and "spec_helper", which includes FactoryGirl for higher-level tests.

I say "higher-level" tests to refer to integration, system, and acceptance tests, because I don't have an opinion of where to draw the line for FactoryGirl. Is there a reason you would consider FactoryGirl for acceptance tests, but not integration or system tests? Just curious, because I might learn something.

Integration tests just mean you are testing the integration of multiple components, say view and controller, model and database, controller and model. Acceptance tests are tests from the end-user's perspective and touch the entire stack like a real user would. Acceptance tests are integration tests. It's alright for these tests to hit the database because part of what you are testing is the integration with the database.

You can use the build method to instantiate the model without saving to the database. Very useful because you only need use create when you need persistence in your test - such as when testing a controller.

Remember to ensure that your build strategy is set on all associations if you want this to work, e.g.

  FactoryGirl.define do
    factory :user do
      association :account, :strategy => :build
    end
  end

Otherwise when you build a user, the account association will still get persisted to the database.

Even if you do this, though, you're still spinning up a whole network of real objects in what is supposed to be a unit test.

I'll go one recommendation further. Just don't couple user and account. When making a factory named after a model, only set model attributes and never set relations.

    factory :user    # sets only user fields, no relations
    factory :account # sets only account fields, no relations
    
    factory :user_account {creates both}

When you have a large # of tests/factories, it's nearly impossible to keep track over time what the :user factory creates. Today it's one relation. Years from now, it could be 10 other tables relations. Someone later on decides they want to reuse the :user factory, but they don't want the account to be create, so then they do :user_only factory. Yada yada. Shenanigans ensue.

If you stick to the above strategy, things are much faster, but more importantly, there are less unintended surprises, like associations doing a create instead of a build, etc.

That sounds like pretty good advice. I do not have a very large base of factories and tests yet but I can see how unwieldy it could get without really sane naming conventions.

My preference is to have the base factories set everything that is needed for the generated record to be valid, and nothing more. If your User model validates the presence of the account, then the account association should be set - if not, it shouldn't. Otherwise, FactoryGirl.create(:user) is going to provide you with an unsaved, invalid record, which would certainly be surprising to me if I were using that factory for the first time. If all you want is a consistent set of initial attributes, then define a constant USER_ATTRIBUTES somewhere with a hash of values, and pass that to #new - using factories for this purpose is a bit sledgehammer/walnut.

I agree that excessive association building can get out of hand (and the latest version of FG even allows you to subscribe to factory events just so you can work out what the hell your test suite is building - a sure sign that you've lost the ability to reason about your tests if ever there was one), but completely abstaining from building associations seems to remove more or less all of the attraction of using factories as a fixture replacement in the first place.

All of the above applies only to integration/acceptance tests, though, where factories-as-fixture-replacements are at least arguably justifiable; my recommendation for unit tests is "don't use factories at all."

One compromise (edit: on the performance problem) would be to use FactoryGirl to generate some type of (yaml?) cache pre-test-run, automatically refreshing only when the model changes.

I think you're missing the point of what makes FactoryGirl an anti-pattern in unit tests. FactoryGirl reduces the feedback that you can get from unit tests by hiding complexity (of creating objects), which your unit tests are supposed to make apparent.

Edited to add: Using FactoryGirl to generate a cache only addresses the performance issues without addressing the complexity-hiding problem.

I don't disagree; I was only trying to address the 'test suite gets very slow' part rather than the 'unit tests should not hit the database' part.

If performance wasn't an issue, the convenience of auto-created objects would mean more 'unit' tests (edit: usually closer to integration tests) that wouldn't exist otherwise. Sure it's best to de-couple things, but there's room for something between no [unit] tests at all and perfect unit tests.

I could also argue that's exactly WHY to use factory girl in unit tests. You're not using FG to create the things under test, only the things to pass to the thing under test.

Complexity is a moot point for those extraneous objects. You want that complexity out of the tests since it's not the focus of the test itself, and makes it more readable.

But I'll take this one step further. To each their own. There is no one exact correct way. Do what's right for your code, domain, and tests. I wouldn't test an API codebase the same I would a site codebase or a gem codebase anyways.

I would argue that those aren't unit tests, because they depend on the collaborators that you're passing into the unit under test. A change to those objects could break your unit test. That makes it an integration test.

If you're using your unit tests to inform your design, then the complexity in the extraneous objects is relevant, because they tell you how coupled your unit under test is to its dependencies. This is what FactoryGirl hides from you.

I tend to use fake objects for the collaborators. Usually just a vanilla test double will do, but sometimes I use RSpec's stub_model if the collaborator has to behave more "model-like".

I'd recommend http://pyvideo.org/video/631/fast-test-slow-test and http://www.confreaks.com/videos/641-gogaruco2011-fast-rails-... if you haven't seen them already.

And I would argue that arguing bout what a unit test is and isn't has missed the topic of this parent post entirely. I have seen those links and I know both people as they worked where I work now. Arguing over speed and/or what a pure unit test is has never been my cup of tea because every single codebase is different for a variety of reasons.

Tests and code don't always fall into these neat cult buckets we've drawn up for ourselves. Good doesn't mean just fast. It can also mean readable, to wit FG usage addresses quite nicely. Just another tool in the toolbox.

Just like ruby, I can write crap tests with or without FG and great tests with or without FG. Same goes for stubs, doubles, and collaborators.

> And I would argue that arguing bout what a unit test is and isn't has missed the topic of this parent post entirely.

You do raise a valid point: we're arguing about a definition. I don't think that misses the point though, because this thread is specifically discussing the use of FactoryGirl in unit tests. Where I disagree with you is on the importance of definitions. I think it's important to understand the definition of a unit test vs an integration test vs a system test, because without agreeing on those definitions, we don't stand a chance of having a reasonable discussion. If we have different definitions for a word, then our arguments will be talking past each other.

> Tests and code don't always fall into these neat cult buckets we've drawn up for ourselves.

(I'll ignore the loaded language, because that's not conducive to rational discussion.) From this point on, you use the word "tests" instead of "unit tests," and your points are valid because you're talking about the broader category of tests. I'm not. I'm specifically discussing unit tests, which has been the topic of discussion for this entire thread, going all the way up to the top-level comment by indrekju.

> because every single codebase is different for a variety of reasons

I agree with you here. Every codebase is different, and so are their testing requirements. For small gems, I tend to write all integration tests with few unit tests. For larger Rails projects, I write mostly unit tests with some integration tests to catch the cases where an interface changed but I forgot to update a fake object. If you're saying that different codebases differ in their relative requirements of different types of tests (unit vs integration, etc), then I agree. In fact, I would take that a step further, and say the testing requirements also depend on the team. But if you're arguing that the definition of a unit test changes depending on the codebase's testing requirements, then I can't agree.

I'm not refuting the usefulness of FactoryGirl in integration, system, or acceptance tests. Just unit tests.

Having definitions for jargon gives us tools to learn and discuss things at a more abstract level. That doesn't necessitate applying those ideas dogmatically. If anything, I think it helps you understand the trade-offs so that you can be pragmatic, because pragmatism depends on understanding.

There's a saying in nursing: "If it's wet, dry it. It it's dry, wet it."

I think we need the same thing in programming.

If it's slow, make it faster. If it's unreadable, make it readable. If it's coupled, decouple it.

Then just stop worrying if we fall into official definitions of unit, integration, collaborator, mock, stub, double, etc. I swear we spend more time as an industry arguing about that stuff than actually writing code, myself included.

Yep, I'm trying to dig myself out from this problem now.

The Rails Tutorial (railstutorial.org) uses Factory Girl and integration tests for just about everything, which is a fast enough approach for its toy application, but I've found that when your application gets more complex your test suite takes forever to run, and setting up the tests gets more and more painful (especially when you start dealing with external APIs).

Does anyone have any opinions on the best resource for teaching effective unit testing in Rails? I'd love to speed up my test suite while not losing coverage.

It's not really focused on unit testing (though it does cover it) so much as it's focused on good design, but the two are related, so I found Avdi's "Objects on Rails" [1] helpful.

[1]: http://objectsonrails.com/

I'm not sure if it's the best resource, though. I'd love to hear other opinions.

A response from a naysayer:

Factories are slow. Yes, they really are. Persisting everything along a chain of associations will always be slower than properly isolating the dependencies of the unit under test, SSD or not. It's easier to enjoy Ruby when not constantly waiting minutes for tests to run.

Factories increase the over-all complexity of your app. Are you kidding me? Yes, they do. Expertise and huge, easy gains have nothing to do with reducing complexity and more often (but not always) increase it. A test suite where every test depends on state in a database is more complex than one that tests units in isolation.

None of these points are arguing against Factory Girl, which is a great tool for doing what it does; they are arguing against integrating with the database in "unit" tests.

As a guy who recently switched from years of ASP.Net MVC to Rails, I read this post and wanted to shout...

"Watch out everybody, he's hitting the database!!!"

Ruby is a great language that I love, but this is one of those cases where Ruby's conciseness and readability hide a TON of complexity. And complexity in tests is death.

I've seen it for myself -- there's a lot of technical debt interest in those tests. It looks manageable at the beginning, especially within the scope of a blog post, but when the app grows and grows and grows, the tests fail.

To get the best form both fixtures and factories, I can strongly recommend http://github.com/rdy/fixture_builder It utilizes factories to build up your fixtures. This way you only need to specify what you actually care about in you r fixture which preserves readability. But you still get the speed of fixtures. The one downside is that you can never assume a complete knowledge of what's in your DB. This might break test for scopes for example.

However I more and more agree with the general notion that your unit test should not hit the DB. However, Rails works against that. Avdi Grimm's "Object's on Rails" (http://objectsonrails.com/) has some interesting suggestions on how to get some good test isolation out of Rails.

Factory Girl has saved me countless hours - I love it.

You can use build instead of create to avoid the db, as per previous comments. You don't have to use it at all if writing a simple unit test.

But for teeing up complex scenarios prior to a fat integration test, it rocks.

I'm coming to the later stages of implementing an MVP with Rails and for me, FactoryGirl and RSpec have been a godsend.

Old news!

Yea, once your model gets more complex than a dozen or so objects, then something like this is awesome. I only with there was an equivalent in ASP.Net MVC which is the current bill payer. (Plant is the closest I've seen, but not quite as elegant, and I'm a little concerned over support).

How is this different to mocking frameworks like RhinoMocks or the new Microsoft.Fakes?

You're creating an actual instance of that object preloaded with whatever you need rather than a mock of that item which implements a fake implementation. They're not really competing since Mocks are most useful when mocking an interface, and Factory Girl is best at creating a number of usable objects.

Factory Girl is good but Fabrication is far better.

Reply on news.ycombinator.com