Ruby Test::Unit sucks (and why I still use it) | Hacker News Reader

Ruby Test::Unit sucks (and why I still use it)

evan.tiggerpalace.com

44 pointssleight4233 comments

His contrived example of i.should have(1).porkchop_sandwiches highlights my (shared) biggest problem with RSpec too: over the top matchers.

I already know Ruby's syntax so when I want to query an array's length, I use length (or count or size). I don't want to learn yet another term like "have" and nor do I want to reverse how I normally do this check. have(1).obj vs obj.length are chalk and cheese.

I want my tests to be as close to existing Ruby language constructs and syntax as possible but clean too. Test::Unit keeps us closer to Ruby as Evan demonstrates:

  assert_equal 1, i.porkchop_sandwiches.count

But it's not clean enough, IMHO, and you have to learn a bunch of assertion methods to cover basic cases. So I have a "middle way" I picked up from I use RSpec and avoid its more fluffy matchers in preference for something like:

  i.porkchop_sandwiches.count.should == 1

Now it's more like an "if" condition but you still get the right error message goodness if the assertion fails. It adds the least new syntax to get results using your pre-existing Ruby knowledge. (Disclaimer: I usually just grin and bear Test::Unit for small projects ;-))

Unfortunately this style of assertions will lead to warnings in your test cases. I find many RSpec projects run their tests without warnings enabled because of this issue.

I MADE A VIDEO WITH FUN MUSIC TO DEMONSTRATE! :D :D

http://www.youtube.com/watch?v=pqc4kxMTCsc

<3 <3 <3 <3 <3

This totally made my day. Many thanks :)

I aim to please! :-D

Interesting to know! (UPDATE: Looks like I'm wrong ;-) See last update.)

It seems this issue was raised in 2008 and a solution (of sorts) suggested: https://rspec.lighthouseapp.com/projects/5645/tickets/504-wa... .. David Chelimsky seemed to think Ruby 1.9 would lose this warning but it didn't and he reopened the ticket (and it's still open 2 years later) :-) The "check" workaround seems generic and viable, if noisy.

But I thought I'd see what actually happens when you try a super simple example with RSpec itself:

  require 'rspec'

  describe "a number" do
    it "should be equal to itself" do
      1.should == 1
    end
  end

Running with warnings gives some warnings in diff-cls (oops) but none for the 1.should == 1 - the mystery gets a little deeper (RSpec masking it somehow?).

UPDATE: I think I found the reason after scouring RSpec 2.0's source. Based on your example but putting should in Kernel rather than Object:

  module Kernel
    def should; self; end
  end

  def omg
    1.should == 1
  end

  omg

You get no warning in this case. Curiously, though, if you ditch omg and just run the 1.should == 1 direct, you still get a warning. Confusinger and confusinger.

LAST UPDATE: I'm leaving the above for completeness but it appears the result is not such a useful one (see tl's response ;-))

No, adding should to Kernel fixes nothing. Your definition of the omg method returns the result of the comparison, thus it is no longer a void context. So the return value of omg is the return value of the comparison.

In your RSpec specs, if you only have one "==" expectation and it is the last one of the block, you will not get a warning.

I HAVE MADE ANOTHER AWESOME VIDEO WITH SUPER AWESOME MUSIC TO ILLUSTRATE!!!!

http://www.youtube.com/watch?v=M8SB7mcQfrY

Hope this makes sense! :-D

<3 <3 <3 <3 <3 <3 <3 <3

Ah, so it does - thanks for clearing it up. I wondered why you were throwing that extra "true" in there so didn't do it on my v2 but should have pondered this more deeply ;-) I've updated my comment with the relevant notes.

All that said, though, this is sucky. But since I never run with warnings on, I'm going to put some headphones on and hop around shouting "LA LA LA" whenever I run my specs next.. or I could partake in this grotesque workaround of temporarily forcing warnings off: http://codesnippets.joyent.com/posts/show/1438

I've got to buy you a beer one of these days.

RSpec 2 introduced an eq matcher so you can do this warning free:

    i.porkchop_sandwiches.count.should eq(1).

Then for object identity you can still use be:

    i.porkchop_sandwiches.first.should be(my_sandwich)

This is actually more in line with the original RSpec, which had should_equal and should_be, and I prefer it because you eliminate the warning, and (I think that) "1.should eq(1)" actually speaks better than "1.should == 1" for someone familiar with ruby but new to RSpec.

Fair enough, though I know what == means in Ruby and eq is additional, duplicated syntax to learn (as with assert_equals). Without hitting the docs, I wonder what the other comparisons are in this alternate DSL. I could have a good guess.. lt, gt.. not sure on less than or equal to, regex matching, etc. I'd have to look up or memorize those rather than use syntax reinforced over years of use that already does the job. How something sounds is not so important to me, AppleScript swore me off the "sounds like English" route :-)

Naturally this is a matter of opinion, but it's testament to your fine work with RSpec (and particularly in the recent modularization work) that it deals deftly with differing opinions without putting anyone at a serious disadvantage.

Thanks for the props.

The other operators are still supported as/is and don't cause the warning that == does.

I consider Test::Unit a testing tool while Rspec is a documentation tool that happens to do testing - hence the verbose matchers.

(thinking about it that's also a good description of TDD vs BDD - TDD is meant to be run, BDD is meant to be read)

Test::Unit is a testing tool. However, it can be used to facilitate design. That's a lot of what TDD/BDD is about!

Domain driven design is supposed to be about creating an ubiquitous language, used by application code, tests, developers and stakeholders. If you want a more fluent style in your code you want it everywhere not just the tests. The RSpec matchers are great for a more fluent style but they're unfortunately siloed in the specs.

I am in the process of rewriting hundreds of rspec tests to work with rspec 2/rspec rails for rails 3. This is killing my "TDD saves time" argument and it is really all the fault of rspec. Rails 3 requires Rspec 2, which has syntax tweaks that seem largely masturbatory. Now I have 183 failing tests, and no confidence whether it is rails changes I need to make, or updates to test syntax that are needed. Essentially I now have a parallel app to maintain and upgrade (the tests) and with test::unit which is stable I would come out much farther ahead effort-wise as I would get the benefit of the tests, without large maintanence cost for the tests.

In summary, Rspec is on a high horse and thinks I should maintain it like it was another app.

edit: And now I'm wondering if I need a test suite for my test suite. TDD Turtles, all the way down...

You can still hook up rspec or test::unit to rails 3...

I've tried liking rspec so many times, but Test::Unit really is where it's at. I don't understand the obsession with tests reading like english (Cucumber is much worse in this regard). I'll stick with ruby.

I find the assert_equal and so on much more readable than the "English" style. What does "should" mean? It's not as intuitive as assert.

Yeah, rspec looks confusing as fuck. Here's how we do it in Perl:

    is $got, $expected, 'is got eq expected?';
    ok $foo, 'foo is true';
    like $some_string, qr/foo bar/, 'some string contains foo bar';
    cmp_ok $got, '<', $expected, 'got is less than expected';

No need to learn some new language. It's just Perl. (The only confusing part is whether it's $got, $expected or $expected, $got. But I guess that part is arbitrary, like calling "ok" "ok" instead of n"is_the_first_argument_some_sort_of_true_value".)

But Test::More and others have other issues. How most modules do it in Perl usually means: by not cleaning global state between tests and without any isolation. It bit me so many and I wonder why people keep doing it that way.

I honestly expect a lot of cpan modules to have some tests that pass only because they don't run on a clean environment since I found some cases like that already.

I don't program with globals, so this have never been a problem. I also feel it's outside the scope of a discussion on syntax.

Finally, I am trying to fix this problem with: http://github.com/jrockway/eval-clean. It basically gives you an object that is a PerlInterpreter object, completely isolated inside. The only issue is keeping the lexical scope between eval calls, and I've almost figured out how to do this. (No, Lexical::Persistence is not even close to an acceptable solution :)

I'm missing this particular feature in T::U that I have in RSpec -- nested contexts + lets/subjects blocks.

    context "parent" do
      subject { Factory(:something, :value => other_thing) }
      let(:other_thing) { "Value" }
      it "has 'Value' value" do
        subject.value.should == "Value"
      end

      context "nested" do
        let(:other_thing) { "Totally other value" }
        it "has 'Totally other value' value" do
          subject.value.should == 'Totally other value'
        end
      end
    end

Have you looked at Shoulda? It lets you use contexts and adds to the goodness of Test::Unit at the same time.

Exactly. I only alluded briefly to Shoulda (and Coulda) in my post. I live and die by them.

Context is king.

BTW, while I may beat up on RSpec, I learned much of how I perceive TDD indirectly from David Chelimsky. I may not be an RSpec fan any more but I'm a huge David fan.

You might be interested in contest[1], which adds a thin layer of sugar over Test::Unit.

In addition to nested contexts, it also lets you provide your test name as a string rather than a method name, to avoid wearing out your underscore key with "def test_new_instance_should_respond_to_foo".

[1] https://github.com/citrusbyte/contest/blob/master/lib/contes...

I completely agree! After realizing that the team at my new job prefers full-on BDD with Cucumber I took a closer look at testing frameworks. My conclusion was that T::U and Shoulda/Coulda in combination with Webrat does just about everything I need it to do. It's simple enough for me to write quickly and I don't struggle to read it 6 months later.

And that's good enough for me.

Thanks! Its always a thrill for me when someone finds utility in a tool that I wrote.

You forgot the most important part... They're MUCH MUCH faster so you get a wonderful productivity boost, esp when you run your tests with autotest.

I'll admit it also: I usually just use Test::Unit. I have nothing against RSpec, but Test::Unit does enough of what I need it to do and I am used to using it.

Riot (https://github.com/thumblemonks/riot) is a nice compromise with just-enough-DSL.

I would like to like riot, but not creating a new test instance per test is a Bad Thing™. I just can't abide by it despite the speed boost.

xUnit testing packages are very simple and intentionally don't try to do much. They provide a basic testing infrastructure that everyone who has done unit testing in any language will immediately recognize.

This does not mean they "suck."

Read my post. I admit, the "suck" referenced was a bit of a troll. It's how I once felt.

Reply on news.ycombinator.com