Learn how to test Rails using Test::Unit instead of Rspec
whatdoitest.com
whatdoitest.com
describe 'Article' do
let(:article) { Article.new }
describe '#slug' do
before(:each) do
article.title = "Testing with Test::Unit"
end
it 'does not contain non-alphabetical characters' do
article.slug.should_not match(/[^\w-]/)
end
it 'does not contain capital letters' do
article.slug.should_not match(/[A-Z]/)
end
it 'replaces spaces with hyphens'
end
end
Note: I almost certainly misinterpreted the first Regex. 'now you have two problems,' etc. etc. describe Article do
describe '#slug' do
before do
subject.title = "Testing with Test::Unit"
end
its(:slug) { should_not match(/[^\w-]/) }
its(:slug) { should_not match(/[A-Z]/) }
it 'replaces spaces with hyphens'
end
end expect(article.slug).to match(/[^\w-]/) article.slug.should == "testing-with-test-unit"* Testing is an advanced topic, yet it seems to be misleadingly sold in Rails tutorial literature as something that everyone else effortlessly does with the same ease of "gem install hairball".[3]
* Good luck finding resources on Test::Unit/Minitest.[1]
* Where the documentation?
* Rails invents its own definitions for "unit tests", "functional tests", and "integration tests". Even though they already exist in testing vernacular. [2] Unfortunately confusing when Rails is your first experience with testing and you think "unit test" means "model test" and "integration test" means fumbling fruitlessly with your jasmine/capybara frankensuite because you're lost like I was and tried to stitch together a few Railscasts because you just don't know any better.
* Instead of embracing tests that don't depend on the Rails stack, the Rails community embraces tools like Spork that keep a Rails process alive. Because Rails takes too long to boot to have a test-driven cycle with tests that depend on Rails. The concept of service objects and wrapping Rails/3rd party libs is brand new to me and I only encountered it after I was good enough to contribute to large Rails apps and read their source code.
[1]: At least Ruby 1.9 changed the name (in essence) to "minitest" which we can finally google. "test unit"? Not so easy to google.
Also, here's the best resource for Minitest I can find: http://www.mattsears.com/articles/2011/12/10/minitest-quick-.... It's a cheatsheet on someone's blog.
Frankly, there are much better arguments for sticking with Rspec. For one, it's the testing library that the community uses and you need community support. The popular gems you'll be using probably have more wiki info on Rspec testing too (like Devise). Rspec is documented and even illustrates best practices. http://betterspecs.org/.
That Minitest is built in to Ruby was never a convincing reason to use it. I already have development dependencies. Trust me that testing isn't going to be the thing that introduces the first 3rd party dependency into my app. Fucking "awesome_print" is already in my Gemfile, so it's not a big deal to add a gem that makes testing better for me.
And for all the hate that Rspec gets for its DSL, Test::Unit/Minitest's DSL (assertion statements) is just as bad and Minitest::Spec introduces an Rspec DSL anyways.
The only way I want to write tests is by writing my assertions in pure Ruby without having to lug around a bunch of different assertion variants just so the testing library knows what kind of error output to show me.
The ONLY solution I've found is a library called Wrong (https://github.com/sconover/wrong) and it works with both Minitest and Rspec.
It lets you write assertions like this:
include Wrong
assert { "abc" == "abd" }
assert { cookie_jar.empty? }
assert { dog.respond_to?(:walk) }
assert { boy.is_a? Human }
Like, actual Ruby. And it will figure out on its own how to display diffs in error output.Meanwhile, here's Minitest:
assert_equal "abc", "abd"
assert_empty cookie_jar
assert_responds_to dog, :walk
assert_instance_of Human, boy
# I know these because I'm looking at a cheatsheet
Here's Rspec: "abc".should == "abd"
cookie_jar.should be_empty
it ->(dog){responds_2}.(:walk).(&:does?)
So all this talk about avoiding DSLs yet you don't. And it wasn't until blowmage (of minitest) told me about Wrong in IRC that I found what I wanted -- the only real way to avoid DSLs and verbosity in Ruby testing that I've come across.[2]: Xavier Shay brings this up in his talk http://www.confreaks.com/videos/815-larubyconf2012-rails-sus...
[3]: Oh yeah, and after three years of Ruby and Rails, it wasn't until I was exposed to functional programming (Martin Odersky, Rich Hickey), OO design (Sandi Metz), and testing screencasts (Gary Bernhardt) that I finally understood how to even fundamentally write code that could be tested. In other words, it took high level a-ha moments for me to be able to write tests that didn't just cripple my workflow and waste my time and slow down my learning process. That's where I found out that testing is an advanced topic, not a bullet point you can throw into a newbie tutorial on how to generate a Rails scaffold.
Test::Unit is simple, comes baked in, and because it's not some faux-English DSL your brain isn't constantly fighting with itself making stupid and incorrect assumptions about syntax because 'hey, it looks like English'.
The worst testing-related decision I ever made was using Michael Hardt's Rails Tutorial (which has RSpec baked in) to learn Rails in the first place. Great tutorial otherwise, but it set my testing back immensely - it pretty much meant I did no testing at all. When I finally bit the bullet and decided to learn how to use Test::Unit, I was astounded at how easy it actually was.
Most Rails resources online assume familiarity with so many not-out-of-the-box tools it's a bloody miracle anyone can learn Rails at all. It's a real problem.
I'm doing the same Rails tutorial right now and I constantly feel like this when it comes to writing the RSpec stuff. It just doesn't feel intuitive and logical, in spite of (or maybe because of) its efforts of being just that. The tutorial doesn't really dive into the specifics of it, it's just "Monkey write, Monkey do" without being able to grasp the concepts of it. This completely leaves me in the dark when it comes to writing my own tests.
My problem with Rspec is in its error message, whether it is showing why the test failed or that the test code syntax is wrong. I noticed that Rspec is more prone to showing cryptic errors than Cucumber. I've seen some mind boggling error messages that doesn't really help user debug the code. For devs beginning TDD, I can understand why working with Rspec could be annoying. Even now, Rspec doesn't feel second nature to me.
And that's where I'd disagree and my tastes differ, I never quite got the Ruby fetishism for almost-sort-kinda English DSLs. It's still baffling to non-developers and I'd much rather read a more regular programming language syntax, never mind that abusing the syntax can lead to some weird error messages. (Not saying that RSpec/Cucumber fall into that trap, as I never had that much experience with them. Like I said, kinda turn-off for me)
A bit like Lisp macro abuse or too much parsing in Tcl.
DSLs are helpful for some, but sometimes they get in my way. A good example is formtastic (http://github.com/justinfrench/formtastic) in ActiveAdmin. I already understood forms when I had to implement this, so it just dragged me down.
If using a DSL makes your life easier, by all means use it.
First I tried testing models and controllers, skipping views. Now I use integration tests with Capybara because it effectively tests the whole stack without writing redundant test code.
What's not clear is the pros and cons of each approach. Over time you end up using your own approach that works best for you. It's one thing to know that you need to "test your code", but with all the various testing libraries created, it can quickly become overwhelming.
In an ideal world, I'd like to learn more about the various approaches. Test::Unit vs Rspec, vs all the additional testing add-ons and how everything works together. When you end up spending just as much time writing tests as you do writing code, you quickly realize how critical getting the right approach to testing really is.
It's been pretty cool working on the testing book because I'm approaching it from a different angle. I'm not testing something, I'm showing someone how to test. I've got to dig into Test::Unit and MiniTest more than I ever have, and it's been great.
There isn't a wrong way to test, so long as tests are being written.
Does this mean you primarily rely on integration tests at the exclusion of model unit tests?
If that's the case, I'm of the opinion you would be well served by revisiting your unit tests. Controller tests are kinda taken care of by integration tests, but you won't get solid coverage of your models with integration tests alone.
At a minimum, model unit + integration (skipping controllers), IMHO. If for no other reason than running an integration test suite can be time consuming, and you don't always need your feedback loop to be that long.
When I learned Rails a few years ago, I was learning programming from scratch apart from some very early fumblings in BASIC and FORTRAN 20 years ago now. I was completely unfamiliar with the idea of testing. I had to learn it from the ground up. RSpec wasn't just more understandable it was, and remains, better documented. It's not just the official documentation. It's that there are tons of examples of how to test different use cases, what one should test, best practices.
Learning to test isn't just learning literally what a few methods do. It's understanding how to use those in myriad mix and match situations in order to provide good test coverage.
RSpec is easier to read before the test, it produces beautiful output that is readable and expressive, when you write good tests and test the right things.
After trying to struggle with Test::Unit and writing uncovered code for long stretches because I couldn't understand what to do, and then discovering RSpec, reading great blog posts, write-ups and documentation on it and finding comfort in RED-GREEN-REFACTOR I would only recommend RSpec for someone starting from scratch, as I did.
As RSpec documentation goes this set of slides open in a window will help you tremendously:
http://kerryb.github.com/iprug-rspec-presentation/#1
It uses simple example and hits all the major points well.
Minitest::Spec is just an RSpec-like DSL built on top of Minitest, which is the successor to Test::Unit.
As a developer, that is my number one question. Not cucumber/rspec/testunit, but really what do I test so the tests can be fast, I can test my core functionality and I don't overlap rails.
I don't agree at all with this premise by the OP:
> Just from skimming over the tests, it’s obvious that Rspec is more readable. The Rspec DSL was designed for us humans to be able to understand what is happening at a glance. This is where Rspec gets my vote. Unfortunately, we aren’t talking about reading or sharing tests. We are talking about writing tests, and this is where Rspec has a heafty learning curve.
I think the MiniTest::Spec API is both easy to read and write...(I'm assuming the OP is criticizing both Rspec and MiniTest::Spec here). I'm just saying this from personal experience, as I learned both the Spec and standard testing syntax at the same time.
I don't recommend MiniTest::Spec for the same reason I didn't recommend Rspec. It requires some extra configuration out of the box. The actual process of switching from Rspec to Test::Unit went Rspec -> MiniTest -> Test::Unit. There's just more stuff that's neccessary to start writing tests.
It's the same reason that new rails devs shouldn't swap out ERB with HAML before they build their first application. Get to know the defaults, then decide.
Since then, I've stuck with RSpec because it's got a ton of community support, and consistently lets me write concise tests. I never feel like I have to unduly repeat myself with RSpec; I constantly felt like that when I was writing testunit tests.
At a high level, I would say that Rspec is just much more thought out. The other libraries all seem to be written by someone who looked at Rspec and though "Ugh, this is too much. I just need X."
But I find, if I'm doing serious TDD, I'm gonna need pretty all of Rspec. And what happens is that the other libs just don't integrate that well.
For example, you may be using one test runner (minitest) with a mock from some other library (mocha) and assertions from a third (test::unit). The question is: are those things all meant to work well together? Are your mocks scoped correctly within your test runner? Is your database cleaner working properly with your before and after hooks?
The answer is sometimes "no". But with Rspec, everything works well together. It just fits. Generally I'm a big fan of focused libraries, loosely coupled. But the state of things with testing is that I don't think we've yet found that good loose coupling.
The second issue is that Rspec just seems to have much richer assertions. I can do things like:
expect {
delete_zombie_users
}.to change(User, :count).by(-1)
Or: thingy.length.should < 4
And mocking is also underdeveloped in other libs. Mocha, for example, can't stub constants. In Rspec I can do: User.stub(:all => [fred, betty, sue])
Or stub_const("File::MAX_SIZE", 10**10)
And it works great. I find I can easily stub out parts of ActiveRecord and write good targetted unit tests. With Mocha I find myself using any_instance all the time because I can't target things on the structure I really want to target them to.Some of these are available in other expectation libraries, but I've found Rspec to have a really nice, comprehensive set of them.
All in all, I just feel like David Chelimsky and the Rspec folks have really thought about testing in a way the other libs' authors haven't. I keep learning things about TDD and finding out that Rspec has already considered them. With other libs the opposite often seems true.
I just started the process of converting our few hundred Test::Unit tests to Rspec simply because Test Unit documentation is terrible. Our tests need a lot of work and I'm not that experienced at writing test scripts. I have to learn this as I do it, and I struggled to find resources for learning Test::Unit.
Every blog post, tutorial and guide I can find was written for Rspec.
As much as I'd love to continue using Test::Unit and save myself the hassle of converting from one to the other, I found learning Test::Unit to be one giant brick wall after another.
No one is discussing it, no one is blogging about it, no one is writing stack overflow questions about the issues. (I say, in a discussion about a blog post about it... but this is the exception not the rule, sadly)
For me it was a simple decision: If I want community support, I need to be using the tool the community is actually supporting.
(Not that Rspec documentation is that much better-- I've now resorted to just reading public github repos and looking for spec/ directories to scrutinize and learn from. If anyone has links to any good repos that use rspec tests, I'd love it if you could send them my way! I need more good repos to learn from!)
Minitest and Test::Unit are part of the same package since 1.9, with Test::Unit implemented as a simple compatibility layer on top of Minitest:
http://www.rubyinside.com/a-minitestspec-tutorial-elegant-sp...
The Minitest assertions are quite similar to the old Test::Unit assertions, a large number are identical:
http://rubydoc.info/stdlib/minitest/1.9.3/MiniTest/Assertion...
This is really more of a general documentation quibble on my part though, since I'm the dope who can't read through my questions and find the answers in the source (or wherever). I've been planning on making a blog post about 1:1 mapping of concepts between Test::Unit and Minitest learned from my own transformation, but as I've mentioned, I haven't been able to undertake that yet.
As a rule, I miss the thoroughness of CPAN/Perl docs. Some ruby packages have amazing docs but often I feel at sea, particularly with regard to high level overviews and examples.
Truth. And the community is really split on this one. The side that makes the decisions about what goes into rails prefer unit tests, while a large chunk of the users prefer spec-based.
Are you confusing Test Unit documentation with 'How/What to test' documentation? I can't think of any documentation required if what you use is a group of asserts.
Test::Unit in a nutshell: `test "should do something"; ...; end` is what gets run in each of the files. `setup` is called before each test. `teardown` is called after each test. use `assert_*` at your discretion.
http://guides.rubyonrails.org/testing.html#setup-and-teardow...