.NET mocking frameworks, a comparison | Hacker News Reader

.NET mocking frameworks, a comparison

codetuple.com

61 pointseljbutler37 comments

Terrible. It's just a list. No examples, no feature comparison, no analysis.

After reading this article, did you know that FakeItEasy is very slow, so is unsuitable for fuzz testing? Or that NSubstitute can't mock statics, but Microsoft Moles (absent from this list) can?

I do, because I had a developer in my team do that investigation before we decided to standardise our unit testing stack.

what did you end up going with and why?

NSubstitute, the team found it easy to use (readable) and it is extremely fast.

I can't review the decision yet because we were been forced to cut our investment in unit testing, instead focusing on Selenium tests (sigh, "management" should know better).

Agreed, it sounds like this guys research stopped at reading the github pages of those tools.

WRT FakeItEasy - we also found that it's not thread safe. Using it with XUnit (XBehave) we had to explicitly tell Xunit to run tests sequentially.

Notable in absence in addition to Moles is Machine.Fakes.

FakeItEasy v2.0 just came out, and it boasts thread safety as the big new feature. I haven't evaluated it yet (we were also bitten by the lack of thread safety in v1.x) so I can't validate the claims.

I didn't know about Microsoft Moles. Is it any good?

I once got so frustrated with the horribly arcane syntax of all .NET mocking libraries that I made my own, FakeThat[0]. It's unoptimized and not very full featured, but it's very easy to use.

The idea is that you don't need to learn yet another domain specific language just to do some assertions on mock call history. So I expose it all as plain old C# data, LINQ'able and all that, which means you can just use the assertion library of choice instead of weird mock-library-specific "verify" methods. The API gets you great autocomplete in IDEs because of static typing, so you discover the API just by starting to type.

Intellisense-driven discoverability of APIs is one of the great USPs of C# and Java programming (compared to most other popular languages), and IMO it's a shame that more TDD related libraries don't make good use of it.

It's not under active development but I'd be willing to take it up again if there's interest. Would love some HN feedback!

[0] https://github.com/eteeselink/fakethat

USP = Unique Selling Proposition For anyone who wonders what USP means (like I did).

When people (such as this author) talk about mocks, are they talking about mocking and testing behavior or just simply fakes / doubles?

My understanding of mocking by example is: you have a class you want to test, and that class has a dependency on lets say a Mail class that has a SendMail() method. You mock the Mail class and assert that SendMail() was called as per your specification.

Given that definition of mocking, I personally don't like mocking. I prefer to test state not behavior, although I can see in some scenarios behavior is more important.

Oh god mocks/fakes/doubles. The best answer to that probably comes from The Art of Unit Testing as explained on SO: http://stackoverflow.com/a/33333064/2136440.

In short yea, you're right.

that's a good thread.

Osherove's example on SO thread on mocks being troublemakers does not apply what adamconroy is asking, though.

Mocking libraries create dummy objects. They can't reproduce inner workings of mocked object unless you set it up to behave like that, which means you should duplicate the original which is pointless. You set up the mock to listen on it's entry points (and to return some predetermined answer), give it to your code under test, and expect the mock to be called in a certain way. This is testing the entry point of the mocked object, not it's internal workings. This way, you're expecting the code under test to behave according to a definition, so you're still testing your code, the way it's calling some other object gives you insights of your code's inner workings.

I was just saying his idea of what differentiates a mock from a fake was right!

Generally speaking, with a mock you are not providing an implementation. You are simply making an assertion that something is called in a certain way. This is great for testing adaptors, for instance. Basically you are saying, "I expect to interact with another outside object in this way".

Stubs return canned data. This is great for testing that the calculations in your unit are correct without having to use the real collaborators. With stubs you are saying, "I expect my unit to produce this output from this input".

Fakes are objects that provide a working interface for your unit and often provide either stubs or mocks. It is not quite correct (IMHO) to say that a fake can be a stub or it can be a mock. It may also be used when you have highly coupled code and you just need the object that does intelligent things so that you can get to the meat of your test. This is a very common technique when you are trying to inject tests into legacy code. With a fake you are saying "I need to use a working collaborator, but I don't want to use the real collaborator for some reason" (usually because it is slow/dangerous/hard to create).

Like I said, fakes often contain/return stubs or mocks. A good example of a fake would be a fake that implemented a REST interface and returned canned data depending on what parameters you sent it. It would replace the exact same object with a REST interface that would be used to talk to the network.

I've seen .NET projects with massive amounts of complexity because of DI introduced to support mocking. It's become a cult/fetish in some parts. There has to be a better solution.

Check out SimpleMock, it attempts to undo DI damage while still allowing for testing. http://deliberate-software.com/simplemock-unit-test-mocking/

There is something other than Moq? I'm only partially joking... But I really haven't found a reason to switch from it. I find the syntax easy enough and if I ever come up on a problem the question was probably already asked.

There are too many unnecessary lambdas involved when using it as far as I remember. NSubstitute syntax feels more natural from my point of view.

I mock .NET all the time. I was unaware I needed a framework?

You don't. It just makes it easier and requires less code.

I think he refers to this meaning of mocking: http://www.urbandictionary.com/define.php?term=Mock

A lot of Java programmers probably mock .NET all the time too. ;-)

I wonder if this secondary(?) meaning of "mock" is an argot more common among some programming languages/communities than others, because the meaning that comes to mind most readily when I see "mock" is "to treat with contempt or ridicule".

Excuse me, no jokes please.

I'm a big fan of NSubstitute. The syntax makes so much more sense to me than Moq, and readability matters a lot in unit tests.

Agreed. I find NSubstitute incredibly easy to use compared to others.

Same here. If all projects were like NSubstitute and Dapper I would be much, much happier.

For anyone that is using Moq, I highly recommend using AutoMoq as well (this is different from AutoFixture). Everyone knows how frustrating it can be when you add a dependency to the class under test and all your unit tests fail to compile. If you use AutoMoq, you just call .Resolve<ClassUnderTest>() and it will inject a mock for every parameter in the constructor, even if you haven't explicitly set it up. Dramatically reduces how much "Arrange" code you need to write.

https://github.com/darrencauthon/AutoMoq

What are peoples thoughts on handrolling mocks\stubs? I tend to do that the most and would love a compelling reason as to why I should switch my behaviour.

I do as well ... I've used mocking frameworks, and about the only time it's useful is if you have code you don't control that includes static methods. But in those cases, there are still ways to abstract that out of the code that needs to be tested

I used to do this before I saw my colleagues tests using Rhino.Mocks. I found myself willing to write far more/better tests as it got easier to generate when using the framework than rolling my own.

I dunno. I've tried mocking frameworks and found that when the interface they were mocking changed I spent more time trying to fix for the arrange step of the test than adding tests that added huge value.

Rolling my own might take fractionally more effort to initially set up but the transparency of what happens and ability to debug through to me is of more value.

Ah so I have luckily not had to experience this as my interfaces have been relatively stable

Because it's a waste of time and it clutters unnecessarily your test codebase. I prefer to invest my time to create better tests/code rather than creating manually test objects when I can simply do something like this:

    var authenticationService = Substitute.For<IAuthenticationService>();
    authenticationService.Authenticate("goofy").Returns(true);
    authenticationService.Authenticate("Donald duck").Returns(false);

In another test I can easily create a mock that returns true regardless of the users to go on with testing other parts of the code. In your case you need to create two concrete objects that will have a different behaviour. And what if you need a test that checks that users with some roles need to be authorized? And other members of your team need to search for an existing manual mock otherwise they may end adding some utterly useless duplicated manual mock, and in big teams you can see that it can be easily a huge mess. If you are the only person to write code it's up to you to waste your time, but in a shared codebase you need to be mindful of other people.

To be honest I don't really like London style tests and prefer to write state based tests using hand written 'fakes', which should be easy if you design your code around interfaces.

I'd recommend looking into TDD without mocks. This video from Mark Seemann is a good place to start:

https://www.infoq.com/presentations/mock-fsharp-tdd

The author should add SimpleMock, it's very easy and doesn't clutter the code with test-only interfaces and DI. You should check out SimpleMock, it's even easier! http://deliberate-software.com/simplemock-unit-test-mocking/

The links in the article are broken because of www prefix.

E.g. NSubsittue is here: http://nsubstitute.github.io/

FakeItEasy is here: https://fakeiteasy.github.io/

And even though it's commercial product, one should mention TypeMock.

Reply on news.ycombinator.com