Tag Archive for Unit Testing

Moq, Callbacks and Out parameters: a particularly tricky edge case

In my current project we’re using Moq as our mocking framework for unit testing. It’s a very nice package – very simple, very intuitive for someone who has his wrapped around basic mocking concepts, but tonight I ran into an annoying limitation that, while rare, impeded my tests considerably.

Consider the following code:

  1: public interface IOut
  2: {
  3:    void DoOut (out string outval);
  4: }
  5: 
  6: [Test]
  7: public void TestMethod()
  8: {
  9:   var mock = new Mock();
 10:   string outVal = "";
 11:   mock.Setup(out => out.DoOut(out outVal))
 12:       .Callback<string>(theValue => DoSomethingWithTheValue(theValue));
 13: }

What this code SHOULD do is create a mock object for the IOut interface that, when called with an out parameter, both assigns a value to it, and does something with the value in the Callback.

Unfortunately, this does not work.

The reason it doesn’t work is that the Callback method has many overloads, but they are all to various variants of the Action<> delegate. Either Action, which receives one parameter, or to Action, which receives three. None of these support out parameters. But none of these delegates has a signature with an out parameter. The result, at runtime, would be an error to the effect of

“Invalid callback parameters <string> on object ISetup<string&>”

Note the highlighted bits – The Setup method referred to a string& (a ref/out param), while the Callback inferred an Action delegate, which expectes a regular string param.

So what CAN we do?

The first option is submit a patch to the Moq project. It’s open-source, and a solution might be appreciated, especially since the project’s lead, Daniel Cazzulino, pretty much acknowledged that this is not supported right now. I might do that later, but right now it’s the middle of the night and I just want my tests to pass. So I manage to hack together this workaround which does the trick:

  1: public static class MoqExtension
  2: {
  3:    public delegate void OutAction (out TOut outVal);   
  4:    public static IReturnsThrows 
  5:                         OutCallback
  6:                                  (this ICallback mock, 
  7:                                   OutAction action) 
  8:                                      where TMock : class
  9:    {
 10:       mock.GetType()
 11:           .Assembly.GetType("Moq.MethodCall")
 12:           .InvokeMember("SetCallbackWithArguments", 
 13:                         BindingFlags.InvokeMethod 
 14:                             | BindingFlags.NonPublic 
 15:                             | BindingFlags.Instance, 
 16:                         null, mock, new object[] {action});
 17:       return mock as IReturnsThrows;
 18:    }
 19: }

 

We’ve created a new delegate, OutAction, that has out T as its parameters, and we’ve added a new extension method to Moq, OutCallback, which we will use instead of Callback. What this extension method does is use brute-force Reflection to hack into Moq’s internals and call the SetCallbackWithArguments method, which registers the Callback with Moq. Yes, it even works. I’m as surprised as you are.

Note and Caveats:

1) This code works for the very specific void Function (out T) signature. You will have to tailor it to the specific methods you want to mock. To make it generic, we’ll have to create a bucket-load of OutAction overloads, with different combinations of out and regular parameters.

2) This is very hacky and very fragile. If Moq’s inner implementation changes in a future version (the one I’m using is 4.0.10827.0), it will break.

3) I don’t like using out params, usually, but this is a 3rd party library, so I have to work with what I got.

Comments? Questions? Scathing criticisms on how ugly my code is? A different, simpler method of accomplishing this that I totally missed? Bring’em on.

The Case of the Unexpected Expected Exception

“NUnit is being problematic again”, they told me when I came to visit the project. “When running unattended it’s not catching assertions properly and the test is coming up green, but when stepping through in the debugger, it works fine.”. It’s nice, when getting a passing test is acknowledged as a bad thing, at least when you don’t expect it to be. In this case, though, the fault wasn’t really with NUnit.

  1: [Test]
  2: [ExpectedException]
  3: public void DoTheTest()
  4: {
  5:     _myComponent.RunMethod();
  6:     Assert.IsFalse(_myComponent.EverythingIsFine);
  7: }

“It’s simple. Either the method throws an exception, or at the very least – the EverythingIsFine property won’t be set to “True”, so the assert will catch the problem. But in their case, no exception was thrown and Everything wasn’t Fine,  but the Assert call wasn’t raising a red flag – unless they stepped through, in which case it did. What’s going on?

The basic problem is that to many developers, NUnit is a kind of magic. You write a self-contained little bit of code, the [Test] method, but you don’t call it yourself, you don’t get a feel for the whole execution flow. The result – developers don’t exercise the same sort of judgement they do on their own application code.

The root of the problem here is that the [ExpectedException] attribute told NUnit to pass the test if an exception is thrown. NUnit’s Assertion utilities, however, use exceptions as the mechanism for failing tests – when an assertion is hit, it raises an exception – it can be an AssertionException. For various mock frameworks, it can be an ExpectationException. It doesn’t matter – it’s these exceptions that make the test fail, and not some behind-the-scenes magic. Because the test had an open-ended [ExpectedException] attribute, these exceptions were caught, fulfilling the condition, and NUnit was happy.

What can we do to avoid this?

  • Be explicit. Don’t try to catch ALL exceptions with [ExpectedException]. If you’re expecting an exception, you’re probably expecting a specific exception. Specify it.
  • Be aware of how your tools work. If NUnit works by throwing an exception, don’t wrap it with a try/catch. Your tests are C# code too, as is the plumbing to enable it. It plays by the same rules.