ReSharper 4.5 gone gold

April 8, 2009 at 10:21 PMAndre Loker

JetBrains has released version 4.5 of their multi purpose productivity plugin for Visual Studio, ReSharper. This updates contains some pretty cool features. For example, solution wide analysis now checks for members that can be made internal. Also, JetBrains promises to have improved performance and memory use. All in all certainly an update that is worth installing, especially because updating from licenses for version 4.0 and above is free. Maybe I’ll give a more in-depth review if I find the time.

If you want to know more about ReSharper, check out my little overview page. There you’ll also find information on how to get an additional 60 days of free evaluation. Have fun!

Posted in: Tools

Tags:

Debugging speed of Gallio fixtures – reloaded

March 25, 2009 at 10:19 PMAndre Loker

Last week I mentioned the slow debugging of Gallio unit tests. After having published that blog post I had a conversation with Jeff Brown about this issue. After showing him in a video how huge the difference in speed between debugging an MbUnit 2.4 and an MbUnit 3 test was he pointed out that this might have to do with the fact that MbUnit 2.4 uses the multi threaded apartment state (MTA) per default, while MbUnit 3 uses single threaded apartment state (STA).

I tested this difference by debugging the test fixture shown below:

   1: [TestFixture]
   2: public class CalculatorTests {
   3:  
   4:   [Test, ApartmentState(ApartmentState.STA)]
   5:   public void Adding2And3ShouldReturn5_STA() {
   6:     TheTest();
   7:   }
   8:  
   9:   [Test, ApartmentState(ApartmentState.MTA)]
  10:   public void Adding2And3ShouldReturn5_MTA() {
  11:     TheTest();
  12:   }
  13:  
  14:   void TheTest() {
  15:     var x = 2;
  16:     var y = 3;
  17:     var expected = x + y;
  18:     var sut = new Calculator();
  19:     var actual = sut.Add(x, y);
  20:     Assert.AreEqual(expected, actual);
  21:     MessageBox.Show("END OF TEST!");
  22:   }
  23: }

And indeed: the STA unit test ran sluggish as reported while the MTA based unit test could be debugged at virtually the same speed as I am used to from MbUnit 2.4 tests.

I reported that back to Jeff and he proposed to change the default thread apartment state to MTA with the option to override this default using an assembly scoped attribute (for unit tests that require STA).

Thumbs up to Jeff!

Update:

Jeff reported that after some further research the solution to the problem might not be as simple as this and he needs to think it over. Anyway, I’m glad to see that this issue is being worked on.

Posted in: Tools

Tags: ,

Slow debugging of MbUnit tests with Gallio and TD.NET

March 18, 2009 at 12:04 PMAndre Loker

No, this is not another rant about the troubles I’m having migrating to Gallio, so don’t be alarmed :-) In fact, the Gallio people have proven to be really bothered about the issues I have (see previous post and especially the comments). Thanks a lot for your commitment!

Sometimes I need to debug unit tests to see why they fail. TestDriven.NET has a nice feature that allows just that.

image

This has always worked perfectly smooth with MbUnit 2.4 when using Gallio (3.0.6.727) with MbUnit the execution during debugging was very sluggish. This problem has been reported in the mailing list.

One workaround was posted on the thread mentioned above: disable the “property evaluation and other implicit function calls” option in the Visual Studio debugging options.

image

Doing this reverted the debugging experience to normal speed. The only drawback is that you need to “manually” refresh the values of properties by clicking on the two green arrows on the right side.

image

This is not the most convenient thing in the world, but I hope that it’s just a temporary state of affairs that the Gallio folks are able to fix soon.

Posted in: Tools

Tags: , ,

Using Gallio with NCover

March 17, 2009 at 11:39 PMAndre Loker

I’m really keen to use MbUnit 3, but I still have some obstacles with Gallio to overcome. I use NCover to check the coverage created by the unit tests in the assembly that is being tested – and only in that assembly. The whole procedure is part of an automatic build process executed with NAnt.

NAnt + NCover + MbUnit 2.4 = Happy Go Lucky!

For the combo I used before (using MbUnit 2.4) my build files contained something similar to this:

   1: <property name="unittest.cmdline" value='/ap:"${build.dir}" /rf:"${testresults.dir}" /rnf:"mbunit-${target.name}" /rt:Xml "${target.path}"'/>
   2: <property name="unittest.runner" value="${tools.dir}\MbUnit\mbunit.cons.exe"/>
   3:  
   4: <exec
   5:   program="${tools.dir}\NCover\ncover.console.exe"
   6:   workingdir="${build.dir}"
   7:   commandline='"${unittest.runner}" ${unittest.cmdline} //a "${coverage.assemblies}" //w "${build.dir}" //reg //x "${testresults.dir}\coverage-${target.name}.xml" //ea CoverageExcludeAttribute'
   8:   if="${ncover.supported}"        
   9: />

While this might look scary, it boils down to calling this:

ncover.console gallio.host.exe SomeAssembly.Tests.dll //a SomeAssembly

(plus a number of switches that I omitted for simplicity). This works perfectly and generates a nice coverage file for SomeAssembly based on the tests in SomeAssembly.Tests.dll – just what I want.

NAnt + NCover + MbUnit 2.4 Gallio = Does Not Compute

My first train of thought was: hey, with Gallio being more or less the successor of MbUnit, moving to Gallio shouldn’t be so hard, just replace MbUnit.Cons.exe with Gallio.Echo.exe, fiddle with the arguments and you’re done. So I changed the build file to do something along these lines:

ncover.console gallio.echo.exe SomeAssembly.Tests.dll //a SomeAssembly

And indeed, the tests were run and a nice shiny coverage file was generated – which however was empty. Erm? What now? I don’t know exactly what causes this behaviour, it seems that Gallio runs the tests in some separate app domain or so which prevents NCover from instrumenting it.

Second chance: let Gallio do it!

I remembered that there was a special switch to set the “runner” used by Gallio. It defaults to “IsolatedAppDomain”, but there’s also a “NCover” runner available. Someone gave me the hint to simply use this runner.

image

Maybe I only need to tell Gallio to use NCover and we’re happy? So I changed my build file to do something like:

gallio.echo.exe SomeAssembly.Tests.dll /r:NCover

And indeed, now it runs the tests and creates a non-empty coverage file. Sweet! A bit slowish, but running at least. The problem is – and you might have predicted it – that the coverage is gathered of all loaded assembly, including the assemblies of MbUnit and Gallio. So even for a trivial assembly being tested I get a coverage file of 5 MB. I couldn’t find a way to pass arguments regarding NCover. Argh! The problem is known by the way, but yet to be fixed.

Update: this problem is fixed in 3.0.6 builds. There you can pass arguments to NCover using the NCoverArguments and NCoverCoverageFile options.

At last: taming the beast

Believe me if tell you that at that point I was just annoyed. As of now all that Gallio gave me is headaches and frustration. This article was initially meant to reflect this.

Up to the point when I reconsidered the problem of the empty log file and remembered that there was a “Local” runner listed in the Gallio.Echo.exe help. Maybe this would force Gallio to run the tests in a way such that NCover can do its magic? Thus, I tried this (note the /r switch):

ncover.console gallio.echo.exe SomeAssembly.Tests.dll /r:Local //a SomeAssembly

Let’s see what we get:

  • Tests are found and executed? Check.
  • The coverage file is non-empty? Check.
  • The coverage file only contains information about “SomeAssembly”? Check.

Could it be? Finally, after several disappointments I am finally able to integrate Gallio in my automatic builds! Well, I have not integrated the Gallio reports fully in my CruiseControl.NET web dashboard, but I’ll keep that for another day. As for today, I’m happy with what I achieved.

Update:

Gallio also provides a NAnt task which you can use instead of <exec>’ing. Starting from version 3.0.6 you can also pass arguments to NCover (as described above). I tested the following task in NAnt and it worked smoothly:

   1: <gallio result-property="testrunner.exit-code" 
   2:         application-base-directory="${build.dir}"
   3:         runner-type="NCover" 
   4:         failonerror="false" 
   5:         report-name-format="gallio-${target.name}" 
   6:         report-types="xml" 
   7:         report-directory="${testresults.dir}">
   8:   <runner-property value="NCoverArguments='//w ${build.dir} //q //ea CoverageExcludeAttribute //a ${coverage.assemblies}'" />
   9:   <runner-property value="NCoverCoverageFile='${testresults.dir}\coverage-${target.name}.xml'" />
  10:   <assemblies>
  11:     <include name="${target.path}" />
  12:   </assemblies>
  13: </gallio>
  14:  
  15: <fail if="${testrunner.exit-code != '0'}" >The return code should have been 0!</fail>

Thanks to Bruno Wouters and Jeff Brown for the hint!

 

Conclusion

To condense the story into two sentences:

  1. If you want to use Gallio with NCover be sure to choose the “Local” runner by setting the /r:Local option.
  2. Don’t use the NCover runner if you need any control over NCover options. Update: or use version 3.0.6+

To the Gallio people

I hope this article isn’t utterly stupid or useless. I looked on the web but did not came across anything useful regarding NCover and Gallio. Also, I couldn’t find any hint about it in the documentation. I hope this article helps people that like me would really like to use Gallio but struggle with issues.

Update: thanks for your help!

Posted in: Tools

Tags: , , ,

Big websites gone small

March 12, 2009 at 11:24 PMAndre Loker

If you need a simple way to generate a thumbnail of a website, have a look at thumbscreator.net. This service created by Jan Welker and Klaus Bock allows you to create snapshots of websites in three different sizes by simply passing its address to a specific URL. Should be worth a bookmark I say.

 

Posted in: Tools

Tags: