Using Gallio with NCover

March 17, 2009 at 11:39 PMAndre Loker

I’m really keen to use MbUnit 3, but I still have some obstacles with Gallio to overcome. I use NCover to check the coverage created by the unit tests in the assembly that is being tested – and only in that assembly. The whole procedure is part of an automatic build process executed with NAnt.

NAnt + NCover + MbUnit 2.4 = Happy Go Lucky!

For the combo I used before (using MbUnit 2.4) my build files contained something similar to this:

   1: <property name="unittest.cmdline" value='/ap:"${build.dir}" /rf:"${testresults.dir}" /rnf:"mbunit-${}" /rt:Xml "${target.path}"'/>
   2: <property name="unittest.runner" value="${tools.dir}\MbUnit\mbunit.cons.exe"/>
   4: <exec
   5:   program="${tools.dir}\NCover\ncover.console.exe"
   6:   workingdir="${build.dir}"
   7:   commandline='"${unittest.runner}" ${unittest.cmdline} //a "${coverage.assemblies}" //w "${build.dir}" //reg //x "${testresults.dir}\coverage-${}.xml" //ea CoverageExcludeAttribute'
   8:   if="${ncover.supported}"        
   9: />

While this might look scary, it boils down to calling this:

ncover.console SomeAssembly.Tests.dll //a SomeAssembly

(plus a number of switches that I omitted for simplicity). This works perfectly and generates a nice coverage file for SomeAssembly based on the tests in SomeAssembly.Tests.dll – just what I want.

NAnt + NCover + MbUnit 2.4 Gallio = Does Not Compute

My first train of thought was: hey, with Gallio being more or less the successor of MbUnit, moving to Gallio shouldn’t be so hard, just replace MbUnit.Cons.exe with Gallio.Echo.exe, fiddle with the arguments and you’re done. So I changed the build file to do something along these lines:

ncover.console gallio.echo.exe SomeAssembly.Tests.dll //a SomeAssembly

And indeed, the tests were run and a nice shiny coverage file was generated – which however was empty. Erm? What now? I don’t know exactly what causes this behaviour, it seems that Gallio runs the tests in some separate app domain or so which prevents NCover from instrumenting it.

Second chance: let Gallio do it!

I remembered that there was a special switch to set the “runner” used by Gallio. It defaults to “IsolatedAppDomain”, but there’s also a “NCover” runner available. Someone gave me the hint to simply use this runner.


Maybe I only need to tell Gallio to use NCover and we’re happy? So I changed my build file to do something like:

gallio.echo.exe SomeAssembly.Tests.dll /r:NCover

And indeed, now it runs the tests and creates a non-empty coverage file. Sweet! A bit slowish, but running at least. The problem is – and you might have predicted it – that the coverage is gathered of all loaded assembly, including the assemblies of MbUnit and Gallio. So even for a trivial assembly being tested I get a coverage file of 5 MB. I couldn’t find a way to pass arguments regarding NCover. Argh! The problem is known by the way, but yet to be fixed.

Update: this problem is fixed in 3.0.6 builds. There you can pass arguments to NCover using the NCoverArguments and NCoverCoverageFile options.

At last: taming the beast

Believe me if tell you that at that point I was just annoyed. As of now all that Gallio gave me is headaches and frustration. This article was initially meant to reflect this.

Up to the point when I reconsidered the problem of the empty log file and remembered that there was a “Local” runner listed in the Gallio.Echo.exe help. Maybe this would force Gallio to run the tests in a way such that NCover can do its magic? Thus, I tried this (note the /r switch):

ncover.console gallio.echo.exe SomeAssembly.Tests.dll /r:Local //a SomeAssembly

Let’s see what we get:

  • Tests are found and executed? Check.
  • The coverage file is non-empty? Check.
  • The coverage file only contains information about “SomeAssembly”? Check.

Could it be? Finally, after several disappointments I am finally able to integrate Gallio in my automatic builds! Well, I have not integrated the Gallio reports fully in my CruiseControl.NET web dashboard, but I’ll keep that for another day. As for today, I’m happy with what I achieved.


Gallio also provides a NAnt task which you can use instead of <exec>’ing. Starting from version 3.0.6 you can also pass arguments to NCover (as described above). I tested the following task in NAnt and it worked smoothly:

   1: <gallio result-property="testrunner.exit-code" 
   2:         application-base-directory="${build.dir}"
   3:         runner-type="NCover" 
   4:         failonerror="false" 
   5:         report-name-format="gallio-${}" 
   6:         report-types="xml" 
   7:         report-directory="${testresults.dir}">
   8:   <runner-property value="NCoverArguments='//w ${build.dir} //q //ea CoverageExcludeAttribute //a ${coverage.assemblies}'" />
   9:   <runner-property value="NCoverCoverageFile='${testresults.dir}\coverage-${}.xml'" />
  10:   <assemblies>
  11:     <include name="${target.path}" />
  12:   </assemblies>
  13: </gallio>
  15: <fail if="${testrunner.exit-code != '0'}" >The return code should have been 0!</fail>

Thanks to Bruno Wouters and Jeff Brown for the hint!



To condense the story into two sentences:

  1. If you want to use Gallio with NCover be sure to choose the “Local” runner by setting the /r:Local option.
  2. Don’t use the NCover runner if you need any control over NCover options. Update: or use version 3.0.6+

To the Gallio people

I hope this article isn’t utterly stupid or useless. I looked on the web but did not came across anything useful regarding NCover and Gallio. Also, I couldn’t find any hint about it in the documentation. I hope this article helps people that like me would really like to use Gallio but struggle with issues.

Update: thanks for your help!

Posted in: Tools

Tags: , , ,

Comments (14) -

Bruno Wouters
Belgium Bruno Wouters says:

Hi Andre,

I also use Gallio and NCover to run tests. Below you can see the NAnt file I use. The runner-extension can be removed if you don’t use TeamCity. I like this kind of setup better than invoking command line applications. The runner-property will be available from 3.0.6. You can download fresh builds from Not sure when the stable will be released.

<loadtasks assembly="${ToolsPath}\Gallio\bin\Gallio.NAntTasks.dll" />
  <target name="test" depends="build">
    <echo message="Started testing ${AssemblyToTest}"/>
    <gallio result-property="exitCode" runner-type="${RunnerType}" failonerror="false" report-name-format="Tests" report-types="html" report-directory="${ReportsPath}">
        <runner-property value="NCoverArguments='//l ${WorkingDirectory}\Coverage.log //ea CoverageExcludeAttribute //a ${TestedAssemblyName}'" />
        <runner-property value="NCoverCoverageFile='${WorkingDirectory}\Coverage.xml'" />
      <runner-extension value="TeamCityExtension,Gallio.TeamCityIntegration" />
          <include name="${AssemblyToTest}" />
    <fail if="${exitCode != '0'}" >One or more tests failed. Please check the log for more details</fail>    

  <loadtasks assembly="${ToolsPath}\NCoverExplorer.Extras\bin\NCoverExplorer.NAntTasks.dll" />
  <target name="gencoverreport" depends="test">
    <echo message="Generating coverage report for ${TestAssemblyDll}"/>
          <include name="${WorkingDirectory}\Coverage.xml" />
          <moduleThreshold moduleName="${TestedAssemblyDll}" satisfactoryCoverage="${SatisfactoryCoverage}" />


Bruno Wouters
Belgium Bruno Wouters says:

I forgot to replace the runnertype variable, runner type should be NCover.
${RunnerType} = NCover

other variables will probably explain them self.

Thanks for your feedback. I used the <exec> approach to switch between unit testing frameworks more easily, but I'll have a look at the 3.0.6 builds and the gallio-tasks.

Using Gallio with NCover has been a bit tricky.  On the one hand, you can pass the runner="NCover" argument to the NAnt task, however prior to v3.0.6 you could not then also pass in command-line arguments for NCover.

Now you can... The syntax looks like this:

<gallio runner-type="NCover">
    <include name="MyTestAssembly.dll" />
  <runner-property value="NCoverArguments='//a MyTestAssembly;MyCode'" />

However beginning with v3.0.6

The point about documentation is well-taken.  We do need some assistance writing out more docs.  However, many people seem to find these blog posts very useful.  Thanks for writing it!

There's actually a bunch of NAnt / MSBuild docs in there.  It's just a bit hard to find at the moment.  So I've just updated to provide some helpful shortcuts and tips for these common questions!  Smile

P.S.  v3.0.6 will probably be released tomorrow assuming I don't hit any show-stoppers.  You can also grab a recent build from

Oops, left a dangling sentence there...

Beginning with v3.0.6 we support passing arguments to NCover, as shown.

You can also turn it around and call echo from NCover

That's what I'm doing, isn't it?

You need to use the //pm option with gallio.echo.exe. This will provide the coverage results on the tests and not gallio. Smile

Joe Feser

The full command line would be:

ncover.console.exe gallio.echo.exe //pm SomeAssembly.Tests.dll

//pm tells ncover to profile the process.
SomeAssembly.Tests.dll is a parameter for gallio.

Hope this helps.


As in
ncover.console gallio.echo.exe SomeAssembly.Tests.dll //a SomeAssembly //pm gallio.echo.exe?

This executes the tests but then does not continue as if it is waiting for gallio to finish or so.

I also tried
ncover.console gallio.echo.exe SomeAssembly.Tests.dll //a SomeAssembly //pm
But this also freezes.

What advantage does //pm have over //a in this context? With Gallio's /r:Local option I can use //a and it works.

I tried
NCover.Console.exe Gallio.Echo.exe //pm Gallio.Host.exe SomeAssembly.Tests.dll
as described in a forum post (

But this also provides coverage for the Gallio code.


//pm is an NCover 2/3 feature and not 1.5.8 so it would not work for you.

You should really look at adding your article to the documentation of Gallio.  I downloaded the code from subversion and they have a documentation section that you could add what you have learned.

Curious do you know how to achieve the same results using MsBuild.  I started out writing our build scripts using Nant but have since moved to MsBuild and have not yet implemented Gallio but will be doing so shortly.  Any pointers?

I was having problems with your /r:local solution.  Gallio was not loading the correct app.config.  Here is the response I got from the Gallio people...

"That said, your problem is in the way you are using NCover.  You can either
use /r:IsolatedAppDomain and wrap Gallio.Echo.exe with a call to
NCover.Console.exe, or you can use /r:NCover and get built-in support for

"Don't use /r:Local though.  Basically this prevents Gallio from creating a
new AppDomain which means your app.config settings will not be available. "

Switching to /r:IsolatedAppDomain fixed the problem.


Pingbacks and trackbacks (4)+