UPDATE 2/1/2014: After I wrote this initial posting, the team at SonarSource who oversee SonarQube and the .NET plugins replied to me saying they have changed their mind on the need for this plugin, and on Unit Test reporting in general. Going forward, SonarQube will not be reporting unit test results, focusing only on coverage details. As such, this plugin is effectively dead. I do not plan on continuing any work on it at this time.

I've made some decent progress with a new SonarQube plugin, intended to be part of the community "C# Ecosystem" plugin package, which I'm calling the "sonar.dotnet.tests" plugin. The intent of this plugin is to support direct parsing of unit testing/coverage tools' reports instead of relying on Gallio, as the current ecosystem plugins do. The first (and currently only) supported framework is NUnit, allowing you to directly import NUnit test results into SonarQube without going through Gallio (see SONARDOTNT-372).

I posted this email to the SonarQube developers mailing list, but thought I'd also post the information here, so that it's easier to find via search engines and so that people can leave comments without having to subscribe to the mailing list.

If you're interested in taking a look at my work in-progress, you can find my code in the "dotnet-tests-plugin" branch of my GitHub clone of the dotnet repository. Note: I have multiple branches in my repo, so make sure you're looking at the "dotnet-tests-plugin" branch. There is not currently a publicly hosted build of the plugin, so if you want to try it out, you'll need to build it (and the other dotnet plugins in the same repo) yourself.

Some general notes:

• There is one known open issue: It currently is unable to associate test results to the appropriate classes when the tests are defined in a base class of the test fixture.
• The plugin utilizes the default SonarQube unit test widget (no need for the separate widget used in the gallio plugin)
• The plugin is using the org.sonar.api.test classes (TestCase, TestPlan, etc), which required changing the sonar API version to v3.5 (from 3.0). Since 3.7 is the LTS version, I think this should not create too many complaints.
• I have only implemented the nunit support so far, but it should be pretty straight forward to migrate the mbunit (gallio) parsing logic from the existing gallio plugin, as well as almost all of the coverage report parsing code.
• The plugin supports only reuseReport and skip modes.

Implementation details:

• This branch also includes Peter Steven's Pull request for SONARDOTNT-382, which is required for the NUnit support to work.
• As part of this implementation, I made some changes to the base dotnet plugins:
  • Added a property (sonar.csharp.squid.includeTestSources) to include Test projects when performing squid parsing for c# projects (building out the AST). (Default is false to mirror prior behavior). This allows consumers to lookup file resources for types/members in test classes. This also required changes to the CSharpSquidSensor so that it will run for test projects and be able to lookup files in the test sources tree but not to include the Test files when saving measures, complexity, issues, etc.
  • CSharpResourcesBridge now also indexes methods using their parameter signatures (ex: My.Namespace.MyClass#MethodName(bool, int)) as a second index. This allows callers to lookup methods based on their usages without having to know the line number they are defined at.
    • The current indexing implementation includes the line number in the index key and the doc for DotNetResourceBridge currently says this: /!\\ Do not use for the moment! For the moment, method key ends with ':XXXX', where 'XXXX' is the line number, so this API does not work. TODO: Need to work on that.
    • As such, the changes I made to include a secondary index for methods could be removed and made the primary index instead. This would remove the need for the custom SourceMethod class as well, but would break anyone who has implemented against the current line number based signature. I would appreciate feedback on whether this should be a second index or the primary index.
  • Similar changes would likely be required in the VB.NET plugin, which I don't have access to, so haven't tested/reviewed.
• New unit test result parsers (such as MSTest and MBUnit/gallio) would need to extend o.s.p.dotnet.tests.parser.DotNetTestResultParser and be added to DotNetTestsSensor.getParsers() at the line that currently says "//TODO: add other parsers here". I would eventually like to refactor this to use dependency injection to discover the parsers, but haven't gotten there yet.

Usage:

For the NUnit parsing, the following properties need to be defined in the project.properties file (or equivalent) :

#
# global flags to enable/disable all parsing. Integration tests have a separate flag.
#
sonar.dotnet.tests.mode=reuseReport
sonar.dotnet.tests.it.mode=skip

#
# each parser will have it's own pair of report path keys (unit and integration):
#
sonar.dotnet.tests.nunit.reports.path=$(SolutionDir)/TestResult*.xml
#sonar.dotnet.tests.nunit.it.reports.path=$(SolutionDir)/AggregateResults.xml

#
# The NUnit parser requires the new c# squid property to be set to true
#
sonar.csharp.squid.includeTestSources=True

As I mentioned, I will be setting this aside for several months as I will be spending my "side project time" working with the That Conference organizers team for That Conference 2014. I would appreciate anyone who could review things as they currently stand and provide any feedback, particularly from the current dotnet plugin maintainers and SourceSource crew.

Update 2014-02-06: The SonarQube .NET ReSharper plugin has been released. See my release announcement.

For those of you following along with the SonarQube ReSharper plugin, here's an update on where things currently stand.

JetBrains released version 8.1 of the ReSharper commandline tools back in early December, so I consider version 1.0 of the plugin to have been "released" around that time... and there are a handful of folks out there using it! However, I was unsuccessful in getting the necessary votes on the SonarQube developers mailing list to get the plugin published to the official Update Center, which would make it uber simple for others to install and use the plugin. I must say, I was quite disappointed that I only managed one "+1" vote of the required three. (For the uninitiated, the rules for getting a plugin officially released require three "+1", or "yes" votes and no "-1" or "no" votes from the existing developers group.) As discussed on the dev mailing list, there's not much incentive for the developers on the list, who are, by definition, Java developers, to try, test and therefore vote on plugins for .NET (or other languages), thus making the bar very high for new developers to make the cut. Generally, the call for a vote stays open for under a week -- in my case, it was open for more than a month before I threw in the towel.

Since then, however, someone else reported an issue with the rule keys used by the plugin (which map directly to the ReSharper rule keys) that cause the Issues Report plugin to fail.

Additionally, several users have asked for the ability to provide a ReSharper dotSettings file while running analysis directly in the plugin.

And I've had a few people contact me to say they'd like to submit pull requests for features they'd like to see as well.

So, I'm planning to work on version 1.1 over the next month or so to fix the rule key issue and add a few of the user-requested features (especially if pull requests are submitted). And at that point, re-try to get the plugin officially hosted in the Update Center. I am putting a somewhat hard limit on a month to get this wrapped up, as my "volunteer" time will be shifting to support That Conference 2014, likely up until the actual conference, Aug 11-13th.