NOTE: This post was last updated for SonarQube v4 in Dec 2013. The support for .NET has changed quite a bit since then,
so this guide is probably not going to be very useful anymore. Sorry.
In an effort to better understand some of the problematic areas of the C# codebase I work on, I recently setup an instance of the
SonarQube code analysis platform. SonarQube is originally written for Java analysis and later added C#
support. This posting walks you through my experience attempting to setup, configure and run the analysis.
Note: SonarQube changed it's name from "Sonar" in mid-2013, so older references to this posting may use the old name.
I periodically update this post to reflect changes with newer versions of the tools. Most recent update was 12/18/2013
based on a fresh install of SonarQube v4.0.
I've also written a SonarQube plugin to use ReSharper as a source for quality metrics. Once you have your SonarQube instance
up and running for your .NET project, see my post
SonarQube .Net ReSharper Beta Release for details on importing ReSharper results
into SonarQube.
SonarQube Overview:
So why did I even do this? Once up and running, SonarQube provides some useful metrics for pointing out hotspots in your code
that may be making it more difficult to maintain and extend your functionality. Through the web interface, you can drill-down on
any of the metrics to the module, class, and method level, including full source code. Some of the metrics provided for each C#
project include (screenshots are from the "nemo" demo site mentioned below, my own project, or other sources):
- General analysis
- Uses several rule-based static analysis tools
- Details and statistics, with drill-downs, on rule violations
- Comments:
- Percentage of code commented
- Percentage of public APIs that are (un)documented
- Duplication:
- Percentage of code that is duplicated
- Counts by duplicated lines of code, blocks, files
- Counts:
- Lines of code
- Count of files
- Count of classes
- Count of methods
In addition, it will track changes over time, so you can see where issues are increasing/decreasing in your code.
SonarQube is open sourced under the LGPL and free to use, however some of the plugins used to perform the analysis are only
commercially available, and in some cases come with steep licensing fees. For this blog, I focus on only freely available (ie: no
fees) aspects of the product. Each of the tools I used are also freely available (FxCop, StyleCop, Gendarme, Gallio, OpenCover,
MySQL -- all have licenses that allow no-fee usage for most people).
There is a demo site provided by one of the commercial plugin providers to demo the system
(including their not-free SQALE plugin) which shows analysis for several open-source Java projects and can give you a feel for the UI
and the data that can be provided. Beware -- this includes data from some of the commercial plugins, so don't expect to see
everything on that site after following this posting.
You can also reference the [official installation notes])http://docs.codehaus.org/display/SONAR/Setup+and+Upgrade).
General SonarQube Technical Architecture:
Generally, there is a "runner" that consumes the source code and analyzes it via plugins. This information is published to the
SonarQube database directly. Before each run, the runner makes a call to the server to download configuration settings stored there,
and mashes those with configuration settings stored in a local config file, as well as a project-specific config file.
The SonarQube (web) server pulls the results from the database and provides a UI for reviewing and managing them, as well as a UI for
managing configuration settings, user authentication, etc. The server also has it's own config file where you define database
settings, user authentication settings, etc.
Step 1: Prerequisites and Assumptions:
This posting assumes you have a working Debug build of your codebase that compiles with no errors and generates .pdb files. You need
access to the source code and the output, as they exists at the completion of your build. In my case, I have a CI server
(TeamCity for my playground site -- free for small/mid-scale configurations, and
Jenkins for our production BI build servers) running and just added another build step at the end to kick-off the SonarQube analysis
from the command line.
For many of the plugins (such as FxCop and Gendarme), you'll need to install the .NET Reference Assemblies, which are part of the
Win7 SDK for versions of .NET 1.0 through 4.0 and the
Win8 SDK for .NET version 4.5. Note that the
FxCop installer is included (exclusively) with the Win SDK. Also note that a full Win7 SDK install is several Gigs in size and will
take some time to download, so plan accordingly. You only need to install the Reference Assemblies (under the .NET Development header
in the installer) which ends up being about 151MB to download and 505MB to install if you use the web-based installer I linked to
above.
If you have any Silverlight components in your build, you will need to install the
Silverlight SDK for the version of Silverlight you are
targeting.
Install the Java JDK
Yes, the JDK. It's big and ugly, I know. You're a .Net dev, I know. Do it anyway. Set the
JAVA_HOME environment variable to point to the JDK root folder. This should be the “jre” folder – for
example, on my machine the value is set like this: JAVA_HOME=C:\\Program Files\\Java\\jre7 Note: You don’t
need to install the Source Code component from the installer, which will save you about 500MB of disk space.
If you’re using Gallio for running your unit tests, you’ll also need to make sure .NET2.0 is installed. For newer versions of
Windows (Win8 and Win Server 2012), this requires going into “Turn on Windows Features” settings and adding the “.NET Framework
3.5 Features” feature (note: in WinServer 2012, this is part of the Application Server role).
Step 2: The SonarQube Server
Download the SonarQube server. At the time of this (revised) posting,
it was version 4.0.
This will give you a .zip file. Decompress it into the location you want; I used C:/sonar-server
For the initial configuration/setup, I'd suggest running it at the command line until you know you have it fully configured.
To do that, open a command window and run bin\\windows-x86-64\\StartSonar.bat (or -x86-32 for 32-bit
machines) This will run in your current window (no real output, but the command blocks) until you hit Cntl-Z or Cntl-Break. You'll
see it pulling down additional packages as needed, such as JRuby, but eventually you'll see something like this, which means it's
working:
For SonarQube 3.7:
jvm 1 | 2012-04-27 08:45:44.254:INFO::Started SelectChannelConnector@0.0.0.0:9000
For SonarQube 4.0:
jvm 1 | 2013.12.18 21:04:17 INFO Web server is started
Once you get things working, I'd suggest installing it as a Windows service. To do this, first register Sonar as a service
using the provided script: bin\\windows-x86-64\\InstallNTService.bat (There's an UnInstallNTService.bat
script too). Then, start the service from the Windows Services control panel, or using the provided StartNTService.bat
script. Personally, I would suggest just configuring it to be Automatic so it will survive reboots, etc.
With my SonarQube install on Windows Server, running the service failed initially, so I used the logs/sonar.log file to troubleshoot
and found this error:
Encountered an error running main: java.lang.IllegalStateException: Unable to create file in temporary directory,
please check existence of it and permissions: C:\Windows\system32\config\systemprofile\AppData\Local\Temp\
Initially, I just took the lazy route and configured the service to run under my user account instead of a system account, so didn't
really "fix" that issue.
Update: In a later installs, I had to create the “Temp” folder in the systemprofile\AppData\Local folder and
provide security rights to the LOCAL SERVICE user.
The server has a sonar.properties config file that drives some of it's functionality, including which database to use.
You can find it in the \conf folder. The only change I made to this file was the port used for web connections (default is
9000 – I changed to 80), and the database (see Step 3) -- but if you want to use the built-in Derby database, no changes are needed.
Step 2a: Verify and Change Password
At this point, you should be able to access the SonarQube server at http://localhost:9000 (or whatever port you set in the
sonar.properties file). The default username/password is admin/admin. I would highly recommend changing the
admin password at this time. Passwords are changed via the user profile page.
Step 3: The Database
SonarQube server comes with a build-in Derby database. While it is quick to install and easy (and pre-configured), I would
suggest moving away from it almost immediately. I had MAJOR performance problems using it. For instance,
for my solution of about 400K lines of code (50+ Visual Studio projects), when using the Derby database, it took about 2 hours to
perform the analysis, 5 hours if the runner was on a different host. During that time, the server CPU would be pegged at 100% for
5 - 15 minutes per project as the runner reported it's results. Once I moved to a MySQL database, it takes about 20 minutes
total.
So, I would suggest downloading and installing the free MySQL Community Edition
from Oracle. At the time of this (revised) writing, it was at version 5.6.13.
Once installed, you'll need to create the SonarQube database instance. This is pretty straightforward, but SonarQube provides
a script to make it even that much easier. For v3.7, from the SonarQube installation folder, the script is in
extras/database/mysql/create_database.sql. It's no longer included in the v4.0 release, but you can find it
on the SonarQube GitHub repository.
The command to execute the query is at the top of the file. This will generate the SonarQube database instance, as well as grant
access from localhost and remotely. The SonarQube schema is created the first time you run the server.
You'll need to configure the SonarQube server to use this db. This is as easy as changing a couple of lines in the
sonar.properties config file. Chances are, the lines you want are already in the file (commented out), as samples are
there for MySQL, PostgreSQL, Oracle and MS SqlServer. (Note: MS SqlServer is not officially supported, although people on the web
have posted ways to get it working. I did not venture down this path).
Update: While getting SonarQube up and running in our production environment, the person doing the work was able to
get up and running with MS SqlServer and provided me with this feedback. Note, he was using v.2.13.1, which is newer than what I was
using at the time:
I followed the link you had on your page (http://www.sezok.de/sonar/sonar.html) but
differed in the following ways:
- The database must be setup with Case Sensitive and Accent Sensitive rules, so I created the database with collate option SQL_Latin1_General_CP1_CS_AS
- The jdbc drivers that come bundled with 2.13.1 actually work, and the ones listed in the above link don’t.
For me, I commented out the Derby/H2 lines and used these for my MySQL instance:
#----- MySQL 5.x/6.x
# Comment the embedded database and uncomment the following properties to use MySQL.
# The validation query is optional.
sonar.jdbc.url: jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true
# Optional properties
#sonar.jdbc.driverClassName: com.mysql.jdbc.Driver
#sonar.jdbc.validationQuery: select 1
Restart the SonarQube server, if it's running.
I should note that there isn't a quick-and-easy way to migrate from the Derby db to another, so expect any data collected in Derby
will be lost when you move. I found this page
where someone found a way, if you really must try.
Note: at one point, I was getting a com.mysql.jdbc.PacketTooBigException error, so I followed
the suggestions here and ran the sql command:
SET GLOBAL max_allowed_packet = 1024*1024*14;
Step 3a: Verify and Change password, again
At this point, the SonarQube server should be up and running again, albeit with no data. I would suggest verifying you can reach
the server at http://localhost:9000 (or whatever port you used in the sonar.properties file). The user accounts will have reset
with the database change, so again I would highly recommend changing the admin password at this time!
Step 4: The Runner
SonarQube was build for Java, so the docs almost all assume you're using Maven. You may be able to get this working with nAnt,
but there's an easier route. A small, java-based runner
has been created that can be kicked-off from the command line. This is what I used from within my TeamCity and Jenkins builds.
There’s also a Jenkins plugin that looks promising,
but I haven’t used it.
To use this runner, you will need to do a few things:
- Download the runner. At this (revised) writing, the version was 2.3. This will come as a Zip file. Decompress it in an appropriate location. For me, I put it in C:/sonar-runner/
- Test that it works. Open a command line to the sonar-runner/bin folder and run
sonar-runner -h. This will either show you the usage statement (if things are working fine), or blow up. If it blows up, check you'reJAVA_HOME is set correctly to point at the jre folder.
- Once it works, set another environment var:
SONAR_RUNNER_HOME to the root installation folder for sonar-runner. (for example: C:/sonar-runner/) This is used when you want to run the sonar runner from a folder other than the /bin folder. (ie: when you run sonar-runner from your source folder).
- Change the
conf/sonar-runner.properties file. This has general configuration items used by the runner, some of which can be overridden in the project's config file (we'll get to that later). At a minimum, you'll need to set the sonar.host.url to point to where you have your sonar server running, and the database config. For the db, just copy/paste the lines from the sonar.properties file you created in Step 2. Again, example values are provided in the file (commented out) which will likely work just fine. Don't forget to comment out the Derby lines if they exist.
Step 5: The C# Plugins
You'll need to install the "C# Plugins Ecosystem" (ie: The plugins to
analyze C# code). Login to the SonarQube website as an Admin user and click the "Settings" link in the upper right-hand corner,
then click "Update Center" in the left-hand navigation pane. You should see an "Available Plugins" tab, under which you will find
all of the officially supported plugins. Search down the list to find "C#" under the Additional Languages section and click on the
"C#" link. You should now see an "Install" button -- click it!
Under the hood, this is installing each of the C# plugins .jar files into the /extensions/plugins folder of your SonarQube server
installation folder. You will need to restart the SonarQube service after you place the files for them to be available.
The plugins consist of the following: (as of this writing, which uses v2.1 of the plugins)
- sonar-dotnet-plugin - .NET Core (support for the .NET-based languages - general API used by other plugins -- required)
- sonar-csharp-plugin - C# Core (support for parsing C# language -- required)
- sonar-dotnet-fxcop-plugin - FxCop support (general analysis)
- sonar-csharp-stylecop-plugin - StyleCop support (formatting analysis)
- sonar-dotnet-gendarme-plugin - Gendarme support (general analysis)
- sonar-dotnet-gallio-plugin - Gallio support (unit testing / coverage)
- sonar-dotnet-ndeps-plugin - NDeps (assembly dependency parsing) (new in v1.3)
Note: If you use VB.NET, there is a separate plugin entry for that, which will reuse several of these plugins.
If you're going to use FxCop, you'll need to install FxCop 10
(the FxCop installer is installed as part of the Win7 SDK install).
If you're going to use StyleCop, you can either use a bundled version included with SonarQube, or install your own.
If you're going to use Gendarme, you can either use a bundled version or install your own.
If you're going to use Gallio, you'll need to install it as well as
OpenCover (or you can use NCover, if you have a license). When installing OpenCover,
I suggest using the Advanced mode and setting it to install for all users. Also note: Gallio does not support NUnit v2.5.4 or higher,
so you’ll either need to update the Gallio plugin
or jump to NUnit 2.6 (or other supported version).
Step 6: The sonar-project.properties File
Each Solution will need to have it's own sonar-project.properties file. This file will need to exist in the folder from which
you execute the sonar-runner. To make this easy, I would suggest putting the file in the same folder as your .sln file.
The file will have a few sections, which I will describe here.
Important Note: Any folder names in the config file will need to either escape the backslash with another backslash (\\)
or use a forward slash (/). I've chosen the latter.
Project Identification:
This section will provide the project key used by the SonarQube server to group analysis results over time, as well as provide a
useful name in the UI, etc. This should be unique across projects. The project version can be used to track different branches, etc.
# Project identification
sonar.projectKey=Jwright:DemoApp
sonar.projectVersion=trunk
sonar.projectName=DemoApplication
Then, describe the source code layout. The "sources" field points to the top-level folder where the source code exists. If you're
.sln and .csproj files have relative paths internally, then this should be the top-level folder. Assuming you don't have any strange
layouts, this will likely be the same folder as your .sln file (which is likely where your .properties file exists), so can just
be ".". Additionally, you need to denote that the language is C# using the "cs" value.
# Info required for Sonar
sonar.sources=.
sonar.language=cs
C#-specific settings:
Here, you'll need to provide information about where the .sln file is located and where key libraries are located, and which version
of .Net you're using.
#Core C# Settings
sonar.dotnet.visualstudio.solution.file=DemoApp.sln
sonar.silverlight.4.mscorlib.location=C:/Program Files (x86)/Reference Assemblies/Microsoft/Framework/Silverlight/v4.0
sonar.dotnet.excludeGeneratedCode=true
sonar.dotnet.4.0.sdk.directory=C:/Windows/Microsoft.NET/Framework/v4.0.30319
sonar.dotnet.version=4.0
# To prevent any issues while analyzing multiple solutions containing projects with similar keys
# Will be set by default to safe starting at version 2.2: http://jira.codehaus.org/browse/SONARDOTNT-339
sonar.dotnet.key.generation.strategy=safe
Plug-in Specific Sections:
For each plugin, there is a "mode" setting. If blank, then the plugin will run. If you want to skip/not run a plugin, set the mode
to "skip".
For Gallio, you can stipulate if you want to use OpenCover (free) or NCover (not free). You can also stipulate the runner mode.
I had trouble using anything other than "Local". You will also need to stipulate the naming pattern (regular expressions, I believe)
for the Visual Studio projects that include unit tests. You can have multiple patterns, separated by semicolons.
#Gendarme
sonar.gendarme.mode=
# Gallio / Unit Tests
sonar.gallio.mode=
sonar.gallio.coverage.tool=OpenCover
sonar.gallio.runner=Local
sonar.dotnet.visualstudio.testProjectPattern=*UnitTest*;Testing*
sonar.opencover.installDirectory=C:/Program Files (x86)/OpenCover/
# FXCop
sonar.fxcop.mode=skip
# StyleCop
sonar.stylecop.mode=
# NDeps
sonar.ndeps.mode=
Step 7: Running an Analysis
From the folder with the sonar-project.properties file, run the command $SONAR_RUNNER_HOME/bin/sonar-runner
You'll see the runner start up, listing some details like the working folder, etc, then it will kick off the source code parsing,
then the plugins (such as running the unit tests). If there are any errors, a Java exception will be thrown. Sometimes these contain
enough details to troubleshoot, sometimes not, so you may need to run with the -X command line argument to get additional details
when errors occur.
One error I run into frequently is UTF-8 Byte Order Mark causing the source file parsing to fail. You will see this in two ways:
the files will not end up in the SonarQube web UI and something like the following will show in the runner log:
07:42:56.938 ERROR - Unable to parse source file : C:\\Users\\...\\CodeFile.cs
07:42:56.969 ERROR - Source Snippet:
---------------
--> n++using System;
2 using System.Collections.Generic;
3 using System.Diagnostics;
4 using System.IO;
5 using System.IO.Pipes;
6 using System
---------------
You can fix this by setting the encoding in your sonar-project.properties or the sonar-runner.properties file:
#----- Default source code encoding
sonar.sourceEncoding=UTF-8
After a successful run of the analysis, you can see the results in the SonarQube server webpage. Unless you have changed the port
in the sonar.properties file, this will be at http://localhost:9000/
The homepage will have a link for each project (the value provided as sonar.projectName), which will take you to the project
overview. Clicking on any of the links on the overview page will take you to the drill-down data for that metric.
References:
I used this page to start my journey, in addition to the SonarQube project docs: http://www.ifunky.net/Blog/post/Install-and-Configure-Sonar-on-Windows-2008.aspx
The C# Plugins page: http://docs.codehaus.org/display/SONAR/C-Sharp+Plugins+Ecosystem
The Sonar installation page: http://docs.codehaus.org/display/SONAR/Install+Sonar
Update: The Sonar Source page, complete with mailing lists, issue trackers, etc. http://www.sonarsource.org/support/support
Hope this helps!
Update 2014-02-06: The SonarQube .NET ReSharper plugin has been released. 
