2006-11-30, if you intend on using QALab with Maven 2, please do read the plugin FAQ, thanks.
2006-11-27, QALab is integrated in JBuilder 2007 as part of the new TeamInsight team-RAD feature.
"We are really excited at the collaboration between ObjectLab and CodeGear" said Joe McGlynn, Product Manager for JBuilder. "Our engineers were searching for an innovative way to consolidate and trend key project metrics and we found a truly wonderful solution in QALab".
ObjectLab has worked with Borland's Developer Tools Group ("DevCo/CodeGear") to make the transition to Maven 2 smooth and rich 'out-of-the-box'. QALab itself will remain independent and carry on supporting all platforms
2006-11-12, version 1.0 is out! After over 3 years of internal use and 15 months as an open source project, we finally ship a version that we consider good enough to be named '1.0'. We are very proud of this little project, it has generated a fair amount of press. We would like to thank our colleagues, friends and users who have helped us reaching this stage. We will also try to help and improve the product and we already have a few more ideas... and some very interesting news coming soon...Watch this space!
QA Tools like checkstyle, pmd, pmd-cpd, findbugs, cobertura (cobertura-branch and cobertura-line) and simian are great build tools but they only take a snapshot of the state of your project. You do not get a sense of the trend of your project.
This project collects and consolidates data from several QA tools and keeps track of them overtime. This allows developers, architects and project managers alike to be presented with a trend of the QA statistics of their project.
The following tools are currently supported:
This project can be used via ant or Maven. There are three main steps to using QALab:
Here is a highlight of the steps available, in green are the input files, generated by the respective QATools. In yellow, the files generated by QALab and each transition is a task in QALab.
PMD, Checkstyle, Cobertura, PMD-CPD, Simian and FindBugs have each their own XML format. This task is to extract some information from each file, typically the number of violations per file and typically to add this information to a qalab.xml file. QALab is gathering statistics info and therefore, we are not interested in the exact 'violation' or error (see checkstyle, pmd, etc for that).
Please note that the qalab.xml will keep growing and should be kept in your source repository or backed up regularly. The more info you have in it, the more valuable it becomes. As such, it is recommended to use QALab only say, once a day when the whole set of checkstyle, pmd, pmd-cpd, Findbugs and Simian statistics are generated.
The QALab team realise that some project may generate very large qalab.xml files if they have lots of classes and lots of violations. QALab was used for over 2 years on a 450,000 lines of code project without any issue. Having said that, QALab offers you the ability to define how to save the consolidated statistics; simply implement the QALabExporter interface, say to a DB? Most of the examples are shown with the XML file.
In its default configuration, QALab will will create a new summary and result entry for each run and file each time you run the 'merge' task. This can be controlled by using a given timestamp (or forcing it to use only a 'date') and an action 'replace', in which case, each run of QALab 'merge' will replace the previous one with identical timestamp and type. This is very useful if you want to run merge several times per day but want to keep only the latest set of stats per day.
The XML format uses primarily attributes as it tries to limit the number of elements in order to save some space. The name of the attributes though is quite explicit.
Please note that Cobertura generates 2 types of statistics in QALab: cobertura-line and cobertura-branch, corresponding to the line and branch stats in the coverage.xml file
<qalab version="1.1">
<summary>
<summaryresult date="yyyy-mm-dd HH:MM:ss" statvalue="n" filecount="n"
type="checkstyle|pmd|pmd-cpd|simian|findbugs|cobertura-line|cobertura-branch"/>
</summary>
<file id="path/to/File1.java">
<result date="yyyy-mm-dd HH:MM:ss" statvalue="n"
type="checkstyle|pmd|findbugs|cobertura-line|cobertura-branch"/>
</file>
<file id="path_to_File2.java" path="path/to/File2.java">
<result date="yyyy-mm-dd HH:MM:ss" statvalue="n"
type="checkstyle|pmd|findbugs|cobertura-line|cobertura-branch"/>
<result date="yyyy-mm-dd HH:MM:ss" statvalue="n"
type="checkstyle|pmd|findbugs|cobertura-line|cobertura-branch"/>
</file>
</qalab>
The date attributes will be the date and time when the data was collected, this is due to the fact that most original XML (checkstyle, pmd, etc) do not have a date representing when they ran.
The DTD for qalab.xml is as follows:
<!DOCTYPE qalab [
<!ELEMENT qalab (summary, file*)>
<!ATTLIST qalab version CDATA #REQUIRED>
<!ELEMENT summary (summaryresult*)>
<!ELEMENT file (result+)>
<!ATTLIST file
id ID #REQUIRED
path CDATA #REQUIRED
>
<!ELEMENT result EMPTY>
<!ATTLIST result
date CDATA #REQUIRED
statvalue CDATA #REQUIRED
type CDATA #REQUIRED
>
<!ELEMENT summaryresult EMPTY>
<!ATTLIST summaryresult
date CDATA #REQUIRED
statvalue CDATA #REQUIRED
filecount CDATA #REQUIRED
type CDATA #REQUIRED>
]>
Once you have a qalab.xml, it is possible to generate some charts:
As a project grows and the number of developers increases or changes, it is becoming more and more difficult to keep track of the changes impacting the code quality. The best way to deal with this is to deal with any deviation as early as possible. Some style violations may remain in your code for months and are accepted as such.
This report will only show the changes over a given time window, it will highlight the Up (baaad!) or Down impacts (good!) on the code.