QALab

QALab Data Consolidation

As described in the overview, the first step is to take the statistics from checkstyle, pmd, findbugs, cobertura, simian and pmd-cpd and consolidate them in an existing qalab.xml file.

Checkstyle

Here is the ant call, we have not described how or where you generate the checkstyle xml file:

<target name="checkstylestat" depends="checkstyle"
   description="Generates a historical stat report of code convention violations.">
 <taskdef name="mergestat" classname="net.objectlab.qalab.ant.BuildStatMergeTask">
  <classpath>
   <pathelement location="${lib}/qalab-1.0.jar" /> 
   <pathelement location="${lib}/xercesImpl-2.6.2.jar" /> 
   <pathelement location="${lib}/jcommon-1.0.5.jar" /> 
   <pathelement location="${lib}/jfreechart-1.0.2.jar" /> 
   <pathelement location="${lib}/log4j.jar" /> 
  </classpath>
 </taskdef>

 <!-- You could checkout the qalab.xml file here -->

<tstamp>
	<format property="TIME" pattern="yyyy-MM-dd HH:mm:ss"/>
</tstamp>

 <mergestat inputFile="${doc}/checkstyle-output.xml" 
            outputFile="qalab.xml" 
            srcDir="${java.src.dir}" 
            handler="net.objectlab.qalab.parser.CheckstyleStatMerge" 
            propertiesFile="qalab.properties"      optional
            mergerTimeStamp="${TIME}"/> 

 <!-- You could check in the modified qalab.xml file here -->

Parameter	Mandatory	Description
inputFile	Yes	The input file, in this case the xml output from checkstyle
outputFile	Yes	The output file, typically qalab.xml; this is the file that will consolidate all statistics and grow over time.
srcDir	Yes	the source directory for the java files, so that it can be linked. It will truncate that part of the source directory in order to just retain the package path, eg if your source is in: e:/project/qalab/src/main/java/net/objectlab... use "src/main/java to retain the information at net/objectlab/.... level (the fully qualified class name)
handler	Yes	this is the class name that stipulates which SAX handler to use for parsing the inputFile. In this case, we chose CheckstyleStatMerge, which handles checkstyle.
mergerTimeStamp	No	Sometimes, if you merge more than one type of statistics, you may want to use the same timestamp for all of them, use this parameter format "YYYY-MM-dd HH:mm:ss". If you leave it out, QALab will use "now" as the timestamp.
quiet	No	Default to false.
propertiesFile	No	may contain some properties how to handle outputs.

The propertiesFile is a conventional properties file that can define the following items, it is optional.

Property	Description
qalab.merge.output.file	The output file where to put the consolidated statistics.
qalab.merge.output.classname	Alternatively, the data can initially come from as a class Resource, this parameter takes the className.
qalab.merge.output.resourcename	In addition to classname, stipulate here the resourceName.
qalab.merge.exporterclass	You can define the exporter! Default is the QALabXMLExporter, but you could create your own to store the consolidated stats in a DB for instance (and use the qalab.properties to store the exporter initialisation details).
qalab.merge.timestampdateonly	Defaulted to false (ie use full date and time for the timestamp, regardless whether or not you have given a full timestamp. Useful, if one wants to keep only a date in the file. Use it in conjunction with qalab.merge.action.
qalab.merge.action	'new' or 'replace' (defaulted to new). If one uses 'replace', the new run will replace statistics for the same 'type' and same 'timestamp'. This is very useful if you want to be able to run the merger several times per day but only wishes to keep the 'latest' one (use it in conjunction with qalab.merge.timestampdateonly=true)

If you want to keep your qalabs.xml in version control, you can do a checkout just before the mergestat call and a checking just after it.

Accepted Checkstyle Format

The Checkstyle report should be XML, for instance:

<?xml version="1.0" encoding="UTF-8"?>
<checkstyle version="3.4">
<file name="E:\project\qalab\src\main\java\net\objectlab\qalab\ant\BuildStatMergeTask.java">
<error line="161" severity="error" message="Line has trailing spaces." 
	source="com.puppycrawl.tools.checkstyle.checks.GenericIllegalRegexpCheck"/>
<error line="162" severity="error" message="Line has trailing spaces." 
	source="com.puppycrawl.tools.checkstyle.checks.GenericIllegalRegexpCheck"/>
</file>
<file name="E:\project\qalab\src\main\java\net\objectlab\qalab\parser\BaseStatMerge.java">
<error line="182" severity="error" message="Line is longer than 80 characters." 
	source="com.puppycrawl.tools.checkstyle.checks.sizes.LineLengthCheck"/>
</file>
....
</checkstyle>

PMD

Here is the ant call:

 <mergestat inputFile="${doc}/pmd-output.xml" 
            outputFile="qalab.xml" 
            srcDir="${java.src.dir}" 
            handler="net.objectlab.qalab.parser.PMDStatMerge" 
            mergerTimeStamp="${TIME}"/>

Accepted PMD Format

The PMD report should be XML, for instance:

<?xml version="1.0" encoding="UTF-8"?>
<pmd version="3.1" timestamp="2005-06-22T18:11:49.769">
 <file name="E:\project\qalab\src\main\java\net\objectlab\qalab\parser\BuildStatForChartParser.java">
  <violation line="179" rule="UnusedFormalParameter" ruleset="Unused Code Rules" priority="3">
   Avoid unused formal parameters such as 'ignoreRaw'
  </violation>
  <violation line="179" rule="UnusedFormalParameter" ruleset="Unused Code Rules" priority="3">
   Avoid unused formal parameters such as 'ignoreUri'
  </violation>
  <violation line="390" rule="UnusedFormalParameter" ruleset="Unused Code Rules" priority="3">
   Avoid unused formal parameters such as 'ignoreQName'
  </violation>
 ...
 </file>
 <file name="E:\project\qalab\src\main\java\net\objectlab\qalab\parser\BuildStatMoverHandler.java">
  <violation line="265" rule="UnusedFormalParameter" ruleset="Unused Code Rules" priority="3">
Avoid unused formal parameters such as 'ignoreNamespaceURI'
  </violation>
...
</file>
...
</pmd>

PMD-CPD (Copy Paste Detector)

PMD CPD is the Copy Paste Detector from PMD, here is the ant call:

 <mergestat inputFile="${doc}/cpd.xml" 
            outputFile="qalab.xml" 
            srcDir="${java.src.dir}" 
            handler="net.objectlab.qalab.parser.PMDCPDStatMerge" 
            mergerTimeStamp="${TIME}"/>

Accepted PMD CPD Format

The PMD CPD report, Copy Paste Detector, should be XML, for instance:

<?xml version="1.0"  encoding="UTF-8"?>
<pmd-cpd>
  <duplication lines="76" tokens="341">
    <file line="27" path="...\IndexFactorLookup.java"/>
    <file line="27" path="...\TemplateMapper.java"/>
    <codefragment>
...
    </codefragment>
  </duplication>

  <duplication lines="59" tokens="225">
    <file line="12" path="...\datecalc\SingleNameDateCalculator.java"/>
    <file line="13" path="...\SingleNameDateCalculator.java"/>
    <codefragment>
...
    </codefragment>
  </duplication>
</pmd-cpd>

FindBugs

Here is the ant call:

 <mergestat inputFile="${doc}/findbugs-output.xml" 
            outputFile="qalab.xml" 
            srcDir="${java.src.dir}" 
            handler="net.objectlab.qalab.parser.FindBugsStatMerge"
            mergerTimeStamp="${TIME}"/>

Accepted FindBugs Format

The FindBugs report should be XDOC, NOT XML! for instance:

<?xml version="1.0" encoding="UTF-8"?>
<BugCollection>
  <file classname="net.objectlab.qalab.ant.BuildStatMoverTask">
    <BugInstance type="DLS_DEAD_LOCAL_STORE" priority="Normal" 
    message="DLS: Dead store to local variable in method net.objectlab.qalab.ant.BuildStatMoverTask.execute()" 
    line="98"/>
  </file>
  <file classname="net.objectlab.qalab.parser.BuildStatMoverHandler$1">
    <BugInstance type="SIC_INNER_SHOULD_BE_STATIC_ANON" priority="Low" 
    message="SIC: The class net.objectlab.qalab.parser.BuildStatMoverHandler$1 could be refactored into a named _static_ inner class" 
    line="0"/>
    ....
  </file>
  ....
  <Errors/>
</BugCollection>

Simian

Here is the ant call:

 <mergestat inputFile="${doc}/simian-output.xml" 
            outputFile="qalab.xml" 
            srcDir="${java.src.dir}" 
            handler="net.objectlab.qalab.parser.SimianStatMerge" 
            mergerTimeStamp="${TIME}"/>

Accepted Simian Format

The Simian report should be XML as below. At the moment, only the <summary> section will be used:

<?xml version="1.0" encoding="UTF-8"?>
<simian version="2.2.4">
    <check ignoreCharacterCase="true" ignoreCurlyBraces="true" 
    ignoreIdentifierCase="true" ignoreModifiers="true" 
    ignoreStringCase="true" threshold="10">
        <set lineCount="12">
            <block sourceFile="E:\project\qalab\src\test\java\net\objectlab\qalab\parser\CheckstyleStatMergeTest.java" 
              startLineNumber="128" endLineNumber="139"/>
            <block sourceFile="E:\project\qalab\src\test\java\net\objectlab\qalab\parser\CheckstyleStatMergeTest.java" 
              startLineNumber="66" endLineNumber="77"/>
        </set>
        <summary duplicateFileCount="1" duplicateLineCount="24" 
        duplicateBlockCount="2" totalFileCount="23" 
        totalRawLineCount="3689" totalSignificantLineCount="1123" processingTime="1572"/>
    </check>
</simian>

Cobertura

Here is the ant call for collection line coverage, this will generate entries with type="cobertura-line".

 <mergestat inputFile="${doc}/cobertura/coverage.xml" 
            outputFile="qalab.xml" 
            srcDir="${java.src.dir}" 
            handler="net.objectlab.qalab.parser.CoberturaLineStatMerge" 
            mergerTimeStamp="${TIME}"/>

Here is the ant call for collection branch coverage, this will generate entries with type="cobertura-branch".

 <mergestat inputFile="${doc}/cobertura/coverage.xml" 
            outputFile="qalab.xml" 
            srcDir="${java.src.dir}" 
            handler="net.objectlab.qalab.parser.CoberturaBranchStatMerge" 
            mergerTimeStamp="${TIME}"/>

Example with Checkstyle

Checkstyle file:

<?xml version="1.0" encoding="UTF-8"?>
<checkstyle version="3.4">
<file name="E:\project\qalab\src\main\java\net\objectlab\qalab\ant\AntTaskLogger.java">
<error line="29" severity="error" message="Line has trailing spaces." 
   source="com.puppycrawl.tools.checkstyle.checks.GenericIllegalRegexpCheck"/>
<error line="31" severity="error" message="Line has trailing spaces." 
   source="com.puppycrawl.tools.checkstyle.checks.GenericIllegalRegexpCheck"/>
</file>
<file name="E:\project\qalab\src\main\java\net\objectlab\qalab\ant\BuildStatChartTask.java">
<error line="1" severity="error" message="Missing a header - not enough lines in file." 
   source="com.puppycrawl.tools.checkstyle.checks.HeaderCheck"/>
<error line="44" severity="error" message="Line is longer than 80 characters." 
   source="com.puppycrawl.tools.checkstyle.checks.sizes.LineLengthCheck"/>
<error line="46" severity="error" message="Line is longer than 80 characters." 
   source="com.puppycrawl.tools.checkstyle.checks.sizes.LineLengthCheck"/>
</file>
</checkstyle>

Consolidated qalab.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE qalab [
<!ELEMENT qalab (summary, file*)>
<!ATTLIST qalab version CDATA #REQUIRED>
<!ELEMENT summary (summaryresult*)>
<!ELEMENT file (result+)>
<!ATTLIST file
	id ID #REQUIRED
	path CDATA #REQUIRED
>
<!ELEMENT result EMPTY>
<!ATTLIST result
	date CDATA #REQUIRED
	statvalue CDATA #REQUIRED
	type CDATA #REQUIRED
>
<!ELEMENT summaryresult EMPTY>
<!ATTLIST summaryresult
	date CDATA #REQUIRED
	statvalue CDATA #REQUIRED
	filecount CDATA #REQUIRED
	type CDATA #REQUIRED>
]>
<qalab version="1.1">
    <summary>
        <summaryresult date="2005-05-23 21:38:28" statvalue="5"
            filecount="2" type="checkstyle"/>
    </summary>
    <file id="net_objectlab_qalab_ant_AntTaskLogger.java"
           path="net/objectlab/qalab/ant/AntTaskLogger.java">
        <result date="2005-05-23 21:38:28" statvalue="2" type="checkstyle"/>
    </file>
    <file id="net_objectlab_qalab_ant_BuildStatChartTask.java"
           path="net/objectlab/qalab/ant/BuildStatChartTask.java">
        <result date="2005-05-23 21:38:28" statvalue="3" type="checkstyle"/>
    </file>
</qalab>

Example with PMD

If we keep the qalab.xml and run in a pmd file:

<?xml version="1.0" encoding="UTF-8"?>
<pmd version="3.1">
<file name="E:\project\qalab\src\main\java\net\objectlab\qalab\ant\BuildStatMergeTask.java">
<violation line="56" rule="UnusedPrivateField" ruleset="Unused Code Rules" priority="3">
Avoid unused private fields such as 'NOW'
</violation>
<violation line="58" rule="UnusedPrivateField" ruleset="Unused Code Rules" priority="3">
Avoid unused private fields such as 'DEFAULT_DATETIME_FORMAT'
</violation>
</file>
<file name="E:\project\qalab\src\main\java\net\objectlab\qalab\ant\AntTaskLogger.java">
<violation line="234" rule="UnusedFormalParameter" ruleset="Unused Code Rules" priority="3">
Avoid unused formal parameters such as 'compiledStats'
</violation>
</file>
</pmd>

the consolidated qalab.xml file becomes:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE qalab [
<!ELEMENT qalab (summary, file*)>
<!ATTLIST qalab version CDATA #REQUIRED>
<!ELEMENT summary (summaryresult*)>
<!ELEMENT file (result+)>
<!ATTLIST file
	id ID #REQUIRED
	path CDATA #REQUIRED
>
<!ELEMENT result EMPTY>
<!ATTLIST result
	date CDATA #REQUIRED
	statvalue CDATA #REQUIRED
	type CDATA #REQUIRED
>
<!ELEMENT summaryresult EMPTY>
<!ATTLIST summaryresult
	date CDATA #REQUIRED
	statvalue CDATA #REQUIRED
	filecount CDATA #REQUIRED
	type CDATA #REQUIRED>
]>
<qalab version="1.1">
    <summary>
        <summaryresult date="2005-05-23 21:38:28" statvalue="5"
            filecount="2" type="checkstyle"/>
        <summaryresult date="2005-05-23 21:43:28" statvalue="3"
            filecount="2" type="pmd"/>
    </summary>
    <file id="net_objectlab_qalab_ant_AntTaskLogger.java"
             path="net/objectlab/qalab/ant/AntTaskLogger.java">
        <result date="2005-05-23 21:38:28" statvalue="2" type="checkstyle"/>
        <result date="2005-05-23 21:43:28" statvalue="1" type="pmd"/>
    </file>
    <file id="net_objectlab_qalab_ant_BuildStatChartTask.java" 
             path="net/objectlab/qalab/ant/BuildStatChartTask.java">
        <result date="2005-05-23 21:38:28" statvalue="3" type="checkstyle"/>
    </file>
    <file id="net_objectlab_qalab_ant_BuildStatMergeTask.java"
             path="net/objectlab/qalab/ant/BuildStatMergeTask.java">
        <result date="2005-05-23 21:43:28" statvalue="2" type="pmd"/>
    </file>
</qalab>

Modules

Related Projects

Links

Search QALab

Project Documentation

Legend

QALab Data Consolidation

Checkstyle

Accepted Checkstyle Format

PMD

Accepted PMD Format

PMD-CPD (Copy Paste Detector)

Accepted PMD CPD Format

FindBugs

Accepted FindBugs Format

Simian

Accepted Simian Format

Cobertura

Example with Checkstyle

Example with PMD