Wednesday, September 24, 2008

Automated vs Manual QA (CodeRuler)

Overview:
Simply compare the validity of human QA and automated QA tools on previous CodeRuler project.

Running Ant + CodeRuler:
QA Tool: QA Violation (Comments)

CheckStyle: Line Type Javadoc comment is missing an @author tag. (JavaDoc issue)
CheckStyle: Line contains a tab character. (Format issue)
CheckStyle: Unused @param tag for 'int'. (Logic issue)
CheckStyle: Expected @param tag for ‘x’ or ‘y’ (Logic issue)
CheckStyle: Line is longer than 100 characters. (Format issue)
CheckStyle: Type Javadoc comment is missing an @author tag. (JavaDoc issue)
CheckStyle: 'static' modifier out of order with the JLS suggestions. (Logic issue)
CheckStyle: Missing a Javadoc comment. (JavaDoc issue)
CheckStyle: '}' should be alone on a line. (Format issue)
CheckStyle: First sentence should end with a period. (Format issue)
**CheckStyle errors: 101152 errors across 10 files

PMD: Avoid if (x != y) ..; else ..;
PMD: Avoid unnecessary comparisons in boolean expressions
PMD: Private field 'castle' could be made final; it is only initialized in the declaration or constructor.
PMD: Avoid using if statements without curly braces
PMD: Avoid using implementation types like 'ArrayList'; use the interface instead
PMD: Substitute calls to size() == 0 (or size() != 0) with calls to isEmpty()
PMD: These nested if statements could be combined
PMD: Document empty constructor
PMD: An empty method in an abstract class should be abstract instead
PMD: Return an empty array rather than null.
PMD: Document empty method
PMD: All methods are static. Consider using Singleton instead. Alternatively, you could add a private constructor or make the class abstract to silence this warning.
PMD: Return an empty array rather than null.
**PMD errors: 60 errors across 3 files

FindBugs: 439 lines of code analyzed, in 11 classes, in 2 packages.
**FindBugs errors: 0 errors and 0 warnings!

Conclusion:
Obviously, this would have been a nightmare without some kind of automated tool system for checking styles and coding rules. Upon receiving the numerous style errors, these QA tools give a descriptive audit of exactly when and where each error occurs. Using QA tools helps our code exist in one or many standards depending on the situation. After seeing what a mess things could be when we go against the standard, I'm now leaning more toward coding in a much safer and higher quality standard environment.

No comments: