Quality Assessment

TIOBE offers a software code quality assessment service based on the official ISO 25010 standard on software product quality. The main focus of the assessment is measuring the reliability, testability and maintainability of your software system. The deliverables of this assessment are:

  • Report. The assessment report contains about 20 pages and explains the findings of the more than 350 metrics that are applied to your code. It also provides recommendations, benchmarks and an overall qualification.
  • Presentation. During a presentation the report is explained in detailed to all stakeholders including software management and software engineers.
  • Dashboard. All data that has been collected is available in the TiCS dashboard. This is dashboard in which you can drill down from high level findings to individual issues in the source code. This dashboard will be made available.

A possible high level outcome of a code quality assessment is:

Assessment Outcome

We believe that one should measure software code quality by using the source code as the source of truth. The reason for this is that source code is the only tangible asset in a released software product. Once code is shipped, it doesn’t matter any more how nice your architecture documents were or how well-defined your software development processes were.

A positive side-effect of taking source code as starting point is that most work on code quality can be automated. This means that the results can be obtained fast and in a relatively inexpensive way. Moreover, the generated data is reproducible and can be benchmarked.

The following 3 ISO 25010 quality factors are measured in detail:

Reliability. Reliability is about the stability of a system. How does it react in undefined situations? How often does it crash? This quality attribute is measured by identifying occurrences of “null dereferences”, “array out of bounds”, and “division by zero”. These kind of crashes are detected automatically by applying state-of-the-art code checkers. Apart from this, also the code coverage of the performed tests is inspected. Code coverage is measured with appropriate test coverage tooling.

Testability. How much effort does it cost to extend an existing software system? The cost will be higher in case a system is hard to change. Measuring changeability comes down to determining the modularity and re-usability of a software system at all levels. Examples of metrics that are applied here are “cyclomatic complexity”, “fan-in/out” and generic architectural rules, like. layering. In the picture below the high level relations between software modules is displayed, which is input for the changeability analysis.

Maintainability. A well maintainable software system is comprehensible, thus preventing the introduction of new programming errors while making changes. Moreover, it is easier to let others (internal or external to your company) take over. The most effective method to calculate maintainability is to apply all kinds of coding standards to the software. TIOBE has developed a special metric to indicate how compliant a software system is to (a set of) coding standards. this metric is called the confidence factor. Apart from this also a manual review is conducted to see whether the code is understandable for others.

It is important to note that other ISO-related quality factors or company specific requests can be measured as well during the assessment. Examples are the identification of performance bottlenecks or judging the longevity of external libraries.

TIOBE offers code quality assessments for a fixed price. In case of any questions please contact us: sales@tiobe.com.