Industrial Software Maintainability Evaluation: Standards, Metrics, and Methods

Industrial software maintainability evaluation is a systematic assessment of how easily software systems used in manufacturing, process control, and industrial automation can be analyzed, modified, tested, and extended throughout their service life. Guided by international quality models such as ISO/IEC 25010 and its measurement framework ISO/IEC 25023, this evaluation helps organizations control lifecycle costs, reduce technical debt, and safeguard the long-term reliability of mission-critical production systems.

Technical Overview

Maintainability is one of the nine product quality characteristics defined in the ISO/IEC 25010 quality model, which forms part of the SQuaRE (Systems and software Quality Requirements and Evaluation) series. In the industrial context, maintainability directly determines how quickly a software system deployed in a plant can be corrected, adapted to new production requirements, or enhanced with new functionality without introducing uncontrolled risks. Because industrial software often remains in service for ten to twenty years or longer, poor maintainability translates into escalating modification costs, extended downtime, and growing operational risk.

According to ISO/IEC 25010, maintainability is decomposed into five measurable sub-characteristics:

• Modularity: the degree to which a system is composed of discrete components whose changes can be implemented with minimal impact on other components.
• Reusability: the degree to which an asset can be used in more than one system or for building other assets.
• Analysability: the degree of effectiveness and efficiency with which it is possible to assess the impact of a change, diagnose deficiencies, or identify parts to be modified.
• Modifiability: the degree to which a product can be effectively and efficiently modified without introducing defects or degrading existing quality.
• Testability: the degree of effectiveness and efficiency with which test criteria can be established and tests performed to demonstrate correctness.

The evaluation process itself is governed by ISO/IEC 25040, which defines the steps of establishing evaluation requirements, specifying metrics, and applying them to produce quality verdicts. Related standards include ISO/IEC 14764 for software maintenance processes, ISO/IEC 12207 for software life cycle processes, and GB/T 25000.10, the Chinese national adoption of the ISO/IEC 25010 quality model. For programmable electronic systems in safety-related applications, maintainability considerations also interface with IEC 61508 functional safety requirements.

Testing Samples

A maintainability evaluation requires a coherent set of artifacts that represent the software product and its supporting documentation. Typical sample types submitted to accredited third-party laboratories or evaluated by internal quality teams include:

Source Code Packages

Complete, buildable source code in languages commonly used in industrial environments, including C, C++, C#, Java, and Python, as well as PLC programs written in IEC 61131-3 languages such as structured text, ladder diagram, and function block diagram. The code base must correspond to a defined release or version identifier.

Architecture and Design Documentation

Software architecture descriptions, module decomposition diagrams, interface specifications, and detailed design documents that allow evaluators to verify structural modularity and dependency management against the implemented system.

Requirements and Traceability Artifacts

Requirements specifications, traceability matrices, and change request records that demonstrate analysability by linking requirements to design elements, code modules, and test cases.

Test Documentation

Test plans, test cases, test procedures, and test execution reports, including unit test and regression test evidence used to assess the testability sub-characteristic.

Configuration and Build Records

Version control history, build scripts, dependency manifests, and release notes that reveal how changes are managed over time and whether reproducible builds are supported.

Maintenance Documentation

User manuals, operator guides, troubleshooting procedures, and maintenance handbooks that document how the system is supported in the field.

Testing Items

The evaluation quantifies maintainability through a structured set of measurement items aligned with ISO/IEC 25023, which defines quality measures for the SQuaRE model. Core items typically include:

• Cyclomatic complexity of functions and modules, measuring the number of linearly independent paths through the code.
• Code duplication rate, expressed as the percentage of duplicated lines or blocks across the code base.
• Comment density and documentation coverage, indicating the adequacy of inline and API-level explanatory material.
• Module coupling and cohesion metrics, quantifying inter-module dependencies and the single-responsibility quality of components.
• Module size distribution, identifying oversized units or files that hinder modification.
• Coding standard compliance rate against documented conventions such as MISRA C for embedded industrial code.
• Static analysis defect density, counting code smells, potential defects, and maintainability-relevant warnings per thousand lines of code.
• Unit and branch test coverage percentages for the testability characteristic.
• Requirements-to-code traceability completeness and accuracy.
• Composite Maintainability Index derived from complexity, volume, and comment metrics.
• Change impact scope, measured by the number of modules affected by representative modification scenarios.
• Documentation completeness and currency relative to the delivered software version.

Testing Methods

Evaluation combines automated measurement with expert-driven qualitative assessment to produce defensible results for each sub-characteristic.

Static Code Analysis

Automated static analysis tools parse the entire code base without execution to extract structural metrics, detect duplicated code, flag code smells, and check conformity to coding conventions. Results are normalized per thousand lines of code so that projects of different sizes can be compared against defined thresholds.

Metric-Based Quantitative Measurement

Raw measurements are mapped to the quality measures defined in ISO/IEC 25023 and aggregated into scores for modularity, reusability, analysability, modifiability, and testability. Threshold-based rating scales convert continuous metric values into categorical quality levels for reporting.

Manual Code Inspection

Experienced reviewers perform structured inspections on statistically sampled or risk-prioritized modules, evaluating naming conventions, abstraction quality, error handling robustness, and adherence to architectural rules that automated tools cannot fully judge.

Architecture and Dependency Analysis

Dependency graphs and component models are examined to detect cyclic dependencies, layering violations, and hidden coupling, all of which degrade modularity and increase the cost of future changes.

Modifiability Scenario Testing

Representative change requests are traced through the system to estimate the scope of impacted modules, the effort required for implementation, and the regression test burden, directly evidencing the modifiability characteristic.

Testability and Documentation Review

Test suites are executed to measure coverage and verify that test cases are automated, repeatable, and traceable to requirements. Documentation is reviewed for completeness, accuracy, and alignment with the current software release.

Applications

Maintainability evaluation delivers value across the full spectrum of industrial software categories:

• Manufacturing execution systems (MES), where evolving production schedules and product variants demand frequent functional adaptation.
• SCADA and HMI software in process industries, where long service lives require sustained change management over decades.
• PLC and embedded control software, including IEC 61131-3 programs governing machinery and production lines.
• Safety instrumented system software, where modification control and thorough regression testing are essential for preserving conformance with IEC 61508.
• CAD, CAM, and CAE engineering software platforms that are extended with plugins and customizations over many years.
• Industrial IoT and edge computing platforms integrating heterogeneous devices and protocols subject to continuous evolution.
• Supplier deliverable acceptance, providing objective evidence during procurement that contracted software meets defined maintainability criteria.
• Legacy system modernization, establishing a quantitative baseline to prioritize refactoring and migration investments.

FAQ

What is industrial software maintainability evaluation? It is a structured assessment, typically based on the ISO/IEC 25010 quality model and ISO/IEC 25023 measurement framework, that quantifies how easily an industrial software system can be analyzed, modified, tested, and reused, producing evidence-based ratings for each maintainability sub-characteristic.

Which standards govern the evaluation? The principal standards are ISO/IEC 25010 for the quality model, ISO/IEC 25023 for quality measures, ISO/IEC 25040 for the evaluation process, ISO/IEC 14764 for maintenance activities, and GB/T 25000.10 as the corresponding Chinese national standard. Coding conventions such as MISRA C may be applied as supplementary criteria.

How long does an evaluation take? For a typical mid-sized industrial application of one hundred thousand to five hundred thousand lines of code, automated analysis, sampling review, metric aggregation, and report preparation generally require two to four weeks, depending on documentation completeness and evaluation scope.

What deliverables are provided? Standard outputs include a metric inventory with measured values and thresholds, sub-characteristic scores with rating justification, a prioritized findings list of maintainability defects and risks, and an overall conformity conclusion suitable for supplier acceptance or internal improvement planning.

How often should maintainability be re-evaluated? Best practice calls for evaluation at major release milestones, before and after significant refactoring initiatives, and at contractually defined intervals for long-running systems, so that maintainability trends and technical debt accumulation can be tracked over time.