NTT Transforms Existing Software Assets into AI-Ready Development Assets
Automatically addressing inconsistencies between documentation and code and gaps in testing to accelerate the transition to AI-Native development[1]

Key Points

Figure 1. Challenges in enhancing existing systems

Figure 2. Overview of the initiative

Background

Generative AI is increasingly improving software development efficiency, but its use has largely focused on greenfield development, in which software is built from scratch. Approximately 60%^[2] of software development, however, involves enhancing, maintaining, or redeveloping existing systems. Productivity gains from generative AI-assisted development are therefore also needed in this area.

In long-running systems, repeated modifications can cause design documentation to diverge from the actual code. As a result, developers must spend substantial time determining what to change and whether a modification will affect other functions. Under these conditions, generative AI also has difficulty accurately understanding system structures and specifications. In addition, when tests for existing functions are insufficient or not automated, verifying the impact of changes and explaining quality based on the results require significant effort. These issues place a major burden on developers responsible for enhancing existing systems.

Key Features of the Technology

NTT has developed two technologies to transform existing software assets, including specifications, design documents, code, and tests, into an AI-Ready^[3] state that AI can understand and use.

The first is traceability analysis technology^[4], which automatically organizes relationships among specifications, design documents, and code, then detects and addresses inconsistencies and omissions across development phases. The second is comprehensive automated test generation technology, which identifies perspectives not covered by existing tests and generates additional tests to fill the gaps.

Together, these technologies enable AI to understand the structure and specifications of existing systems and establish an environment for verifying the impact of changes on existing functions, supporting a future transition to AI-Native development.

■ Traceability Analysis Technology

Based on international standards and guidelines, including ISO guidance on requirements engineering, this technology automatically structures the composition of and relationships among development artifacts. In existing systems, information is distributed across records of development intent, such as meeting logs, as well as specifications, design documents, code, and test cases. For AI to understand this information correctly, each artifact must be organized into meaningful units, such as individual functions, and the connections among them must be made explicit.
Because the format and amount of information vary among documents, simple segmentation cannot properly capture functional units. The technology evaluates how strongly each passage relates to a particular function and dynamically adjusts the segmentation granularity according to the content, preserving meaning while dividing artifacts into functional units.
Even when specifications, design descriptions, and code address the same function, differences in information volume and expression make accurate mapping difficult. The technology reconstructs downstream information based on key points from upstream phases, aligns information volume and granularity, and then determines dependencies. This approach enables highly accurate identification of omissions and inconsistencies across development phases.
The technology converts large volumes of differently formatted artifacts into traceability information that enables AI to understand system structures and design intent.

Figure 3. Overview of the traceability analysis technology

■ Comprehensive Automated Test Generation Technology

Existing systems accumulate large numbers of tests over many years of development, but it is difficult to understand comprehensively which functions and conditions each test covers and whether important perspectives are missing. Complex dependencies among functions also mean that a change in one area can unexpectedly affect existing functionality. Manually retesting all existing functions requires substantial time and cost.
Using mutually consistent specifications, designs, and code, together with ISO quality characteristics, the technology builds a test perspective space that organizes which system functions must be verified and from which perspectives. Mapping accumulated tests to this space visualizes both covered areas and areas where testing is insufficient. This process specifically identifies inadequately verified perspectives, including abnormal conditions, boundary values, combinations of multiple conditions, and behavior during interactions among functions, which tend to receive less coverage than normal conditions. The technology then systematically fills the gaps using testing techniques^[5] such as decision tables, generating automatically executable tests that cover the impact of changes while making use of existing test assets.

Figure 4. Overview of the comprehensive automated test generation technology

Preliminary Experiments

NTT evaluated the two technologies in preliminary experiments using documentation and code prepared for ATRS^[6], a sample application of the Macchinetta Framework.

■ Evaluation of Traceability Analysis Technology

The evaluation examined whether the technology could detect 41 errors intentionally embedded in design documents, including contradictions with specifications or code and missing information. The conventional method, retrieval-augmented generation (RAG)^[7], detected 11 errors, or 27%, while the proposed method detected 33 errors, or 80%. Previous research on software reviews reported an average defect detection rate of 68%^[8]. Although the evaluation targets and conditions differ and do not permit a direct comparison, the 80% detection rate in this experiment exceeded that reported level.

This result is attributed to structuring the relationships among specifications, designs, code, and other artifacts as traceability information and checking their consistency against the expected relationships, rather than only searching for and referring to relevant documents as in conventional RAG. The technology thus detected cross-phase inconsistencies and deviations from design intent that are difficult to identify by reviewing individual documents alone.

■ Evaluation of Comprehensive Automated Test Generation Technology

In an evaluation involving six intentionally embedded bugs^[9], the technology detected four, or 67%. It also identified three latent bugs that had not been intentionally embedded. These latent bugs were found in areas where tests for input validation and error messages did not provide sufficient coverage. By contrast, none of these bugs was detected when generative AI generated tests directly from specifications and other documents.

This result is attributed to systematically organizing test perspectives based on ISO quality characteristics and applying suitable testing techniques to each perspective, rather than leaving test generation entirely to generative AI. The approach generated tests with broad coverage and enabled verification from a wide range of perspectives while reducing gaps in test coverage.

Future Development

NTT will evaluate the technologies on existing systems of various sizes and characteristics, expand the range of supported artifacts and programming languages, enhance the extraction and supplementation of tacit knowledge, and broaden the quality perspectives and testing techniques covered. NTT will also apply the technologies to actual software development and maintenance projects to verify the effectiveness of integrating generative AI across the entire process, from system understanding and impact analysis to modification, test generation, and test execution. Through these efforts, NTT aims to keep existing systems in a state that AI can continuously understand and use, contributing to the adoption of AI-Native approaches throughout software development and maintenance.

Notes

^[1]AI-Native development: An approach that places AI at the center of development and optimizes the development environment so that AI can perform to its full potential.

^[2]Share of enhancement development: Based on Software Development Data Analysis Collection 2022. The approximately 60% figure in the text is based on this source.

^[3]AI-Ready: A state in which existing software assets are organized so that AI can understand and use an existing system.

^[4]Traceability: A state in which relationships among requirements, design, implementation, and testing are clarified, enabling consistency and completeness to be checked and the impact of changes to be tracked.

^[5]Testing techniques: Methods for organizing test perspectives and conditions and systematically deriving the test cases to be created.

^[6]Macchinetta Framework/ATRS: https://github.com/Macchinetta/atrs

^[7]RAG: A technology that searches external databases or internal documents for relevant information before generative AI produces a response, enabling the response to be based on that information.

^[8]"What We Have Learned About Fighting Defects," Proceedings of the 8th IEEE International Symposium on Software Metrics (METRICS 2002).

^[9]The number was set with reference to actual defect occurrence in software development projects, taking into account the size of the target application code, approximately 32 KSLOC.

About NTT

NTT is a leading global technology innovator, providing a broad range of services to both consumers and businesses. As a mobile operator and provider of infrastructure, networks, and services, NTT is dedicated to promoting a sustainable future through cutting-edge innovations. Our portfolio includes business consulting, AI-powered solutions, application services, global networks, cybersecurity, data center and edge computing, all supported by our deep global industry expertise. Generating over $90 billion in revenue and employing 340,000 professionals, we allocate 30% of our annual profits to fundamental research and development. With operations spanning more than 70 countries and regions, our clients include over 75% of Fortune Global 100 companies, alongside thousands of enterprises, government organizations, and millions of consumers.