Home

Recently, for the purpose of establishing and improving processes for improving software quality, many companies have actively introduced process models such as ISO 9000, CMM(Capability Maturity Model), CMMI(CMM Integration), SPICE(Software Process Improvement Capability determination). Domestic A company also develops quality manual and e-INNOVATOR based on these international standards and presents them as company standard process and methodology, and it shows much improvement in development productivity through process improvement. In addition, A company has conducted three part inspection of the product before shipment as part of the "ZERO DEFECT 21" campaign since 1999, improving the defect rate by 247%. In addition, 86% of the respondents said that product inspection activities contributed to the improvement of reliability. However, in the case of a project that fails the internal standard at the time of the first inspection, it will be re-inspected. In this case, the higher the reject rate, the more the additional cost will be incurred[1][2].

Fig. 1 shows the change of the acceptance rate at the first inspection until the introduction of the product inspection. Although the passing rate is improving, it is not remarkably improved. In other words, the quality of the final product delivered to the customer could be increased, but it did not help to strengthen the project's own testing capacity. The cause can be found from various factors. One of them has not been able to improve the actual test process because the existing process model does not provide a detailed guide to the test, which is a key activity for improving the product quality. In other words, it is not enough to provide appropriate processes and methods for carrying out the test itself. Therefore, the necessity of introducing a separate process model to improve the test process has been raised, and A company has been making efforts to improve the product quality by improving the test process by introducing TPI(Test Process Improvement). Section 2 briefly introduces TPI, a test process model introduced by A company, and section 3 describes a test process improvement example based on TPI[3][4].

Fig. 1. 
Change of acceptance rate at the first inspection by year

The TPI model was developed by Kooman and Pol in 1997, and it is designed to make it easier to improve the test process[2][3]. By examining the organization's test process maturity, improvements can be made to the test process by determining areas of strengths and weaknesses and providing appropriate improvements.

The TPI model consists of Key areas, Levels, Test Maturity Matrix, Checkpoints, and Improvement suggestions as shown in Fig. 2. The TPI model was developed by Kooman and Pol in 1997, and it is designed to make it easier to improve the test process[2][3]. By examining the organization's test process maturity, improvements can be made to the test process by determining areas of strengths and weaknesses and providing appropriate improvements. The TPI model consists of Key areas, Levels, Test Maturity Matrix, Checkpoints, and Improvement suggestions as shown in Fig. 2.

Fig. 2. 
Structure of TPI

The TPI model has 20 key areas, most of which relate primarily to improvement of system and acceptance testing. The 20 key areas are divided into five areas: Life cycle, Technique, Infrastructure, Organization and others. Details are shown in Table 1[5][6].

Each key area can have different levels, the lowest being is A and the highest being is D. In order to obtain a particular rating, the key area activity must satisfy all checkpoints of that grade. In addition, TPI has three maturity levels Controlled, Efficient, Optimizing - and 14 subscales.

With the satisfaction level of detail 1 to 5, a control level is reached, indicating that the test process is being controlled, which corresponds to level 3 of the CMM. At the control level, a test step is defined, and the test depth in each step is matched with the defined test strategy. Also, at the control level, test specifications are made, faults are reported, communication is take placed, and testware and test operations are managed. With the satisfaction level of detail 6 to 10, the efficiency level is achieved, and the test process is automated, integrated, and settled in the development organization. Finally, the optimization level is achieved by satisfying the level of detail of 11 to 13[7], which is perceived as an everyday improvement of the organization.

The process and necessary activities that improve the maturity of the test process of an organization or project using TPI are shown in Fig. 3[8].

Fig. 3. 
Change process

3.4 Plan and execute

TPI provides improvement proposals for achieving the next grade of each core area, and S has decided to pursue the following three plans through the improvements provided by the model and its own analysis. It also prioritized improvements that were made to facilitate smooth and rapid improvements.

11. Promise and motivation → Lack of awareness of test → Strengthen test education

12. Reporting → Insufficient defect management → Proliferation of defect management automation tools

1. Test strategy → No test strategy according to project characteristics → Provide guide

In order to plan and implement the implementation of improvements, it is necessary to find an approach that is appropriate to the context of the project and the organization. In case of A company, we have recently rely heavily on the partner companies for development of external SI projects.

As a result, there is a continuing decline in test perception due to collaborations with vendors that lack process awareness and lack of awareness of software engineering. Therefore, we made improvements that strengthen the test training to be a top priority. In order to strengthen the test education, we have opened a new regular curriculum and continue to test the field project.

In addition, in the standard process of A company, the stepwise defect report preparation is set as a mandatory item, so the project itself performs these activities. However, due to the fact that most of the projects are being scheduled, there is little real control over the defects found. In other words, management is not systematically carried out until defects are found, corrected, and finally finalized. Thus, by supporting automated defect management tools as shown in Fig. 4, defect management is strengthened, So that the effect of real time reporting can be obtained.

Fig. 4. 
Change process

Among the several models proposed to improve the test process, we briefly introduced the TPI, and introduced the test process improvement activities and results of the A company. By performing improvement activities based on the process model, we were able to perform progressive, effective and continuous process improvement.

Through this, we found that choosing an appropriate model to improve the quality of the process, rather than one-time improving the process for achieving certification, is crucial to improving the process of improving processes. Although TPI has the advantage that it can be used in organizations that do not apply enterprise-wide process models, further research is needed to improve TPI model and application methods to perform a complementary role and avoid overlapping roles in organizations that already apply models such as CMMI.