[1] Wang Ziyuan, Chen Lin, Wang Peng, et al. Improved metrics for evaluating faultdetection efficiency of test suite [J]. Journal of Southeast University (English Edition), 2014, 30 (3): 285-288. [doi:10.3969/j.issn.1003-7985.2014.03.005]
Copy
Improved metrics for evaluating faultdetection efficiency of test suite(
)
改进的测试用例错误检测效率度量方法
)Journal of Southeast University (English Edition)[ISSN:1003-7985/CN:32-1325/N]
- Volumn:
- 30
- Issue:
- 2014 3
- Page:
- 285-288
- Research Field:
- Computer Science and Engineering
- Publishing date:
- 2014-09-30
Info
- Title:
- Improved metrics for evaluating faultdetection efficiency of test suite
- 改进的测试用例错误检测效率度量方法
- Author(s):
- Wang Ziyuan1; 2; Chen Lin2; Wang Peng3; Zhang Xueling1
-
1 School of Computer, Nanjing University of Posts and Telecommunications, Nanjing 210006, China
2 State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210093, China
3School of Computer Science and Engineering, Southeast University, Nanjing 210096, China
- 王子元1; 2; 陈林2; 汪鹏3; 仉雪玲1
- 1南京邮电大学计算机学院, 南京 210006; 2南京大学软件新技术国家重点实验室, 南京 210093; 3东南大学计算机科学与工程学院, 南京 210096
- PACS:
- TP311
- DOI:
- 10.3969/j.issn.1003-7985.2014.03.005
- Abstract:
- By analyzing the average percent of faults detected(APFD)metric and its variant versions, which are widely utilized as metrics to evaluate the fault detection efficiency of the test suite, this paper points out some limitations of the APFD series metrics. These limitations include APFD series metrics having inaccurate physical explanations and being unable to precisely describe the process of fault detection. To avoid the limitations of existing metrics, this paper proposes two improved metrics for evaluating fault detection efficiency of a test suite, including relative-APFD and relative-APFDC. The proposed metrics refer to both the speed of fault detection and the constraint of the testing source. The case study shows that the two proposed metrics can provide much more precise descriptions of the fault detection process and the fault detection efficiency of the test suite.
- 分析了在测试用例优先级问题中被广泛用于度量测试用例集错误检测效率的APFD度量标准及其变种, 指出APFD系列度量标准存在物理意义模糊、对错误检测过程描述不清晰等缺陷.针对这些缺陷对已有度量标准进行改进, 提出2种新的测试用例集错误检测效率度量方法relative-APFD和relative-APFDC.新的度量方法在评价测试用例集效率时, 综合考虑了错误检测速度和测试资源约束问题.实例分析表明, 新方法可以更为清晰地描述测试用例集错误检测过程, 并更为准确地反映不同测试用例集的错误检测效率.
References:
[1] Rothermel G, Untch R H, Chu C Y, et al. Prioritizing test cases for regression testing [J]. IEEE Transactions on Software Engineering, 2001, 27(10): 929-948.
[2] Qu X, Cohen M B, Woolf K M. Combinatorial interaction regression testing: a study of test case generation and prioritization [C]//Proceedings of IEEE International Conference on Software Maintenance. Paris, France, 2007: 255-264.
[3] Elbaum S, Malishevsky A G, Rothermel G. Incorporating varying test costs and fault severities into test case prioritization [C]//Proceedings of the International Conference on Software Engineering. Toronto, Canada, 2001: 329-338.
[4] Chen X, Gu Q, Zhang X, et al. Building prioritized pairwise interaction test suites with ant colony [C]//Proceedings of the 9th International Conference on Quality Software. Jeju, Korea, 2009: 347-352.
[5] Walcott K R, Soffa M L, Kapfhammer G M, et al. Time-aware test suite prioritization [C]//Proceedings of 23rd International Symposium on Software Testing and Analysis. Portland, Maine, USA, 2006:1-11.
[6] Harrold M J, Gupta R, Soffa M L. A methodology for controlling the size of a test suite [J]. ACM Transactions on Software Engineering and Methodology, 1993, 2(3): 270-285.
[7] Weiβleder S. Towards impact analysis of test goal prioritization on the efficient execution of automatically generated test suites based on state machines [C]//Proceedings of the 11th International Conference On Quality Software. Madrid, Spain, 2011: 150-155.
[8] Qu B, Xu B, Nie C, et al. A new metrics for test case prioritization in parallel scenario [J]. Journal of Southeast University: Natural Science Edition, 2009, 39(6): 1104-1108.(in Chinese)
[9] Zhang X, Qu B. An improved metric for test case prioritization [C]//Proceedings of Web Information Systems and Applications Conference. Chongqing, China, 2011:125-130.
Memo
- Memo:
-
Biography: Wang Ziyuan(1982—), male, graduate, associate professor, wangziyuan@njupt.edu.cn.
Foundation items: The National Natural Science Foundation of China(No.61300054), the Natural Science Foundation of Jiangsu Province(No.BK2011190, BK20130879), the Natural Science Foundation of Higher Education Institutions of Jiangsu Province(No.13KJB520018), the Science Foundation of Nanjing University of Posts & Telecommunications(No.NY212023).
Citation: Wang Ziyuan, Chen Lin, Wang Peng, et al. Improved metrics for evaluating fault detection efficiency of test suite[J].Journal of Southeast University(English Edition), 2014, 30(3):285-288.[doi:10.3969/j.issn.1003-7985.2014.03.005]