Software Testing: Review on Tools, Techniques and Challenges
The software testing process enables the verification and validation of a software program, to ascertain it works as per the users’ expectations. Software testing provides a means to minimize errors, cut maintenance, and reduce the overall software cost. In recent years, various software development, testing tools, and techniques have emerged to enhanced software quality. In this paper, we conducted a review to examine the most utilized tools, techniques, testing challenges addressed by the research community. In total, 70 articles were selected through our study search and selection process based on our defined inclusion and exclusion criteria. Finally, the selected studies were classified based on testing tools, techniques and challenged they addressed. The result of our study shows that mutation testing is the most utilized software testing technique, in terms of programming languages choice, java is the most utilized programing language, and manual testcase generation is testing challenge addressed by most of the reviewed studies. The identified research challenges are also highlighted with direction for future work.
Bentley, J. E., et al. (2005). Software Testing Fundamentals—Concepts, Roles, and Terminology. Proceedings of SAS Conference.
Bhatti, I., et al. (2019). Towards Ad hoc Testing Technique Effectiveness in Software Testing Life Cycle. 2019 2nd International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), IEEE.
Claessen, K. and J. J. A. s. n. Hughes (2011). "QuickCheck: a lightweight tool for random testing of Haskell programs." 46(4): 53-64.
Do, T., et al. (2012). Scalable automated test generation using coverage guidance and random search. 2012 7th International Workshop on Automation of Software Test (AST), IEEE.
Do, T., et al. (2015). "Goal-oriented dynamic test generation." 66: 40-57.
Everett, G. D. and R. McLeod Jr (2007). Software testing: testing across the entire software development life cycle, John Wiley & Sons.
Ferrari, F. C., et al. (2008). Mutation testing for aspect-oriented programs. 2008 1st International Conference on Software Testing, Verification, and Validation, IEEE.
Garg, D. and A. Singhal (2016). A critical review of Artificial Bee Colony optimizing technique in software testing. 2016 International Conference on Innovation and Challenges in Cyber Security (ICICCS-INBUSH), IEEE.
Harman, M., et al. (2015). Achievements, open problems and challenges for search based software testing. 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST), IEEE.
Jaygarl, H., et al. (2010). OCAT: object capture-based automated testing. Proceedings of the 19th international symposium on Software testing and analysis.
Khan, M. A. and M. Sadiq (2011). Analysis of black box software testing techniques: A case study. The 2011 International Conference and Workshop on Current Trends in Information Technology (CTIT 11), IEEE.
Khatibsyarbini, M., et al. (2019). "Test Case Prioritization Using Firefly Algorithm for Software Testing." 7: 132360-132373.
Kim, Y. and M. Kim (2011). SCORE: a scalable concolic testing tool for reliable embedded software. Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering.
Kitchenham, B. and S. Charters (2007). "Guidelines for performing systematic literature reviews in software engineering."
Landhäußer, M. and W. F. Tichy (2012). Automated test-case generation by cloning. 2012 7th International Workshop on Automation of Software Test (AST), IEEE.
Le Goues, C., et al. (2013). "Current challenges in automatic software repair." 21(3): 421-443.
Omar, E. and S. Ghosh (2012). An exploratory study of higher order mutation testing in aspect-oriented programming. 2012 IEEE 23rd International Symposium on Software Reliability Engineering, IEEE.
Pałka, M. H., et al. (2011). Testing an optimising compiler by generating random lambda terms. Proceedings of the 6th International Workshop on Automation of Software Test.
Patel, D. and A. J. I. J. o. A. R. i. C. S. Patel (2017). "Mobile Applications Testing Challenges and related solutions." 8(3).
Quadri, S. and S. U. J. I. J. o. C. A. Farooq (2010). "Software testing–goals, principles, and limitations." 6(9): 7-10.
Rodrigues, D. S., et al. (2018). "Using genetic algorithms in test data generation: a critical systematic mapping." 51(2): 1-23.
Shamshiri, S., et al. (2015). Random or genetic algorithm search for object-oriented test suite generation? Proceedings of the 2015 annual conference on genetic and evolutionary computation.
Tracy, M., et al. (2002). "Guidelines on electronic mail security." 800: 45.
Zhang, S., et al. (2011). Combined static and dynamic automated test generation. Proceedings of the 2011 International Symposium on Software Testing and Analysis.
Zheng, W., et al. (2010). Random unit-test generation with MUT-aware sequence recommendation. Proceedings of the IEEE/ACM international conference on Automated software engineering.