Software Testing: And the Challenges (and Opportunities) Keep Coming!

Abstract

Disruptive shifts in software application types and software development environments create challenges to software testing that need to be addressed to ensure software quality and reduce the cost of software development time. Over the years, the size and complexity of software have grown as well as the need for fast-changing codebases, fault detection strategies, and test case generation and selection. To meet these challenges, techniques such as regression testing, selection/prioritization, and fault localization have been developed as well as specialized testing techniques for GUIs, object-oriented software, mobile computing, and continuous evolution of software to name a few. This talk presents an overview of these challenges and solutions and references Mary Jean Harrold’s achievements in these areas. The talk then explores current challenges and opportunities that bring problems that cannot be solved by state of art techniques, including applications that are machine learning applications or use machine learning as part of a system where components interact and evolve. Other challenges that need to be explored involve autonomous systems, cloud applications, and data churn. As software becomes more autonomous, its operations and outputs become less predictable at test writing time; hence, the traditional nature of assert (Actual, Expected) test oracles does not work and needs to be addressed.