How many of you know what the “blue screen” is? Or Ctrl-alt-delete? We used to test functionality (“Does it work as intended?”), but as software has become more complex and distributed, we’re faced with different software quality challenges. This is getting even more complicated with what’s now called The Internet of Things, or IoT. With IoT, software and hardware work together more than ever. How do you diagnose the problem? Where is it? In my recent keynote in San Diego at the Practical Software Testing Conference, I had the opportunity to present some of the most critical challenges facing us as software engineers.

Software Quality Challenges and Approaches in the Next Decade

I didn’t have all the answers then, and I still don’t, but in summary, the issues and possible approaches include:

  1. Data-accuracy and actionability of the data. Big Data raises its head again. Mostly unstructured and produced in real time, these characteristics make it more difficult to test than standard ETL (Extract Transform and Load) methods. Later, if we can ensure the integrity of the data, we’ll need to think of real use cases where the user gets value. How many of you actually still care about the data from your fitness wearable?
  2. Working within an ecosystem of many pieces that you can’t see leading to integration and platform configuration expansion. Most of us are developing and testing behind a wall. Sometimes the wall is opaque and sometimes it is translucent. Rarely is it transparent. Testing and developing, and thus, interfacing with many other pieces that we don’t understand fully, presents challenges in understanding behavior. The possible misinterpretations and assumptions are innumerable.
  3. Data privacy and security issues across interfaces. Because of the translucency of the interfaces, we cannot be sure of what is going across them. We need to ensure that aggregation happens when it should and access is protected after the information moves from one place to another.
  4. Updating the devices in the field. One of the biggest issues with IoT devices in the field is that they will need updating. If they are not directly connected to a smartphone, there will be some other type of interface or device to make this happen. It’s just another piece in the puzzle.
  5. Algorithm use cases. As software testers, we need to do more than test the basics to make sure it works. We’ll require domain expertise and knowledge of what’s important to the end user in those domains. It’s not about the buttons.
  6. Integration. IoT welds together hardware, firmware and software. Putting those pieces together in a laboratory and testing these different configurations in the field will require us to understand all the pieces.
  7. Getting QA in close alignment with customer service. We’ll need to do this because that’s where your defects and use cases come from. We cannot foresee most use cases and we can only test basic combinations of hardware-software and behavior in advance.
  8. Designing and testing simple hardware interfaces. There aren’t standards for use or design patterns – look at printers versus cars for design patterns, or even wearables. Each works differently.
  9. Performance issues. Networks can rarely keep up and batteries are always an issue even with low power CPUs. Testing uptime and CHO (Continuous Hours of Operation) will be important as many devices will be left out in the field in the real world for long periods of time.
  10. We’ll need new tools and software to test performance and security across an ecosystem of software and hardware.

I’ve really only listed a few of the software quality challenges that come to mind. It’s really just touching the surface. As software professionals, I think the biggest change we need to make is to open our minds to the other pieces of the puzzle that we are connecting to and not connecting to (security) in order to provide an optimal experience for the users where they can access other services and products (interfaces) in a seamless manner yet still have their information protected or exposed (with their explicit permission).