In the April 3 webinar “Top IoT Testing Challenges,” XBOSoft had the opportunity to host Jon Hagar, one of the leading experts in IoT software testing and author of the book “Software Test Attacks to Break Mobile and Embedded Devices.” The webinar covered a variety of subjects, including testing combinations as well as security and privacy issues. Philip Lew, founder and CEO of XBOSoft, said the webinar boiled down to three major points:
- Testing with patterns and models will help us discern and discriminate what to test. With so many combinations of platforms, devices, and operating systems, it will be difficult to determine what to test first given all of the variables for each of the systems. Jon mentioned the use of familiar test patterns to attack this problem, as well as the use of combinatorial techniques to help us narrow down useful test combinations. Not only is it Machine to Machine protocols and communication that will need testing, but it involves humans working with and interacting with the different systems in different combinations. Jon mentioned attack-based exploratory testing, but I’d make sure that any exploratory testing had very specific charters, timelines, and goals to keep it well-defined and productive.
- All companies are software companies. IoT software testing, Jon mentioned how these companies and we software testers have to concern ourselves with how to test these devices for such things as updating and configuration management. The real problem lies in so many “software companies” making up the IoT infrastructure with no set standards for device intercommunication. Many are implementing Bluetooth Low Energy, but implementation can still have many variables. As we know, cars now have a big software component, and autonomous cars will take it a logarithmic level higher. It’s hard to believe that automobile manufacturers will also be software companies in a way. They’ll have the additional problem of safety, which could result in death if software is defective.
- Data is the next frontier in software testing. IoT devices are collecting mounds of data. Everyone wants it, but they’re not sure what to do with it. Those who do know are going to be the frontrunners in integrating data from different sources and deriving meaningful information. So, Extract, Transform and Load testing will be exponentially expanded. We won’t just be testing IoT software releases for whether we are able to get the data from one place to another. Rather, we will be testing algorithms and interfaces galore. And because everyone wants data, security and privacy will also be huge issues. But it will be across multiple devices, not just a single system or device. Sometimes these interfaces can be rather opaque, so IoT software testing won’t be easy.
I thought it was an outstanding webinar that covered a lot of subjects. All of the specialized hardware and software running on it make IoT software testing a challenge. I guess Jon will be busy for a while. Jon lives in a bunker — literally — and takes great pleasure in all of his security measures. I could describe his house in detail, but that’s classified information. :)