By Mike Urbanovich, Head of test automation and performance testing labs at a1qa.
Teaser: With IIoT at the core, smart factories are becoming mainstream in the industrial world, but maintaining flawless operation is challenging due to their sophisticated architecture. In this article you will learn how to test them properly to support manufacturing operations in the post-pandemic era.
By introducing advanced technologies, the tech world is witnessing the rise of Industry 4.0. While reimagining the manufacturing processes and bringing digitization in place, smart factories provide interoperability and data transparency, decentralized decision-making, and real-time capability.
IIoT transforms factories into smart ones that are able to monitor the performance of smart equipment, foresee potential failures, and ensure high product quality.
And like in the case with any other software, IIoT-enabled solutions require thorough testing to provide intelligent industrial management, maintenance, and control of the equipment and processes, and avoid any time or money losses.
But how to do it right considering the overwhelming complexity of smart factories and the need to continuously perform operations even during testing?
In the article, let’s discover how a proper QA strategy assists in releasing highly reliable IIoT solutions.
Why test IIoT-enabled solutions: 6 reasons
While setting up smart factories, companies encounter the issues of cybersecurity, interoperability, and handling a constantly growing variety of information flows.
QA is one of the ways to address them and ensure high performance and safety of the manufacturing equipment.
IIoT-based software has at least 6 core aspects that require rigorous testing. Among them are:
- 1. The devices, the way they operate, and how they send signals to other items.
2. The environment as well as the programming language used for coding and providing their smooth interaction.
3. System performance in the case of simultaneous requests from multiple devices.
4. Core system logic, including system administration, configuration, and data processing.
5. The delivery of data to end users.
6. Device interaction security.
To ensure the smooth operation of IIoT-driven products, it’s important to implement the top 4 essential testing types — functional, performance, security, and automated testing. Let’s delve deep into each of them.
4 cores: Building a wise testing process for IIoT-based solutions
Ensuring faultless operation of software solutions within smart factory is quite challenging. However, traditional QA methods and best practices can save the day.
To tackle the complexity of the process, the team may decompose this IT solution into smaller parts and verify their quality individually. Along with performing the 4 core types presented below, other tests are also applicable as the approach depends on several factors, including the customer requirements, the software quality state, and more.
Core 1. Functional testing
Functional testing guarantees that the software operates correctly, works in compliance with defined requirements, and checks the way a smart factory makes decisions. During this process, it’s necessary to focus on verifying 2 pillars of smart factories:
- The main system, which controls all smart devices and interacts with end users.
- Software embedded into smart devices that fulfills diverse functions, like monitoring the environment or taking pictures.
Starting with functional testing, it’s vital to follow the 4-step procedure to test the IT product operation and avoid potential bugs:
- 1. Testing API and the system of messages to check the data transferring process between the devices and the main system, considering an ample variety of IIoT protocols ― from MQTT to M2M.
2. Fulfilling integration testing to guarantee proper interaction between various parts of the core software.
3. Conducting storage and database testing to ensure data consistency, as IIoT-based systems produce a tremendous volume of data and metadata stored in DBs or cloud storages.
4. Verifying the logic of generating graphs and reports for end users and checking how they interact with the UI.
While testing, harnessing real equipment ensures stable communication with other devices and helps the signals reach their destination.
However, this scenario isn’t always feasible.
First, the equipment can be huge. So, the costs of placing it in a test lab and providing every QA engineer with individual testing environments skyrocket. Unforeseen circumstances like COVID-19 limitations can also complicate delivering it to a required location.
Second, it’s hardly possible to apply test automation for specific physical production devices.
So, what’s the way out? I see 2 means of overcoming bumps in the road:
- 1. Using an emulator, so that to run software in a virtual environment and test it for all required combinations of input and output parameters.
2. Applying a device control unit to check installations and software updates. As it’s small, this hardware device simplifies the setup of a test stand that serves to connect various sensors to the control unit.
Result: Functional testing provides proper operation of IIoT-driven software and a smooth manufacturing process.
Core 2. Of course, cybersecurity testing
Robust security is another mission-critical issue regarding the stable functioning of smart factories. Cybercriminals enhance their hacking techniques and invent new types of malware, so even a well-protected system is vulnerable to cyber incidents and sensitive data compromise. The Global Risk Report indicates that cyberattacks have become the 5th top-rated risk in 2020 while forecasting that IIoT cyber incidents will double by 2025.
In the industrial environment, data maintenance plays the first fiddle as companies store information in the cloud or legacy databases.
Within smart factories, 3 types of objects are prone to cyberattacks ― smart devices, network infrastructure, web and mobile apps that are a part of the data processing and output systems.
Thus, to identify vulnerabilities, the team can apply two utilities — Censys and Shodan Search Engine.
Shodan searches for the open ports of the main web protocols such as HTTP/HTTPS as well as FTP, SSH, TELNET, etc. The system checks the settings and availability of these ports and primarily aims at locating devices on the map and gaining access to them.
Censys, being based on the same principles, provides more emphasis on finding and highlighting vulnerabilities.
Result: Cybersecurity testing eliminates sensitive points in the system and secures the code, environment, and communication channel.
Core 3. Performance testing
Imagine the situation where several machines report on the equipment breakdown, sending the signal to the main system to process this data and take appropriate recovery steps.
But what if more than 100 machines simultaneously indicate failures? The system may not cope with processing large amounts of data and eventually crash.
To prevent potential flaws, it’s significant to conduct server- and client-side performance testing.
QA engineers should emulate the traffic between the components of the IIoT system and focus on:
- 1. Testing the database, data storage, and the volume of stored data. Then, check the system resilience reaction to package loss that usually happens as a result of poor network or Internet connection.
2. Identifying the system behavior in terms of responsiveness, stability, and scalability under a certain number of concurrently working users or smart devices.
3. Validating the request-response time between the equipment and the server that requires to fulfill any action because it directly impacts the uptime of IIoT operations.
4. Emulating real-life scenarios to mimic heavy data load on the system and achieve more accurate testing outcomes.
Result: Conducting performance testing helps companies estimate the load that the IIoT software withstands and the aspects that require strengthening for faster operation.
Core 4. Test automation
With automation being the cornerstone of achieving quality at speed, testing industrial IoT becomes easier, faster, and more reliable due to eliminating the human factor.
And first, I’d suggest automating API testing. APIs allow the equipment to connect and ensure a smooth data- and task-sharing process. Therefore, it should be the main aspect for test automation, as IoT uses networks and specific protocols to connect smart devices.
Second, the engineers can automate UI testing. As the interaction between the IIoT and end users is implemented with web or/and mobile interfaces, the QA team needs to provide accurate user workflows using UI.
Result: Test automation brings the companies both business and operational benefits, including accelerated time to market, predictable ROI, mitigated business risks, and reduced QA costs.
The bottom line
Alongside proper planning and development, a system of industrial IoT-based solutions requires comprehensive testing.
By introducing functional, cybersecurity, performance, automated testing, depending on business needs, companies ensure the smooth functioning of their smart factories.
This culminates in optimized asset management, increased operational efficiency, and better visibility into the operations.
About the author: Mike Urbanovich is a Head of test automation and performance testing labs staffed with more than 180 QA engineers at a1qa ― a software testing company. Through 9 years of vast experience in quality assurance, he has performed multiple roles, including a QA software engineer and a QA manager. Currently, Mike is responsible for high-level team coordination, projects management, accounts management, and coaching. A huge technical background in the field and advanced communication skills help him successfully support the range of projects for the Fortune 500 list clients representing diverse industries and coordinate technical and non-technical specialists.
