The World Quality Report (WQR) 2024-25 looks at the current state and future trends of Quality Engineering—albeit from a particular perspective. It is based on a survey of 1,775 executives across multiple sectors and regions. While it is a potentially interesting source of information, I’m always left asking the question: Is it useful?
Those who have followed Testing Times for a while will know I don’t always agree with everything stated in this annual report. Last month, for example, I drew attention to its AI recommendation that could derail your QA strategy.
Dare I suggest that the WQR has a narrow, potentially even elitist perspective? It focuses on global corporations and does not reflect what is happening within most companies.
They don’t speak to regular companies with limited IT budgets, small test teams, and incredibly tight timelines. Also, only 18% of those interviewed are in a QE/QA role! I find this surprising in a Quality report” in the title.
It’s not that I believe the WQR authors are lying or manipulating answers; I have complete faith in their integrity. I am sure the authors accurately reflect the respondents they talk to; it’s just that those respondents seem to exist in a completely different realm from the majority of companies. Also, I suspect that the results are skewed by the questions they ask.
In many ways, the WQR is a bit like the classic BBC show Tomorrow’s World. Sure, the tech breakthroughs and grandiose predictions made for passable TV, but very few made a difference in the real world.
The WQR, like Tomorrow’s World, is mostly concerned with edge cases. They talk to the top 1% of companies with time and money to spend on elaborate programs and grandiose ideas.
Because of that, I have reviewed the WQR with regular companies in mind, and in this edition of Testing Times, I have picked three specific points to critique on this basis.
1. The Evolving Landscape of Quality Engineering Skills
The WQR 2024-25 highlights a shift in the skills considered critical for Quality Engineering associates.
Surprisingly, only 70% of respondents identified “Quality Engineering skills” as the most crucial. Surely, that should be 100%. Aren’t all of the other skills mentioned, by definition, just constituent parts of that broader skillset?
Also, a keen eye will notice that ‘AI/ML and Gen AI Skills’ appears twice in the graphic below:
Those issues, while fun to point out, are not my major bugbear with this section.
To quote the WQR, “Quality Engineering (70%), Artificial Intelligence (AI)/Machine Learning (ML), and Gen AI skills (66%) have surged to prominence, reflecting a clear shift in priorities.”
While this emphasis on core QE competencies is encouraging, there’s a glaring omission in the report: soft skills.
The Missing Piece: Soft Skills in Quality Engineering
The absence of soft skills in the WQR’s discussion of critical QE skills is concerning. As quality engineers, we often find ourselves at the intersection of various teams, requiring strong communication and interpersonal abilities. Consider the following scenarios:
- Delivering potentially unwelcome news to developers about defects
- Efficiently extracting information from busy business users
- Maintaining composure when faced with shortened timelines or resource constraints
- Advocating for the importance of quality during high-pressure go/no-go meetings
These situations demand excellent people skills, adaptability, and the ability to work effectively under pressure.
Omitting soft skills suggests a potential disconnect between the report’s findings and the day-to-day realities of quality engineers and testers.
It’s worth noting that technical skills, while crucial, can often be learned more easily than the human touch and resilience required in QE roles. As the industry evolves, we must not lose sight of the importance of these soft skills in delivering high-quality software and fostering effective collaboration across teams.
2. Rationalising Quality Engineering Automation Tools
On a more positive note, the WQR 2024-25 provides valuable advice on rationalising automation tools in Quality Engineering. This shift is driven by the need for more comprehensive and efficient testing solutions to keep pace with rapidly evolving application landscapes.
62% of respondents stated that complex tooling was one of the top three challenges in integrating test automation into their CI/CD pipelines—a task that is made even more challenging when you must integrate multiple disparate, unintegrated automation tools.
The Challenge of Automation Tool Fragmentation
This is not just a challenge for Agile or DevOps teams running CI/CD pipelines. In my daily life, I’ve seen an increasing number of organisations grapple with a fragmented tooling landscape, with multiple automation tools, each testing a different application.
This approach presents several challenges:
- Increased complexity in managing and keeping multiple tools up to date
- Difficulty in achieving consistent test coverage across different applications
- Higher training and licensing costs
- Reduced efficiency due to context switching between tools
- Challenges in integrating test results and metrics from disparate systems
The Case for Automation Tool Consolidation
To address these issues, the report emphasises the importance of adopting a more strategic approach to test automation tooling.
Tools like UFT One, UFT Developer, UFT Digital Lab, and ValueEdge Functional Test offer extensive technology coverage and feature sets that make them attractive options for consolidation.
Tools like these can handle various applications, from web and mobile to desktop and API testing. They also incorporate AI and machine learning to enhance test creation, execution, and maintenance.
These solutions provide better integration with other development and testing tools, supporting a more cohesive testing ecosystem. They simplify and are likely to save money.
Your automation needs will evolve, and your tools must support that evolution.
It’s no use having a huge library of automated tests that you can’t quickly adapt and add to when you integrate additional applications.
3. Sustainability in Quality Engineering: A Misplaced Focus
My final comment on the WQR is considerably less favourable.
This might be controversial, but I’m just going to say it. The fact that the WQR has a section on sustainability in Quality Engineering is a glaring example of virtue signalling.
They state that “58% of organizations prioritizing their efforts here.” What, so over half of the companies they speak to are prioritising efforts on the environmental impact of software testing?
Well, actually, no, that’s not what they’re saying, although it is how it comes across, especially considering they’ve included this statement in this global testing report.
The figures relate to their overall company priorities—even then, they look high.
Regardless, the fact that they are taking time to discuss/report on the environmental impact of testing activities is absurd.
The Futility of Green Testing
Green testing or optimising test environments for sustainability is, to steal a quote from Sean
Lock, akin to turning up at an earthquake with a dustpan and brush.
The environmental impact of testing is so negligible that any efforts to reduce it are meaningless. This misguided focus wastes valuable time and resources that could be better spent on addressing real environmental challenges within the tech industry.
The Elephant in the Room: Cloud Providers
While the report frets over the minuscule environmental impact of testing, it completely overlooks the massive carbon footprint of cloud computing infrastructure.
Data centres and cloud services’ energy consumption dwarfs any impact testing activities could have.
If organisations are serious about sustainable IT, they should focus on pressuring major cloud providers to accelerate their transition to renewable energy sources, not micromanaging testing.
WQR’s focus on sustainability in Quality Engineering is not just misguided—it’s potentially harmful. By diverting attention and resources to such a trivial aspect of IT’s environmental impact, we risk ignoring the real issues that demand our attention and action.
It’s time for the Quality Engineering community to reject superficial environmentalism and focus on initiatives that can make a genuine difference.
Conclusion
The WQR offers some valuable insights into Quality Engineering trends, but many of its findings do not reflect typical realities. Yes, companies should consolidate their test tools, but they shouldn’t prioritise sustainability.
Technical skills are important, but most companies would benefit from hiring testers with effective communication, adaptability, and interpersonal abilities. Then, you can train them in the technical skills you need—This is typically cheaper and a more consistent way to grow a team.
I have barely scratched the surface of the WQR’s 106 pages, but I hope this has helped you understand the report’s perspective and that you will take its recommendations under advisement.
Of course, these are just my opinions, and I am interested in hearing yours. Am I being too harsh, not harsh enough, or do you agree?
