Tech Trends & Future of Work
To keep the organization running, decision-makers must ensure that business-critical apps and services are successfully deployed from development to end users. Deployment testing is one way to limit crucial errors that lead to deployment failure.
In this report, Pulse asked over 400 tech decision-makers about deployment testing in their organization. Decision-makers can use this report to benchmark against their peers.
Data collection: Aug 26 - Nov 29, 2021
Respondents: 441 tech decision-makers
Most decision-makers attribute their team’s deployment errors to human error (54%) and a lack of testing time (51%).
Most (63%) decision-makers report that 1-25% of their deployments fail testing the first time.
“Testing is really necessary at every point of deployment, there is no alternative.”
- VP, large professional services company
During testing, most errors are found in functional tests (38%).
However, even after testing, most (57%) occasionally see errors in deployment, though more than a quarter (27%) report that this happens frequently.
“Although we are using ‘best practices’ to test code and deployments, we are not at a 100% ‘pass’ rate yet (I don’t think anyone will be at 100%) so we are continuing to improve the processes and drive deeper automation to drive down the human errors.”
- C-suite, large software company
For desktop testing, most (72%) turn to external tools.
Of those who use external, third-party tools, Selenium (15%) is the most popular (the top 5 most popular tools are listed).
However, decision-makers turn to many different desktop testing tools, shown by the pool of responses in all other tools outside of the top 5 most popular (77%).
When it comes to testing for mobile, there’s a similar story. Most use third-party tools (62%).
Appium (21%) was the most cited tool among decision-makers (the top 5 most popular tools are listed).
Though, again, 77% of respondents cited other tools outside of the top 5 most popular.
“No matter how good the testing tools [are], debugging and testing are necessary and need to be repeated over and over again in different environments and ways to get a bug free app.”
- Director, small software company
On the accuracy of deployment testing tools most feel ambivalent, with 40% feeling neither confident nor unconfident, though more feel positively (43%) than negatively (16%).
Most (38%) report that 1-25% of their deployment tests are automated.
“Automation will reduce the errors related to deployment testing.”
- Director, large manufacturing company
“Security is one of the most important factors in deployment testing.”
- Director, medium-sized software company
Almost two-thirds (62%) of decision-makers describe their app deployment as more challenging during the COVID-19 pandemic due to the remote work situation.
“COVID has negatively influenced our ability to retain competent employees.”
- Director, small education company