Google released the results of its Accelerate State of DevOps report on Tuesday, finding that respondents who use hybrid or multicloud were 1.6 times more likely to exceed their organizational performance targets.
Elite performers in the survey deploy 973 times more frequently than low performers, have a 6,570 times faster lead time to deploy, a 3 times lower change failure rate and a 6,570 times faster time-to-recover from incidents when failure does happen.
Google has worked on the report for seven years, querying more than 32,000 professionals worldwide over the last few years.
Dustin Smith, research lead with Google Cloud’s DevOps Research and Assessment (DORA) team, said the study continues to show that excellence in software delivery and operational performance drives organizational performance in technology transformations.
“This year we also investigated the effects of SRE best practices, a secure software supply chain, quality documentation, and multicloud — all while gaining a deeper understanding of how this past year affected team’s culture and burnout,” Smith said.
“Based on key findings from previous Accelerate State of DevOps reports, we again used four metrics to classify teams as elite, high, medium or low performers based on their software delivery: deployment frequency, lead time for changes, mean-time-to-restore, and change fail rate. This year we saw that elite performers continue to accelerate their pace of software delivery, increasing their lead time for changes from less than one day to less than one hour.”
Smith said they asked respondents to rate their ability to meet or exceed their reliability targets, finding that teams with varying degrees of delivery performance see better outcomes when they also prioritize operational performance.
Smith added that this year, 1,200 working professionals from a variety of industries around the globe shared their experiences with the researchers.
More than half of all respondents said they used a public cloud, a 5% bump compared to 2019, and 21% additionally said they deploy multiple public clouds. About 21% said they used data centers or on-premises solutions instead of the cloud and 34% said they used hybrid clouds.
The study found that those using hybrid and multi-cloud were 1.6 times more likely to exceed their organizational performance targets than those who did not and 1.4 times more likely to excel in terms of deployment frequency, lead time for changes, time to recover, change failure rate and reliability.
One in every four respondents said they used multiple cloud providers because of the unique benefits offered by each one, with the second most common reason being availability.
Nearly 75% of respondents use on-demand self-service, a 16% increase from 2019, and 74% used broad network access, a 14% increase from 2019.
How teams implement their cloud services was also a major focus of the report, with the researchers finding that elite performers were 3.5 times more likely to have met all essential NIST cloud characteristics.
“Only 32% of respondents who said they were using cloud infrastructure agreed or strongly agreed that they met all five of the essential characteristics of cloud computing defined by NIST, an increase of 3% from 2019. Overall, usage of NIST’s characteristics of cloud computing have increased by 14-19%, with rapid elasticity showing the largest increase,” the study found.
“73% of respondents used resource pooling, a 15% increase from 2019, 77% of respondents used rapid elasticity, a 18% increase from 2019, 78% of respondents used measured service, a 16% increase from 2019. In analyzing the results, we found evidence that teams who excel at these modern operational practices are 1.4 times more likely to report greater SDO performance, and 1.8 times more likely to report better business outcomes.”
More than half of all respondents said they use SRE practices to some extent. The Google study found that regardless of performance, teams saw benefits from the increased use of SRE practices.
Quality documentation is also important, and the report found that teams with higher quality documentation are 2.4 times more likely to see better software delivery and operational performance.
Teams with good documentation also are 3.8 times more likely to implement security practices, 2.4 times more likely to meet or exceed their reliability targets, 3.5 times more likely to implement Site Reliability Engineering practices and 2.5 times more likely to fully leverage the cloud.
Continuous testing and continuous integration are also both indicators of success, according to the report. Trunk-based development is key as well, with elite performers who meet their reliability targets being 2.3 times more likely to use it.
Maintaining databases is very important, Google researchers found, with elite performers being 3.4 times more likely to exercise database change management compared to their low-performing counterparts.
Observability was cited as another metric that separated elite performers from the rest. Teams who successfully meet their reliability targets are 4.1 times more likely to have solutions that incorporate observability into overall system health.
The study takes time to note that the COVID-19 pandemic forced significant changes on how teams worked. Nearly 90% of respondents worked from home and just 20% said they had ever worked from home before the pandemic started.
“Respondents who worked from home because of the pandemic experienced more burnout than those who stayed in the office (a small portion of our sample). Inclusive teams with a generative culture were half as likely to experience burnout during the COVID-19 pandemic,” Smith said, adding that security was also an important part of the survey.
“Security can no longer be an afterthought—it must be integrated throughout every stage of the software development lifecycle to build a secure software supply chain. Elite performers who met or exceeded their reliability targets were twice as likely to have shifted their security practices left, i.e., implemented security practices earlier on in the software development lifecycle, and deliver reliable software quickly, and safely.”
