If you needed any proof about the lingering disconnect between business and information technology departments, the data is in. We often associate the disconnect to a failure to communicate changing business requirements, but business leaders also are off the mark when it comes to assessing the productivity or performance of their software teams. Almost nine in ten (89%) executives in a recent survey believe they can accurately measure the performance of software engineering teams, but in reality, they don’t have a handle on what’s going on. For starters, many still consider lines of code produced to be an accurate measure of software team productivity — a very imperfect means of measurement.
Photo: Joe McKendrick
These are among the takeaways of a recent survey of 2,013 executives, released by CircleCI, which estimates that businesses lose up to $126 million per year because executives don’t understand the essentials of developer productivity .At least 40% of business leaders are measuring the performance of their teams using measures like lines of code and story points, which are units of measurement that estimate the effort required to complete a software project. Neither of which accurately prove the ROI of engineering outcomes, the survey’s authors point out. “Both measures have some niche utility but are ineffective at measuring a software teams’ success and impact on business.”
Eighty percent of executives say the abilities of their software engineers are critical to their businesses these days. More than one-third, 36%, say they could increase revenues significantly (by more than 50%) with efficient software delivery. “In total, the leaders we surveyed estimate a potential uplift of $126 million per business per year could be achieved by strengthening the critical relationship between business and technology,” the authors state. “Yet, without a deep understanding of whether engineering teams are well-poised to deliver results and achieve their full potential, that revenue increase will remain out of grasp for many.”
This lack of insight into software operations has many implications, especially since only one-third of companies in the survey are led by executives with some IT background on their resumes. “Executives of digitally-driven businesses are often hard-pressed to explain how software engineering headcount and tooling costs return for their bottom line,” the researchers point out. “Without this understanding, seemingly straightforward questions like ‘should we hire more developers?’ are hard to answer.”
The survey showed how much work is still required to close these gaps. For example, DevOps and continuous integration/continuous deployment are still nascent. At this time, 30% are planning to prioritize DevOps, and only 15% will put CI/CD into practice for the first time.
The report stresses the importance of aligning on goals and outcomes, shortening feedback loops, and using proper metrics as methods that can bridge these critical gaps between business leaders and their development teams. The report’s authors advocate for service level objectives (SLOs), which are “simple, numerical measures that can bring business context to a particular part of your output, and commonly include things like error rates, uptime, and time to recovery.” Organizations that still base activity metrics on basic output such as lines of code are advised to “think about your software delivery pipeline, set a baseline, and set goals to optimize your ability to ship quickly, get customer feedback, and iterate – offering more chances for
success.”
Outcomes matter, the report’s authors emphasize. Measuring outcomes are critical, “both for engineering leaders to own the story of their impact on the business, and for business leaders to make accurate judgments on where and how to invest in engineering teams. The gold standard is to understand how changes in software impact business performance: did that new feature increase sales? Did a reduction in downtime lower churn rates? Were customers in the new version more likely to expand their usage? Yet tying software team work to business impact may be difficult, and the challenge of defining good outcome metrics is that they can be very individual based on the needs and goals of each organization.”
