hn.today

Engineering Vitals in the Age of AI

octopus.com10 points5 comments
Screenshot of Engineering Vitals in the Age of AI

Octopus Deploy’s engineering vitals are designed to answer whether AI is helping the company deliver software faster without sacrificing reliability or quality. The central argument is that customer value cannot be measured directly or attributed to engineering alone: it emerges over time as teams ship changes, observe results, and adjust. Engineering outputs are therefore useful proxies for progress through this “value discovery loop,” and velocity means completing that loop quickly. Metrics such as generated lines of code or agent usage measure activity, while claims about value per hour of human attention are too difficult to quantify reliably.

The company groups its measures into delivery objectives, quality objectives, and organization-wide trends. Delivery measures include deployment lead time, pull-request delays and churn, time to recovery, build performance, and deployment batch size; quality measures include monthly high-severity defects and developer-experience NPS. Pull-request volume tracks whether the volume of change is growing, but has no target, to avoid distorting the metric. Octopus rejects change-failure rate as its main quality signal: a stable percentage can conceal rising numbers of defects as deployment volume grows. Instead, it counts customer-impacting Sev 0-2 defects and investigates them. Together, rising pull-request volume, steady lead times, and stable severe-defect counts would indicate more throughput without worsening customer outcomes.

Read on octopus.com5 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.