Rolling out AI in the SDLC: process standard, measurement and pilot

Handing out AI tools alone brings no visible speedup

Handing teams tools such as GitHub Copilot, OpenAI Codex or Anthropic’s Claude Code does not by itself speed up the work in any real way, and the only visible effect is the cost. The speakers hear this in the market and when they exchange experience with peers.

One of the main topics clients raise is AI wired into processes that are neither measured nor standardized. In conversations with the management of several organizations, two situations come up. In the first, the company is still working out which processes get AI first and how to measure success. In the second, developers start using AI on their own and the manager agrees. Paid, more expensive and supposedly more secure versions of chat apps then appear, next to custom interfaces wired to the API. None of it is standardized, and at some point the organization sees neither an effect nor a gain.

Speeding up one stage clogs the process elsewhere

The SDLC (software development life cycle) is the whole path from gathering a business need to working, deployed software.

Reports from companies that focused on writing code alone speak of gains of around 4–5%, sometimes less, depending on whom you ask. In some very small organizations you hear of much higher numbers, because they have no organizational overhead. In a small product, such as a small SaaS, a developer can write a change, test it and deploy it within a day. In the large organizations Protopia works with, that is very often not possible.

Łukasz Kałużny: “if we speed up just one stage, then realistically it will get clogged somewhere” (translated) 03:39

That is why, when you roll out AI in the SDLC, you standardize the whole process.

The gain from AI comes only after a process redesign

In 1987 the Nobel laureate Solow observed that you could see computers everywhere, but nowhere could you see a rise in value from them. Organizations waited about 15 years for a return on their investment in computers. With the electric motor in factories it took about 40 years, because processes had to adapt to the new technology.

AI follows the same path: the gain will show only after the process and the approach are redesigned and the organization matures. The payback cycle will probably be much faster than with computers or the electric motor.

If you want to control this return, you have to measure it, that is, assess the state before and after you improve the process. You plan what to measure and at which points before the rollout starts, whether it runs all at once or in stages.

The rollout starts from the current process and evolves it gradually: it adapts what exists to working with AI tools. The goal is a repeatable, scalable effect for the whole organization. The success of one team may sometimes be enough, but then you have to check whether you can repeat it later.

The 2025 DevOps Research and Assessment (DORA) report describes AI as an amplifier that lifts mature teams and intensifies the dysfunctions of weak ones.

The standard covers the specification, agent work and the feedback loop

Spec Driven Development is a method in which the team gathers requirements in a repeatable way, generates a specification from them, then a plan for the agent’s work on a given feature, and only at the end the code. It is the opposite of vibe coding. It builds on open standards such as GitHub Spec Kit or OpenSpec, or on a lighter version prepared for a specific organization.

The goal is one consistent standard for working with the agent across the organization’s repositories. The team gets a base it can reuse in a new project or in a project that the standard covers.

Skills are sets of commands and prompts that tell the agent how to do something. They can be company-wide, for example “review the requirements against the global policy”, or project-specific. One of the popular skills that Protopia rolls out at clients analyzes a bug report from production to locate the bug faster.

Documentation goes into the code repository, stays up to date and is readable by both the agent and a human.

Harness and guardrails are the whole environment around the agent that gives it a short feedback loop. Just as a developer uses unit tests, the agent gets automatic formatting, linting and testing, blocking checks and quality gates. In some places the gate is test coverage or cognitive complexity (cognitive load), a measure of how hard the code is to understand and how complex it is. You choose the metrics so that the code is easier to remove or maintain in the future, and you wire them in automatically. This way the agent gets feedback as fast as possible and works in the direction you set.

You choose the tools last, because many of these practices carry over between models and tools. Tools currently change every quarter, and some say the trends change even every month. That is why you choose them together with the models to fit your tech stack and your compliance requirements.

The standard covers the whole SDLC. Spec Driven Development handles requirements. The agent’s tools, together with Spec Driven Development, handle code, tests and review. Maintenance relies on integration and unit tests that run locally and in CI/CD.

The rollout starts with a readiness assessment and the choice of a pilot team

The first step is an assessment of the current state and of readiness. It checks whether the process is ready for AI and assumes from the start that it will need adjusting. It shows what your SDLC standard looks like and whether it has bottlenecks already on paper: how CI/CD and the build servers work, what the release cadence is and whether the company has standardized it. It also covers testability hygiene and deployment frequency. If frequent deployments are not needed, it checks whether deployment is at least automated. The assessment draws on the DevOps approach and its feedback loop.

The second step is choosing the pilot: one or two teams, systems, modules or applications. Teams that volunteer are the best choice, because the pilot needs the commitment of the whole team. The pilot team should be active in testing and later carry the solution further in the organization as its evangelist.

Measuring the state before the pilot is a diagnostic with no set targets

The third step is measuring the state before the pilot, without KPIs. KPIs bring the risk of window dressing and gaming the metric. The measurement is a diagnostic: it shows the bottleneck and the place to start, so you speed up wisely.

You do not measure vanity metrics: tokens used, lines of code generated, the number of pull requests or the share of code generated by AI.

Łukasz Kałużny: “This is not productivity, and we will not reduce it all to one number; it will be a multidimensional assessment.” (translated) 17:27

The measurement has 3 levels, or 2.5, depending on how you count. The goal of the whole measurement is faster or more stable delivery.

The pilot runs on a real application and ends with a comparison to the state before

The fourth step is the pilot with the chosen team or teams on a real application under active development and maintenance, not on made-up cases tested on the side. At the start, Protopia supports it with mentoring and consultations for 1–2 production release cycles, and fits the length to the sprint, so the team goes through and tests the whole path end to end. Once everything is configured, the approach can start working after 1–2 sprints.

After the pilot, you compare its data with the state you collected earlier (the baseline) and check whether anything moved forward. You have to be ready for nothing to move. Some research found, for example, that very mature teams saw no significant speedup.

The pilot can also reveal areas to improve that do not depend on AI, such as acceptance processes. For example, the team builds and tests changes faster on its side, and the queue clogs at acceptance because someone cannot keep up with testing. At the start, this is a very welcome scenario: you then consider how to speed up or improve that stage. If the gain turns out small where AI was supposed to help, you may in the end decide against AI in that place. Success then means that the process got measured and improved in another part.

The pilot result decides on scaling and its reach

After the comparison, you have two paths. If you do not accept the improvement, you do not scale: you diagnose the constraints and perhaps go back to the foundations. You check which processes in the organization block the change, where the bottlenecks are, whether the problem lies in the pilot setup, and how review, testing and observability looked in this area. If you accept the improvement, you turn what worked into standards, templates and a definition of done.

A rollout to the whole organization may not be needed. Depending on company size and criticality, part of the organization or a few development teams are sometimes enough. You update the standard and prepare an adoption plan for the projects that are ready and willing to adopt it.

One way to scale is ambassadors: selected people from the teams attend dedicated ambassador workshops, then show their teams day to day how the process should work. Teams that adopt the standard get expert support: ad hoc consultations and mentoring.

The standard sets the general direction and the main guidelines. Every project still needs final polishing, so the team always has some work to do before it gets value from the standard.