Nov 28, Paris: AI Health Hack #2 with Alan, Qovery and AWS. Applications close Nov 21

From Task-Driven to Impact-Driven: How AI Agents Changed the Way We Build Qovery

We replaced two-week sprints with one KPI per engineer, let agents write the code, and gave every pull request its own environment. How our R&D works now, what changed in six weeks, and where people still make the call.

Romaric Philogene
CEO & Co-founder
NOV 20, 2026 · 11 MIN
From Task-Driven to Impact-Driven: How AI Agents Changed the Way We Build Qovery

Key Points:

  • We stopped counting tasks and started measuring impact. Every product engineer owns one KPI. Work is done when the number moves.
  • Everyone is a product builder. Engineers read the data, choose the bet, let agents build it, test it on a preview environment, ship it and measure the result.
  • Agents write the code. Testing and review are the new bottleneck. That is where we put our engineering effort.
  • The missing piece was data. We tried this model last year and it failed, because engineers could not see customers, revenue and usage on their own.
  • Six weeks after the KPIs went live, p95 build time went from 11.5 to 6 minutes, and median time to value for a new organization from 8.3 to 2.3 hours.

Qovery · Infrastructure control plane
Build with Claude Code, Deploy with Qovery
Learn more

Yesterday I spent an hour with the R&D team of a French scale-up. They wanted to know how we work with AI agents day to day. Their situation is common. AI already writes most of their code, preview apps are started by hand, every change waits for one shared staging environment, and they are rebuilding their CI.

We had the same constraints a few months ago. Qovery is six years old, and until last year we ran R&D like most software companies, with two-week sprints, specs, estimates and velocity. Today we work in a completely different way. This post is the long version of what I told them.

Before and after

BeforeNow
Unit of workA two-week sprintA KPI with one owner
Big projectsMonths in a tunnel before a customer sees themSmall changes, in production within days
Who writes the codeEngineersAgents, reviewed by engineers
TestingA shared staging and manual QAAn environment per change, tested by agents
Done meansMergedThe KPI moved

The tunnel effect was our biggest problem. When you start a company, you ship lots of small things fast. Then projects get longer, and impact gets slower. By the time a customer saw a big feature, months had passed and we had no idea whether it mattered.

From engineers to product builders

We no longer split people into engineers, product managers, designers and infrastructure people. Everyone in R&D is a product builder.

Almost nobody at Qovery writes code by hand anymore. Agents are better at executing. What people do now is review, decide and take responsibility.

I always made a distinction between coders, developers and engineers. The job of an engineer has always been to understand a problem, break it down and find the best solution. It was never really to write code. Sometimes the best solution is to write no code at all. With agents, this is truer than ever, and it means engineers have to move up the value chain, toward the product and the business.

That move was hard at first. Many of our engineers liked writing code, and we got a lot of pushback. Today most of them see that writing code is no longer enough to have an impact, and they like the new job better.

One engineer, one KPI

Every product builder owns one KPI. I define the pool of KPIs with the team leads. Each lead then lets people in their team pick one, with one or two people per KPI at most. If you think your KPI is useless, you challenge it. That is healthy, because people need to own their number. Once we agree, you commit to it: you improve it, you defend it, and your lead unblocks you within the week if you get stuck.

Every KPI falls into one of three buckets:

  1. Churn. Revenue that should have stayed with us and left, often because a part of the product is not good enough. This is the top priority.
  2. Retention. Keeping customers happy enough that they stay.
  3. Expansion. Bringing more value to existing customers so they spend more.

We have two teams, with different KPIs:

  • The Experience Team owns time to value, activation and support load. How fast a new team gets its first app running, and how often people need us to get unstuck.
  • The Platform Team owns failures, speed, uptime and security. Deployment failure rate, build and deploy time, 90-day uptime, and CVEs fixed within 10 days.

The Platform Team can work on Experience KPIs, but the Experience Team cannot take Platform KPIs. We are an infrastructure product. When an agent proposes a change to the infrastructure, someone with deep infrastructure and backend skills has to judge it. On infrastructure we are a bit like a bank. One incident and we can lose the trust of our customers, and that is a business problem long before it is a technical one.

Part of this is about trust, and trust moves. Three years ago, GitHub Copilot was a better autocomplete, and nobody would have let it refactor a whole platform. Today we let an agent work alone for three hours on a large refactor without much worry. We are not there yet for infrastructure, but I expect that line to move too.

The KPI is end to end, whatever your specialty. A backend engineer who owns time to value also fixes the onboarding copy if that is what the number needs. We recently pushed time to value further up the funnel, to include the docs and the website. A year ago, engineers would have said the website is not product. Today they see the experience starts there.

When a feature needs skills nobody on the KPI has, like design, someone with that skill helps. People build these skills over time. We do not tell anyone "you are a product builder now, do everything alone."

Why we failed the first time

We tried this model last year, and it did not work. It only started working about four months ago.

The missing piece was data. To own a business number from start to finish, you need to see the business. Last year, an engineer who wanted to know which customers used a feature, or what they did with it, had to ask a product manager. We had PostHog and other tools, but the data had to be pulled together, cleaned and explained by a few people. Everyone else waited.

So we connected our tools to Claude. We have about 20 integrations today, plus a few MCP servers we built ourselves: CRM, billing, support, product analytics, meeting notes, Slack, docs and code. Each person gets access based on their permissions. Someone in sales can ask the same questions as an engineer, and an engineer can ask which customers a KPI affects and who they should talk to. That is what makes the full cycle possible.

On top of that, we built an internal dashboard where all the KPIs live, with a separate view for sales, customer success and product. We also rebuilt our billing back office so contracts, invoices and offers are managed from one screen, connected to HubSpot and our billing system. People and agents read the same dashboard.

The features that made this work for us were data retention control, access control per user, cost control, and above all the integrations.

What replaced sprints

We dropped sprints and long backlogs. Work is continuous, with a weekly rhythm:

  • Monday. Each lead walks me through every KPI on their team and what is blocked.
  • Friday. The weekly KPI review. Two people per team present the number, what shipped, what they learned and the next step.
  • Monthly. An all-hands where we turn what shipped into customer and revenue impact for the whole company.
  • Quarterly. We keep, change or drop each KPI and set the focus for the next quarter.

At first people hated the Friday review. They said a week was too short. Now they like it, because they can see it works. The review is mostly about sharing knowledge. People explain their assumptions, what surprised them, what did not, and the hypotheses they want to test next. That is the work a product owner used to do.

A KPI is never finished. We keep improving it until the gains become marginal, and then we move on.

Estimation stopped making sense

The team I spoke with asked how we estimate work now. Their story points and time estimates were getting harder to trust every sprint.

We still plan, but the time is so compressed that estimates in person-days no longer mean much. The funny part is that agents still estimate in human days. An agent will tell you a task takes three days and then finish it in twenty minutes. The real question today is whether we have enough tokens and credits, not whether we have enough time.

How we keep KPIs honest

Another question was how we make sure a builder's changes move the KPI in the right direction. Two rules.

First, the direction of each KPI is agreed upfront with everyone. If the KPI is well defined, improving it is the right thing to do.

Second, do not game the KPI. Think of the paperclip maximizer thought experiment: give a system full autonomy to maximize one number, and it will find ways to raise the number that have nothing to do with what you wanted. Agents behave exactly like that if you let them. The KPI is a proxy for impact, never the goal. When someone tells me they improved something by 40%, I want to know how and why. That is the other job of the Friday review. I have already had to step into a few threads to say "guys, do not start gaming the KPI."

We do not review a builder's plan before they start. Hypotheses matter more than plans, and we want people to move. I push the 80/20 rule everywhere. If a cheap shortcut gets most of the result without putting anything at risk, take it.

One example. We could not understand why so few visitors to our console signed up. We had plenty of hypotheses. One of them led an agent to take the open source code of our Console, replace the API calls with fake data, and publish a full demo of the product that people can try without signing up. Nobody would have planned that in a sprint. It came out of someone owning the number.

The product builder loop

With a KPI instead of a ticket, an engineer does the work of an analyst, a product manager and a builder:

  1. Analyze. Read the KPI, usage and support tickets to see where it hurts.
  2. Hypothesize. Write the bet: changing X should move the KPI.
  3. Build. Agents write the code and open a pull request.
  4. Experiment. The pull request gets its own environment, and agents test it live.
  5. Ship. The builder validates the change on the preview and releases it.
  6. Measure. Customers and the KPI say whether the bet paid off.

The product builder loop: analyze, hypothesize, build, experiment, ship, measure. Agents build and experiment, the product builder decides the rest.
The product builder loop: analyze, hypothesize, build, experiment, ship, measure. Agents build and experiment, the product builder decides the rest.

Each product builder runs a small fleet of agents and orchestrates them. Last week I migrated our billing back office, with all its data, from one cluster to another. I did it from my bed, on my phone, with Claude and Qovery. The agent did the migration, tested it and told me when it was done. It took 17 minutes, and I was watching something else in the meantime. Then I checked the result and finished the job.

Agents or not, the person who presses the button is responsible. Only the way we do the work has changed.

Give every pull request its own environment
Qovery clones a full environment for each PR in your own cloud account, lets agents test it with a scoped token, and deletes it at merge.

Six weeks of results

We compared the week of 28 September 2026, when the KPIs went live, with the week of 9 November:

Four KPIs six weeks after they went live: p95 build time down 48%, median time to value down 72%, new organizations deploying within 7 days up 83%, how-to tickets per 100 paying customers down 25%.
Four KPIs six weeks after they went live: p95 build time down 48%, median time to value down 72%, new organizations deploying within 7 days up 83%, how-to tickets per 100 paying customers down 25%.

KPIWeek of 28 SepWeek of 9 Nov
Build time, 95th percentile11.5 min6 min
Median time to value for a new organization8.3 h2.3 h
New organizations deploying within 7 days4.1%7.5%
How-to tickets per 100 paying customers12.89.6

We also shipped 87 features in October 2026 alone. These numbers move week to week, and sometimes they go back up for a reason we do not understand yet. Then someone owns finding out why.

Testing is the new bottleneck

Agents produce a lot of code, and that is great. What we see at every company we talk to is that producing code is no longer the hard part. The hard part is testing and fixing it without a human in every step. If you cannot do that, all you have done is move the pressure onto your engineers, who now review agent output all day and burn out before the agent finishes.

Building that testing loop is our job at Qovery, so we use our own product for it. Every pull request gets its own environment:

  • Created with the PR. Qovery clones the whole environment from a blueprint: services, databases, configuration and secrets.
  • In your own cloud account. AWS, GCP, Azure or Scaleway. The data and the bill stay with you.
  • Deleted at merge. Ten open PRs never become ten forgotten servers.
  • No queue. Ten agents, ten PRs, ten environments side by side. Nobody waits for staging.

A change then goes through four gates before it reaches customers:

The four quality gates: agent checks, owner review, team check and release, all running on a per-PR preview environment.
The four quality gates: agent checks, owner review, team check and release, all running on a per-PR preview environment.

GateWhoWhat happens
Agent checksCoding agentRuns the tests and end-to-end flows on the preview, reads logs and errors, pushes fixes to the same PR
Owner reviewKPI ownerOpens the preview URL, uses it like a customer and checks it serves the KPI before reading the diff
Team checkThe teamThe change runs for our own team before it reaches customers
ReleaseQoveryMerge, deploy to production, changelog entry. Friday's review shows whether the KPI moved

For code review we use AI review tools from the market, plus our own setup that reviews the reviews. For design, agents compare before and after screenshots on the preview against our design system, so nobody has to hard-code visual checks. Our designer rarely checks screens by hand now and spends that time improving the design system skill that everyone in the company uses.

At the end, a person still presses the button. Our compliance frameworks, like SOC 2, ISO 27001 and HDS, require it, and auditors check it. Technically we could go further, and we will as soon as the frameworks allow it.

Infrastructure is harder than code. For code, there are tests and checks that can tell an agent "do not do this." For infrastructure, the question is how to let an agent do some things and make sure it cannot do others. Every agent gets its own scoped API token, limited to the projects and environments it works on. It can create and delete preview environments, but production needs a person's approval. Tokens expire and rotate, and every action lands in the audit log with the agent and the person behind it. I wrote more about this in Agents Write the Code. Our Infrastructure Is Not Ready for Them.

Shipping is useless if nobody knows

This part comes at the end of the chain, and most teams forget it. If nobody knows you shipped something, it is as if you did not ship it. With agents shipping as much as they do, it gets worse.

Qovery is my third company, and we always had a culture of communicating a lot, inside and outside. Agents solved most of the work behind it. A communication agent reads the merged pull requests and the KPI each one targets, and drafts:

  • a note for sales on what shipped and which deals it unblocks,
  • the public changelog, every other Wednesday at 3 PM, with the main four changes and short product videos,
  • documentation updates for each release,
  • product videos and screenshots that follow our brand,
  • LinkedIn posts,
  • and a note to each customer who asked for a feature.

We track read time and read rate on the changelog, and people do read it. The docs also feed our own agents, which we benchmark on how well they use the docs to help users. So better docs make better agents, which make better docs. This work used to take two people full time. Agents now do about 95% of it.

I have one rule here. I validate everything that is generated before anyone reads it. It catches small mistakes and things that are not quite precise, and it keeps us from publishing AI slop that helps nobody. We also have tone-of-voice skills. I talk in a certain way, everyone at Qovery talks in their own way, and the skills follow that. If you get an email from me, there is a good chance my agent wrote it from a voice note. You will not notice.

Marketing works the same way

Our marketing team is autonomous on qovery.com. Apart from the pricing page, they can create, edit and optimize any page on their own. They work from Claude, which edits the site and opens a pull request, with the same software factory as the product.

We went from our own code to Prismic, then to a stack an agency built for us, and then back to a React stack we own. A headless CMS was great when marketing needed a way to publish without touching code. Agents close that gap, so we dropped the extra layer.

Our design system and brand rules are loaded as skills, pushed to everyone through a shared skill registry, and some sync straight from the repository. One skill generates complete motion videos with our design system and real React components. The video on our homepage was generated this way, and a new one takes two or three minutes.

Budget: be stingy on purpose

People always ask about the AI budget. Most of our spend goes to the product itself, through APIs like Bedrock. Those have caps too, if only because people try to steal keys to get around them.

For internal use, everyone has a standard or premium license, and I am stingy with it on purpose. I do not let people go over budget. I want everyone to be responsible for their usage and focus on impact, 80/20 again. When we raised the caps, we did not see better output for much higher usage. If someone hits the limit of the standard plan, they need a real reason to move to premium.

I think 2027 will be the year companies want to see ROI. Last year AI was a new budget line and passed without many questions. This year CFOs are starting to push back. Next year an open tap will not pass anymore.

The hardest part is the people

There are many possibilities, and you have to go step by step. It is also an organizational change, and everyone in the company has to contribute.

In every company I know, people fall into three groups. A few ambassadors who see the opportunities and push without waiting. A passive majority who follow. And a few people who resist with all four feet. Every company has them, and so did we.

What helped us most was a simple internal ritual. Every week, then every two weeks, we hold a one-hour session where three or four people show what they automated. The first sessions were rough. Someone typed three questions into ChatGPT and found it amazing, which felt like 2023 in 2025. But it is a necessary step. Seeing beginners and advanced users side by side raises everyone's level, and people discover how much of their recurring, boring work they can hand off.

At first we asked who wanted to present. Now we pick who presents. At some point this stops being voluntary. It is a transformation of the company, and not doing it is a mistake. You still have to bring people along gradually.

Where to start

You do not need a dedicated platform team for any of this. Here is what I suggested to the team I spoke with, a product team of nine:

  1. An environment per pull request, on your own cloud account, triggered from your CI. Shared staging stops being the queue.
  2. A KPI per engineer. Start with two or three numbers your team already watches, and review them every Friday.
  3. One agent on watch. Start with security patches, with a scoped token and a person approving production.
  4. Release notes on autopilot. An agent turns each release into notes for sales, customers and docs, and a person approves them.

We are still learning, and some of this will change by next quarter. If you are going through the same change and want to compare notes, book a call with me.

Romaric Philogene
About the author
Romaric Philogene

Romaric founded Qovery to make Kubernetes accessible to every engineering team. He writes about platform strategy, developer experience, and the future of cloud infrastructure.

Next step

Give every pull request its own environment

Qovery clones a full environment for each PR in your own cloud account, lets agents test it with a scoped token, and deletes it at merge.