The Experimentation Bottleneck Is Disappearing. What Happens Next?

For years, experimentation teams have faced the same problem: more ideas than they have the resources to test. A CRO manager has a hypothesis. A marketer wants to try a new message. A product team spots an opportunity. But before any of those ideas can become an experiment, someone has to build it.

That often means relying on developers, waiting for design support and competing with other product priorities. In some organisations, practitioners interviewed by Speero reported waiting up to eight weeks for a test to move through the development queue.

The consequence is simple: teams don’t necessarily test the best ideas. They test the ideas they can get built.

Agentic Experimentation is starting to change that. Research conducted by Speero and commissioned by Kameleoon, involving 19 senior experimentation practitioners across six industries, explored what happens when AI removes some of this technical friction. The findings suggest that the impact goes well beyond speed.

It changes who can build experiments, where teams spend their time and how experimentation connects to production.

1. The person with the hypothesis can become the person who builds the test

Traditionally, having an idea and building the experiment were often two different jobs. The person who identified an opportunity might need to explain it to a designer, translate it into requirements, wait for development and then review the final variation. With AI-powered variation building, that distance gets much shorter.

With PBX, for example, a practitioner can describe the change they want to make in natural language. PBX uses the context of the live page, including its DOM, CSS and structure, to generate a working variation that can be previewed directly on the website.

The question shifts from:

“Can we afford to build this?”

to:

“Is this worth testing?”

That matters because it changes which ideas can enter the experimentation pipeline. Smaller ideas, more uncertain hypotheses or ideas coming from people without coding skills no longer face the same technical threshold.

It also changes how ideas are shared internally. Instead of describing an experience or showing a static mock-up, teams can put something tangible in front of stakeholders and let them interact with it.

AI doesn’t just make experimentation faster. It makes it more accessible.

2. More capacity shouldn’t just mean more experiments

Once tests become easier to build, the obvious temptation is to run more of them.

But more experiments ≠ better experimentation.

The value of an experimentation programme comes from the quality of the questions it asks and what the organisation learns from the answers. The time saved on implementation can instead be reinvested where human judgement matters most:

• understanding customers and identifying meaningful problems
• developing and prioritising stronger hypotheses
• defining the right success metrics
• interpreting results and deciding what to do next

Speero’s research reinforces this point. Rigour doesn’t come from the tool used to build a variation. It comes from the research, hypothesis, measurement methodology and analysis surrounding the experiment.

The same shift applies to developers. If practitioners can handle more straightforward variation building themselves, engineering resources can focus on complex experiments, technical infrastructure and changes that genuinely require their expertise.

AI can make execution easier. It can’t decide which questions are worth asking.

3. Removing one bottleneck reveals the next one

Making experiments easier to build doesn’t solve everything. Because once a test wins, another familiar bottleneck appears: getting that winning experience into production.

Today, the workflow often looks like this:

Idea → Build → Experiment → Winner → Rebuild for production

The experimentation team proves that something works, but engineering may then need to recreate it before it can become part of the product. This is where the next evolution of AI-powered experimentation becomes important.

With PBX Ship, for example, the variation created during experimentation can connect with an engineer’s development workflow and provide a starting point for production implementation.

The ambition is to connect the lifecycle more directly:

Idea → Variation → Experiment → Learning → Production

That is a much bigger change than simply generating variations faster. Experimentation starts becoming a connected system for turning ideas into learning and learning into product changes.

The real opportunity isn’t speed. It’s capacity.

Agentic Experimentation is often framed as a speed story. Speed matters, but the more important question is what teams do with the capacity they get back.

When execution becomes less of a constraint, more people can participate in experimentation. More ideas can be explored. Developers can focus on higher-value technical work. Experimentation specialists can spend more time on research, strategy and decision-making.

But none of this automatically creates better experimentation. The organisations that benefit most won’t necessarily be the ones running the most experiments. They’ll be the ones using that additional capacity to ask better questions, explore more possibilities and learn more systematically.

The experimentation bottleneck is disappearing. What matters now is what teams choose to do once it is gone.

Kameleoon is sponsor of Experimentation Heroes 2026. Get your tickets for Experimentation Heroes taking place on 5 November at Hotel Casa in Amsterdam.

Kameleoon

• LinkedIn