The Experiment Builder

For a very long time, product teams organised themselves around the core assumption that shipping is expensive.

And when it was, that made sense. You specialised, created handoffs, planned carefully and added process to protect UX and engineering time. When getting something live took months, the structure was justified.

That assumption doesn't hold the same way anymore.

With AI-assisted development, going from idea to something testable can take days. Sometimes hours. That doesn't mean building is easy, it's still a lot of work but it's no longer the slowest part of product development.

The slowest part now is: What are we actually trying to learn? And what's the smallest thing we can ship to find out.

What got cheaper and what didn't

This is where I think teams are getting tripped up.

Building got cheaper. Learning the truth didn't.

Teams still lose months the same old ways, by debating in meetings, polishing Figma files that never touch a real user, shipping things that looks complete but is built entirely on assumption. The waste hasn't gone away, it's only just wearing different clothes.

Lean Startup introduced build -> measure -> learn. Growth teams have been running experiments for years. None of that is new. What is new is the gap closing between "we think this might work" and "we can actually test it out." That gap used to be measured in sprints. Now, it can be measured in one afternoon.

When that's true, the most valuable thing on the product team isn't the ability to execute at scale, it's the ability to get to evidence fast.

Enter the Experiment Builder

I've been thinking a lot about a particular kind of person, one who can run the whole learning loop without waiting for perfect conditions.

They can frame a hypothesis, shape something usable, build a test, launch, instrument it, and make a call. Not hand off at each stage, but run the whole thing.

I want to be careful here, because this is where the "PM who codes" conversation usually derails. That's not what I mean. The point isn't skills. It's the timing.

Early-stage discovery work is mostly uncertainty. And in uncertainty mode, the co-ordination cost of involving five people can be higher than the cost of one person just building the thing to find out. When you're trying to learn something, speed to evidence is what matters most. Perfect division of labour can come later.

An Experiment Builder compresses the cycle. They reduce the time between "what we think" and "what we know."

A real example

This isn't abstract for me anymore. I felt it click a while ago.

I lead a product team working on how higher education approaches career readiness with AI. We'd just wrapped up discovery phase for revamping our onboarding, stakeholder interviews, Miro boards, Figma prototype. But I didn't have what we actually needed: concrete evidence.

Specifically, we needed to know whether AI could reliably handle a part of the onboarding flow. Not in theory, in a way that stakeholders could actually go through and react to. That's not a "test different button copy" kind of question, it needed to feel real.

Instead of pulling an engineer off more important work, I took what I had into Cursor, stitched together a testable flow from our existing artefacts, and had something shareable in under four hours.

That's when the idea solidified for me. The point wasn't that I'd done four people's jobs. The point was that I hadn't needed four people to get to the thing worth learning from.

This isn't a "replace your product team" argument.

I want to head off the obvious pushback, which is: Isn't this just asking one person to be PM, designer, engineer and growth analyst?

No. It's about when each of those roles add the most.

When you're deep in discovery, when you're still testing whether something is even worth building, the overhead of co-ordination can slow down more than it helps. An Experiment Builder keeps that loop tight.

When something starts working and you need to integrate or scale it properly, specialists become more valuable, not less. You need the designer obsessing over details, the engineer building something that lasts. The two modes are different, and the people best suited to each are different.

Experiment Builders don't replace teams. They protect them from spending months executing on assumptions that a two-day test could have killed. Teams should hire for this role, and I think that's the right call. But the better question is whether you already have someone capable of it who's been slowed down by process, or never given the room to just go find out.

Where it fits and where it doesn't

This kind of person thrives in early-stage startups, growth-stage product teams, internal innovation work, anywhere the goal is to validate new bets before committing serious resources. It's a harder fit in highly regulated environments, or where deep domain expertise is genuinely the constraint.

My conclusion

As building gets cheaper, the teams that win won't be the ones who ship the most. They'll be the ones who guess the least.

That's what an Experiment Builder actually does. They collapse the distance between idea and evidence. Not by working harder, but by keeping the loop short enough that the team is always making decisions based on something real.

That ability, right now, is sitting unused on most teams. And I think that's about to change.