Samsung are huge… and experimenting across such a large organisation can come with its challenges. When these challenges arise, the thing that’ll fix them is unlikely to be a new testing framework or a smarter hypothesis template. It’s senior stakeholders deciding that experimentation matters. That one shift, leadership visibly backing the programme, can remove blockers that teams quietly absorb (for months or even years in some cases).
That’s the difference between experimentation as a tool and experimentation as a culture. The tool is what you run tests with. The culture is what happens when a test produces an uncomfortable result OR when a senior stakeholder wants to override the data OR when the programme has had three losses in a row and someone in the room asks whether it’s “working”.
Most teams have the tool. Very few have developed the culture. And the gap between them is where many programmes stall.
The signals that tell you which one you have
You don’t need a maturity audit to know where you sit. The signals are usually visible in the day-to-day.
Experiments that get overridden by opinion are the clearest one. A test completes, the data points one direction, and someone with seniority points the other. Not because they’ve questioned the methodology, but simply because the result doesn’t match what they expected or wanted. Once that happens, the team learns something… the data is negotiable. And once that’s the lesson, the programme starts optimising for approval rather than truth.
Results that get reported but not acted on are the quieter version of the same problem. The team shares a results deck, everyone nods, nothing changes. The insights don’t connect to the roadmap, the findings don’t reach the people who control the budget, the programme continues running tests while the business continues making decisions without them. This is where a lot of CRO programmes live. They’re generating knowledge with no reliable path into the decisions that knowledge should be informing.
Testing only when it’s convenient is the third one. When there’s traffic to spare, when the dev team has capacity, when there’s no campaign running and the site is in a quiet period. Experimentation as a culture means testing is how decisions get made, not a parallel track that runs alongside them when the conditions are right.
What culture actually requires that process cannot create
You can document a hypothesis framework. You can build a test prioritisation scoring system. You can run weekly readouts and set up a Slack channel, publishing a results newsletter. None of that creates culture. It creates infrastructure. Infrastructure is necessary but it’s not sufficient.
The thing that process cannot manufacture is psychological safety around losing experiments. A test that loses isn’t a failure. It’s the answer to a question you were previously guessing at. While working with The Wine Society, quantifying what a losing variant would have cost if it had shipped without being tested was more persuasive to leadership than any of the wins the programme had generated. It reframed the programme as risk management. That framing landed because leadership could see the value of the question being answered, regardless of which direction the answer pointed.
That only works in an environment where the losing result is treated as useful data rather than a huge red flag or a signal that something’s gone wrong. Building that environment is a leadership job, not a practitioner job.
Leadership has to celebrate the question, not just the answer.
They have to be seen treating an inconvenient result with the same respect as a convenient one. When they do that consistently, the team learns they can bring honest results without softening them. When they don’t, the team learns to manage the narrative instead, which honestly doesn’t help anyone.
This is also why the ambition to embed experimentation across the business, across product, marketing, engineering, content, rather than keeping it within one specialist team, matters so much. CRO as a discipline has sometimes been its own worst enemy here. When practitioners position themselves as the gatekeepers of experimentation knowledge, they make the programme feel like a service function rather than a capability the business owns.
The goal isn’t for everyone to become a CRO practitioner. The goal is for enough people across enough teams to understand how to ask a credible question and test it honestly that experimentation starts to feel like how the business thinks, not a separate workstream that reports into it.
What quietly breaks it
Culture is slow to build and fast to damage. Two things reliably break it.
The first is a single high-profile result being overridden. It doesn’t have to happen often. It just has to happen once, visibly, and a very loud message is received by the team. People stop bringing the results that might generate that response. The programme starts filtering toward tests it thinks will be accepted, which means it stops testing the things that are actually uncertain.
The second is a programme that only reports wins. This one is subtler because it looks like good communication. A results newsletter that celebrates the uplifts, a quarterly review that leads with the revenue attributed to testing, a case study deck that only includes the experiments that worked. The problem isn’t the reporting. The problem is what it signals about what’s valued. If the only results that get visibility are the ones that went the right way, the team will orient toward producing those results, which means taking fewer risks, asking safer questions, running tests where the answer is already fairly obvious. The programme becomes incrementally less valuable while looking increasingly successful on paper. It’s also why I’m a bit skeptical when Win Rate is the only KPI that an experimentation team is measured on.
What actually earns lasting buy-in
The engagement tactics that get proposed in experimentation culture conversations often sound reasonable in theory.
- A company-wide idea intake form.
- A results newsletter.
- Gamification
- Leaderboards
- Experiment-of-the-month awards
Some of these are fine. None of them are what actually builds buy-in from the business.
What builds it is governance that removes blockers. At Samsung, that meant getting to a place where the experimentation programme had a stakeholder who could cut through the friction with brand and dev teams – the teams who had the ability to stop experiments in their track. It’s extremely hard for someone in the experimentation team to fight their through these blocks with persuasion alone. It’s a politics problem, and treating it as a capability problem is why a lot of programmes spend years stuck at the same ceiling.
What builds buy-in is a shared vision with the people who control decisions. They need to feel that it helps them to hit their goals or is generally in the best interest of the business. And if they just don’t really know what CRO is, then that needs to change. Not a Slack message dropped in the CRO Slack channel. Not a PDF attached to an email. A conversation, in the room where the roadmap gets decided, where someone with influence over that roadmap understands what the data is saying and why it matters for the next call they have to make.
And what builds it, over time, is the programme being honest when it doesn’t know something. Saying “we ran this test and the result was inconclusive and here’s what that tells us about our next question” builds more trust than a string of positive results that start to look too clean. People who make decisions for a living know that answers aren’t always clear. A programme that reflects that reality feels credible. One that doesn’t starts to feel managed.
Where to start if you’re not sure where you sit
The honest answer is that most teams don’t know which side of the line they’re on until something goes wrong. A result gets overridden and they realise there was no governance to protect it. A programme reports record wins and they realise the business still doesn’t consult the data before making major decisions. The signals were there earlier. They just didn’t have a frame for reading them.
If you want to benchmark where your programme currently sits, specifically on the culture dimension rather than just the technical one, the Experimentation Maturity Quiz is worth five minutes of your time. It’s built to surface the gaps that don’t show up on a test velocity dashboard, and the culture questions in it tend to be the ones that produce the most useful answers.
