Joshua Kreuger, DPG Media: ‘We do several hundreds experiments a year across multiple teams’

What is your role at DPG Media?

‘I’m program manager CRO at the Circulation department, which is responsible for DPG Media’s subscription sales programs for newspapers and magazines in Belgium and the Netherlands. It’s an exciting and dynamic role in a rapidly growing organization. DPG Media embraces an entrepreneurial mindset and experimentation plays a pivotal role in its goals. On a day to day basis, I divide my time between supporting our online subscription sales teams in scaling up their experimentation efforts whilst also putting my Conversion Rate Optimization experience in to practice in optimizing our paid content subscription sales programs on our websites such as AD.nl, volkskrant.nl and HLN.be. Next to my role at DPG Media, I’m a member of the CRO taskforce at DDMA.’

How CRO Mature is DPG Media?

‘DPG Media consists of many labels and markets, which have different experimentation teams, so maturity levels differ depending on where you look in the organisation. Notable labels are nu.nl and Independer, which have pretty advanced experimentation programs each with distinct optimization strategies. When looking at the experimentation program at our department – responsible for subscription sales for newspapers and magazines – we’re somewhere around a medium maturity – according to our revised DDMA CRO maturity model. We’re all set to grow towards a highly mature program in 2022!’

Which steps did you take in the last years to become more mature?

‘It feels like we have come a long way in the past three years, starting off with one or two simple experiments per month and now we’re doing several hundreds a year across multiple teams. Getting there demanded us to shift our focus from merely doing our CRO activities well and showing impactful results, to more technical and cultural challenges. We were faced with questions like ‘what should our next hire be?’, ‘what tools should we invest in?’, ‘how are we going to get colleagues’ involvement?’. It was at that time we went looking for a suitable maturity framework on which we could monitor whether we were making progress and what goals to set ourselves.

Using the DDMA maturity model, we quickly learned that scaling the experimentation process from an individual to a multi-disciplinary team was just one facet of doing both more and more impactful experiments. Many other aspects turned out to be symbiotically responsible for growth of the program. For instance if we wanted to do more advanced and impactful experiments, we not only needed a more advanced server side experimentation tool and the required funding. We also needed the time and support from colleagues at the development department and the approval of product owners as it would often cross with planned features.

We convinced colleagues mostly by just trying to speak their language and placing the use of experimentation in respect to their goals. Product owners showed a lot more openness when we showed a business case with the additional subscriptions we won through experimentation and the potential for their domains. Our development team was susceptible to the amount of tickets we aimed to reduce validating features step-by-step through experimentation, rather than going all in on a project that might turn out to be a flop.

On the organizational level, we restructured the CRO team to a Center of Excellence or expertise – a team of CRO specialists, UX-researchers and web analysts, that centrally supports the experimentation efforts across for instance e-mail marketing and online advertising. We see it fosters a lot more experimentation velocity and exchange of insights and ideas.’

What does a proper culture of experimentation look like according to you?

‘I think a signal of a good experimenation culture is when experimentation is no longer seen as an optional hassle when trying to develop or innovate, but rather the way to do so. More tangibly speaking, I think it’s when non-CRO colleagues approach the CRO team for advice on a matter they want to test. For me it was when colleagues and managers started showing sincere interest in experimentation results and the perception of it being a chance to learn something new instead of being proven wrong.

Also management’s involvement in experimentation is a good indicator of the test culture. A CRO team should have a mandate in what they want to test. If a CRO team finds itself battling against opinionated decisions of managers rather than based on insights and research, it is often blocking for the rest of the organisation to adopt it. If necessary, I think your director is one of the firsts that should be onboard, as this person’s approval often sets an example for the rest of the organisation and often is the one who decides over additional funding.’

What developments in the CRO field do you expect in the following years?

‘There seems to be a lot of discussion about the scope of CRO, its embedment in the organisation and even what it should be called. I expect CRO teams- lets still call it CRO for now 😉 – to become facilitating rather than executing, like a center of excellence. Yet, I don’t think the profession of experimentation will entirely dissipate into the activities of for instance marketeers or product analysts. In my experience maintaining CRO programs requires ambassadors within the organisation that keep CRO activities and knowledge in check and I think that need will remain.

Additionally, I’m excited about what I see in terms of new customer experience- and product analytics platforms that provide both qualitative and quantitative insights in a very accessible fashion. I expect CRO teams and marketeers to be able to gain much more insights in user behavior in a shorter amount of time. I think we have a lot to look forward to in our industry!’

How mature is your organisation in the field of Conversion Rate Optimization en experimentation? Find it out in 3 minutes and compare yourself with other companies. Start the CRO Maturity Test.

Erik Molkenboer

Voormalig Teamlead content & community | DDMA