Back to Episodes
Published: May 6, 2024

The Tester’s Dilemma: When to Test, When to Trust

Published:May 6, 2024
Pixel Font:On
People in this podcast
SummaryIn this episode, we dive into the nuances of A/B testing, guided by Christian's latest side project. Alex shares his insights on conducting effective A/B testing, striking the perfect b
#123: The Tester’s Dilemma: When to Test, When to Trust
00:00 / 14:32

Full Transcript

I mean, yes, of course, there's obviously like different ways to look at what makes a good CPO, right? Or not only the CPO, we can probably talk about a lot of executives. You can translate it. Yeah. But I mean, let's talk about your perspective. I mean, when it comes to effectiveness, I mean, I've been recently talking to too many CPOs to better understand their challenges. And one question I'd love to ask the CPO, and if you're tuning in today, whether you are a CPO or in any other product leadership role, I think the question that I like asking is what's the difference between an effective and ineffective CPO? So and the answers are quite interesting, because on one hand, they're all the same, but they're also varying on a wider level. Do you think it depends on how effective or ineffective the CPO is? The actual CPO is definitely, totally, right? I'm sure you only talk to the best CPOs in the world. I mean, generally, my goal is not to validate those CPOs, it's to better understand generally what's going on in the world, right? But I think what is interesting, actually, is the difference between European CPOs and US CPOs. I think we can maybe start there. I think the product world is extremely different in the two countries. Totally. I mean, as you know, Product Bakery is based in Berlin, in the heart of Europe. But where all the product innovation happens worldwide. Actually, most Unicurrents or Fortune 500 companies are Berlin, or Berlin-founded. I mean, we have a good number of Unicurrents, but yeah, totally unfortunate. Oh, that was a nice pay-per-hectare mix. We'll get to it. It's late. It's late. It's true. Okay. But I think, interestingly, something that I have observed is – and it's very important to make a statement here – I cannot generalize what I'm saying, but there's like a kind of pattern, I would say, that is observable, that, for example, the CPOs in Europe are still focused on tactical and operational things, while something I've realized is that the US CPOs are much more focused on strategic topics and driving business impact. Question here. Do you think that it has to do entirely with the CPO or general with how companies are run and how companies are being founded? I mean, while I agree, I think it also has a lot to do with the CEO itself, where I think the European culture might be not as flat, potentially, from a collaboration perspective in C-suite. Like there's a lot of companies where the CEO is just the one who gives the vision or who asks the team or tells the team what to do, and that then automatically has this trickle-down effect of, okay, as a CPO, I've been told to build feature X, Y, Z. That's why I fall into the trap of working mostly operationally. But I think even from a cultural perspective, I think there it already starts. Let's just look at the educational systems. So I mean, there's already a big difference, especially in the US. I don't remember how long it was, but I think a couple of years ago, Google was still hiring only MBAs in product positions. So I mean, that's a big difference if you hire a product person, a potential or an aspiring product person, or an experienced one with an MBA and a lot of business background, while we have in Europe and Berlin, and as we know, even ourselves, right? I mean, I don't have an MBA. I studied business administration, but I never made the MBA. So – I didn't study at all. You didn't study at all. See what happened to us. So there it already starts, right, for different cultures. And I think also something that is very big to me is when it comes to the cultural point of view, I have the feeling, and it's also a feedback that I've gotten while I was talking to the CPOs, is that the networking aspect and the network itself in the US is a complete different one than in Europe. So I mean, you know the stories, right? You're sitting at a party, you're telling someone, hey, I'm planning to build this product, and two people say, hey, I know this guy, I know this guy, you should talk to him or to her. And connections are made up, while I have the feeling, especially as a German in Berlin, and I mean, we Germans are special, yeah? So we don't have friends, we don't laugh, we just work. We don't maintain social interaction. We're the German here. Yeah, I'm half German, like you. So however... Yeah, let's not talk about the specifics, but I'm 100% North African. Okay, that's also true. Sorry about that, it's almost offensive for me. I made a DNA test recently. That's on my to-do list, actually. 75% North of it. Like literally, you can spot my own city, like my family didn't move too much. I have 1%, no, I have 4.5% North Africa, 1% Nigeria, and some Asia in there. I have to do this test. 500 bucks, right? No. No? What test did you do? With blood and all that kind of thing? Like, we need to beep it here, to not make some advertisement. There's like... I don't remember the name, so we don't beep it, but it's like 40 bucks. Okay, good. Yeah. You do like a swipe in your mouth. Yeah. Okay, that way. I know only the blood test thing, but... No, yeah. That's cool. Okay, but this was like a short context switch. Yeah. But yeah, so my point is, talking about the different styles and types of CPOs and also the effectiveness. Yeah. I mean, one key aspect to me is that, especially looking at the US and also talking to CPOs, product leads from the Silicon Valley, is that there is such a big drive on strategical work and driving business impact. I mean, this is what you always see, business impact is super crucial, while there's still this big challenge of the tactical operational work, empowering engineering teams, building an organization around that whole kind of strategic topic. Now the one thing that's interesting and that we should look at a little bit closer, I mean, we already opened with cultural differences, right? Like I think culture plays a big role, but you just mentioned one thing, right? Like building the organization, enabling and empowering the organization. I think there is also a big difference when it comes to overall leadership style. I think this also has a big impact, because I feel like if you look at the feedback culture, if you look at communication styles and so on, like America has a very direct approach to feedback, and I think this really changes a lot based on the cultures and also like if teams have like multiple different cultures in there, but the leadership is also extremely important when you work with a team. And if you have good leadership, it will be easier for you to empower the team and to enable the team and to generally build and grow the team to work on these more tactical and operational tasks, so that you have more time to work on the strategic and visionary aspects. I think if you take the leadership part, I think one key ingredient to add is the strategic background, right? If you have a clear strategy in place and you approach a very direct direction giving leadership style, right, I believe also a lot of things will fall in place automatically that you have to struggle with if the strategic direction is not clear, because then you have to go much more into tactical operational things, which is, in my opinion, very easy to describe with the word firefighting. So I mean, there's a lot of firefighting. But it's like you need your team. Your team needs to free your back also to some extent to really work in the direct. Your team needs to give you input also on things, right? You need to have this bottom-up and top-down direction giving and taking everything into consideration. But yeah, the team itself plays a massive role. And that's a direct reflection also of the leadership style, of your knowledge, all these sort of. And one thing that I was also hearing from a couple of CPOs in Europe, actually, which I really liked was that an effective CPO knows the team personally, trusts the team, right, gives guidance, while an ineffective CPO wants to be involved in every decision and micromanage the people. We know these are basic leadership things we're discussing right now, but still, we still see, I mean, you and I have seen a couple of companies these days, and we see it still happening. So it's not like… Conspiracy. And this is real. And I mean, also to be real here, right, I've also been in situations where I felt that I need to step in and I need to micromanage, right? And this comes back down to, okay, when I'm in that position, what can I do to help my team get to a point where I trust them? Trust is something that needs to be built, right? It's not just, oh, you want to be an effective leader, you need to trust your team and give them complete autonomy, because that can go in the completely wrong direction. But yeah, if you're there, you need to find out how to grow them and to also work with them so that they understand what you need to trust them, so that over time, you can grow that, right? So I think, like, generally, every company that hires a new CPO, you cannot hire an effective CPO, and then he comes in and everything runs smooth from the first day. The trust needs to be built, the team needs to be developed, the vision needs to be developed, the processes need to be developed, and in order to do that, you need to first observe and understand and do your research, so you can slowly build out the right structures. And I mean, the question is also, I mean, why are we having this discussion? What are the takeaways, right, for people who are tuning in? I mean, again, let's put the CPO title aside for a minute. I mean, the question is always, and I think it's something that we're regularly talking about is, as a product person, you're solving problems, right? So there are people out there who have problems, and you believe that you have a way to make their lives better. So how do you approach that? So what is the strategy behind it? And I mean, we are all not doing things for free. I mean, there's a business behind it, we want to be profitable. So we need also to combine this problem-solving aspect with the business mindset and see where we will be and how we want to be in the future, and how we want to position ourselves in the future. And I think it's a very, very important step to start with. I mean, this is why we have product discovery, right? You need to find product market fit, and with a good product market fit, you can move on. And with a good product market fit, you're able to derive a good strategy. If you are stuck in too much tactical operational work, it's recommended. And this is the hardest part, in my opinion. So you have a team, you have to do a lot of firefighting, and you're maybe not trusting them. You maybe need to jump in to give guidance and directions, but not because you're not trusting them, actually. But you have a lot of operational work on your plate. And at the same time, there is much more strategic direction needed. So it sounds like a kind of devil's loop, but the only way to get out of it is to go the extra mile. Yes. And like everything in product prioritization, right? Like there might be times where you just deprioritize the operational work, because you need to focus on the team, on the strategy. And even though if you know that that shit will hit the fan, sometimes it's a much better investment than continuing driving to the wrong direction with no strategy. Prioritization is such. And we also have this podcast episode about the one thing, right? The great book that you should, can read if you want to better prioritize and not lose focus. You will find it in the resources of this podcast, as usual. And we also have a couple of conversations with CPLs lined up. So you will see some of them here and there in the next couple of months. See or hear or add here. Make sure to press the follow button, subscribe and see you soon. Bye bye. This was the Product Bakery. All links can be found in the podcast description and make sure to follow and subscribe for weekly episodes on all podcast platforms, as well as YouTube. Bye bye.

Play The Product Game

START GAME