Panel Discussion: Running The Product Discovery @Product Conference Munich

Welcome to the Product Bakery Podcast. My name is Christian, I'm Alex, and together we present you the ingredients to build the right product. I think what's like interesting is also to see, okay, how do these different companies compare to each other when it comes to the development of the product and especially the discovery. So maybe, I mean, starting with you because, I mean, you're working for ID now, I think a product that's probably many of you living in Germany have used when like verifying your ID or like it's this app where you have to show your passport and show your face. So how do you approach product discovery in your company? Sure. One fact about me that I, my first language was Fortran, by the way, those who remember. So anyway, on the question of the product discovery, I think in ID now we have taken it to a different level right now. So there are three different aspects of the product discovery. So typically we divide it into three phases. And for those who know ID now, we produce the software for identity verification, KYC, which can differ quite a bit from country to country because each country has a different regulation. You have to make sure that you comply with those regulations. What that also means is that you cannot simply take the product easily from, let's say, Germany to France to the Spain. You have to really understand the dynamics of those countries. When I talk about the product discovery, there's a standard framework that we are using. So as part of the framework, we have multiple steps and those steps would be things like understanding your customer, understanding the market, things like TAM and the SAM, all those data points and all, understanding the competition, super important, and not only competition, direct competition, but also indirect, because we also tend to compete with the cases where people could be going to the post offices and doing the ID verification, right? So it's just not the software. And the way we are doing right now is in three ways. The first one is, I call it as a step change, which is understanding the market or GTM part. Second one is product step change, and the third one is, I call it as a low end. So depending on the phase that you are in, you really want to be careful in where you want to invest. So, for example, if I have a very simple feature of the function that I want to add in the product, we go through something called hypothesis based product discovery function. What that means is you build your hypothesis based on certain data points, and you start proving or disproving your hypothesis. Make it very simple and easy, and if you are going, let's say, higher, where the mistakes can be very expensive, you want to spend quite a bit of time, the things which I said, looking at the market. So we talk to quite a bit, Gardner and the Foresters, we talk to, through GLGs and all, we are talking to the market experts quite a bit in these countries, and also we do this paid engagement to understand what is the landscape. What is also important to understand is, is the market green or is it red? Because the whole method could differ quite a bit, how you want to take your product to those markets. If you are in a green zone, then it's all about building the features of the function and pushing, whereas if it is already competitors are existing, it's a displacement, which means it's all about pricing, it's all about positioning of the product and all. So these three phases, I would say, differ, so you have to be clear, at least this is what we do. We are very clear that what we categorize as discovery level one, discovery level two, and discovery level three, and decide on how much effort we're going to put on those. And so when you're also going through the discovery, obviously you're serving a lot of companies more like B2B, well, there is always the end user at the end using the tool itself. So who do you look at as your main users? Is it actually the company integrating right now, or is it the final customer using it? That's a very good point. So for those who do not know, we are B2C, so it's like a LinkedIn a little bit, right? So people like us, we want to rent a car, we want to open a bank account, but the people who are buying our software would be like Commerce Bank or UBS, right? So effectively, I think it's important to understand the persona. So we have both personas, the end user persona and the customer persona. So in the research phase, I would say the customer persona for us would be the development team, the people who are integrating our APIs and mobile SDKs and all. It's a very different mindset when talking to these people, understanding their requirement. When it comes to the end user, it's almost like working in a consumer company, booking.com or other places where we have embedded quite a bit of analytics in our software to understand the funnel, right? So what's going on when the users are starting, are they struggling in showing their ID card? And we see very interesting things a lot. We see things like, you know, show me your passport, but my passport is maybe in my room and I have to go back and put it in the room. And actually we see some crazy things, I don't want to tell you, we see some people actually sitting topless also sometimes and doing their ID verifications and all, and don't even realize that they have to do that. You have to be also sensitive about culture, right? So if you're from Arab or something, you'll be putting on the scarf and all, right? So you have to make sure that you cater to those requirements also. So you see quite a bit of these funniness, I would say, but you have to cater to it, right? At the end of the day, these are the users, you have to adopt your software to those requirements and the data analytics becomes super important. You need to really know where people are struggling, why they're struggling, and we even see people are actually trying to fool the system. So I'm asking for your passport, I would simply take somebody else's ID or take a printout and try to fool the systems. And also we have seen things like people stealing car and all, so it's quite interesting to look at all those aspects. I mean, working in financial technologies, we do a lot of verifications and we see that also quite regularly. You mean the topless people, right? Everything, topless people, people driving while they verify their ID. It's definitely an interesting space. Indeed. Cool. But I mean, maybe moving on to you, Sophie, how does this compare to how you run product discovery at Compute? Very differently. So maybe for everyone who's not yet such an expert in product discovery, you do not have to get scared that it's so far away, so you can also start small. And maybe because I just got a question before, are you working for a startup? Because you're talking about product discovery. So I know some of our products, we had one of the first multi-posting engine for recruiting in Germany, so it's 20 years old. So we're not a startup, but we're doing product discovery. And if we're working with product discovery, it's not only something for startup, it's really integrated in the flow. It's what you're doing as a product manager. We like to picture at Compute our product management process as a funeral. So really taking, in the form of a cabin board, our ideas from idea to draft to development to finish. And basically it enables us when we select an idea or draft, so prepare a concept to make, to raise attention for everyone in the team. Oh, that's the space where we do product discovery. And that's actually a very nice thing because a lot of the HR techniques, they are based basically on peer pressure and transparency. So it means once an opportunity is coming into this column, we know we should challenge each other. Oh, that's where we do product discovery. But what does it mean, basically, product discovery? It's about aligning and framing the problem at the beginning, taking the time to analyze what are the real risks in this opportunity. And then it guides you to define what do you really want to do in this product discovery. And actually what you have to do is very different based on the risk. Is it mostly a market risk, a usability risk, or technology risk? And I like to understand product discovery in this wide term of addressing risks by thinking in alternatives and confronting brutal facts. It means not only doing it internally, but really confronting with the reality, the reality of the technologies, the reality of the customer's opinion, and the reality of your sales opinions. So for me to be very important. That's really what you're doing in product discovery. And I think it's important for everyone not to get scared, to think it's just, it's a normal thing you do in your product management everyday practices. It's just reflecting and taking the time to stop when you get an idea from someone, a product from someone, to really deeply understand the risks and the hypothesis behind it, and how, and think of different alternatives on how to tackle it. That's very interesting to see, like, kind of two different ways how you approach, but at the end of the day I think we can all agree on that we need to make sure that we validate what our hypotheses are and what we want to build. So looking at you, Gopi, I'm very interested because you are not only CPO, you're also co-founder at Bitcrunch. So just to drop a couple of buzzwords here, we're talking about crypto, blockchain, and Web3, which I'm very passionate about, by the way. So I'm just curious to hear, since you also founded the company, how did you initially set up the product discovery process? We did it through a hackathon. So that's it. So we did pack some ideas together and we went to a hackathon and we came out successful and we said, oh shit, let's do this. So yeah, that's simple. I think, yeah, as you know, we are a very young company. We're just two years into, you know, this month is actually completing two years, officially starting, but we were actually doing bootstrapping for about three years, one year beyond that from the pandemic time, 2020. We started on general data science, doing general analytics. And then when this moment, we just said, let's try something different. What's happening? You know, everyone was at home. They got a lot of money and they started investing in crypto. You guys know, well, you guys do that. So we said, OK, maybe there is some opportunity there because there's a lot of money going in. There's a lot of assets moving. There's a lot of data being created. Why not try something there? And that's how we started. So a lot of gut, right? There's a lot of gut feel, right? Typically product, even today, we do a lot of gut feel based selling point for product discovery, even feature discovery. Then we validate that with customers. So that's how we did the validation through hackathon. We went to the hackathon. We validated that. We got a price there. It's a global event. Second price, not first, but second is good enough. And then it was not, we didn't succeed immediately after that. We had a rough time. We had six months of trying to find an investor for the idea. And then we also realized that maybe the way we framed the idea was not good. So it also took us some time to frame the idea. And most of us were working in a full time job. So we only did this as a part time. So, you know, it's not like we have eight hours per day to work on this 24 hours per day because startups is like 24 by seven life. And then like 2021, mid of 2021, that's when we really started and a lot of people believed on us. And then that's how we groomed the product. I would say we're still grooming it. Every day we discover something because the market, we cater to NFT, analytics and forensics. And this is a market that's only two years into mainstream. Only in 2020, end of 2020, NFTs became a big thing. And so it's also not a matured market. So we can't use any traditional approach to do product discovery because any of the traditional methodology will not work because all those participants, if you want to go and do any survey, you don't get people because this is a small segment of user base. So we have to go through a lot of trial and error. In this case, the trials are not really just, you know, doing mocks and then showing it because if you want to show data, it's meaningful. And so you need to put a lot of effort. And then, you know, as a manager, you guys know that you have to have a thick skin, right? Because failures are always there, right? Every second day you will fail on something and there's someone there waiting to point the finger at you. So you have to be really thick skinned for that. So, yeah, we have to go through this. And yeah, product discovery, as I said, it's a lot of cut for you and then validation after that. And when it comes to the involvement of the process, you said at the beginning there was the hackathon and you managed to hit the nail on the spot. So how has that evolved over time? So when we look at the process, is it like someone has an idea, you start prototyping and then you test it or how is it working, especially in the crypto world? Yeah, a lot of time we do the same, what you said, prototype. Most of it, even the post-hackathon, we did the prototype, we had a customer with the prototype and we were able to validate it. But it's not, it doesn't happen often as in other mature industries because there's not that many customers to go and, you know, get them on board. It's also, you know, the whole industry itself in a curvy place, right? You know, everyone now thinks of NFTs as what monkey picture, right? That's what is popular for. I lost a lot of money with that. Yes. Let's not talk about that. Yes, and that, right? Everyone lost a lot of money. But there's a lot of utility around NFTs and a lot of organizations have realized that we had like a few weeks back, Siemens Innovation CEO was at our office actually talking about how they can use NFTs in Siemens. So it's not something that, you know, it's not just about monkey picture. That's what I want to say. But then it's also a lot of things that we try. Maybe there is customers that we can really, you know, evaluate that and then validate it. But also sometimes we do a lot of gut feel and then figure out if we get some traction. If we don't get traction, let's try something different. I think it's anyone who is in startup, it's probably they know that this is something that you do it. It's like the pivoting is like you do micro pivoting every second week or every month. It's not something crazy. But yeah, we have to do that to stay alive. I want to go back to Sophie here because like I think there's probably also a big difference if you look at, okay, I'm bootstrapping a company. I have this idea. I can like really pivot a lot and often versus, I mean, you mentioned your company has been around for quite some time, right? So how was it for you to join the company and like also in your role to then like set up product discovery or adjust the process? Interestingly, it was about introducing product discovery. It's difficult for anyone of you working in B2B SaaS. B2B SaaS is most of the time coming out of the idea of one founder that is coming out of a specific industry and recognize a specific need. And then this founder is finding the first and the second favorite customer and they are starting developing the product based on the requirement of these first customers. And it makes the product management is not really existing as product management. It's basically requirement engineering of the wishes of the first big customers. And it means at some point this big B2B SaaS has to make a transition from this sales based requirement management to product management. And actually product discovery is a very powerful tool to free up your product team from the sales based product development perspective. And because you can justify and explain why you're doing things, why you're still customer centered, why you're no longer answering each customer wish. And how I introduce it, I think the difficulty is that you only get good at product discovery by doing it. That's the reality. So for my team, I offered some books. I can recommend it. It's called Discovery Discipline from Tristan Chavalli-André Miglio, I think. That's a former CPO of BlaBlaCar. And I really like the book because compared to a lot of book about product management, they are really displaying their experience, how they use it. And it's a European based company. So a lot of examples from product management are based from U.S. companies, as you know if you read a bit. And I found it great to have one book and examples based from a European company. But I like the guidance. I do not say follow the rule, but I like the inspiration and the guidance. What I think is important still is to hire or find an external person, a senior person to guide the first product discovery as a co-pilot so that your team gets this guidance during the first discoveries to feel comfortable, reassured and to learn how to run it. However, I think in terms of adjustments, I believe a strict process, or it's my view, a strict process about product discovery is like scrum to a child. So it's very good to start. But basically, you also integrate it in your daily thinking. So I once came to one of my product managers with an idea. I said, hey, not an idea. I said, at the end of the week, we should do that. And he challenged me and said, Sophie, we didn't do a product discovery. So I liked it that he challenged me. And I said, OK, fair point. I didn't do a kickoff and say, no, we do the product discovery. But let us revise what we did in the last four days. And I told him, OK, we fixed the problem in our last meeting on Monday. And then we built up alternatives. And we checked with our peers. Basically, it's a nice thing about B2B. SAS, a lot of our colleagues working in support are former recruiting agencies employees. So they are actually our target users, which is very nice to get very good feedback on small ideas. And it's something you're then naturally integrating in your way of thinking of products. So, yes, it's good to have a formal process of product discovery. But I like to think that in your everyday work, it's also about doing little product discovery and keeping this process in mind also when you're checking on small ideas, small improvements. Yeah. And I think maybe it also makes sense to talk quickly about two things. I think if we look at the companies that we have, there are companies that are either very product-led or companies that are more sales-led, sales-centric, especially on the B2B side. So, I mean, with you coming in and introducing product discovery, moving a little bit away from the sales-centric approach, I'm just curious how sales took it and how you actually managed them. It's a process. We're still working on it, but I think it's true. So, I mean, any product managers know this ambiguous relationship with sales. So it's love and hate. So I think one of an important piece is to still display that what they are bringing on the table is important. And every two weeks, we are prioritizing together on the production so that they have the space to bring their idea from the market and the space to understand why we prioritize things. So I think this transparency is important. And especially in B2B, a product manager should go from time to time to customers with the sales guys to understand what they are confronting with during sales pitch so that we do not create two worlds, but these worlds have to work together. And I actually like that you said it is a process. So, especially not only for sales team, but also for the top management, for example, founders and CEO, I can imagine it's not easy to introduce a product discovery process, right? By coming in and saying, hey, we cannot just build something that you have in your mind or the person who's shouting loudest. We need to validate this based on ideas and other topics. So, Vikas, going back to you, I was just wondering, what KPIs do you use to measure the impact of product discovery? Because it usually sounds very great if you do it, and I believe we can all agree that it's needed. But how do we measure the impact of a good product discovery process or the work in general? Sure. And I think it's a valid point regarding the sales. So one of the things, just to take on her previous answer, I think it's super tough. If you have a loud sales guy with a high quota, I think it's like fighting every day. I think one thing we did was to get convincing the CEO quite a bit, meaning making sure that people are also educated. So educate what that means and get through the process. I think this also helps a lot. I think the KPIs are a super important one because at the end of the day, it's all about data, right? So we can prove whatever we are doing through the data. I keep telling people, remember this word, your opinion is important as long as you keep it to yourself. So this is what I keep telling people. I think it's awesome. But please keep it to yourself. Let's look at the data. So what I did when I joined the company was also to start looking all the places where the data did not exist. So just say, here's the data, here's the data. And what really helped was to exactly to your point, to collect these KPIs and all. And typically, like in the three stages that I described, if you're talking about launching a product in a new market altogether, you're going to be waiting for a year, even a half before you see these results. And also super hard to convince people also. What really helped was to exactly in this meeting, what she talked about, not only looking at the roadmap and isolation, but also looking at the things that we did, what we had agreed on the KPI, and where do we stand with those features. In some cases, I have to be honest, just to please the ego of the people, I said, OK, fine, your idea is awesome. Let's test it out and see if it works. OK, maybe I'm wrong. So effectively, that also helped us to sometimes fail to prove that there is a better life out there. I mean, typical KPIs would be standard depending on the company. In our case, it would be things like conversion rate and all. That's very obvious how many in the funnel, how many people are entering, how many people are converting. It's also important we are using quite a bit of AI in our technology. What is the automation rate of the AI? Because if the AI doesn't work, it has to be done manually by somebody else. So looking at the data around the automations of the AI or looking at the fraud, because fraud is super important. The examples I gave people, stealing car and other things and all to these discoveries and measure the data on an ongoing basis. So effectively, depending on the industry, KPI could change. And for the larger business level, we were always looking at the revenue. So what we also did was for feature level also, we tried to estimate quite a bit. What is the impact of the feature on the top line or the bottom line? Top line meaning revenue or the bottom line meaning cost savings. And not to consistently, as you said, you have to have a thick skin. Sometimes you have to really beat all the doors to get the data. Whether it's the controlling team, whether it's the sales team, account management team to really get the data, get the data. Be persistent about getting the data if you want to be successful in the whole discovery process. If the data is missing, you have to fight it out with the engineering team. So what we also did was in the user stories and all, I mean, typical user story, if you search and Google doesn't exist, is the success criteria for the story, which by default means what are the data points you're going to add for this feature or the function? And how would you measure the success also? So be really persistent on that. So that's how we went about collecting these KPIs. And be honest, if that doesn't work, raise your hands, I was wrong. It didn't work. Let's move on and do something else. So you said many times that you need to fight. So it looks like mixed martial arts as a skill can help somehow. But what I'm understanding is that you use product KPIs as the main KPIs to validate during the discovery process, whether a feature makes sense or not. That means the value of what you want to build based on KPIs is what you measure against different features. Is that correct? Yeah. So, I mean, what I did was when I joined the company, I clearly defined the KPIs. Like what are the product PIs and the product KPIs like in consumer companies could be your MAU and TAU kind of things. And of course, like I said, was conversion rate, but also the business KPIs. So if I'm launching a feature, let's say we added something called PST2 Open Banking as a way to also do the identification, because I can ask you to log in and I get your name and I know you are who you are. So you're not fooling because you don't have somebody else's password and two-factor authentication. So that was a big thing. Everybody was pushing. We need it. We said, OK, how much value we can get out of it. We did quite a bit of discovery and that's telling the people. They say, hey, why are you asking me to log into a bank account when I just want to rent a car? So we did quite a bit of discovery as part of it. So we did the final analysis that we expect out of 100 people would try to do this process. Only 20 people would, because this data we had already collected through the discovery process. But the success criteria, we say, OK, for us, the success would be 30%. If 30% people use the method, this is success for us. If not, this is a failure for us. So effectively, on one side, pure product API. On the other side, the business API, as I call them, as a revenue of the gross margin. But not only looking at the revenue at the end of the year, but also looking at on a quarterly basis, working with the sales and the finance teams also. So effectively, what I'm getting at is that our notion of the product discovery is not just the technical side, but quite a bit commercial side, meaning commercial product management and looking at everything you do from the commercial perspective. What is the novelty in whatever you're doing for the success of the business? So I think we're already close to the 30 minutes mark. So I think as a final question, it could also be helpful for everyone who's starting out with product discovery and implementing this. Where do you think that most teams actually fail in the process? Yeah, I think, again, it's different perspective because I've been a product manager in SAP before this. So it's a big company. So the perspective of where things fail there is different from where things fail in startup. So most often, sometimes you misread the data. It could be because you didn't look at the correct target group or you didn't do enough sampling of the data. When I say this, it could be empirical data or it could be data feedback or user testing. So it could fail there. That's failing early. But of course, if you don't see it and then if you let it go through the development and you develop, and then you see that it fails at the commercials because you're not going to sell it. So yeah, most often you fail early, but you don't discover it. And typically, you go through the whole journey and then you find it out at the end when the product is not bringing in revenue. And then it sucks to accept it. But then you fail or you realize you failed. But then the failure happened pretty early. You just didn't figure it out. Sure. I think you can fail pretty much everywhere. I mean, this is what I've seen. Average human has, guess how many biases? Anybody knows? 600? Almost 600 biases. And one of the biggest one is called confirmation bias. Confirmation bias, meaning you buy a fridge or a car and then you are only looking for signals to prove your theory that, okay, you are right. And this is, I would say, the biggest failure you see. Typically, which means if you have a hypothesis, you get so attached to it, you start only looking for the positive signals and start ignoring the negative signals. If somebody is screaming behind, some data is screaming, no, this is wrong. But you say, okay, I invested so much, this whole sunk cost theory also quite a bit. This is where I've seen the biggest issue. I think product merit need to have a scientific mindset. So you have to be really accepting the defeat. So, okay, I'm looking for not only positive signals, but more important, I'm looking for the negative signals. This is one area consistently you will see. I think you learn quite a bit through the maturity as you go through. Second thing I have seen, especially in the B2B, where you do not necessarily have access to the customers that often. It's easy to say, talk to customers and all, but then you talk to the sales guy, he will take you to the customer. She will take you to the customer only if he or she feels something comes out of it. Next thing you start, they start talking about the bugs or escalation or the SDK crashing. And then you really talk to the customers who are really positive about the company. These five customers, you always talk to them because the guy or the girl likes to talk. So not necessarily talking to the unhappy ones also. So having access to the right customer, I think it's important, is the toughest part. So people always end up picking these five customers always and then building their hypothesis based on that. And you realize for remaining customer, it doesn't matter anyway. So this is what I have seen. Third area I have seen quite often is the discovery process. Meaning if you're not prepared with your interview guide with the customer, you will fail. Because in the discovery process, you will understand your pain, pain, pain, right? Why, why, why? Oftentimes, if you're not prepared for it, you start immediately jumping into the solution space. So you start on the problem and then you get distracted and start talking about the feature, feature. Because people like it, right? What if we can do like this? So this is also I see people failing quite a bit. And the persistence. I think whatever you do, you just get tired after a while. Okay, I tried three times not working. Let's leave it like this or accepting defeat too early on some of these things. As they say, you know, out of 100 startups, how many survived? Two. So 98 startup failed. I think the same also goes for your discovery process. Which means if you're setting up 100 hypotheses, be ready that 90 of them would fail. Right. So it's a it's a tough one for your heart to take. Know so many times you start believing maybe you're not competent anymore to do anything. So I think this persistence is also important vis-a-vis the failures. I will just highlight not a failure, but maybe something that is challenging for product manager is to move from product discovery to product delivery. So this little transition phase, because you discover so much great stuff and you have a great concept. And then you need to translate that into smaller artifacts, smaller sprints and deliver iteratively. And the exercise to stream down all this great view of what you discovered into a user mapping of what is this first minimum valuable product you want to build. And still feeling, OK, is it still a line? What is the product discovery I run? And I think it's you have to take a piece of your heart away or what you learn and what you design. And still, I think it's very valuable to design a concept and view for the target picture and then stream down. OK, what is the minimum I need to validate it with the first ship on the market? And this is really a hard exercise. Maybe not failure, but a place where you can get lost. Beautiful. I would say that was the first session around product discovery. Thank you for joining us here in the panel. It was a pleasure. Maybe we can have an applause for the game. Oh.