The Importance of User-Centricity in Product Development - Panel Discussion @BerlinProduct Conference2023

Hello everybody. This is a very special episode because this time we had a format that we actually never had before. So the Product Bakery Podcast was invited to moderate a panel discussion at the Berlin Product Conference 2023. Yeah, unfortunately, Alex didn't make it to the conference. He decided to be at a better place and be on vacation, but I made it and I had the honor and the pleasure to interview three great minds, three great people and talk about the importance of user centricity in product development. So who did we have at the conversation? So first of all, there was Henning Witzel, who is a head of product design at a company called Newstore, which is a omni-channel retail platform with the mission and the goal to change the way people buy online and offline. Then next to Henning, we had a person called Vamsi Krishna Vupalapati, who is an interim chief product officer at a company called Pynthelligence. Pynthelligence is a company that has the mission to electrify the micromobility and last mile transportation, as well as Benjamin Kaiser, who is a chief product officer at a company called Latana. And Latana is a platform that helps you to get insights and measure and track the impact of your brand performance. So with these great people, we had an amazing discussion. And next to that, we had also a 20, roundabout 20 minute Q&A session with the audience afterwards. So everything is recorded, but unfortunately, we had some issues with the microphone. So we tried to do our best to make the soundtrack as clean as possible. Nevertheless, I'm pretty sure you will get out something when you tune in today. So with that being said, have fun and don't forget to press the like button on your favorite podcast too and follow the Product Bakery podcast. So let's jump right into it. The setup today is to me amazing, having like three people with different backgrounds, product and design sitting together is the perfect lineup. So it's great to have you on one table or actually three chairs. There's no budget for the table, I guess. However, I have prepared some bad-ass questions, but I just want to start slowly. So Benjamin, first of all, the topic is the importance of user centricity in product development. I thought it's getting more and more important today to not ignore all these important data that you can get. So I was just thinking to start with the first question on how do you approach a user-centric development approach usually when you work in product teams? So Benjamin, you have a lot of product experience and I was just curious how you do things. First of all, I just want to extend my welcome and thanks as well for being invited today. It's fantastic to see such a big crowd and to try to create a community around product. User centricity is, it's kind of a, you can think of it as a methodology, you can think about it as a set of principles, you can think about it as an approach that you want to take, but really at the end of the day, what you're trying to do is match what you're doing with the product with what the customer is actually willing to pay for and needs to make their lives better. So everything that you do in product should be, by definition, customer centric, even if it's a refactor of a piece of software that's a platform that the customer will never really see or touch. It's in the end of the day that gives you better availability or better performance that still fits into giving the customer a better experience, making your product more valuable to the customer. And so the question is not, I think, how do you get user centricity in? It's how do you get all of the other stuff out? Because taking everybody's opinions and ego out of the process is absolutely crucial for lining up with what customers really want. People can have moments of inspiration and have opinions and hunches. And those are really valuable because they can be that person in their subconscious putting together a bunch of dots that they've seen from the world. But if those dots don't end up being actually good for customers, if those things don't, if they imagine something or use too small of a sample in their passive data collection, that isn't going to be a customer centric solution that you're creating. So when I think about customer centricity, I think about talking to customers all the time, thinking about the mechanics of their day, not just the big grand vision that you have, and trying over iteration and iteration to keep taking the work out of the customer's hand, doing the things that you can do to make their lives better and focusing the product around them. And I try to think about metrics for this. And there are a bunch of them, but I'd be really curious to hear what other people are doing for metrics. But just a couple of things like how, I mean, sort of long tail ones like NDR, do they expand with you? That's an obvious one. Like obviously they love you. So they're paying you more money and doing more things. But what is, how much effort do they have to invest in order to get the value? How do you measure that effort? How many different journeys are they taking in your product? Do they just keep going back to the same one and it's only infrequently or are they frequently coming back and trying new things and doing new things because they can see the value that you're trying to provide and they're finding their way into it. So finding metrics around those, I think are helpful, but I'd love to hear what other people have to say about it. Fancy, go for it. Can you guys hear me? Yeah. Okay, that's great. Thanks a lot, Benjamin. So I come from pretty much product strategy and management background that also says that three more than 10 years working very closely with the designers. A few aspects that I would just want to top it up, right? So because you have covered up most of it, I will go from the user metric, user-centric metrics perspective. So whenever we are building any sort of a design, we are, a lot of times we tend to think that, okay, this is only for the end consumer and then all the stuff is done. If we talk about Airbnb, as an example, you have suppliers who are the host and you have the guests who are like end consumers. Now, the design that you're working on along with the designers, whatever you're putting across, you've got to check on certain amount of metrics. Okay. From the supplier angle or the host angle, what are those metrics that we need to look at? The same way, if it is from the consumer angle, what are those metrics? Now, if you have to think from those lines, if it is largely from the revenue-based ones, then we will look into the return on investment. If it is largely from the user engagement, then you look into two aspects. One is the retention of both the hosts and also the consumers or the users and also the churn that is happening. So a lot of times, I think, you know, I'm with Amplitude, the gentleman have spoken elaborately about the importance of data analytics. So that's where with these two factors, you are able to understand, okay, where the churn is happening. Let's say for the end consumer, when they have gone through the entire funnel of choosing a different apartment or a place, at the payment, there is certain amount of an issue that is coming from the design aspect. And because of that, there is a 20% downturn that is happening. So I don't want to get into details. So like this, there are multiple metrics that we can look into it elaborately, but everything goes on top of, you know, how, whether you are looking to the engagement versus revenue. So, and you go a little top up, it's largely about, you know, how the product guy is actually working very to other product managers, specifically because product designer and product manager are both product guys. So how they're actually working together to look into, not just the design aspect, but also, you know, from, you know, what is going to get achieved in terms of metrics. So I'm just trying to add all the other methods, the rest of it, I told you everything. Yeah. And we have to make a deep dive regarding the metrics, but Henning, also from your experience, because you are the head of product design, we have like the other side, you're sitting to the dark side. So maybe you can share some of your thoughts around that. So, yeah, thank you, Christian. Also, thank you for being here. I'm Henning, I'm the head of product design at Newstore. I've been at this company for eight years. So I hope I get my degree at some point and graduate. Yeah. Anyway. So yeah, we've been through a lot to actually be able to, you know, be user centric. Yeah. Okay. I have user centricity. These microphones are not good. Anyway. So maybe just quickly tell you what Newstore does. So we are a B2B to C company. And what we're going to do is we provide software for retailers to actually operate their stores, the way that Apple does this. So you walk into an Apple store, do you see any checkouts? Do you see any big machines? No, you don't. Because they pull out the iPhone and they make everything on this. So that's one of our missions to really like disrupt this industry and, you know, give store associates, sales associates iPhone. in their stores and empower them. All right, so you can see this as our main use case where we actually exist. We always say this is like the married couple. We all go to the wedding because of the two of them, the consumer and the sales associate, and you are all consumers here in this room. And over all these years, we realized there's one thing you bring user centricity in, is actually, you need to get your organization right. Because we all have great ideas, but the thing is, can we execute them in a way? And we did everything. Feature teams, we did context teams, and what we've done more than a year now is actually organized by user experience. So we have teams that just think about the end user's experience. We have teams that think about a merchandiser that needs to put products into stores, or think about the in-store sales associate, or think about a store team. And what we realized is, if you really rally around user experiences, we finally think about improving someone's life, you know? And that's the overarching thing. Like, we need to think about outcomes and not outputs, and that's easier said than done, because at the end of the day, we produce things, right? We build things, we design, we code. But the question is, how do we improve someone's life? And I recently heard something that really got me thinking, is how can we reduce someone's cognitive node so they actually have the mental capacity to do innovative things, you know? If I don't have to do this task over and over again, literally, you know, I don't have to dance, you know? Thank you, that works very well. No, I have a lot more I can do. Yeah, yeah, I'm immersed in this. I'm immersed in this problem. So if we reduce this, right? And now I'm nervous, so this is a problem. We can really become innovative, because you have time to think, actually, and do this. So I hope that helps you. Yeah, and I mean, you all mentioned it a little bit, and more explicitly, the whole metric part is just super important. And I think Adam just also said, right, you first of all need to get your organization right to be able to have also user-centric approach, in my opinion. And I was just wondering when it comes to measuring user-centricity, what are your, and actually, I don't like this word anymore, the favorite KPIs, but I would rather say, how do you measure it? Maybe you can share a little bit about that. Yeah, there's one thing I wanted to reiterate that Adam had said earlier, which is ownership of the data. And it goes beyond that. It actually is ownership of the data pipeline and ownership of the data. One of the things that you could do to shortcut, or to get to this self-service versus data analyst. So the reason why this is important to user-centricity is because you're gonna need this data in order to know if you're succeeding down this path or not. And that's a foundation for it, getting the data that matters, having it accepted across the organization as the facts. These are the things that we agree are the facts, and there are 15 different versions of revenue and 15 different versions of what an event means that you agree on that. And then you can really use that data as a source of truth. And that is an organizational thing. You have to agree at the organizational level that data is important, that you will own it. And I would go even further. I've never really seen this work, but it's kind of my wish that one day I could make this come true. I'd like a product manager to own the data for end-to-end, own the entire journey from the organic search all the way through to expansion, and be able to provide insight across the journey that the customer takes and where they're having trouble and where they're not, and really consult with other product managers as an expert in that area. I have not been able to make this come true yet, but if anybody has an organization that does this, I'd love to see a show of hands. Wow, fantastic. Maybe we can talk afterwards. But anyway, as a foundation for being user-centric, you have to understand what that journey is. And specific metrics, I think, are important. I'm literally day two as CPO of a company called Latama, and I'm trying now just to figure out how to dig into the metrics that matter, what would matter to our users. And so, because I don't know the product very well yet and learning it, I don't know exactly what frames the user experience the right way. So you have to actually understand the interaction, and that goes to qualitative, looking at how users use the product, talking to users. And then from that, you can decide what are the most important data points that might proxy for that and represent that. And then you can use that as an ongoing sort of look view on the product. But things like breadth of product, like how many of the features, of all the features that you're offering on a certain surface, how many are they using? Within those features, how deep are they going in that feature? How much time are they spending and what's their frequency of return to the feature? Those are some just quick things that might be relevant, but again, you have to really understand how your users are using your product and why, and then pick the metrics that are really relevant for you. It won't be the same in every single company. It won't be the same in every product. There's just one more thing I would like to catch up on, because you said it's just super important to put your ego out. Yes. And I think, at least back in the days when I was working as a product manager, I caught myself many times being very biased. And I think bias is a big problem, especially when it comes to at least my ego. I'm not sure how you feel. So how do you try to clean out the bias to reduce the ego? That is such a terrible mic. That is such a great question. I mean, I think the thing that you can rely on is don't trust yourself to not be biased. Don't trust your team to not be biased, but instead continually bring the conversation around to the customer and their voice and bring that into the discussion, because that can help keep you honest. So if you're hearing feedback from customers on an ongoing basis about an issue that you're trying to address, a solution you're trying to create, or a problem to solve, make sure that the customer input on that is constant and you're getting as much as you can. I was recently at Contentful. I have some colleagues here tonight, which is great to see. And in that role, we actually had an assistant whose job it was. I'm not sure how successful we were 100% of the time, but her job at least half time was just to schedule calls with customers to work with us to make sure that we had a constant stream of discussion with customers. And it actually takes resource and time and investment to get that customer conversation going, but there's actually nothing more important to product manager or product management than that understanding and connection with the customer. So however you need to do it, whether you're kind of in a scrappy startup environment and you have to make those calls yourself, or if you have a customer success person that you can work with, create that channel and repeat, repeat, repeat, because that story will change over time. When you release a new feature, it changes again. And just really understand your customers and try to set aside your preconceptions. I know this is a lot of work, but it's very, very worth it in terms of the speed at which you're able to become more successful with the features that you're building. Yeah, thank you. I want to share a metric that we started tracking, and I wish everybody would do it. And I want to share it, so hopefully you guys do it. I have one question for, who of you has ever left a comment on Amazon about a product? Okay. I'm hating the product all the time on Amazon. So what was a good comment that you loved? What was a good comment that you liked it? What was a bad comment that you said this wasn't good? Like from the ones that put their hand up, which one was a good comment or a bad one? What was a bad one? Okay, so we realized the thing while we were trying to find a way to get user feedback into our apps and actually learn. And you know, there's one thing people do when their app doesn't work. And it's not crashing this app. What do you do when an app doesn't work? You kill it, right? You put a background, you kill it, you open it up again, and you start all over. So we were thinking, okay, let's make this a restart rate and start tracking this. So we started this. We're just like, okay, next 20 seconds, you open this app again. And actually we have amplitude on this, by the way. The engineers and the product manager couldn't believe the data. Everybody was saying the code is wrong. This can't be true that people restarting this app so often. So we put another tool in to actually ask someone, so what was wrong? So now we actually even getting comments every time. And we even asked the users, hey, maybe you could categorize your feedback. Were you stuck at the payment or you were stuck there? And this became a health metric that we literally track and we can even foresee when issues are arising. We shipped a version where you couldn't send an email receipt. We opened Amplitude in the morning and the spike was increasingly high. And we fixed the same data issues and we had a new version for the customers running the next day. And we actually prevented even a whole escalation topic. So I can highly recommend you guys, so if you ever build software, please build this restart metric. And yeah. Just really quickly, it reminds me, support tickets are a great way to understand where your customers are at. So there's some metrics around support tickets that you can start to, because they're complaining at that point, right? They're giving you feedback on your product through reporting bugs and problems. And that can be a really valuable source as well. Amity, how about you? Yeah, that's great. I really like, so putting forward, I'll share your story with my team. Thanks a lot. So similarly, how many of you guys have been using the YouTube and Facebook? Almost all, right? Almost, okay. So what are the metrics for, what are the KPIs or metrics for YouTube? Anything, you can give a shout. Sorry? Retention. Retention, okay. So how are you measuring the retention? What are the key metrics? Keep watching, views. Views. So now the similar sort of metrics have been built by Facebook from looking at YouTube and also, you know, the same things have been added into Instagram, the same things similarly added into TikTok. So then where is, you know, the key metric actually coming in when you're actually designing it? So that's actually goes back to the user research aspect. So as a product team, specifically, you working with your user research team to understand the benchmarking of different companies in order to make it better. So is that, so when Instagram has come, Facebook was already there. So it's not just like, so probably they wanted to create a little more, which is, which is probably, you know, additional sharing Facebook was Facebook, if you have seen in the recent times, I think six months back, they have, they have with a new version. It's not just the like it is, whether it is a love symbol or a like symbol or a hate symbol or a funny symbol, whatever it is. So there the metrics are largely involved in that. So all those things are also coming from the, from the user research perspective. So what I'm trying to also highlight over here is metrics, because we have already spoken about the metrics. So where they're coming from, the user research, coming up, coming with the user centric design plays a very, very crucial part. And that goes, that shall go in sync with the product managers, product metrics, or you can talk about organizational metrics, now we are putting, putting it level up. So just wanted to share that as an example. Yeah. I mean, we talked a lot about metrics. We talked also about generally how to approach the development process. And I was just wondering, because it's so important to make sure that the research team, then the design team, as well as the product team are in line. I was just curious to hear your opinions on how to make sure what's actually your number one secret, maybe you have even more to you want to share, to build up a good collaboration between those departments and making sure that you have like a seamless and common understanding on aha moments, for example, and what's going on with your product or products, especially multi-product companies. I think it's even more interesting. I've been thinking about this because we got the questions beforehand. It's a really good practice, by the way. Thank you so much. The one secret that I have is, and I want to split it up to the, you know, the person itself. So let's say you're a designer, do the other person's job. If you're a designer, start thinking, yes, I can say that again, do the other person's job. If you're a designer, learn more about business, think about what does it mean when something is viable, right? If you're a product manager, you should go out and do research and try to uncover demand. Don't ask, hey, do you want this? No, you should ask like, why you don't want something or what else do you, what are you struggling with? Your designer's going to love it when you do this, because they're the ones that also don't really, sometimes don't want to do the research. And lastly, if you're an engineer, actually start using your own product. There's the saying, like, use your own dog food as a bit of a negative connotation because who wants to eat dog food? But at the end of the day, if an engineer has used their own product, and we did that and do so, we put an engineer into a store and worked as a sales associate, and it was the most awkward situation I have seen. Like the engineer didn't say a word, the consumer was standing there, and they're waiting for the receipt to be emailed. And then the engineer said, well, this takes way too long. Yes, it does, right? It hurts, right? It hurts. It's so awkward. Honestly, when you're in your simulator, you click through like, oh, yeah, works, works, works, works. Right? That's not a real world situation. So my one secret is do the other person's job. I don't know if I have any secrets, I think, honestly, I think the leadership has to equally involve the functions in a real way. So no politics, no BS between the functions. Really try to involve them. And really, at the functional team level, that designer, researcher, and product manager should all be working together. Thank you. Should all be working together. So it really shouldn't be about a secret that I have. It really should be about that trio of people that are working together to create that solution. Designers are really, they're really magical at what they can do, right? So we respect the designers, because they solve the problem in a way that can be really brilliant. Engineers, same. They do things that product management is there to identify the problem, figure out how to match what you can solve with value, the customer will give you back in return. We're not there to do those jobs. So if we frame the problem the right way, and we make sure that the customer's voice is there, so that the designer and engineers can benefit from that perspective, that bakes in customer centricity. And it really sets the team up for success. But as a product manager, you shouldn't be doing the design job, you should not be doing the engineering job. Your job is really to find the problems, make the world a better place for your customer by proposing the problems that we could solve, and matching those against the business value that you think that it will have for the company. It's all about building relationship as well, right? So for the product guys, there are two main teams or team people, whatever you may call your counterparts, if you're working with a design engineer and an engineer, right? So what you can do is like, treat your engineer like your spouse and become best friends with your designers. So anyways, jokes apart, but it's not largely secrets, it's definitely relationship building. I have learned with a lot of mistakes as well. So one thing that I really, really made great amount of relationships and also built trust for the designers on me was, from day one, giving them the entire ownership and throwing the same problems that were thrown on to you, by your CEO or your investors or your, you know, your CPO, or even your CTO, just discuss with them, because those are the guys who are actually thinking about, you know, coming up with a very creative solution. So if you understand the psyche of designers getting into their persona, they're very creative people, right? Doesn't mean that, you know, the tech, the engineers and product guys are not, but largely they're coming from the creative, so people who have been, you know, working on different arts, design. So where are they coming from? So they're coming from very creative fields and all that. So giving them that sort of liberty has really helped me multiple times and also getting them inclined towards, you know, what are we actually doing with the business? I would like to add something to this, because yesterday I spoke at the Town Hall in our company, and we talked about the word design driven, what does it mean? And I don't like the word per se, because it focuses so much on a function, but if you think about design, it's actually a capability we as human beings have, it's like we can project ourselves into the future and can imagine how this would be. Like if you move into a new apartment, you can imagine, ah, this is how I want the kitchen to have to be, this is how the living room should look like, and everybody has this capability. So literally everyone is a designer, most people just suck at it, but you can get better as I was, because I was, I'm actually a computer scientist, I actually studied computer scientists, I was coding, and I found my way into design. And now, you know, it's easy for me to relate to an engineer. And like talking about design driven, I love this, because, of course, you can listen to users, and they got to, you know, want to increment value, and they tell you that's what I want. And, you know, there's this famous quote of Henry Ford, if I would ask my consumers what they want, they would fast horses, but then you have actually technological innovations, and like very tech driven. If you combine this and push something to the market, like we did with the iPhone, or we did with a Polaroid camera, or we did with a Wii, or actually did with a car, it's like people all of a sudden create meaning into these things and make them like, oh, that's like, I didn't even know this can exist. And I see this every day with, let's say, the customer success team we have. And you know what, they're going to start, they love amplitude now. We literally like we haven't had it for quite a while, because we need to get into a contract, we were like past our trial period, and they were begging to get it back on. Because finally, they could actually bring something to our consumers who are actually able to track this. And we as tech people, we often do this, we victimize all the other non-tech people. We imagine they can use it, and it's like super easy. And for them, it's like, I don't know even how to withdraw cash from my ATM machine. So it's really up to us to make this also, you know, usable and combine this with modern technology. I just want to add one other thing, which customer centricity is not, and you were kind of alluding to it. You don't just do what customers say their pain is, or you don't just do what they tell you their idea of to make your product better. As product managers, we really have to dig way down into the what is the underlying thing that is hanging up their day or making their business results less good, and get to that and solve that problem. It's not about just, oh, I listen to a customer, and then I just do that thing, then we wouldn't need to be in the role, that would be unnecessary. And it is this idea of, you were calling it projecting into the future, but really, it's a kind of a deep empathy. And I know that word is really overused, and, you know, apologize for using it, but really trying to put yourself in your customer's shoes in a deep way that maybe is more imaginative than the way your customer is even thinking about their issues, and getting to that problem, and then giving that to the team to tackle. really when you have the chance to build some real value with customers and build loyalty and engagement and all the things that you can measure, but you don't measure those until after you really nailed the problem and then the team comes up with a great solution for it. Yeah. And one thing I just still want to catch up on is, we're talking about these things and they make sense, but there's still this problem to me, the real world, right? I mean, it sounds, everything sounds ridiculous. Don't be ridiculous. Exactly. I'm sorry. I just had this nice quote. What was it again? Cocaine isn't courage. Porn isn't real sex and business books aren't business. So I really like that. So I was just wondering, because you said you would love to see one time a team that has the end to end ownership of data. And then right after that, Henning was talking about the experience teams that you have. So for you, Vamsee, I was just curious, first of all, Henning, do you have end to end ownership of data? And I would also like to hear if yes, how have you done it? And if not, why not? I'm used to people sitting there. So now be careful what I'm saying. I would say, yes, we do have the ownership. You would say, or you say? So now we beat to the bone. Yeah. Words are important. I also like that. I say, we do have this. The question is, does everybody make use of it? You know, is it really like evangelized and that people like understand what they can do? We even have a data, you know, a data team that makes us accessible and their only purpose is to actually enable product teams to make use of this. So we literally just talked about this, like, this is your purpose in your store, like nothing else. And what was the other part of the question? So if the answer is yes, the second part doesn't matter. I don't know yet because I'm only on day two, so I haven't actually figured out who owns what yet. So I'm still working on that. Yeah. I'll probably talk on that end to end data and giving access. I'll tell you at least 85 to 90% of companies, when I say companies, let's talk about startup here, at least 80% on an average of startups are not really data driven. I know I might get it backlashed a lot. All the teams I'm talking about, so that we were also talking sometime back and it has to come from the organizational, you know, culture and all that, right? So, of course, there's a huge amount of realization where every team, it's not just the design. It's not just the product. It's not just engineering. Everyone in the organization, customer experience, of course, analytics guys, marketing, performance marketing, they deal with a lot of data as well. Payments team, support team, you know, everyone having the access towards data, specifically towards, you know, what are they trying to achieve and where they have been going, has been changing slowly, not slowly, sorry, fastly. But it's very, very important to have an end to end facility or a view towards what's how the data is actually moved. There are a lot of startups now in Berlin also, where within the product engineering and design means you have the boards, you know, whether it is internal metrics data, whether it is from the support angle or whether it is, you know, how many customers you've got, how many daily active users, monthly active users, depending on the organizations and the team's metrics, it is happening. But if you guys have got this sort of, you know, inch that, you know, you have to be data driven, definitely talk to your managers, talk to your leaders and tell them, hey guys, we got to be, you know, getting to the data driven aspects. We need to understand at least, you know, where at least the data driven 101 we can build. There's just one thing I want to say about data driven. I see it more, you should be data informed and not driven by it because I have seen the bad examples where you just try to perform, improve your checkout and get people in to do things that you don't even want to do. So use it as another insight. As you look at customer support tickets, as you look at insights you generate, when you go to talk to consumers and use it just as another source, right? Otherwise it makes you blind and you just look at it. Thank you. First of all, you don't need to clap right now because we're not done yet. Because so as you said, I mean, I would say a lot of startups in Berlin start changing and shifting their mindsets because data becoming more important. AI is going to eat the world, I would say, or maybe not. And yeah, I mean, I work as a product coach and I see more and more emphasis on that whole topic. And I think it's maybe a good bridge to open the Q&A. We have also like the next 20 to 30 minutes for people to ask questions. I was going to say I would love to hear some questions from these folks. Yeah. So do we have a microphone that we can share around? Maybe you can just tell us your name and where you work. That would be interesting. Yeah, I'm Sabir. Hi everyone. I'm founder of Fade Group. My question is, currently I'm doing a lot of survey from the customers and we ask, try to define the problem. My question is, when I ask the customers, I see a lot of reactions like face reactions or body language. So I can understand if it's true what they are saying or they try to sound good. You know, this happens a lot in service. About the data, it's really hard because when I do the service, I'm not sure if this data is reliable or the right data and should I use it really? Because in some service, maybe they click wrong or something. Can I ask you a quick question about that? Did you design the survey as a researcher or was a researcher involved in structuring it or was it a product manager driven survey? It was designed with the team. I mean, a scientist, me and also two advisors together. So the questions were formulated from different fields. I guess one piece of advice I would have just to start with is it's really hard to do a survey that actually gives you good data. And if you have a researcher that you can draw on as a professional in how you formulate questions and take the bias out of the questions, it's super helpful. Not everybody has that. And I'm sure you guys did a great job. No, my question is not about doing the survey, but rather to understand in the digital world when we ask the questions, do the survey or try to check the data or metrics. How to know how reliable is that? That was my point. The better the survey, the more reliable. But surveys are also not 100% reliable. But to try to bulletproof it, one thing you can do is have a professional researcher work on it. That's one tip. Second tip, I don't have, I don't know. I have one more question. One question. How about qualitative feedback? Just to understand your situation, is there any chance to talk to people and do interviews? Yes. When I start the conversations, the answers are very generic. When I deep dive and then I figure out it's very different. So when he was talking about user researcher, right, obviously it's always great to have a researcher because they are very specialist to do that particular aspects of surveying, talking to multiple people. This is largely about whether it is qualitative. It's largely qualitative research. But at the same time, when you're asking the questions, let's say you have a hood and you're a designer of this hood or you're a brand and you want to go and talk to X number of customers. Right now, you have to identify that X, what is the sample space? Is that going to be just three people or is it going to be at least a minimum or greater than or equal to 50 people? Now, that particular sample space should come from your right target audience, which means it can be the demographics of it, which is your age, group and all. And also specifically about, you know, what are the problems that you're able to tackle there? For example, if you're having this food and you want to check with them. So if you go and say like, hey, we have great food and it has got like, you know, you can put on these things and it has some chips inside. What do you think? Do you like it or not? So there is a psychological inference, which is proven totally that when you go to the customers, if you say like, you have a problem, I have a solution, just buy it. Most of them say, hey, hey, yes, this is, I'll probably, I'll change. But rather, you may tell like, you know, hey, this is around minus five degrees. What kind of dresses you wear? What kind of clothes you wear? What is the thoughts that you're looking at? Where do you shop? Where are you coming from? And in a year, what is your purchase? If you're asking these questions, that provides you the insight. So that's where he was talking about the researcher. So that's where you would get the insights. And probably you can, with those insights and a sample space, you can at least come on to a certain amount of a thesis or an inference, or you can also call it a hypothesis. So it also goes back to the way you're asking the questions. And what are the, who are the audience? I can share one example of maybe like how unreliable it can actually be. Remember the Walkman? They wanted to figure out which color would really like, you know, would people have been drawn to. And then they invite people in a testing lab and they put a black Walkman and a yellow one, and everybody describes the yellow one as this super beautiful thing. And they could imagine wearing this. And then at the end of this, they say, hey, you know what? Since you're joining here, you can pick one. Grab one, it's yours, you can take it. And everybody took the black one. Although they just, they described the yellow one as the most beautiful thing. So this is how reliable this can be. Honestly, maybe one thing to add to you, I mean, if you start gathering the data, you should see patterns. You should see things emerging and then you start digging in. Then you talk to people about this specifically, and you may even show them something to actually validate if that's what they were talking about. The thing is, yeah, thank you very much for the feedback. The thing is, the output from the physical service and also digital service is very different. That's something. With the same audience? With the same audience, demographics, the age. My product is between 30 to 44 years. You've got to choose whether you want to target now on online stores or offline stores, or something like that. Digital business or offline journey? Anybody else? I'm sure more like about the offline. I mean, because I see the expression and the answers are actually quite relative or relevant. But when I see the digital data, then I feel like, oh, it's very different. Then you go and do your A-B test or multivariate test to get further. I would love to understand more about your product. Maybe we can catch up later and do a deep dive afterwards, if you don't mind. Thank you, yes. There was a young man raising his hand. Yeah. How's that one? I had just a question about your thoughts on gamification in product design for 2023 and how it will influence product design applications, platforms, and inside, not in landing pages. If it's B2C, maybe it has a future. If it's B2B, probably not. I guess that's my rule of thumb. I work mostly in the B2B space. In fact, exclusively. I have worked in consumer work before. Usually, stuff like that just gets in the way of getting a task done for somebody whose job it is to produce results. They probably don't want to be amused or entertained by the product. They're probably trying to get to a goal. Anything you do that causes friction or derailing from that goal is usually not the right thing to do. There may be cases where it can work, but I've never seen it be amazing in a B2B product. Just one small tidbit. Gamification has been there for the past 20 years. It's very sad to see why it really did not get its way out. But if you look into apps, specifically into different segments, if you're talking about dating, there is a lot of gamification. Working? Yeah. It's not working. Working is not working. If you look into dating business, dating apps, a lot of gamification happens over there. If you look into EdTech, specifically K-12, there's a tremendous amount of gamification. I can talk to you about an app called Byjuice in India. It's the largest EdTech app in the world, both by revenues and volumes and size. Specifically, they have been building on gamification design for the last eight, nine years, as I know. But once again, it is very limited to certain businesses. In my projects, we've been working with thinking about the user as the protagonist in the story. In their narrative, you can define goals. From a user-centric perspective, you can actually think about the user as having their own goals. How do you display their goals inside your platform? I was thinking perhaps that could influence how you build trust in applications and in product design. It's hard for me to agree, because usually people try to get the job done. This thing is just a tool I use to do this. I think I can imagine this, and we are not doing this very well. Learning something new. There's a new functionality we just launched. How do you get this installed quickly? There you could probably use some gamifications. There's one funny example. I was with my girlfriend. She's also here, by the way. She's supporting me, because you get a little nervous in front of 300 people. We were walking into a store of Rafa. We have Rafa Cycling. There was a sales associate looking at our dashboards on the web using a computer, which is against our vision. We want people to have the iPhone, right? I asked this associate, until I said I'm from Newstalk, why do you do this? This person says, I want to see how my colleagues are doing in Australia or in New York, because I want to be the best. I want to have higher sales numbers than them. I was like, okay. I was just walking out of that thing. I was like, we need to give this data to the sales associates. They need to see this. We have brands that do gamification. They make sales goals. You have to hit the numbers. That person had completely different motivations. Therefore, you can use it. I can see that. Thank you. From a psychological point of view, I hope gamification won't make it, because I'm just super addicted to all this kind of stuff. There's another question over there. Yes, I have another question. I'm a UX researcher. We as a team collect a lot of user data, qualitative and quantitative. My question is, how do you make this data and insights more accessible for the stakeholders, different designers, product managers? Normally, after the delivery, in a formal presentation, the insights are there, but they are reusable. They are recyclable. How do you make them really valuable for other people, too? That's my question. I guess I have a quick thought. I think Adam was kind of pointing to this. You have to take action on the insight you get from data. Otherwise, it's not worth collecting it. It's a really important, very simple, fundamental point. I guess the question is, why is your team not acting on it? If you're having engaged conversations with product managers to explain the aha moments that you've seen, to use Adam's words, if that's not leading to action, then that's a different kind of a problem. I don't think there's a magic visualization that you can provide that is going to suddenly change people's minds. If only it worked like that, but it doesn't. It may be a conversation you need to have with the wider product team. It may be a conversation you need to have with your management about how you upgrade that insight from data in the organization. As a human being trying to work with another human being to explain that stuff and still no action is taken, it's probably not something that you're going to be able to affect right at that moment. You're going to have to make it a wider conversation at that point and advocate for what you've discovered and how you think it could help customers, maybe to other people. Wow, that worked so much better than this one. Without knowing how you actually do it. One thing you should really pay attention to is be inclusive. Take people along the journey. If you go out and do all the research and come back and put this on someone's table, how much do they care? They don't. They weren't even part of this. If I, as a designer, just designed something and put it on an engineer's table, they do not feel empowered. But what you could do is before you even go out and research, you just go to, let's say, a product team. You have engineers, designers, product managers saying, so what do you want to learn? We're going out. What questions should we ask them? And then you come back and then you present the early first results and then you say, we're going to refine this and we do it together. Just bring them along the journey. They're going to, I promise you, they're going to be so much more involved the next time. They're going to beg you for it. Can you go out? Can you ask these questions? Because we don't know. And I'm not on the panel, but I just want to add one thing to what you just said. It was really cool. I actually did this one time. I took a designer and basically without them using the data, had them use the actual product step-by-step. And as they hit each step, I asked them, how many people, what percentage do you think go from here to here? And they would guess and they would be wrong. And they went to the next one and the next one. And I found for some reason that weird technique really got buy-in from them. There you go. That's the answer. So I would say time-wise, two more questions. Hi, my name is Pearl and I work for Metro Digital on the product management team. So my question is, how do you draw the line between your vision? For instance, you want to start something and you have this vision, but you also know the importance of user research and the user-centric approach. How do you draw the line between what you want to put out there and what the customer is saying? Because probably maybe your vision is new. And so it's kind of like, how do you draw the balance? So I think it's a very important question. A lot of us always think like, hey, this is the problem and this is what the solution is. But we just want to go and validate. But as our friend was telling, the moment you get on to the research, the moment you get on to the customers, you build your MVP or anything, and then you validate it. Most of the times, I might be subject to once again over here, but it's like, you know, you gotta have this attitude of learning new things. And there is nothing wrong if you have to change your vision as long as you know that you have a vision. But the directions might be a little different because you would have laid your own path towards the vision. So what you're going to change now is, or what you're going to be flexible towards is, while learning from the information data or, you know, as you grow with your product, you know that, you know, have to take different paths, but you can still have the same vision. Sometimes it may also vary depending on so many other forces, external forces, but you don't have to draw any lines, that's what I feel. Actually, I want to add to this, yes. How likely would you recommend this? Microphone to your friend. No. Hold it higher up. Hold it higher up? No, covering the antenna bit. The iPhone antenna gate? No. Okay, so like if you think of big corporations and you might wonder like why the startups are being disruptive, why they actually can, you know, bring them to fall is because they have a build, measure, learn cycle that, you know, when you read the lead startup or whatever, they fail fast and then they change and then they just keep learning and building and measuring and they do this constantly. But if you just pay attention to it, you will get to your vision at some point. Slightly challenging. If you have a vision, you should immediately question, wait, do I have visions when I drink too much or do I have visions when like, what you want to do is start asking questions. Is that vision real, right? It's that vision is usually your intuitive brain putting together a lot of data points that you've collected, but then you still have to validate it. You still have to go back and ask questions. So the line, I think, starts to become, okay, so I have this idea of what I think is going to solve a problem or idea of what really captures that problem, then go test it. And then you'll know, is that really something that you want to spend time on and invest? And I think that helps clarify some of the lines and you might go, that vision that you had might get smaller along the way or might get even bigger, but at least you'll know that the steps you're taking have been validated with customers. And you need to get lucky sometimes. One more. So one more question. By the way, thanks for the tip with the antenna at the end of the conversation. It took us about 15 events to figure that out. Why should we have it easy, right? I can understand it. Hi, my name is Pauline. I am CEO of Grace Agriculture and we're building a software for farmers. We are quite a small team, so four founders and in total we are two product designers and two engineers. So my question is, when do we start implementing the advice you've given us today? Because we don't have a data analytics department. We don't have a separate department for that, but at some point when people are starting to come in, where do we, I would like, I want to build a perfect system that can integrate people and realize all the advice you've given today. Customer centricity, product ownership and to avoid all the 10 mistakes that were mentioned in the first keynote. So how do I do that? The first impulse I had was hire a product manager, but I'm not sure if that's right now in the budget. But as you said, you try to make a perfect, you try to have the perfect, what was it again? The perfect solution, the perfect system. Maybe first of all, don't try to be too perfect to this thing. So build, measure, learn. I would try to make as many mistakes as you can actually, because there is no one solution fits all out there, right? I mean, I think we have very experienced people who will support that. You have a great advantage at that stage because you are so closely connected to the farmers in your case, right? Where you actually need all the rest of this stuff that we've been talking about is as you grow as a company and you scale and you have lots of customers, that's when you start to get detached and that's when you have to reattach and really work on it. So you actually have, you're blessed in a way with that advantage. You're going to be talking to those farmers and you're going to really understand. What you have to worry about is in 10 years after you've changed the entire game, how far away you are from farmers. Then you have to worry really about, are you still connected with them in a realistic way? Okay, so you mean that our customer centricity system has to grow along with the company? Yeah, exactly. Do you have engineers in the team? Yeah, we have two software developers, one drone operator, one drone designer. Okay, drones. Yeah, we're working with AI drones, cameras and so on. That's a cool company. So I mean, I don't get any money by Aptitude, but you can actually put this in tomorrow. Literally, last time I was at this event, I was sitting next to a PM and he was like, this Aptitude sounds interesting. I said, you can actually just use it tomorrow, put an event in and start tracking. You don't need a big data team, just start tracking, but not everything. I think we did the same mistake, which is that everything and then it was just a mess. No, just start in, put some meaningful points in and you can actually use it for free at the beginning and just make sense of what you're seeing. Okay, nice save speech. Thank you. Yeah, I should get money. So Henning is not getting 15% if you sign up for Aptitude, by the way. So I'm not sure how you feel, but I have like a very dry throat. So there are two more things we need to do. So number one is go to your favorite podcast tool and please follow the product bakery podcast. And number two is enjoy the night, network and get some drinks. Okay, cheers. Thank you very much.