Why case studies make you lose your best candidates

Welcome product people to another episode. I'm here today with Alex, my dear co-hosts on a sunny and very hot day in Berlin. Alex, how's it going? Similar than last time, it's still very hot, but it's doing great. I think it's also the German attitude that we always complain about. Too hot, too cold. I never complained about too cold. I am. You can just wear a very thick jacket. Yeah, it's not enough. No, I can't do product placements. But yeah, with the right jacket, there is no too cold. Guys, if you have any recommendations for hot jackets, please share them with us. I'm always looking for something like that. Considering the heat now, it's just a matter of a couple of days or weeks and we're back to minus 20 degrees in Berlin. I think rather it's good to advertise some good air conditioning at the moment, but... It's too late. Amazon has already sold out on everything that's cooling you. And they're too expensive. Anyway, there's one topic you have a strong opinion about and we touched it in one of our last episodes, which is the topic of case studies during the interview process. How do you feel about it? Deep breath. It's no secret, right? We talked about it, that I had to think about my process in hiring people a lot, especially as I'm trying to hire more and more people at the moment and actually just managed to get the first offers out. One of them signed. So yeah, I will keep on pushing. Hopefully by the moment you listen to this episode, I managed to hire one of my most promising candidates that I just had in my pipeline this week. But yeah, obviously like a big part of it is like the process is like candidate experience is like finding the right profile. And I think here the understanding of what's best in the process varies a lot. It varies a lot of what do applicants expect from the process. It varies a lot from what do businesses expect from a process all the way down to how it actually works. And I think it's still best practice to have case studies for designers. I also see it a lot for product managers. I keep sitting in a lot of product management case studies also at my company when they are hiring. And when I joined, I was actually talking to our people team and to our recruiters. And we have this process in the company that a case study is like a must. There is no way around it. That was a little bit like also where it started for me because whenever I talk to candidates and candidates who interview for companies, the case studies are like usually the worst part of the experience of interviewing with a company. It's adding a lot of stress to them. Those are designers who most of the time are still in a full-time position. They are anyway working. They are anyway stretched. And then maybe they are interviewing with different companies. And imagine all of these companies ask them to do a case study. There are some companies who are smart enough to say, okay, we do a more general case study that doesn't feel like something or that doesn't feel like you're working for my company specifically and giving me some ideas that I could use. I think that's already better. And there are some that are terrible and even ask you to do something directly on your product. And I think then you have the sense of people who, first of all, don't want to work for free. People feel like you are trying to get a lot of ideas out of an interview process. And even worse, and that's now speaking me from my perspective, I always feel like there is only a very small amount that I can actually get out of these case study interviews. Because, of course, these sort of interviews help you understand how people think. Or that was also what I thought when I kept running them back in the days at SumUp. But at the same time, I believe that there are a lot of other ways to look and to understand how people think. A lot of other ways that are more efficient. Because how much can I actually get out of a case study that someone did in a couple of hours, maximum a couple of days in their free time without having all the data or the input or the way to collaborate with team members compared to how they would actually work in their real life, right? We could also argue that understanding how someone is thinking is a quite crucial part of the process, right? In product management, if you give someone a case study to, for example, choose one of two features and explain why he or she did that and how they would proceed and how they would plan, etc. I don't need to make up a case study now out of my mind. But I think it's very important to see how people are thinking and what rationale they use to get to their decisions and to their decision making. What better steps would you follow in order to understand how someone is thinking? Let me put it like this. Most of the time we talk to people who have some sort of experience in their field. Even if it's just like experience that they took out of university maybe or out of some sample projects that they did there. What is more valuable to you? Me explaining you, for example, on a project from my previous employer? How we did it? What the challenges have been? Who I worked with to actually get to it? If there was anything that I did wrong that I'm going to do better next time? And what the results have been at the end? Or is it more valuable to hear, I don't know, how I would design an ATM for kids under five? At the ATM, obviously. No, you're absolutely right. And I think making the deep dive when someone is explaining their past experiences and also trying to understand what the dependencies were, the challenges were, helps a lot also to make a deep dive into how is someone collaborating and understand what is their rationale and how do they react under pressure. Because for everyone who's regularly interviewing people, you know exactly where you need to dig into when someone is dropping certain, I don't want to say red flags, but yellow flags, where you better try to understand, what was going on there when you decided X, Y, Z? Or what was the problem with your supervisor not launching the project, etc. So I agree, that's definitely something that is maybe not seen enough at the moment. But I also try to understand how can you convince your organization or your leadership team to get rid of a case study process and rather replace it with more in-depth interviews. I probably found the easiest way around, but it could also be that my organization was easier to convince. So I still kept the case study in the process, if you look at the process. But the case study is an email where I ask people to just prepare one or two past cases and where they walk me through actual real-life work and where I can challenge them and where I can ask the questions that help me understand how good of a team player are they, who pissed them off, where do they think they knew it better than someone, where are they open to collaborate, like all these sort of things that I can't get out of something that a designer does and probably applies for a product manager, that a designer does in two afternoons alone at home, completely out of context, without a chance to talking to a developer, and so on and so forth. I'm just wondering, sometimes you have positions where you are looking for a particular skill. Let's say you are going to open up a new department or a new position and you need someone who is taking the existing product and thinks forward because you need a certain domain expertise. And I still believe that a case study that is also tailored around this specific use case saying, hey, you are becoming the new, let's say, head of design because we haven't in the past. And you come up with a case study to really see how someone is thinking and also digging into your actual product and doing the research by him or herself to better understand, okay, what is going on and how do I see the vision for the company can be very insightful to understand not only how is someone working but also what could be the benefit of this person for our company. So the thing is, So I don't need to make a case study and ask you specifically for it. Okay, I mean, we discussed it in a previous episode, right? You should always be prepared when you go to an interview. So if you don't know that, I don't need to ask you. I think it needs to be whoever applies who also has the drive and can convince me that they want to work for this company because of X, Y, Z reasons. So this is one. Secondly, I think it's always hard to have a case that is around your organization. I would feel bad and maybe I can bring in a little twist but I would feel bad to ask someone to do actual work on my product. Now, the twist here is I once applied or I once was interviewing for a company and they asked me to literally one-on-one work on their company vision. Now, while I would usually say they paid me for it. It was like me working as a consultant for them. They gave me access to their whole team. I was on their team slack. I could talk to team members, I could talk to developers, other product managers, spin ideas, completely open, transparent. I had all the information to actually make it quite real and quite realistic also in terms of collaboration. What do I try to understand from others? What do I do? Because that's the real work but then it's just very time intensive. But it also sounds almost like a trial work you're doing, right? It's less a case study than rather being on-site in quotes for sure but really collaborate. Yeah, and I think you can only do it with certain positions. I agree, I agree. Imagine you need to have every to get them onto your platform on Slack, give them access to all the tools for 1-2 weeks. I think it's generally very challenging hiring product people because I'm not sure if I mentioned it once in one of our episodes but if you hire an engineer, you will have a person who's going to join, who's going to write code for sure attends all the meetings but this person will do the job. But if you're going to hire a product person you're going to have someone who's scheduling tons of meetings, asking many questions, having a huge impact on your company's strategy, so it's a massive change that is going to happen to your organization once you're going to hire product people. Therefore, I'm fully with you. You need to understand how this person is thinking and trying to get as much out of them as possible and if there's even one message out of this episode for today, the worst thing you can do as a company is giving people the feeling you're working for free. Yeah, it's a lot also about respect and I think while you can find ways to make these case studies, take-home exercises whatsoever, however you want to call them you can use them in a good way. I still feel like that the notion most of the time is negative and it puts candidates off a bit and I heard it a lot, you can read a lot about it and it's a little bit like this one-sided thing where candidates talk about it but companies still follow the old best practices that they somehow found and everyone is applying. Let's call it old practices. Old practices. It's just a matter of also adjusting to that. I think it's a matter of I need to respect the time of people that applying for a company is taking time, that competition is big, that candidates are probably talking to more than one company. Maybe if I am Google, I can afford to have processes that run three months, five months with multiple different rounds where I need to do a lot of this sort of work but then people do it because they are intrinsically motivated to work for Google and I think as a startup you're sometimes probably not in that position and then you need to find ways around and that's where just iterating again, obviously the past work is super relevant to me to understand how people think but then I want to see their motivation, I want to see their mindset most of the time, their thought process how they talk to me and I also try to move maybe not the first call but the second call that I might have with a person to something much more around also having an open conversation to get their thoughts to brainstorm which could be to some extent also a case study because it also means that you need to be prepared and I might get some ideas out of it that I could use in real life but it's really more about your talking, about your thinking, about your approach to things and I think that's something that I can get out of a normal conversation without asking someone to spend a couple of days in preparation of a presentation that then goes pretty much into the bin. Agreed. And there is this war of talents going on and not every company is one of the big players so therefore the leaner the process is the better. Alex, any closing from your side or any important message that you want to share when it comes to case studies? I think we shared already too much. I hope I'm not pissing anyone off but also more than happy to hear any thoughts or different approaches maybe some companies are running whiteboard exercises that before COVID was something that I really liked to do because it meant no preparation and still understanding more about general thinking and asking and so on but also that one can be hard for different personality types if you put them on the spot but yeah, if you have some good thoughts around it, please share them with us. And other than that feel free to follow us on social media and in case you like this episode we always appreciate if you share it with your network and your colleagues and friends. Perfect. Dan, let's continue sweating and have a nice day. Bye-bye.