Hiring PMs is hard for a number of reasons and some of these are applicable to other roles as well. In this post I’m going to focus on the evaluation aspect of the PM hiring process and talk about why it’s challenging to evaluate PM skills.
To understand why evaluating a PM is hard, first we’ll briefly look at what a PM does. We are all aware of this famous infographic. I think it’s illegal to write about Product Management and not include this graphic so here it is — A Product Manager’s role lies at the intersection where UX, Tech and Business meet (whatever THAT means! :-))
Then a PM came along and added value to the above infographic:
The jury is still out on this one.
But what do some of the industry experts have to say about Product Managers?
Ben Horowitz, founder of Andreessen Horowitz and a serial entrepreneur, thinks that a Product Manager is akin to being CEO of the product. Try telling that to PMs at organizations with more than 30 people! 🙂
The reality for PMs is often times remarkably different:
(yes, a lot of cleanup is involved!)
Ken Norton, Marty Cagan and Ben Horowitz have some seminal writing on the role of a PM, but some of it is still vague to interpret for a lot of folks. To top it off, various companies have their own flavour of the role, making it more challenging to describe and assess what PMs do. Unlike engineers who write code and produce tangible output, or sales folks who can show $$$ sales done, the PM role is still in the abstract magic category. (Exaggeration intended)
“Product managers need the following four things: the brain of an engineer, the heart of a designer, the tongue of a diplomat and the herding skills of an Australian Shepherd” – Noah Askin, INSEAD (credit)
A quick google search for PM skills will throw up everything from “technical background”, “communication skills”, “execution chops” and “conflict resolution”, to “user centricity”, “data driven”, “business skills” etc.
Simple then: let’s throw hard engineering, design, business and project management problems at them and see if they can solve them all, right? And do it in four to five limited interactions. This is how most interview processes for PMs are designed, in the hope of finding these unicorns. For most organizations and PMs working in the industry, this process remains inefficient and painful. Hiring Managers are finding it hard to create a repeatable and consistent framework for recruiting PMs (it’s subjective!). PMs feel they often get passed on randomly and the feedback they get after the process is very vague.
So to address this, PM hiring process at most organizations has evolved over time to focus more on problem solving. Tasks such as case studies and collaborative brainstorms on a problem statement are good additions to the process. If used well, of course.
Prepared Case Studies
What: Candidates are given a problem statement and asked to prepare a deck addressing the problem covering market assessment, high-level product spec, risks etc.
Why they fall short: This process has a lot of false negatives and biases built in. The problem(s) are often product/industry specific and evaluators have already made up their mind on what they want to see in a proposed solution.
Pro Tip(s):
- Choose a generic product problem or ask the candidate to pick one from a few choices
- Use this as a screening method (pre-interview) as opposed to a qualifying method
On the Spot Problem Solving
What: Similar to case studies, a medium size problem statement is given to a candidate and she is asked to brainstorm the solution with the panel or the interviewer.
Why they fall short: Although thinking on the spot is a good skill to have, you are hiring a PM for her ability to think through problems deeply and not superficially. A lot of smart candidates are not able to do this quickly in an interview setting sans the context, research, thinking and prep that goes into shipping great products. Simulating this in a two to three hour case study is not foolproof.
Pro Tip(s):
- Instead of choosing a neutral problem, pick a project/feature from the candidate’s resume to delve deeper on.
- Look for a) Depth in thought process, b) Pride in their work c) Honesty about their role (it takes a village to ship a good product)
Beer Test
What: Some teams have a Beer Test, often after a candidate has passed hard skills tests, to assess “culture fit”. The question is simple—would it be fun for the interviewer to spend a couple of hours over beer with the candidate?
Why they fall short: This kills diversity in a team. Period. There are several ways to find out value system fit and that’s important. However, tests like the Beer Test just ensure teams hire people who are like them and it’s a sure-shot path to mediocrity.
Pro Tip(s):
Here are a couple of questions more useful than asking “would it be fun to hang out with this person?”
- Would this person bring a different perspective to the team that’s unique and missing today?
- Would spending more time with this person make me think differently and challenge my own assumptions?
So what do I over-index on during the PM hiring process? Keeping aside some hard skills, in my experience if a candidate checks the following three things, there’s a very high likelihood of a good PM hire. They may still sound subjective, but that’s the nature of the role and you can’t completely eliminate subjectivity from any hiring process. Using some pointers below, you’ll be able to make the assessment as objective as possible.
User Centricity/Empathy
The most obvious one. Every organization/team aims to be user-centric. The trick is to devise the right questions to assess whether the candidate understands real life trade-offs of user centricity in a business. Some sample questions to get to the heart of this:
Ask the candidate to design a returns policy for an e-commerce marketplace.
Give the candidate the settings screen of a popular app and ask them to pick default values.
These questions will address how well the candidate thinks about the reality of business constraints w.r.t. user experience. In the e-commerce example, do they come up with a blanket policy that works for all and results in good UX at a prohibitive cost? Or does the candidate think deeply about cohorts and use data/tech to devise smart policies that are different based on the kind of users and their propensity to game the system etc.
In the settings example, it’ll become obvious how they are approaching defaults and whether business comes first or the user.
Curiosity
Again, it’s super important for a PM to be curious and it’s extremely hard for anyone to assess curiosity. Curiosity is important because you want PMs to ask questions, learn continuously, push the boundaries of what’s possible etc. Here are a few ways we judge this:
Check the number of apps they have on their phone. Ask them about apps/products they love that more people should know of.
Ask them about the last book/article/podcast they consumed about PM. What did they learn?
What skills do they have today that they didn’t six months ago?
Ask them to walk through your app/product and ask questions as they go. Are they curious about choices made?
Taste
By taste it’s not to say that your tastes (in music, arts, tech, etc.) should match. By taste I mean if the candidate has a strong point of view and opinion on the aesthetics of things. It tells you they understand and appreciate craftsmanship and will bring the same bar to the products they work on. To gauge this:
Ask them to critique your product. Ask them to critique their favourite product. Ask them for the most delightful and horrible product experiences they’ve had.
So, there you have it. There are no direct answers and it’s an ever-evolving process but with some conscious tweaks and choices, we can all make the PM hiring process a bit more thoughtful and efficient for everyone involved.
[Written by Rahul Ganjoo, Vice President/Head of Product Management at Zomato.]
