This week’s post is about preparing customer discovery interviews. When to run them? How to engage? And what strategies to use during the interview?

I have been prepping to run customer interviews for a new feature. This was a great opportunity to review my previous experience on running custommer interviews and to review the best strategies to tackle the task.

Part art, but also part science

Customer Interviews are one of those part art, part science activities. A big part of running successful customer interviews is having the ability to read the customer and guide the interviewee to provide clarity and validation over the problems, opportunities and desired outcomes. The exercise of running the interviews will be different each time. Different people will likely have different personalities and think about the same problems in different ways. Even if two customers have similar feedback and thoughts about a product area, extracting this information may require completely different approaches.

Even though it is hard to come up with a formula to run successful interviews due to its variable nature, it does help to prepare them. We should understand what success means and work backwards to understand how to get there.

The easiest would be to simply ask what customers want, but often that is not the best path to get to the problems, opportunities and desired outcomes. That is simply the quickest way to get the customers perceived solution.

Some people say, “Give the customers what they want.” But that’s not my approach. Our job is to figure out what they’re going to want before they do. I think Henry Ford once said, “If I’d asked customers what they wanted, they would have told me, ‘A faster horse!’” People don’t know what they want until you show it to them. That’s why I never rely on market research. Our task is to read things that are not yet on the page.
— Steve Jobs, CEO of Apple, in Walter Isaacson’s Steve Jobs

When to run interviews?

If you read Teresa Torres’ book Continuous Discovery, you know probably have already missed your weekly customer interview last week. The book calls for continuous discovery and very frequent interactions with customers.

Like anything that is important, discovery is something that needs to be prioritised. However, to extract the most value I believe it makes sense to adjust the frequency of the discovery to be a factor of the frequency of product iteration and pace of the overall industry.

At a lean startup where product iterations happen all the time and the focus is on validating learnings, customer interviews should happen all the time. It is likely that new information and fresh feedback will be available. In contrast, companies that have long delivery cycles and operate on steady industries are likely to struggle to extract value if they talk to customers every week. Especially in cases of companies with short customer pools.

How to engage?

Customer Proximity: B2B vs B2C

Generally, there are differences between running customer discovery on B2B and B2C businesses. B2B typically has longer sales cycles which requires proximity to the customer. Sales and customer success teams often have existing relationships and can act as intermediaries to schedule the interviews. Even without booking discovery specific meetings, product managers can hop in pre-existing touch points with customers like QBRs, customer success meetings or other periodic operational, support, implementation catchups.

In cases where there is not proximity with direct customers, and when specific 1-2-1 time is needed to run discovery, there are other options:

• Surveys and Polls – you can run surveys and polls on the product directly or on product related webinars;
• Rewarded Interviews – you can go deeper on the survey/poll feedback and offer rewards for 1-2-1 time with customers;
• User Testing Services – you can even engage with user testing companies (like usertesting.com) that have pools of users that are paid to use products and provide feedback.

B2C businesses often face the challenge of lack of proximity with customers. Some times there is not even a way to contact them directly, like web publications for instance.

Another challenge that is more common in B2C business is the difficulty of finding a customer pool sample that represents the universe of users. Especially in services that have a lot of users, it is hard to find customers that think and behave in similar ways to the rest.

Cohort Analysis: Understand your customer before you start

A good way to tackle the problem of user representativeness is to run cohort analysis on the user base before scheduling the interviews. This will inform you of the type of customer you would like to talk to, how you should guide the interview and which customers you should prioritise speaking to.

The are several metrics that can be used to segment users. Interesting cohorts to analyse could include:

• Early adopters
• Highest spending customers
• Large/Medium/Small Companies (in case of B2B)
• Engaged users
• One time users
• Users that are likely to churn

Understanding the user persona will help you guide the interview and allow you to understand the specific needs of the cohort, which may differ from the others. Depending on your objectives and the value of each cohort, you may want to prioritise who to talk to.

What to ask?

Once you have established the engagement with the customer and opened up a communication channel, it is time to define how to conduct the interviewing process and decide what question to ask.

At this point it is important to remember that the goal of doing discovery is to get clarity and validation over the problems, opportunities and desired outcomes of existing or potential customers. This is not the type of information that can be conveyed in a set of yes or no questions. To set expectations, it is better to inform your participant that you would like them to share their full story, to share as many details as possible and to leave nothing out.

You should focus on getting an open ended conversation. The broader the questions, the more likely you will be to unravel new opportunities. Depending on the customer, the stage of the product and the stage of the discovery, you may want to narrow down the interviews as they go. This will allow you to get details around specific features.

The ATS Funnel

When I was prepping for my own customer interviews I had a bunch of aspirational questions I wanted to ask about what were the needs and desires that mattered most to customers, to understand if that could represent opportunities for the company. In addition, I also had some tactical questions about how customers interacted and valued specific features of the platform and how could I delight them. Finally, I was also death curious to know what was their best perceived solution for their problems the “faster horse” from Ford’s quote above.

So I decided to group my questions based on these 3 categories: Aspirational, Tactical and Solution. The world is fertile with product frameworks, why not one more? Let’s call this the Aspirational, Tactical, Solution funnel, or simply ATS funnel.

Aspirational

My aspirational questions had little to do with our existing product. They were mainly focused on the customers perspective. What drives him/her, what are his/her main concerns, what does he/she prioritises. As I was making this exercise for B2B customers it was easy to create questions around the customer’s business motivations:

• What do you wish to drive with your investment in X?
• Tell us about the last time you felt the need of having Y?
• If you had all the information/tools/resources how would action on area Z?
• What would be the first you would change about this industry?

These questions are very open ended and could lead to unexpected insights about the product but also about how companies see their businesses and even the industry. In addition, these questions help you put yourself in the customer shoes and understand their sentiment about their business, industry and market. This is where the gold is, this is what would allow Ford to understand that to disrupt the transportation market, a faster horse would not be enough: you needed something completely unrelated with what existed so far.

Tactical

It is not everyday that you get to reinvent the wheel (or a motorised set of wheels in Ford’s case). Sometimes you just looking to optimise an existing product. You may get interesting insights about a product feature by asking aspirational question, but it is not that likely that you will get all the interesting juicy details of the product experience.

Once you have established a high level understanding of what a customer cares about, you may want to consider drilling down the details that could inform the shaping a concrete solution.

Tactical questions do not necessarily mean narrow questions, they are definitely narrower then aspirational, but they should be broad enough and solution agnostic to let the customer elaborate about how thinks/feels about a product area. Examples of tactical questions would be:

• Do you or your team use functionality X ? Tell us about your experience?
• What are your pain points around area Y? Have you ever run into any issues doing Y on our product?
• Would you like to interact with our product in a different way?
• Do you use other tools or resources to supplement the features of our product?

⚠️ Solution ⚠️

Be careful and proceed with caution on this section.

In some cases, it may make sense to ask customers directly about a given solution. This is the equivalent of asking the customers what they want. Solution questions look like this:

• If X existed do you feel that would be valuable? How do you see using it?

The answer for this question can have problematic consequences. Not only customers may not necessarily know what they need, like in Ford’s case, but also the answer may give product managers a false sense of security. They may think that they will be successful delivering exactly the features requested by the customer, which may not be true.

In addition, solution questions are often the narrowest and by consequence the ones that will have the shortest answers with little context. There is a high chance of misinterpreting the answer or simply to get a vain response that will not necessarily hold through later.

Ultimately, between a fast horse and a car, the car will be the solution that will generate longer term value. Most customers would eventually opt for the car, even if they considered the fast horse to be the best solution before. Customers will make their choices based on the solutions available on the market, not necessarily the opinions they had before having to make a choice.

Having said this, there are some insights to be drawn from solution questions. Hard core users and early adopters are more likely to know exactly what they want and need. In some cases, they may know more about existing market gaps than the companies providing the services. They are also likely to have strong opinions over what can work and what may not.

Solutionising with your early adopters and most loyal customers is also a way to delight them. They will likely feel they are part of the process, have their say in the end product, be willing to participate on beta tests for the MVPs and will be less likely to churn.

The other interesting aspect of solution questions is the reason behind the answer. Remember that even if a customer tells you that he would fancy something, that does not mean you have to do it. You may simply be interested in the story behind the preference.

Let’s say we asked a transportation company in the 19th century what would the best enhancement for their business and the answer was a faster horse, instead of a car/truck. This company could have all their logistics setup around the horse itself: horse-carts instead of trailers, stables instead of garages, horse food instead of fuel. Does this mean the horse is the best solution? I guess we can clearly state in the 21st century, that it is not. Nonetheless, the example is very informative to understand the challenges that transportation companies would have in adopting the car. Car companies could have used this information to thrive in the segment.

Finally, the other interesting result to solution questions is if the answer is: no! “Mr. Customer, we feel like solution X could be a great fit for you, what do you think?” – says the PM. “Mr. Product Manager, thanks for the suggestion, but I don’t think that will work” – replies the customer. If explored this conversation could unravel juicy details about the opportunity and even anticipate future issues on product-market fit.

Going down the ATS funnel

In most cases, especially if it is the first you meet a given customers, it makes sense to start with aspirational questions. As we have seen Aspirational is where the gold is. It will allow to understand your customers motivations and needs.

As you need to get more detailed insights about a given product area, you may chose to go down a level and start drilling into more tactical questions. It may be hard to be tactical with less engaged users or prospective customers, so not all interviews get into a tactical stage.

Finally, solution questions may give the ultimate validation over the implementation of a feature, however as product manager you need be mindful of the pitfalls you can get into, when interpreting the answers. It will not be feasible to ask solution questions to most customers. But if you decide to ask one, you should probably do it towards the end of the discovery to avoid bias answers on other questions.

Bonus: Getting answers without asking questions

If addressing customers is out of question, do not despair. There are still ways to get insights about how users interact with your product. It is not the same as running an interview, but it is probably the next best thing.

We currently have this challenge @ideashortcut. We have no direct relationship with customers, it is hard to get insights outside our close network. Instead, we use Hotjar for to record all user interactions. Hotjar allows you to run surveys, recordings and even heat maps with the user’s interactions with a given page.