Mastering Product Discovery
Discover more from The FishmanAF Newsletter
Mastering Product Discovery
What discovery isn’t. What discovery is. And a discovery journey from Patreon.
Hi there, it’s Adam. 🤗 Welcome to my weekly newsletter. I started this newsletter to provide a no-bullshit, guided approach to solving some of the hardest problems for people and companies. That includes Growth, Product, company building and parenting while working. Subscribe and never miss an issue. I’ve got a podcast on fatherhood and startups - check out Startup Dad here. Questions? Ask them here.
Today’s newsletter is sponsored by Segment. A product I’ve used extensively and highly recommend. Being a founder is HARD. Finding product-market-fit is one of the toughest challenges and poses questions like “Are people using my product? Are they returning? How are they using it?” When they started Segment the co-founders didn’t understand the importance of these questions or how to use analytics to answer them. Now they do. Even better: they built a product to help you get there faster.
Segment helps you collect, clean and act on your customer data. You can now build fast with a reliable, performant and compliant stack. Apply now to get $50k in Segment credits!
Product Discovery or “discovery” as I’ll refer to it in this series is a loaded concept. There are lots of competing philosophies around how to do it and how to do it well.
At its core the role of discovery is to understand. Understanding customer problems and opportunity areas, understanding and developing opinions on what to build, and even understanding how to build it. It is rooted in the idea that building a fully scalable product solution (product, feature, service, etc.) is expensive, time consuming and uncertain.
If the majority of product solutions fail to achieve meaningful outcomes then improving your understanding of what important opportunities exist and what solutions will address those opportunities should improve your odds of success. Effective product discovery helps us make better decisions and (eventually) deliver better products and services.
Over the next few newsletters, joined by Behzod Sirjani of Yet Another Studio, I’ll share an overview of discovery practices, quantitative discovery methods, qualitative discovery methods, and how to choose the right tool from your discovery toolkit. I’ll do my best to include several examples that can help you with your discovery about… discovery (see what I did there).
Part 1: My Discovery Journey and Debunking Discovery Myths ← This post
Part 2: Quantitative Discovery Methods
Part 3: Qualitative Discovery Methods
My Journey with Discovery and Debunking Discovery Myths
When I started in product development I approached everything from an experimentation lens. I had spent the earlier part of my career steeped in analytics, rode the wave of A/B testing in the early 2000s, and felt like the only way to make a decision was to have empirical, statistically significant data to support it. I didn’t want to have opinions; I just wanted customers to tell us (through experiments) what worked and didn’t. I had an experimentation hammer and everything looked like a nail.
Then I spent more time in product development and I realized that not everything can be an experiment. There is a real cost to building an experiment, some of the time you don’t have enough data to run one reliably, you still have to generate hypotheses from somewhere and experiments don’t really tell you why something is the way it is.
So I developed a newfound appreciation for qualitative research methodologies. And I’ll admit that when I first heard about “product discovery” I started off thinking that it always meant talking to customers before doing any work. I don’t think this way any more, but I know a lot of other product managers do. I know this because they tell me that it’s hard to balance building products (delivery) with talking to customers all the time (what I used to believe discovery was).
This has been furthered by concepts like continuous discovery and corresponding best-seller books. I have read all of these articles and own the book. They are excellent. What is challenging to many is that most hear “continuous discovery” and think “I must be speaking with customers, users, prospects, etc. constantly.” My theory is that most people equate discovery with generative research. This is not the fault of the authors.
Generative research is important, but it’s just one slice of the pie.
And speaking of pie, Behzod uses eating as a metaphor for discovery:
“I think about it a lot like eating. Even if your body regularly needs fuel, it doesn’t need full meals all the time. When you hear things like ‘oh I need a little bit of course correction,’ I think ‘great go get a snack’ or whatever the snack equivalent is of discovery.”
A Discovery Story
In May 2019 we launched new creator plans at Patreon, including a “good / better / best” pricing model (note: they have since settled on two pricing plans with their recent product launches). One of the features in the “pro” plan was something called Special Offers.
How we arrived at Special Offers is an example of product discovery.
It started with one of Patreon’s core goals: helping creators make money and helping them make more money over time. We followed this goal with a quantitative observation from our data on creators. We observed a plateau in creators’ earnings after an extended period of time; almost like they hit an artificial ceiling. We didn’t know what caused it but we knew it existed.
We thought: what if there are some creators who experience this plateau and then recover and start growing again?
So we ran another quantitative analysis to see if this phenomenon existed. It turned out that for hundreds of creators it did.
Our next step was manual inspection of the list of creators who overcame the earnings plateau. I looked at ~50 creator pages, dug into their posting history, their earnings history, and their promotional strategies to try to figure out some patterns. A handful of patterns emerged from my research. But I needed more information.
Next I commissioned a quick survey of all creators from our original list. It included asking about specific strategies that I had uncovered in my research. The survey responses matched my observations.
This was great, but still not enough information. We now knew what a lot of these creators were doing, but we didn’t know why. And the underlying challenge was to have an impact on a creator’s behavior. In order to understand why creators did what they did I conducted one-on-one interviews with ~6-8 of those creators. My goal was to understand what was going through their head when they were making those changes; what caused them to make the change and what difficulties they experienced when trying to do so.
A lot of what caused creators to make the change was some extrinsic motivation—the loss of a job, changing circumstances, or the desire to reach a certain new goal. But what we uncovered in the conversation was that they had wanted to make changes to their Patreon page for a while but hadn’t really known how or what to do. The extrinsic event gave them a real urgency to figure it out; without it they wouldn’t have. We also learned that there was a real desire to offer their long-time members something special and they felt bad if they offered something to new members only.
With those insights we came up with the concept of Special Offers (internally called “limited time benefits.”) We built the bare minimum experience and helped a few dozen creators run them manually—we had observed some creators attempting to do their own version of this from time to time and we knew it was really complicated with how our product worked. The prototype was a success and we continued learning what creators loved and hated about it. We iterated further and shipped another version of it; then another. Finally we arrived at the product that still exists today.
Goal. Quantitative analysis x 2. Qualitative analysis. User research. Prototyping. Experimenting. Iterating.
We covered a lot of the bases of discovery with this product experience.
So What *IS* Discovery
Discovery follows a continuum from qualitative to quantitative; from deep, embedded ethnographic field research all the way to data analysis and experimentation. You don’t have to use every method in the continuum, but you should be aware of them all.
The snack equivalent of discovery comes in many flavors (okay, I’m done with the metaphor now, I promise). Sometimes you need the quick data snack; sometimes you need the Royal Caribbean Cruise buffet (I mean it this time).
What matters is that you understand the whole toolkit and use the right tool for the job. And that you are also able to gather data in a timeframe that works for you and your team. Discovery is more than just talking, surveying or experimenting with customers. It’s all of those things when the time is right.
Product discovery is like crafting a new recipe. Before you start cooking, you first understand what your guests like to eat, explore different ingredients and flavors, test a small dish to see if they enjoy it, and then refine the recipe based on feedback to ensure the final meal is a hit.
In my next newsletters I’ll unpack the multiple types of discovery methods, provide some templates to support them, and include some other examples of great discovery and the outcomes it leads to.