Doing SaaS Cancellation Interviews (the Jobs-to-be-Done Way)

The other day I saw this great chart by Tomasz Tunguz on churn:

The numbers above are exactly what I’ve seen and heard over the years. Since Bidsketch customers are usually small businesses, we’re constantly working to improve churn.

A while back I improved our cancel feedback by changing the cancellation process.

While this was better than a free-form text field, we still had a big problem: over 50% of our churn was in the Not Using it Enough category.

When we followed up, we found that there were multiple reasons for this, but I felt like we weren’t getting the whole story.

This is where Jobs-to-be-Done comes in.

If you’re not familiar with Jobs-to-be-Done, it’s a framework created by Clayton Christensen which helps you figure out what “jobs” your customers hire your product for.

I’ll admit, when I first heard about Jobs-to-be-Done I thought it was obvious and kinda dumb.

What jobs do my customers hire my products for?

Easy:

• Quickly creating sales proposals
• Closing sales
• Getting new clients
• Eliminating mistakes in proposals
• etc.

But there’s actually a lot more to it than that. What I listed above is all surface stuff without something important: context.

Context is what will give you the real insights.

To get a better understanding of what Jobs-to-be-Done is, check out the milkshake story by Clayton Christensen (5 min video):

Also, when you get a chance, I highly recommend checking out “the mattress interview” with Jason Fried of Basecamp (47 min audio).

My favorite way to implement this framework is by doing “switch interviews.”

I learned how to do these interviews by attending a workshop by Bob Moesta and Chris Spiek. (They help Clayton Christensen implement Jobs-to-be-Done, and do a great job teaching the framework.)

Switch Interviews

What’s so special about switch interviews?

Unlike other types of interviews, you don’t ask people about how they usually do things. You focus on a single story. The story of how they stopped using one product and started using another.

So if you’re trying to find out why people bought, you wouldn’t ask them this question:

“What convinced you to buy today?”

The problem with this question is that you’ll just get surface level stuff. We’ve asked these types of questions and customers usually give us fluffy answers:

“We wanted to streamline our proposal process.”

“Save time on proposals.”

“Spending two hours on proposals, so trying to cut that down.”

While this stuff sounds reasonable, there aren’t any real insights here.

Doing a switch interview you’ll hear stuff like (this is part of a real story uncovered using switch interviews):

“I was traveling and had a big proposal due the next day. My connecting flight was delayed, and by the time I got to my hotel, I was exhausted. There was no way I’d be able to write a detailed proposal from scratch. I did a Google search for starter content and found a blog post that recommended you guys. When I saw your website promising to create proposals in minutes, I just had to try it.”

This is a condensed snippet of the actual story, but even in this small piece we get some key pieces of information besides this customer just wanting to “save time”:

• Next day deadline (does our 14-day trial really matter here?)
• Needed sample content fast
• Guest posting on high ranking sites is a good thing

And this is just from a single interview. Once you have around 10 interviews you’ll start seeing some obvious patterns.

We’ve done over 30 of these types of interviews for customers that switched to Bidsketch. But for some reason, we never did them for people that switched away from Bidsketch.

It wasn’t until I had a conversation with the always helpful Hiten Shah, that I knew I had to do them for cancellations.

We were talking about Jobs-to-be-Done interviews for churn, when he just smiled and said, “Trust me, you want to do those. You’ll learn different things than the normal switch interviews.”

I’m glad I listened. He was 100% right.

We discovered a lot of useful stuff from those interviews, and we’re using what we learned to make big changes in our product and positioning.

What follows is the process we used to recruit interviewees, the questions we asked, and an example of insights learned from one of the actual interviews.

Recruiting People

Switch interviews typically last 45 minutes, so recruiting people can be a little tricky. It’s even harder to recruit someone that just canceled.

If you’ve ever tried to email people that cancel, you know it’s almost impossible to get them to even reply.

So instead we used an incentive and recruited inside the app.

After someone canceled we showed them this page:

You don’t have to use an incentive but I’ve found that it really speeds things up if you do, so I often use them.

The Timeline

Switch interviews focus on the actual story of someone getting fed up with a product enough to switch to something else.

You have to keep in mind that the decision to cancel didn’t take place the day they canceled — it happened weeks, months, or even years before.

So you can’t just get the story of the day they canceled, you need to build the entire timeline of key events that led to the cancellation.

You want to build a timeline that shows the following:

• First Thought – This is when they first had a thought that maybe the product wasn’t working for them. An early struggling moment.
• Event One (Passive Looking) – Something happens here that gets them to start considering alternatives, though probably not actively looking.
• Event Two (Active Looking) – Usually a time-sensitive event that triggers active looking.
• Buying/Cancel Moment – When they evaluate options and actually buy/cancel. Pay attention to tradeoffs.
• Consumption – When and how they consume (product usage).
• Satisfaction – How do they like it? Could it be better?

Seems like a lot of stuff for someone that simply cancels a $29/month app. Does everyone really go through all of this just to cancel a web app?

In my experience, surprisingly, most people actually do go through these stages.

Cheat Sheet

To help guide our interviews we find that there are some questions that we often ask (learned in the switch workshop).

Below is our cheat sheet for these interviews…

Setting the Stage

• Thank them for taking the time and explain that we’re doing these interviews to learn how we can improve Bidsketch. Mention: “We love blunt feedback, so don’t worry about hurting our feelings, it’s not going to happen.”
• Tell them: “We’re going to focus on the story of how you started using Bidsketch, first started struggling or not finding it helpful, and then eventually canceled. So we’re going to ask detailed questions about where you were and what was happening, to just help paint the picture.”

Key Questions

• “Why did you initially sign up for Bidsketch? Did you evaluate other tools?” (This is a fluffy question designed to ask them what they expect so they can focus on what we want to know after.)
• “When was the first time you had the first thought that maybe Bidsketch wasn’t going to work? Or realized that you weren’t using it?”
• “What happened the last time that you used Bidsketch for a proposal?”
• “What happened the first time you had a proposal that you didn’t use Bidsketch for?”
• “Why did you cancel the day that you canceled? Why that day, and not the day before or after?”
• “What are you using now? How did you start to prepare to use the new product (export data, etc.)?”
• What happened the day you started to move to the new product/solution? Why did you start that day and not after?”

Keep in mind that most questions follow the story and ask details about that, so the above questions are mainly used to keep things on track and dig further into things.

A Real Cancel Switch Interview

Let’s make this concrete by digging into one the cancellation switch interviews we did for Bidsketch.

We’ll call this customer Tim.

Here was the feedback we received from Tim when he canceled his account:

Category: Other

Comments: “Not using anymore”

Yikes. Not much to go on here. Why wasn’t he using our product?

Early in the interview, Tim told us why he canceled:

“I don’t do many proposals since they’re mostly for new clients, and I work with a lot of repeat clients. Also, I couldn’t figure out how to get the design to match my brand.”

Seems straightforward, right?

At this point, it’s easy to just go along with what they tell you, but if you want real insight, you’ll have to dig.

We interviewed Tim for 35 minutes and ended up with this final timeline of events:

First Thought	He didn’t need new clients for several months and got his monthly invoice email from us.
Event #1 (Passive Looking)	Saw a well-designed proposal from another designer and realized he needed to update his design. Started a rebranding project to update the look of his website, invoices, contracts, and proposals.
Event #2 (Active Looking)	Finished up rebranding project and now it was time to implement the new design into everything they had. Struggled to implement specific elements of new design in Bidsketch. Searched and found another app but also couldn’t implement what he needed. Two months passed and paid for two accounts. Downgraded to a cheaper plan during this time to save money while finding an alternative. Spent a lot of time implementing new design in MS Word.
Cancel Moment	Was reviewing quarterly expenses and saw the $29 charge and remembered to cancel.

 

Some of our takeaways from this:

• Custom template designs should be shown during onboarding (this would have let him implement his design).
• Frequency wasn’t the issue, making him feel better about his proposal design (compared to other designers) was the actual “job.”
• We should reach out to customers that downgrade, since it may mean they’re getting ready to cancel.
• He churned months before the actual cancellation happened. There were warning signs and ways we could have saved the customer.

Recording and Reviewing Everything

When I do these switch interviews, I record the audio and write/review at a later time.

Each interview gets a document outlining the main stories, takeaways, and pressures at work:

 

You can get that Google Doc here.

I also write up a more detailed version of the switch interview, which is basically a summarized transcript of the interview.

The last thing I do is go through each of the interviews and try to put everything in a spreadsheet to help identify patterns:

End Result

The real goal isn’t just to collect interesting stories about your product, but to get a deep understanding of the jobs customers truly hire (or fire) your product for.

From this, you’ll often come up with some obvious big wins, but other times you might discover that some of the fundamentals of your product need to change.

What was the end result for us? We were able to cut down churn by over 30%.

Reducing churn is great, but that’s just the quick win. Longer term, we’ve got huge changes and opportunities that we’re even more excited about 🙂