Five tips for selecting a win/loss analysis service

Most B2B technology providers seek to learn from their lost deals. They analyze the closed reasons in their CRM, hold monthly or quarterly loss reviews, or both.

But when they need more accurate data and analysis, they consider hiring a win/loss analysis service.

In this article, we take a look at the five things you need to do when choosing this win/loss analysis service:

Know the delivery models: subscription vs clustered
Choose a program you can sustain
Ensure your data store stays usable
Dig into project management capabilities
Focus your due diligence on interview, domain, and business experience

What is a win/loss analysis service?

Win/loss research consultants design and deliver win/loss programs, from buyer outreach to analysis and reporting.

During design, the service should address these four key issues:

Opportunity selection: setting parameters for product or use case, firmographic attributes, consideration set, and recency to ensure the deals are comparable.
Data collection method: varying the method used for data collection at each phase of the program’s lifecycle will optimize efficiency.
Sample size: for an initial win/loss analysis, we recommend at least 20 buyer interviews, but this increases when multiple deal types or buying decisions are included.
Data sought: 'no decisions' are caused by different issues than late-stage competitive losses, for example, so the interview discussion guide or survey questionnaire must be adapted to the situation.

During delivery, the win/loss agency is responsible for:

Outreach: contacting buyers in the selected opportunities to request their participation.
Data collection: gathering the required data.
Analysis: identifying the most important causes of losses and wins is essential for making the analysis actionable.
Reporting: communicating the analysis and recommendations with reports, readouts, and workshops.

As a source of unbiased, fresh data sourced directly from buyers, a win/loss analysis can have a strong, cross-functional impact.

So, the selection of a win/loss analysis service is an important decision.

Tips for choosing a win/loss analysis service

Here are five tips that will help you make a good decision:

1. Know the delivery models

Services for win/loss analysis typically complete the buyer interviews within 1-2 months in a clustered delivery model, or spread them out over 1-2 years in a subscription model.

While both approaches produce accurate data about each deal that’s been interviewed, they have different use cases.

If you plan to use the buyer interviews in loss reviews or sales coaching, then each individual interview is valuable and a subscription model is suitable.

If instead you need to quickly find and fix the issues that are driving wins and losses overall — not just for individual AEs — choose a clustered model.

While a subscription delivery model can also be used to find overall win/loss patterns, that requires waiting several quarters, until a critical mass of interviews has been completed.

Consider as well that that analysis will be a less accurate portrayal of today’s market, because it’s based on data about opportunities that are several quarters old.

2. Choose a program you can sustain

A one-and-done approach to win/loss analysis doesn’t work because technology markets and buyers change too quickly. So, program efficiency is an important consideration. Make sure to compare the cost/benefit of programs over a one or two-year horizon.

While buyer interviews are the gold standard for collecting accurate data, they’re also the hardest to sustain because they’re the most expensive approach.

Happily, sustaining the highest accuracy year-round isn’t necessary in most cases because win/loss analysis has a lifecycle.

While most programs begin with a period of broad change that requires insights that are incontestable, they quickly enter a period of narrow change when we want to benchmark the improvements just made with new buyers.

Do they perceive the improvements? Have we actually improved things, or do we need to iterate?

In the matrix below, we’ve mapped win/loss data sources by accuracy and efficiency.

Qualitative techniques like interviews provide the highest accuracy, but are the least efficient option. They are best suited to periods of broad change. More efficient, but less accurate, quantitative techniques are appropriate during periods of narrow change.

Win/loss data sources mapped by accuracy and efficiency

By using a mixed-method design that alternates between qualitative and quantitative data sources, a win/loss program can optimize this trade-off between accuracy and efficiency.

3. Ensure your data store stays usable

The output of win/loss analysis is a valuable data store about your buyers, competitors, and how they’re evolving over time. This includes: interview transcripts, survey responses, topic indexes, charts and graphs, etc.

Your selection of a win/loss analysis consultant may inadvertently limit the usability of this data store. That’s because the services that provide win/loss analysis use custom software to varying degrees to collect data, analyze it, and report their findings.

If you later need to move this data store - to change consulting services or bring the program in-house - report formatting and functionality may be lost when exported to standard file formats.

Make sure the demos you receive include the export of data and reports so you know how hard it will be to move on, if that becomes necessary later.

4. Dig into project management capabilities

Win/loss analysis requires a choreographed sequence of actions by dozens of people. Internally, product marketing, sales management, sales operations, and AEs are all involved. And, of course, the participation of champions/sponsors from closed deals is essential.

Everyone’s busy, so there’s little room or tolerance for mistakes and do-overs.

As you assess agencies for the win/loss analysis, dig into their project management capabilities. How confident are you they’ll be able to keep things moving and avoid stepping on toes? Have they developed a repeatable process based on learnings from prior projects? What documents and meetings will they use to keep you up-to-speed after the program has begun?

5. Due diligence on interview, domain and business experience

The perceptions your team and your buyers will have of a win/loss analysis is correlated with the knowledge and skill of the consultant you select.

Project management experience chiefly affects the collection of data and everyone participating in that process, as described above. Interview experience and domain experience affect the champions/sponsors who are interviewed or surveyed, and the quality of the data that is collected. Business experience affects the quality of the findings and recommendations and how confidently they are delivered.

Assessing interview experience: To assess interview experience, it’s important to understand how interviewing actually works.

People sometimes think interviewing is just asking a question, getting an answer, and then asking another question and getting an answer. That’s not how it works. It’s actually a lot of follow-up, probing, and improvising new questions in the moment.

But not every question can be asked in an interview of 30 minutes or less. So the interviewer must prioritize questions and moderate the dialogue for the time remaining.

Assessing domain experience: A good interview also requires domain knowledge. If you’re in B2B SaaS, how much experience does the consultant have there? How much experience does she have with your buyer? And how much homework will she do on your category and offering before interviewing your buyers?

Ask what materials the win/loss analysis agency requires from you to prepare. Beware if they don’t ask for a product demo, sales playbook, ICP definition, etc.

Assessing business experience: To assess how well the win/loss consultant leverages her business experience, sample reports are a good starting point. And there are clues in other places.

Do client case studies describe making changes based on the findings and recommendations? Or does it feel like the client got a report and let it gather dust?

Check the words used on review sites like G2. Also, consider the source. Will your team perceive this consultant as trustworthy and credible? As a reliable source of information they should act on? Without holding back for fear of offending, the win/loss consultant needs to foster a desire to use the buyer feedback to improve.

Choosing the best win/loss analysis service for you

Netting it all out, look for a win/loss analysis service that will support you with a program that’s adaptive. Your program does need to be continuous, but it doesn’t need to be inefficient.

The program should adapt - switching between qualitative and quantitative data collection - as your needs evolve. There will be periods when deep dive interviews are needed, and times when a quantitative tool is needed to gather a larger sample and benchmark the improvements you’ve made and monitor the market.

Make sure that custom software used by the research service doesn’t impair the usability of your win/loss data store. And, pay attention to the consultant’s project management capabilities, interview skills, and the quality of the insights she’ll generate.

With these five steps, you should be well positioned to select a vendor whose win/loss findings and recommendations you’ll want everyone to see.

Wanna know more about nailing your win/loss strategy?