What product marketers need to know about running multi-country user surveys

Surveys are an important avenue for marketers to know about their customers. But often, companies sell products globally and this makes the process of understanding their customers complicated. As someone who has conducted a survey simultaneously in over 10 countries, I know how complex such surveys can be.

So, what are the things you should know when running a multi-country user survey? While there are a number of important factors to consider, product marketers will want to keep a few of those top of mind.

In this article, let’s explore the important ones in detail:

Deciding which countries the surveys will be run in
Localizing the survey questions
Designing survey incentives
Timing and accessibility
Testing the survey before launch

1. Deciding which countries the survey will be run in

If you have customers across the world, it’s not easy to survey them all. So, you might have to decide which countries to include and which to exclude from your survey but you’ll also want your survey to be representative of your customer base. You could use the following aspects to decide the countries:

Customer base

Look at which countries have the most customers. From experience, I can say that 70 - 80% of your customers are likely to be in your top five to 10 countries. As you dive deep and identify those countries, prioritize them.

Revenue contribution

You should also look at which countries are critical from the standpoint of revenue generation. The countries that contribute the most revenue should ideally be included. This will help get data to maintain or increase those revenue streams.

Growth potential

As technology marketers, growing users and revenues is an important part of our job description. So, when you are choosing countries, this is something you should be conscious of.

You should include locations in your survey that may be important for acquiring new users in the future, even if they currently have a low number of users or generate little revenue.

Gathering data on these areas now will provide valuable insights to guide your future strategies for expanding into new markets and attracting more customers.

Representative base

This part is quite difficult. Your top few countries likely belong to either a single geographic region (like Europe) or be of the same level of economic development (like developed economies as defined by the United Nations or World Bank).

If you want your sample of countries to be diverse, you need to select them intentionally to be of different traits and characteristics.

Operational challenges

There are operational and financial aspects to consider as well. For instance, besides being expensive, survey translations will also need to factor in cultural nuances, so that the translated survey reflects local norms.

This large expenditure of time and resources will need to be weighed against whether a country is important enough (in terms of growth potential, revenue contribution, etc.) to be included in the survey. You could choose to do the survey solely in English across all selected countries in order to be able to launch the survey quickly.

To facilitate this, you could even include countries where the main spoken language isn’t English but a majority of your users interact with your website or services in English. You could also screen your users from such countries for their proficiency in English as a first step to surveying them.

2. Localizing the survey questions

In general, surveying a population in its local language not only boosts survey completion rates but also improves the accuracy of the collected data. But translations will also necessarily need to be sensitive to local norms and customs.

For instance, if you simply translate a question that is originally designed for, say, the United States, into Hindi and run it in India, the particular question could be considered invasive or inappropriate by Indian norms.

It can get complicated if you also have to grapple with questions that involve monetary amounts (for instance, a monetary question could be “how much would your users be willing to pay for a proposed product?” – an important part of a marketer’s role).

In this situation, you’ll have to decide whether to show a single US dollar amount to everyone across the world; the alternate options would be to either show a tooltip that readily displays currency conversion rates or, even better, show the survey questions with amounts only displayed in local currency.

The latter option might require you to field the survey using different links for different countries – this will depend entirely on the software you use to run the survey and its functionality. But keep in mind that if you do use different survey links, it could complicate analysis of the final data (as the data likely would get collected separately for each country).

3. Designing survey incentives

Incentivizing the survey would likely help you get the desired level of responses. You could incentivize in a number of ways though typically you will either run a sweepstake (a lucky draw that gives, say, gift cards to a randomly chosen few) or provide a guaranteed incentive (likely a gift card) to everyone who completes the survey.

However, it’s important to understand the legal rules surrounding such incentives – it’s outright illegal in some countries to run sweepstakes. In some others, it’s illegal to give sweepstake prizes in some parts of the country while not in others. And in others, you’ll have to be physically present in the country and conduct the sweepstake.

You could decide to do a combination: sweepstakes in some countries (like the United States) and provide guaranteed incentives in a few others. If there’s no distribution cost, you could also run the survey in some countries without any incentives.

Overall, you need to dive deep, understand the rules and design incentives that are legally permissible and help you in maximizing responses.

4. Timing and accessibility

There are good times to send out surveys and… not so ideal times. Christmas might not be a holiday in some countries, thus opening up a window for you. In some others, there might be local holidays that you are not aware of (like Diwali in India) that will significantly hamper your survey’s response rate. Look out for such festivals when finalizing your survey’s launch date.

When it comes to accessibility, you’ll have to be conscious that in some countries, mobile devices are used more than desktop-based devices. If you do run the survey in a country where mobile penetration is very high, you’ll need to optimize your survey questions so that they can be conveniently accessed on smaller, mobile screens.

This might require reimagining and re-drafting questions that have eight to 10 answer choices displayed left-to-right (horizontally) and will be difficult to read on a mobile device.

5. Testing the survey before launch

While a soft launch to a small population is usually best practice for any survey, it becomes absolutely imperative for a multi-country one. You should test the survey with a small group of individuals while you watch them answer your questions (either in person or over a video call).

Testing live with a very small population will help you check whether the questions are landing as you intended. You should use this to identify questions that your testers find confusingly worded or difficult to comprehend. There might also be questions that have a specific, obvious meaning to you but mean something entirely different to others.

Once you have surveyed a small group, you’re now all set to do a soft launch for a small subset of your intended survey population. This will not help you in identifying whether users comprehend questions but will help you analyze response patterns, whether your survey logic works as intended, and completion rates.

This will alert you to the fact that some countries need more incentives than others, for instance, in order to achieve your desired response rate.

Conclusion

It’s important to note that these are aspects specific to multi-country surveys and are to be considered in addition to all the other things you'd care about (such as sample size) when conducting any survey.

The tips in this article are meant to draw your attention to the important factors that you'll have to consider and act upon when you are running a global survey, and I hope they set you up for success.