1.  Are you asking the right questions?

Take the time upfront to find the right questions and you’ll have a better chance of delivering on your research objectives.

Ask them in the right way by following best practice question design. Think about using simple language, keeping questions and options neutral, avoiding double-barrelled questions and using balanced response scales.

Look for proven questions from industry and national surveys, such as ONS studies or DCMS research. These will have already been tested and offer contextual or comparison data to your research. You might also have questions you can take from past projects – the more you do, the more you build up an understanding of what works.

Work with and involve the right people at the right time to co-design your research project – who will use the data to make decisions and what will they need to do next. Make sure your questions work by testing them with the end user, giving you an opportunity to make those final tweaks. Once it goes live, you’re stuck with it.

2. Are you confident in how your data has been collected?

Once you have some questions, it’s then about building a tool to collect quality data. It goes without saying how important it is to base decisions on accurate data. What the tool looks like will depend on the aims of your research. In this article we’re focusing on online surveys.

There is a wide range of online survey platforms to choose from, but what they can do for your research varies. Whatever you use, get to know the technical features and how they can improve both the quality of your research and, importantly, the quality of experience for your audience. This might involve building routing and logic rules, setting limits on multiple choice questions, considering when to make questions ‘must answer’ or piping fields based on answers to previous questions.

Be careful of poor-quality data sneaking into your sample. For example, ‘speedsters’ who race through a survey or unreliable response patterns such as always selecting the first option in a question. You can use algorithms to quarantine these so you can be confident that you’re working with clean data.

Thinking about the experience of your audience, don’t forget about data privacy. If you’re asking for any kind of personal data in a survey, you need consent from the respondent. You should also make it clear why you’re collecting data and how it will be used. Developing a Privacy Notice for your survey can be a good way to explain all of this.

3. Is your data fit-for-purpose?

“How many responses do we need?” is probably the question we get most at the start of a survey project. But it’s not all about sample size.

Make sure you collect the right data to understand if your sample is representative of the audience you are trying to reach. This might include questions in your survey such as key demographics or how often they visit or use something. But it might also involve using some hidden or url variables in your survey – data that the respondent doesn’t see but that you can use for analysis. An example of this might be tagging the social platform you share the survey on.

When considering a reliable sample size, think about the ability to look at different groups and audiences within it. Have you got enough responses within these groups to run some effective analysis? For example, 1000 responses might sound great until you look at it by age and find it is dominated by one group.

Often when running surveys, you will find certain groups are over- or under-represented. If this is the case, you could look at weighting your data. This is a statistical technique where you apply a weight to your data based on one or more questions. For example, you might weight your data by demographics like age or gender, or key behaviours linked to your research such as visit frequency or website usage. Ultimately, the purpose of weighting is to ensure overrepresented groups in your sample aren’t skewing your findings.

4. Are you making the most of your data?

Your data is in and the temptation is to jump straight into pulling out the key findings and analysis. But the chances are you won’t be getting the full picture. There’s work to do first to get your data into shape so it reveals extra insights and depth to your analysis.

Look at how your data is structured. Just because you asked questions in a certain way, doesn’t mean that’s how they need to be set up for your analysis. Explore how you could group response options or combine responses from different questions to create segments.

The chances are you will have included an open question or two in your survey. At first glance, there’s a lot of information in these and it can be difficult to understand what the patterns and common themes are. You can code comments into key topics or analyse them by another question to break them down.

Most online survey platforms will offer some basic analysis tools. But if you want to get more from your data, consider running it through specialist analysis tools such as SPSS. This will allow you to recode variables, run statistical tests, apply complex filters, cut your data by key questions and more. If that doesn’t feel like something within your reach, get to know pivot tables, filters and lookups in spreadsheets like Excel and Google Sheets.

5. How do you make your analysis meaningful?

You’ve done as much as you can with the data, how do you present it in a way that tells the story of your research and reveals key insights?

Think about different ‘views’ of the data. If you ask the right questions and gather a reliable sample, you should have the luxury of cutting up your data in different ways. For example, you might want to build some motivation groups or home in on what a particular demographic group are saying.

You will probably want to visualise your data in some way. This isn’t just about making it look good, but how you start to bring out the patterns in your data. Make sure you pick the right charts for the right data points. This Chartmaker Directory gives you an idea of just how many different options there are. 

You’ll likely want to report on your data in one way or another too. This might be a written report, summary slides, infographics, online reports or maybe even some kind of interactive dashboard. The way you report should have your intended audience in mind. Not just finding a format that works for them but guiding them through your work and how the analysis you’ve undertaken relates to the research aims.

6. How will you get to action?

You’ve done the thinking, gathered your data, got it into shape and turned it into something that is meaningful. Now it’s about returning to the reason for doing the research in the first place: to make a difference.

Get it in front of the right people. Show them what you are seeing. Use the data to back this up. Did you know that 95% of CEOs feel more confident about a decision if they have the data to back it up? (This is made up, but you get the idea)

Use those key facts and figures to raise awareness of your research and then work with your team to co-analyse the data and understand how it effects what they have to do. Data literacy is one of the biggest skills gaps in the Museum sector but taking time to review, understand and action the data will build those muscles.

It’s not unusual to find your research leads to more questions. Quantitative data doesn’t always give you all the answers, but it will help generate facts that help you to ask better questions.

About Adam

We’re partnering with Adam Pearson from Pearson Insight on Insight For Change our package of research and support to deliver robust insights to the cultural sector.