Non-linear questionnaires, gamification, social questionning and co-collaboration

Despite the explosion of methods and means for collecting feedback from respondents, many market research surveys still resemble the paper-based surveys of decades ago. Obviously they are still effective, and fine for bread-and-butter or DIY research. However, as audiences become more inured with websites and online marketing, it becomes more important to find ways of keeping (and measuring) attention. Gamification (an awful word) to make the questionnaire fun is one method, but there are also other ways forward.


Non-linear questionnaires

With traditional market research, because samples were hard to obtain, market researchers would typically try to get an answer to every question from all the people who took part in the research. However, because the survey itself forces people to have opinions and so mixes up people with very strong opinions with those who may not have thought about the question or issue before this leads to a 'positivist bias' in market research (not a positive bias which is more related to yea-saying). In particular, those who have weak or non- opinions are likely to give their impression of an area based on hearsay or experiences from long ago, when often the business really needs to know the opinions of those who really have something to say. Even though the answers of different groups might be separated out later by looking at different segments, it's not always possible, so strong opinions can easily be mixed up or neutralised by weak opinions. Worse is that in forcing individuals to give answers to things that, to them, aren't important the questionnaire becomes less relevant, less engaging and boring.

With large sample sizes now available, particularly for consumer surveys, this force-everyone-to-have-an-opinion approach is less necessary. Non-linear questionnaires enable people to answer the questions they want to answer, and to answer the questions in their order they want to respond. From information about the order in which questions were answered and which questions are chosen, inferences can also be made on factor like the importance of different areas to those taking part. In other words, by forcing the collection of less information per individual, more valuable data can be obtained.

Obviously, non-linear questions can be mixed with standard linear questions which must be answered in order, but the benefit, for areas like customer satisfaction, is that if respondents only answer what they want or are most willing to answer, the survey is more relevant to the respondent and we can ask about more things in aggregate, without increasing the work for any individual.

Demonstration of a non-linear questionnaire on SurveyGarden


Gamification

The second direction/trend for improving questionnaires is 'gamification'. This means making tasks more interesting, or making questions themselves more fun to answer. Factors can also include awarding points or bonus incentives for certain tasks or completing extra sections. To make questions or tasks more interesting, a simple method is to frame them creatively: 'If an alien landed on earth and asked about X, what would you say?', or to make the task more engaging for instance drag-and-drop, click-to-rank, timed or animated questions. Obviously this type of questionning has always been encouraged - even for face-to-face surveys before computer-aided interviewing (CAPI) things like shuffle packs and boards were used for encouraging participation. A variety of methods are now available to make surveys more interesting. Unfortunately, many have been developed using Adobe Flash programming language which can cause problems particularly as more devices (eg iPad or tablets) do not have Flash. Our toolkit is Javascript-based providing animation and smart effects (and collecting back-data like timing) and can be tuned and adjusted to create 'tasks' and not just questions.


Social questionning and co-collaboration

The third method to increase engagement is through what we call 'social questionning' - a particular view of the general trend towards co-collaboration (see also virtual communities). That is where the survey, or parts of the survey, or follow-ups to the survey can be reviewed, replayed or commented upon by respondents at a later time. This can be done in a variety of ways. For instance as the survey is running live, once the respondent has given their opinion, we can show results (or a subset of results) and ask for comments, or allow the respondent to go back and change answers - we collect both before and after data. We can show other respondents comments and ask for them to be rated, ranked or additional comments to be given, so making use of the social impact to explore the subject in more creative ways. This can extend to allowing respondents to dip back into the survey results (or subset) at a later date for additional review - eg for business-to-business where data or findings can be part of the reward for taking part.

An extension of this social questionning is to use a social snowball. A snowball sample is one where one respondent is asked to provide contacts to other potential respondents - for instance in minority sports, one volleyball player is likely to know other volleyball players, so to build a convenience sample from a difficult to reach group, snowballing uses social connections to get a broad set of participants. A 'social snowball' is an extension of this, but with the added twist that shared link to the survey includes a snowball ID. Others within the snowball can see the group's results or comments separately from any overall results given (subject to a minimum for anonymity).


For help and advice on carrying out projects requiring innovative or advanced survey designs contact info@dobney.com