Data Integrity Series. Chapter 4: Cracking the Bot Code

In the fast-paced world of online market research surveys, the rise of bots poses a significant challenge. We have talked about what bots are in our previous blog post, Bot Awareness 101. Bots can skew data, compromise the integrity of research, and waste valuable resources.

To ensure the accuracy and reliability of our survey results, we use a range of tools and strategies. Here are our top five tips to spot bots and overcome their implications in online market research surveys.

5 Tips to Spot & Stop Bots in Online Market Research Surveys

1. Implement CAPTCHAs and Human Verification:

One of the most effective ways to weed out bots is to implement CAPTCHAs (Completely Automated Public Turing test to tell Computers and Humans Apart) and other human verification methods. These tools are designed to distinguish between human users and automated scripts and can significantly reduce the likelihood of bot interference. While this may introduce an additional step for respondents, it is a small price to pay for the enhanced data integrity that comes with ensuring human participation.

2. Utilise Geolocation and IP Tracking:

Leveraging geolocation and IP tracking tools can provide valuable insights into the geographic origin of survey responses. Bots often generate responses from the same IP addresses or exhibit a lack of diversity in terms of location. An unusually high concentration of responses from a specific region or if IP addresses seem suspiciously repetitive could be signs of bot interference.

3. Include Honeypots and Trap Questions:

Honeypots and trap questions are strategic tools in online market research surveys to identify and thwart bots. Honeypots involve creating fields that are invisible to humans but bots unwittingly complete, while trap questions deliberately confuse automated scripts. By analyzing responses to these elements, we can quickly spot and flag potential bot activity, and when used alongside other security measures, they provide an effective defense against the infiltration of automated entities in survey data.

4. Analyse Response Patterns:

Some bots can still find a way to bypass the above security measures, gaining access to the survey and completing it. Thus, it’s crucial to know how to identify and remove them from the data set. Thankfully, bots often exhibit distinct response patterns that differ from those of genuine human respondents. By closely monitoring response times, completion rates, and the consistency of answers, we can identify anomalies indicative of bot activity. We also utilise data analytics tools to conduct in-depth analysis and identify patterns that may indicate the presence of bots.

5. Include Open Questions:

Bots typically struggle with providing meaningful and contextually relevant answers to verbatim open-ended questions. Genuine human participants are more likely to offer diverse and thoughtful insights, while bots may produce generic or nonsensical answers. Analysing the content of verbatim responses can be a valuable qualitative tool to complement quantitative data and identify potential instances of automated participation. Similarly, incorporating open numeric questions with a wide range of values can pose a unique challenge for bots. Unlike closed-ended questions with predetermined answer options, numeric questions demand a level of comprehension and mathematical processing that bots are not prepared for as they often lack the ability to interpret the context of numeric inquiries.

Concluding Thoughts

Maintaining the integrity of online market research surveys requires a proactive approach to identify and counter bot interference, as new tactics are constantly emerging. By implementing a combination of technical tools, strategic analysis, and continuous vigilance, we can ensure that the data we collect accurately reflects the sentiments and opinions of genuine human respondents. In the ever-evolving digital landscape, staying ahead of the curve by incorporating the latest advancements in bot detection is not just a best practice but a necessity for reliable and actionable market insights.

In addition to bots, there are other types of respondents that can jeopardise the data’s reliability, including fraudsters, fake respondents, AI-generated respondents, ghost respondents, and even ‘click farms’ and WhatsApp scam groups. Check out this blog post ‘The Evolution of Data Quality Issues in Online Market Research Surveys’ to delve deeper into this worldwide conundrum for market researchers, and check out our ‘Bot Awareness 101’ blog post to learn more about bots.