3 Causes of Bad Data in Survey Research

By: Amanda Barna


Survey research is a way to learn about a large population by studying a small random sample of that population.  Survey Research is used for many reasons including reducing a company’s risk of making wrong decisions, providing companies with a “picture” of its clients or customers, and “weighs” the public’s perception of a particular issue.  In order to make data driven decisions, you first have to ensure that the data that you are basing your decisions on is both valid and reliable.

Below are three common sources of bad data and how you can avoid them.

  1. Sampling Bias- A good sample is one that is representative of the population as a whole. For example, if you are doing a survey of Stark County adults and you know that 48% of Stark County adults are women, then a good final sample will be 48% women. Sampling bias is introduced when a sample does not accurately represent the population. An example of sampling bias is undercoverage.  This occurs when some members of the population (like young adults or minorities) are underrepresented in a sample. How to avoid sampling bias: Monitor your sample demographics throughout the data collection process so that you can ensure that the final sample is representative.
  2. Questionnaire flaws- A questionnaire is a series of questions asked to your sample to obtain information about a particular topic. How a questionnaire is designed can have an effect on the quality of data that is collected. There are many, many ways that a questionnaire can be flawed. Some of the most common flaws include questions that are written above or below the knowledge level of the sample, leading or biased questions, double-barreled questions, long questions with a long list of response choices, and questions with response choices that are not mutually exclusive. How to avoid questionnaire flaws: Make sure that your final questionnaire is short, clear and understandable to your target sample. There are many resources available on-line and at the local library on how to write a good survey instrument. Also, it is important to pre-test your survey instrument prior to full implementation.
  3. The Interviewer Effect- The Interviewer Effect is the variation in answers associated with the person conducting the interview. The Interviewer Effect can happen when interviewers do not read questions thoroughly, direct respondents to certain answers, or record answers incorrectly. How to avoid the Interviewer Effect: Make sure that your interviewers are properly trained on the importance of consistency, standardization, and neutrality.

Remember, if you are trying to make data-driven decisions, you must make sure that you are not using bad data. Making decisions using bad data is many times worse than making decisions on no data at all.