A survey is a method of gathering information through well-designed questions from a group of people with the aim of understanding the population as a whole. Surveys provide data and insights for anyone engaged with people in any capacity.
Surveys at Champlain are subject to the College’s Survey Policy and must be approved by the Office of Institutional Research. Survey projects that are part of a research study may be subject to approval by Champlain’s Institutional Review Board (IRB).
There are many good reasons to gather information through surveys. Surveys can (1) provide a relatively easy and efficient way to gather information; (2) allow you to reach a large audience; (3) provide quantitative, statistically significant, analyzable data; (4) give a voice to, and perhaps evoke discussion among, a group of people; (5) provide a benchmark for comparison; and (6) obtain information not available through other means.
It is important to keep in mind that conducting a survey is doing “research on human subjects,” and is therefore subject to ethical guidelines. When you ask people to take a survey, you are asking for their time and cognitive energy. Therefore, before embarking on a survey journey, it is important to ask yourself if conducting a survey is the best approach to obtaining the information you need.
A survey is not a good idea when:
Alternative ways to collect information include:
If you do decide to go ahead with a survey, you are required to submit a Survey Approval Request Form to the Office of Institutional Research.
Identify purpose and goals
Start by considering the main goal of your survey, how you will gain information from the results (by analyzing the data), and what decisions will be made or actions taken based on the findings. Are the decisions or actions subject to a deadline?
Define participant group
Consider what group of people you want to hear from and how best to reach them. Will you need to advertise or provide incentives to get people to respond? When is a good time to reach out to the participants (and are there inconvenient times)? How will you distribute the survey to potential respondents?
Create a timeline
Items on the timeline could include:
Select survey method
Design questions
Decide what information you will need through analyzing the data. For example, will you need to compare results among different sub-populations or based on answers to specific questions?
Create outreach plan
Distribute survey
Analyze data
Act on findings
Champlain’s Survey Approval Request Form takes you through all the planning steps, as does this post from SurveyPlanet.
Define a clear, attainable purpose for your survey–the reason you are organizing a survey in the first place–which is supported by three to five goals. The goals will help you determine your target audience, how best to distribute the survey, and how to design questions that are relevant and lead to actionable outcomes.
Begin by asking yourself what you hope to learn and why. The purpose is a broad statement of the primary survey aim or outcome. The goals are specific and measurable ways to attain the survey purpose.
Questions to ask yourself include:
The statement of purpose should begin with an action verb, such as describe, explain, explore, identify, investigate, gauge, measure, assess, or test. The purpose and goals will help you provide context and set expectations for your survey audience, which is vital to engaging them with the survey and obtaining valid results.
**What is the survey’s main purpose?
Gauge the effectiveness of a conference through feedback from attendees in order to make improvements for next year.
**What do I want to learn?
How did attendees rate the conference overall?
What parts of the conference did they like best?
What parts of the conference need to be improved?
**What do I need to know about my target population?
How do the responses differ among students, teachers and administrators?
Start with transparency. Explain the purpose of your survey. Tell respondents how survey results will be used to make decisions or take actions. Describe how long the survey will take and what the respondent can expect. Explain how their information will be treated and if it is anonymous or confidential. Let them know if they are eligible for any incentives. Thank them for their time!
Writing good survey questions and organizing them sensibly makes it easier for respondents to answer truthfully and complete the survey. Your aim is to make completing the survey as easy as possible.
For gathering opinions and behaviors, consider what type of rating scale will produce the best, analyzable results from your multiple choice questions.
Important considerations for designing a survey include the following:
Survey Design Considerations (from Qualtrics):
You can find lots of advice online from experts about question types and how to write good survey questions, including from Nielsen, Hubspot, Pollfish, Qualtrics, and SurveyMonkey.
Champlain’s Institutional Review Board (IRB) is the body that reviews research involving human subjects conducted under the auspices of the institution, in order to oversee whether such work is being conducted according to the ethical standards of academic research. The rules governing IRB and human subjects research are controlled by the federal government, regulation 45 CFR 46.
If your research involves interacting with human beings in any significant way, it could qualify as human subjects research. Research in this context is understood to be systematic investigation, including research development, testing, and evaluation designed to develop or contribute to generalizable knowledge. Research methods that entail significant interaction with humans include but are not limited to surveys, interviews, participant observation, focus groups, and experimentation.
If your survey results are to be used for internal institutional purposes, then your project will likely not need IRB review. If your survey is part of a systematic investigation designed to contribute to knowledge in a field of study or a published paper, then your survey may need IRB review. If you are unsure, you can check with the Champlain College IRB Chair at irb@champlain.edu.
Who are you asking to take your survey?
Selecting the target audience is vital to obtaining the actionable insights you are seeking. The target audience is based on a set of demographics, behaviors, or characteristics. Suppose you are surveying students. Do you want to include undergraduate/graduate, traditional/adult, part-time/full-time, residential/commuter, in-person/online? Students who studied abroad, had an internship, were in a club? This is just one example of “audience segmentation.” Targeting the correct, specific population is crucial to the success of your survey project.
It is important as well that the target audience be of sufficient size to allow you to generalize your findings in a way that accounts for diversity and heterogeneity within the population. The sample of respondents needs to be representative of the population’s important characteristics in order to allow for valid inferences from the data. (A representative sample “accurately represents the characteristics of a larger population.”) Consider how publicity and incentives can bolster your survey’s response rate and the likelihood of obtaining results from a representative group.
How are you going to get your survey to the target audience and collect their responses?
There are multiple ways to administer a survey, including one-on-one interviews, mail, and online. Chances are, you will be using an online platform. Self-administered formats include Google Forms as well as freeware such as SurveyMonkey. For a high-stakes survey, you will want to work with the Office of Institutional Research. We will build the survey and administer it for you through Champlain’s licensed Qualtrics account.
This post describes ways to collect data through an online platform.
Whether you are emailing survey links, conducting a survey in class, or posting a QR code to provide survey access, you will want to promote your survey in order to get a good response rate and a representative sample. In promoting your survey, it is helpful to emphasize the survey’s value to the respondent or to something they care about. You can advertise, for example, on social media, The View, digital signage, the Stall Street Journal, the Academic Brief, the People Center Brief, The Crossover, posters, flyers, or verbal announcements (at Faculty Senate or Staff Council). It may be appropriate (for a high-stakes, institutional survey) to send a pre-survey announcement (from a high-placed administrator) by mail or email.
Incentives can improve response rates, but also pose potential negative consequences. Incentives should be large enough to entice the respondent, but not so large as to overshadow the importance of the survey itself, or worse, constitute “undue influence.” A small incentive for each respondent who completes the survey (a $5 gift card to Zime) or a chance to win a larger prize (a $30 Amazon gift card, an Apple watch) through a lottery constitute acceptable incentives.
If incentives are used, promotional materials as well as the survey invitation and reminder communications must describe the prizes, their value, and an estimate of the odds of winning.
Surveys ask respondents for their opinions and personal information. As the initiator and owner of a survey project, you are responsible for making sure your project respects privacy and data security. If you collect personal data, tell survey takers how you plan to use their data and how it will be stored, protected, and potentially destroyed. Sharing your privacy practices with survey takers may help ease their privacy concerns and increase your response rate. Describe your privacy practices as well as the anonymity or confidentiality of the survey results in the survey introduction and in the email inviting people to take your survey.
If the survey is built on the Qualtrics platform, it is secure; “Qualtrics complies with applicable data privacy laws in its role as a data controller of personal data it collects,” and is GDPR compliant. For survey data collected through Google Forms, security depends on the user.
According to Anton Chernikov (Sept 4, 2023):
Google Forms should not be used for collecting sensitive information unless additional security measures are in place. The confidentiality and privacy of your data largely depend on the settings you choose, making user vigilance an essential aspect of data security.
Once you have chosen the platform, you can choose from different options for distributing the survey, for example:
If you are unfamiliar with analyzing quantitative and qualitative data, ask for help from the Office of Institutional Research.
There are four different measurement scales that shape data analysis:
Learn more about Question Scales here.
Data analysis allows you to make inferences from the survey data that can lead to insights or actions. Analysis is done by conducting operations on the data. A good survey–which asks about measurable, observable characteristics, activities and behaviors–will provide analyzable data. You can use statistical analysis to determine if comparisons or trends have statistical significance rather than being due to chance.
Most likely, you will be conducting descriptive analysis, which summarizes and describes the important characteristics of the data set, mainly frequency, central tendency (averages, including mean, median and mode), and dispersion.
Cross-tabulation allows you to break out the data according to sub-groups within the survey population for comparison. For example, you may want to compare an engagement measure between students who live on or off campus. Also known as contingency tabulation, cross-tabulation is subject to tests of statistical significance including chi-square. Cross-tabs can be created in Excel using pivot tables.
You will also want to analyze the textual data collected in any open-response questions. Whereas quantitative data answers the “what,” qualitative data enables you to dig into the “why” and “how.” The most quick and easy approach is to create a word cloud, an image made of words that illustrates the most frequently used terms. More rigorous qualitative analysis methods include:
Once your data analysis is complete, you can share the story. A formal report should include: the survey objective and goals, including the impetus for and story behind the research question; the survey method used, including target audience, data collection and analysis methods; survey respondent descriptive statistics (representativeness); survey results, including statistical findings and takeaways; and a summary that includes important findings and next steps.
Be sure to combine visuals (charts, graphs and tables) with text. You may wish to track how the findings for particular questions have changed over time (trend analysis). Take care to contextualize and not overinterpret data. Do not confuse correlation and causation.
Contact the Office of Institutional Research for survey approval or help with survey design, administration and analysis.