William R. Molasso
Counseling, Adult and Higher Education
Northern Illinois University
Posted: November 2005 Student Affairs Online, vol. 6 no. 4 - Summer 2005
In the past few years, one area of emerging technology that is rapidly expanding among student affairs professionals is the use of the Web to collect assessment and evaluation data. Because it is quick, relatively easy to use, and inexpensive, student affairs practitioners and scholars are using Web surveys to provide evidence of program effectiveness, determine student needs and interests, and conduct theory-based research. As accountability in higher education increasingly becomes a regular part of daily campus life, the future success of student affairs professionals will require expanding our own assessment skills to include Web survey administration.
Several articles have appeared in Student Affairs On-Line related to the use of the Web surveys. Upcraft and Wortman (2002) have already briefly outlined some of the advantages and disadvantages of Web based data collection methods. Malaney (2002) reminded us that response rates are equally important in Web surveys as they are in more traditional paper-and-pencil instruments. Chatman (2002) hoped his article “goads the researcher to explore and experiment in the use of this new powerful tool for university surveys of student populations” (para. 1). While many student affairs professionals learned about research methods and traditional survey instrument design in their graduate preparation program or during professional development workshops, some may be asking how they can successfully integrate Web surveys into their current assessment plans. The purpose of this article is to explore several options to have Web surveys developed for student affairs assessment strategies.
Are We Ready for Web Surveys in Student Affairs?
The massive expansion of the Internet and Web based technologies over the last ten years has been well documented. Sills and Song (2002) have identified Web surveys as an important method for public pollsters, social scientists, and government agencies. They believe that the capacity of Web surveys to collect credible and usable data for research studies should be taken seriously.
Collecting data entirely through Web based approaches may not yet be possible in the general population of the United States. However, in subsets with near universal Web access, it is currently effective (Couper, 2001). Crawford, Couper, and Lamias (2001) believe that the “high penetration of Web and Internet usage” among college students makes Web surveys especially popular with this population (p. 146). Computer technologies have already saturated campus communities for such uses as online enrollment for courses; classroom instruction using software such as Blackboard and WebCT; and delivery of grade reports, bills, and other official correspondence via email. Therefore, using the Web to collect assessment data could be an effective method for student affairs professionals.
How Does it Work?
Using survey questionnaires to collect information from a particular population is a regular method of social science research and program evaluation/assessment practice. With traditional paper-and-pencil surveys, the researcher prepares the survey instrument; formats it using a page layout program; duplicates the questionnaire, cover letter and other items to be sent to the sample participants; labels and stuffs the envelopes; mails them via the U.S. Postal Service; and sits back and waits for the completed survey questionnaires to be returned. Web surveys use the same general process, but integrates the use of Web pages and email as a replacement for the more traditional mailed paper-questionnaires. Currently, there are four different ways to collect Web based data—self-created HTML, information technology (IT) departments, stand-alone software packages, and online Web based survey servers.
As personal Web sites have gained in popularity over the last few years, many students and professionals have developed extensive skills in designing their own Web pages. Those with the requisite skill level can develop a Web survey questionnaire from scratch. It should be noted that designing a Web survey in this way requires skills well beyond those needed for designing home pages. Java scripting, database management, security issues, and integration of respondent tracking are a few of the skills needed for self-created HTML Web surveys. Unless you are a seasoned Webmaster with extensive experience in crafting and managing Web surveys, other methods described below may be a better choice. Even if you have a student leader eager to prove his/her skill by creating a Web survey, what happens when he/she graduates and you need to collect data again?
Increasingly, many universities have information technology departments (IT) that provide services in designing and managing Web surveys. This process requires that you give all your questions, lists, and other information to a staff member in IT. The IT staff member then creates and deploys the Web survey. For assessment needs that require more complex survey-design, IT departments have well-developed skills and the necessary knowledge. However, Web surveys created by IT can be both expensive and slow to schedule. Although this method can be useful, care is required in the planning stages to determine the costs and expected turnaround time associated with IT development of the Web survey. IT staff availability for emergencies or changes once the Web survey is deployed should also be considered.
A number of stand-alone software packages are currently available that allow Web surveys to be created and managed. Essentially, the internal features of the stand-alone software package (very similar to those found in Microsoft and Adobe software) can be used to create the Web survey itself. Then, it is exported into a stand-alone Web site for copying to a server or mainframe. Stand-alone Web survey software includes packages such as SurveySolutions and EZSurvey. These programs have become very popular and are worth exploring if your assessment plan calls for a number of individual surveys that are used to collect data frequently. However, there is a steep learning-curve with these programs, and it is important that you fully explore the availability of a server to house the Web survey once it is created. The security of the Web survey and its data can also be a concern.
Online Web Based Survey Servers
A number of companies have capitalized on the need for easy-to-use Web surveys and developed Web sites that you access using your Internet browser. Using your browser (such as Internet Explore, Safari, and others) you access the company’s Web site and create a survey online using the company’s Web pages. Examples of online Web survey companies include www.surveymonkey.com, www.zoomerang.com, and others.
Each of these methods has advantages and disadvantages. If you plan to collect data only once, for one large assessment project, working with your IT Department may be a valuable option. However, if you plan to conduct more extensive assessment projects, it is worth the effort necessary to learn how to create and manage Web surveys on your own. Online Web based survey servers are probably the easiest for beginners to understand and use and are the least expensive in startup costs, regardless of whether you plan to do only one study or many.
Creating a Web Survey
After you select the method for creating the Web based survey, what’s next? In paper-and-pencil surveys, the first step is formatting the survey itself using a page layout program. Web surveys also need to be created and formatted, but instead of using a layout program such as Adobe InDesign or Microsoft Word, you use the Web survey software or service. Using the package, you choose question formats (e.g., multiple-choice with radio buttons, scaled questions, or fill in the blanks) and create the sequence of questions, answers, and pages of the survey. Each package or service will have a different way to accomplish these tasks, but most of these packages (especially online Web survey servers) are actually very simple to use.
After the survey has been designed, the next step is to get the Web survey to the respondents. In paper-and-pencil surveys, staff would prepare a set of address labels for the participants and stuff envelopes with the necessary materials. Web surveys are very similar, except email will invite the sample to participate in the survey. For example, in www.surveymonkey.com, you copy and paste a list of the names and email addresses of the participants in the List Management section of the Web site. Once this information is entered, you create an email message within the company’s Web page, which is sent to the list of people you have selected. This email contains all the information you would like to send to the participants.
The software adds a unique Web link for each person. Participants simply point their mouse to that link and are taken to the first page of the Web survey. The software package keeps track of who has already completed the survey, so that follow-up email invitations are sent only to those participants who have not yet participated. After all responses have been completed, you close the Web survey and download the data into a Microsoft Excel spreadsheet, SPSS, or other format for later analysis.
This description is very general and is intended simply to provide a rough sequence of events when using Web survey mechanisms. Although each software package or server will use its own format and processes, the overall flow is generally the same.
Providing data-based evidence of behaviors, attitudes, and trends will increasingly become important for student affairs professionals as they improve their programs and services, and fight for limited staff, financial resources, and exposure. Applying your existing skills in developing traditional survey instruments and collecting data to Web surveys can assist you in collecting the information you need more quickly, and less expensively. Student affairs professionals have a number of choices to begin exploring Web survey design on their own campuses. The best way to learn about the value of this expanding method is to try!