Lamont A. Flowers
Distinguished Professor of Educational Leadership
Director, Charles H. Houston Center for the Study of the Black Experience in Education
Clemson University
lflower@clemson.edu
Richard F. Massie
Vice President of Student and Administrative Services
Shawnee Community College
richardm@shawneecc.edu
Posted: February 2006 Student Affairs Online, vol. 7 no. 1 - Summer 2005
Using the Internet to collect and analyze data has become an increasingly popular technique in recent years (Ballard & Prine, 2002; Bandilla, Bosnjak, & Altdorfer, 2003; Dillman, 2000; O'Neil & Penrod, 2001; Sills & Song, 2002). This has led to a fairly sizable body of research that compared the effects of utilizing web-based surveys versus paper-based surveys to collect data (Cook, Heath, & Thompson, 2000). One primary finding that has emerged from research examining the differences in web-based and paper-based surveys is that the response rates are similar and in some cases higher for web-based surveys (Carini, Hayek, Kuh, Kennedy, & Ouimet, 2003; Cook, Heath, & Thompson, 2000). Other benefits have also been observed from using web-based surveys (Roztocki & Morgan, 2002). For example, researchers can disseminate surveys to participants in diverse locations using automated procedures (Sills & Song, 2002). Additionally, the cost of using the Internet to collect data from large samples may be significantly lower than using mail-based surveys to obtain data (Best, Krueger, Hubbard, & Smith, 2001; Schaefer & Dillman, 1998; Weible & Wallace, 1998). Also, using the Internet to collect data has the potential to increase efficiency in the data analysis process with respect to cleaning and coding data for use in data analysis software programs (Berry, 2005; Cronk & West, 2002). While the findings in this line of research are not conclusive (Best, Krueger, Hubbard, & Smith, 2001; Crawford, Couper, & Lamias, 2001; Cronk & West, 2002), a substantial amount of research indicates that there are important benefits to be gained from properly and appropriately using web-based methods to conduct research (Cook, Heath, & Thompson, 2000).
Given the widespread use of computers and the Internet in virtually every aspect of postsecondary education (e.g., course delivery, course evaluations, disseminating information, etc.), it is not surprising that the utilization of the Internet and information technologies have proliferated among higher education and student affairs professionals in recent years. As evidence of this fact, several articles have been written about the need for student affairs professionals to use online technologies to collect and analyze data pertaining to student development issues as well as other areas of student affairs (Molasso, 2005a). Upcraft and Wortman (2000) discussed some of the prominent advantages and disadvantages of using the Internet to collect data in student affairs settings. Chatman (2002) encouraged novice and experienced web-based survey users to consider the comprehensive features of web-surveys in order to maximize the outcomes of the data collection and analysis processes as well as make more informed decisions regarding the specific manner in which web-based surveys could be best utilized for particular purposes. Flowers & Moore (2003) advanced a method for using AOL Instant Messenger to collect qualitative data from students. Molasso (2005b) discussed strategies for increasing the effectiveness of web-based surveys to enhance student response rates.
It is clear that a substantial amount of scholarly literature has explored the use of the Internet for research activities in student affairs. However, despite the research and practical guidance associated with using the Internet to collect data, to date, the literature is incomplete with respect to assessing higher education and student affairs professionals' survey preferences. This important issue is as relevant as discussing the technical issues surrounding the use of web-based technologies to collect and analyze data in postsecondary institutions. As such, the aim of this exploratory study was to determine which mode of survey administration higher education and student affairs professionals at 2-year institutions preferred (web-based surveys or paper-based surveys).
Methods
Data for the present exploratory study were collected as part of a comprehensive study designed to examine higher education and student affairs professionals' perceptions of job satisfaction, employee empowerment, perceived organizational support, and organizational commitment at 2-year institutions. All data were collected via an online survey. Due to the sensitive nature of the questions posed in the survey (e.g., job satisfaction, perceptions of support in the work environment), the survey did not pose questions regarding the name of the institution where the participant worked nor were any items included that could be used to identify the specific location of the institution. While the primary focus of the survey addressed issues of organizational dynamics and administrative behaviors in higher education and student affairs settings at 2-year institutions, one item was included on the survey that specifically explored preferences regarding survey administration procedures.
To obtain participants for the study, a letter discussing the purpose
and goals of the study was distributed on four listservs that were known
to be used by higher education and student affairs professionals at
2-year institutions. The letter was posted to each listserv in March
2005. Four weeks later, a follow-up letter was re-posted on each
listserv. Then, four weeks later, a second follow-up letter was posted
on each listserv. Throughout this period, data were obtained from higher
education and student affairs professionals who agreed to participate in
the study. Overall, the data collection period lasted approximately 14
weeks. A total of 335 participants agreed to participate in the study,
signed (electronically) the informed consent form, and completed the
online survey. The data analysis phase revealed incomplete data as well
as data obtained from individuals at 4-year institutions. Given the
purpose of the comprehensive study, which was to focus on 2-year
institutions, incomplete data as well as data pertaining to 4-year
institutions were not analyzed. Thus, for the present study, complete
data were available for analysis from 284 higher education and student
affairs professionals. Ninety-eight percent of the sample was employed
at public institutions and 2% of the sample was employed at private
institutions. Seventy-three percent of the sample was female and 27% of
the sample was male. Fourteen percent of the sample had a bachelor’s
degree, 58% had a Master’s degree, 13% had a Ph.D., and 15% had another
degree (e.g., Ed.D.).
Results
To assess higher education and student affairs professionals' survey preferences, the following question was asked: Do you prefer to take web-based surveys or paper-and-pencil surveys? Basic descriptive statistical analyses showed that 95% of the higher education and student affairs professionals in the study reported that they preferred taking web-based surveys compared to approximately 5% who preferred taking paper-and-pencil surveys. Table 1 highlights these data.
Table 1:
Percentage distribution of higher education and student affairs professionals' survey preferences (N=284)
| Survey Preferences | N | Percentage |
| Web-based survey | 271 | 95 |
| Paper-and-pencil survey | 13 | 5 |