|
Higher Education and Student Affairs Professionals' Survey Preferences: A Research Note
Lamont A. Flowers
Distinguished Professor of Educational Leadership
Director, Charles H. Houston Center for the Study of the Black Experience in Education
Clemson University
lflower@clemson.edu
Richard F. Massie
Vice President of Student and Administrative Services
Shawnee Community College
richardm@shawneecc.edu
Posted: February 2006 Student Affairs Online, vol. 7 no. 1 - Summer 2005
Using the Internet to collect and analyze data has become an increasingly popular technique in recent years (Ballard & Prine, 2002; Bandilla, Bosnjak, & Altdorfer, 2003; Dillman, 2000; O'Neil & Penrod, 2001; Sills & Song, 2002). This has led to a fairly sizable body of research that compared the effects of utilizing web-based surveys versus paper-based surveys to collect data (Cook, Heath, & Thompson, 2000). One primary finding that has emerged from research examining the differences in web-based and paper-based surveys is that the response rates are similar and in some cases higher for web-based surveys (Carini, Hayek, Kuh, Kennedy, & Ouimet, 2003; Cook, Heath, & Thompson, 2000). Other benefits have also been observed from using web-based surveys (Roztocki & Morgan, 2002). For example, researchers can disseminate surveys to participants in diverse locations using automated procedures (Sills & Song, 2002). Additionally, the cost of using the Internet to collect data from large samples may be significantly lower than using mail-based surveys to obtain data (Best, Krueger, Hubbard, & Smith, 2001; Schaefer & Dillman, 1998; Weible & Wallace, 1998). Also, using the Internet to collect data has the potential to increase efficiency in the data analysis process with respect to cleaning and coding data for use in data analysis software programs (Berry, 2005; Cronk & West, 2002). While the findings in this line of research are not conclusive (Best, Krueger, Hubbard, & Smith, 2001; Crawford, Couper, & Lamias, 2001; Cronk & West, 2002), a substantial amount of research indicates that there are important benefits to be gained from properly and appropriately using web-based methods to conduct research (Cook, Heath, & Thompson, 2000).
Given the widespread use of computers and the Internet in virtually every aspect of postsecondary education (e.g., course delivery, course evaluations, disseminating information, etc.), it is not surprising that the utilization of the Internet and information technologies have proliferated among higher education and student affairs professionals in recent years. As evidence of this fact, several articles have been written about the need for student affairs professionals to use online technologies to collect and analyze data pertaining to student development issues as well as other areas of student affairs (Molasso, 2005a). Upcraft and Wortman (2000) discussed some of the prominent advantages and disadvantages of using the Internet to collect data in student affairs settings. Chatman (2002) encouraged novice and experienced web-based survey users to consider the comprehensive features of web-surveys in order to maximize the outcomes of the data collection and analysis processes as well as make more informed decisions regarding the specific manner in which web-based surveys could be best utilized for particular purposes. Flowers & Moore (2003) advanced a method for using AOL Instant Messenger to collect qualitative data from students. Molasso (2005b) discussed strategies for increasing the effectiveness of web-based surveys to enhance student response rates.
It is clear that a substantial amount of scholarly literature has explored the use of the Internet for research activities in student affairs. However, despite the research and practical guidance associated with using the Internet to collect data, to date, the literature is incomplete with respect to assessing higher education and student affairs professionals' survey preferences. This important issue is as relevant as discussing the technical issues surrounding the use of web-based technologies to collect and analyze data in postsecondary institutions. As such, the aim of this exploratory study was to determine which mode of survey administration higher education and student affairs professionals at 2-year institutions preferred (web-based surveys or paper-based surveys).
Methods
Data for the present exploratory study were collected as part of a comprehensive study designed to examine higher education and student affairs professionals' perceptions of job satisfaction, employee empowerment, perceived organizational support, and organizational commitment at 2-year institutions. All data were collected via an online survey. Due to the sensitive nature of the questions posed in the survey (e.g., job satisfaction, perceptions of support in the work environment), the survey did not pose questions regarding the name of the institution where the participant worked nor were any items included that could be used to identify the specific location of the institution. While the primary focus of the survey addressed issues of organizational dynamics and administrative behaviors in higher education and student affairs settings at 2-year institutions, one item was included on the survey that specifically explored preferences regarding survey administration procedures.
To obtain participants for the study, a letter discussing the purpose
and goals of the study was distributed on four listservs that were known
to be used by higher education and student affairs professionals at
2-year institutions. The letter was posted to each listserv in March
2005. Four weeks later, a follow-up letter was re-posted on each
listserv. Then, four weeks later, a second follow-up letter was posted
on each listserv. Throughout this period, data were obtained from higher
education and student affairs professionals who agreed to participate in
the study. Overall, the data collection period lasted approximately 14
weeks. A total of 335 participants agreed to participate in the study,
signed (electronically) the informed consent form, and completed the
online survey. The data analysis phase revealed incomplete data as well
as data obtained from individuals at 4-year institutions. Given the
purpose of the comprehensive study, which was to focus on 2-year
institutions, incomplete data as well as data pertaining to 4-year
institutions were not analyzed. Thus, for the present study, complete
data were available for analysis from 284 higher education and student
affairs professionals. Ninety-eight percent of the sample was employed
at public institutions and 2% of the sample was employed at private
institutions. Seventy-three percent of the sample was female and 27% of
the sample was male. Fourteen percent of the sample had a bachelor’s
degree, 58% had a Master’s degree, 13% had a Ph.D., and 15% had another
degree (e.g., Ed.D.).
Results
To assess higher education and student affairs professionals' survey preferences, the following question was asked: Do you prefer to take web-based surveys or paper-and-pencil surveys? Basic descriptive statistical analyses showed that 95% of the higher education and student affairs professionals in the study reported that they preferred taking web-based surveys compared to approximately 5% who preferred taking paper-and-pencil surveys. Table 1 highlights these data.
Table 1:
Percentage distribution of higher education and student affairs professionals' survey preferences (N=284)
| Survey Preferences |
N |
Percentage |
| Web-based survey |
271 |
95 |
| Paper-and-pencil survey |
13 |
5 |
Discussion and Implications
Internet technology has impacted that way in which research is conducted in general and in student affairs settings in particular (Berry, 2005; Couper, 2000; Flowers & Moore, 2003; Mollasso, 2005a). This study provided additional support for this contention. More specifically, an overwhelming percentage (95%) of the higher education and student affairs professionals who participated in our study designed to examine their perceptions of the organizational environment in student affairs settings at 2-year institutions reported that they preferred taking web-based surveys over paper-and-pencil surveys. The findings of this study are also consistent with Roztocki and Morgan's (2002) study, which included 299 researchers within various field of academia (e.g., engineering, business/management, education), who found that researchers overwhelmingly agreed that web surveys were more efficient than paper surveys.
It should be noted that because the survey was administered via the web the results reported in this study may have been biased. Stated differently, it is possible that the particular survey mode we employed might have attracted individuals with a natural tendency and preference for technology, computers, and web-based information. While this consideration is relevant, there is research-based evidence that shows the effects of survey modes are trivial (Ballard & Prine, 2002) especially with participants that are "highly educated" (Bandilla, Bosnjak, & Altdorfer, 2003) and suggests that the survey mode does not influence the responses of survey participants with advanced degrees. In contrast, research also exists that suggests survey respondents answer questions differently on the Internet than on paper, particularly when the questions relate to computers and technological resources (Carini, Hayek, Kuh, Kennedy, & Ouimet, 2003). Because over 70% of our sample had either a master's degree or Ph.D. and given the inconsistency among the findings in this line of research, we are reasonably assured that our results are valid and represent an accurate reflection of the survey preferences among 2-year community college higher education and student affairs professionals. Given the results of this study, in the years to come it seems plausible to suggest that utilizing computer-based resources and related survey procedures will continue in student affairs. However, because this study was small in scope, contains inherent limitations, and represents an initial investigation in this line of research, more research is needed which utilizes larger scientific random samples in order to confirm this finding. Also, more research is needed to explore the effects of using web-based surveys in student affairs at 2-year and 4-year institutions.
References
Ballard, C., & Prine, R. (2002). Citizen perceptions of community policing: Comparing internet and mail survey responses.
Social Science Computer Review, 20, 485-493.
Bandilla, W., Bosnjak, M., & Altdorfer, P. (2003). Survey administration effects?:
A comparison of web-based and traditional written self-administered surveys using the ISSP environment module. Social Science Computer Review, 21, 235-243.
Berry, R. M. (2005). Web-based survey research: Lessons from the University of Akron study. International Journal of Public Administration, 28, 57-72.
Best, S. J., Krueger, B., Hubbard, C., & Smith, A. (2001). An assessment of the generalizability of internet surveys. Social Science Computer Review, 19, 131-145.
Carini, R. M., Hayek, J. C., Kuh, G. D., Kennedy, J. M., & Ouimet, J. A. (2003). College student responses to web and paper surveys: Does mode matter? Research in Higher Education, 44, 1-19.
Chatman, S. (2002). Going beyond the conversion of paper survey forms to web surveys. StudentAffairs Online, 3. Retrieved December 15, 2005, from http://www.studentaffairs.com/ejournal/Winter_2002/surveys.html
Cook, C., Heath, F., & Thompson, R. L. (2000). A meta-analysis of response rates in web- or internet-based surveys. Educational and Psychological Measurement, 60, 821-836.
Couper, M. P. (2000). Web surveys: A review of issues and approaches. Public Opinion Quarterly, 64, 464-494.
Crawford, S. D., Couper, M. P., & Lamias, M. J. (2001). Web surveys: Perceptions of burden. Social Science Computer Review, 19, 146-162.
Cronk, B. C., & West, J. L. (2002). Personality research on the internet: A comparison of web-based and traditional instruments in take-home and in-class settings. Behavior Research Methods, Instruments, & Computers, 34, 177-180.
Dillman, D. A. (2000). Mail and internet surveys: The tailored design method (2nd ed.). New York: John Wiley & Sons.
Flowers, L. A., & Moore, J. L., III. (2003). Conducting qualitative research on-line in student affairs. StudentAffairs Online, 4. Retrieved December 15, 2005, from http://www.studentaffairs.com/ejournal/Winter_2003/research.html
Molasso, W. R. (2005a). Exploring options to create web surveys for student affairs assessment plans. StudentAffairs Online, 6. Retrieved December 15, 2005, from http://www.studentaffairs.com/ejournal/Fall_2005/StudentAffairsAssessmentPlans.html
Molasso, W. R. (2005b). Ten tangible and practical tips to improve student participation in web surveys. StudentAffairs Online, 6. Retrieved December 15, 2005, from http://www.studentaffairs.com/ejournal/Fall_2005/StudentParticipationinWebSurveys.html
O'Neil, K. M., & Penrod, S. D. (2001). Methodological variables in web-based research that may affect results: Sample type, monetary incentives, and personal information. Behavior Research Methods, Instruments, & Computers, 33, 226-233.
Roztocki, N., & Morgan, S. D. (2002). The use of web-based surveys for academic research in the field of engineering. Proceedings of the 2002 American Society of Engineering Management (pp. 33-38), Tampa, FL, October 2-5.
Schaefer, D. R., & Dillman, D. A. (1998). Development of a standard e-mail methodology: Results of an experiment. Public Opinion Quarterly, 62, 378-397.
Sills, S. J., & Song, C. (2002). Innovations in survey research: An application of web-based surveys. Social Science Computer Review, 20, 22-30.
Upcraft, M. L., & Wortman, T. I. (2000). Web-based data collection and assessment in student affairs. StudentAffairs Online, 6. Retrieved December 15, 2005, from http://www.studentaffairs.com/ejournal/Fall_2000/upcraft.htm
Weible, R., & Wallace, J. (1998). Cyber research: The impact of the internet on data collection. Marketing Research, 10(3), 19-24, 31.
Click here for printable version
|
