Models for Evaluating Student Affairs Web Sites

Will Barratt
Indiana State University

Posted May 11, 2001         Student Affairs Online, 2 (Spring)

A good web site is a delight and a poor web site is worse than no web site at all. But, what makes a good web site? This is an evaluation question, and an answerable one. A literature is emerging to address these kinds of evaluation questions (Center for Instructional Technology, Academic Technology and Networks, 2000; Cornell University Libraries, 2001;and Schrock, 1998). While there are many "best practice" statements and models of good and bad web sites (Flanders, 2001), these are generally values driven and developed in the absence of data.

Information Technology (IT) creates asynchronous computer mediated interactions between people. Computer mediated interaction does not contain valuable visual communication cues, in contrast to face-to-face interaction. The loss of these cues can lead to misunderstandings on many levels. The lack of feedback mechanisms, inherent in face-to-face interaction, can severely limit communication. Student affairs professionals can never be sure that students are reading our web pages in the way that we intended, and students can never be sure if we are responding to their needs. Student Affairs is essentially student focused and interpersonal, and our web sites should reflect our core values.

Understanding what makes a good web site is difficult; especially understanding what makes a good web site within a specific campuses institutional and organizational context. Introducing the reader to multiple strategies for enhancing web site quality is the heart of this material. All of the evaluation questions are focused on elements that can be changed to better meet student need.

Using multiple strategies for evaluating your web site will result in a stronger evaluation, and consequently a better web site. While each evaluation strategy can be used alone, the strategies presented here were not designed to stand alone, but to reflect different evaluation questions. Because each campus will have a unique web site, the strategies presented here contain general questions that form the beginning of an evaluation plan. Campus specific questions should be developed and explored.

The strategies presented here are introduced as models. Much as a model car is a miniature representation of a real car, lacking size, detail and the ability to drive, these models are miniatures of full-scale evaluation strategies. To use a full scale evaluation, some assembly is required.

Basic Values For Student Affairs Web Sites

Our work in student affairs is values driven. In evaluating a site, determine the basic values that should be reflected and determine if these are reflected in the site. The following list of basic values are proposed as a useful start in developing a list of values to be used in evaluating student affairs web sites:

Evaluating A Web Site As Campus Ecology

Environmental theory and practice is a strong tool to examine the features of your web site (Banning, Davies & Quick, 2000; Wallace, 2000). Asking what, how, who, when and where questions are important in evaluating a web site. Web pages can succumb to the same analytical techniques as the campus environment.

The questions below reflect a human aggregate and a perceptual model of campus. There are undoubtedly many additional questions that can and should be asked on different campuses.

As an evaluation exercise to learn how students perceive your web site, have five students from a different campus look at all of your pages and write a brief description of your campus based on your web pages.

Web Sites as Student Development Tools

Student development and learning provide wonderful means to examine features of your web site. These questions should be answered as a degree, not as yes-no questions, and specific web features should be associated with student development and learning goals.

Evaluating Web Sites as Learning Tools

Concepts and practices of student learning are effective in evaluating a student affairs web site. Learning is the acquisition of knowledge and skills as the result of experience. The web site can be seen as an experience and as providing information about experiences on campus.

Evaluating Web Site Design and Features

Web sites can be seen as a projection of the institution's values and structure. Most web sites in student affairs use the organizational structure as the web structure. Unfortunately, most students use the organization in different ways. Organizational structure is designed for certain managerial goals; web site structures should be designed with other goals in mind.

Student Evaluation of Student Affairs Web Pages

One method of evaluating a web site is to have a novice user try to find specific information on the site. To do this have a student who does not know your web site try to find specific items starting from the campus home page using only links. If these cannot be found, then the student should use the campus search feature to find them. Watch as the student tries to find the information, and pay attention to how the student uses the site. If you need to answer any questions from the student, the search is a failure, since all information and directions for getting it should be on the web site. 

The student should try to find active information that is subject to regular changes, such as the intramural sports teams and schedule, the name and phone number for the Director of the career center, a photo of the CSAO, and a list of student organizations. The student should also try to find regulatory information that is not subject to regular changes, such as the Student Government Constitution, the campus Code of Student Conduct, and the Student Affairs Mission Statement. Links are an important part of any web site, and the student should try to find links from the student affairs pages to affiliated student affairs sites, professional organizations, and to specific campus academic activities.

Evaluating Web Page Design and Content for Accessibility

Accessibility is a campus requirement and a core student affairs value. Building accessibility and web page accessibility should be of equal concern. There is extensive literature and sets of recommendations for web page accessibility. The Web Accessibility Initiative (WAI) at the World Wide Web Consortium (W3C) provides excellent and thorough information and accessibility guidelines, which are summarized below. A visit to their web site is required reading when evaluating web pages.  

 

Web Accessibility Initiative (WAI) Guidelines that Web Content Developers Must Satisfy.

In General

  • Provide a text equivalent for every non-text element.
  • Ensure that all information conveyed with color is also available without color, for example from context or markup.
  • Clearly identify changes in the natural language of a document's text and any text equivalents (e.g., captions).
  • Organize documents so they may be read without style sheets.
  • Ensure that equivalents for dynamic content are updated when the dynamic content changes.
  • Until user agents allow users to control flickering, avoid causing the screen to flicker.
  • Use the clearest and simplest language appropriate for a site's content.

And if you use images and image maps

  • Provide redundant text links for each active region of a server-side image map.
  • Provide client-side image maps instead of server-side image maps except where the regions cannot be defined with an available geometric shape.

And if you use tables

  • For data tables, identify row and column headers.
  • For data tables that have two or more logical levels of row or column headers, use markup to associate data cells and header cells.

And if you use frames

  • Title each frame to facilitate frame identification and navigation.

And if you use applets and scripts

  • Ensure that pages are usable when scripts, applets, or other programmatic objects are turned off or not supported. If this is not possible, provide equivalent information on an alternative accessible page.

And if you use multimedia

  • Until user agents can automatically read aloud the text equivalent of a visual track, provide an auditory description of the important information of the visual track of a multimedia presentation.
  • For any time-based multimedia presentation (e.g., a movie or animation), synchronize equivalent alternatives (e.g., captions or auditory descriptions of the visual track) with the presentation.

And if all else fails

  • If, after best efforts, you cannot create an accessible page, provide a link to an alternative page that uses W3C technologies, is accessible, has equivalent information (or functionality), and is updated as often as the inaccessible (original) page.

Checklist for Evaluating a Student Affairs Web Site

Web sites can be designed with many things in mind such as stakeholders, expense, color scheme, and many other web features. Four main areas, web site navigation and design, web site technical details, web site aesthetics and web site content will serve as a beginning point for the evaluation process for student affairs web pages. While complicated web site evaluation schemes exist, simple is always a good starting point.

Presented in table form below is a checklist that includes details for each of the four areas and which can serve as a starting point for evaluating a web site.  The point assignments are arbitrary, but serve as a vehicle to summarize the data collection process.

Navigation and Design

 

Points

(4)
(3)
(2)
(1)
(0)

Material should be found within three clicks from the home page and take no more than 30 seconds to load, even on a slow modem.
     

3
clicks
4
clicks
5
clicks
6
clicks
7
clicks

All pages should list a most recent update date, and have been updated within the past 12 months.
     

All
     
Most
      
Some

There should be an appropriate and consistent tool bar/navigation set on pages.
     

All
     
Most
      
Some

It should be very easy is it to find the Student Affairs pages from the school home page.

*Main link - not a pull down menu item.

Home
page
link*
Second
page
link
Menu
item
link
 
No
link

Material should be organized or indexed by function (department).
     

All
depts.
      
Most depts.
      
Some depts.

Material should be organized or indexed by population being served - Information for students, faculty and staff.
     

All
       
2 of 3
      
Stdts.
only

Sum of points in each column
     

       
      
      
       
       

Navigation and Design Points
     

       
 

Technical Details

 

Points

 

(4)
(3)
(2)
(1)
(0)

There should be a search function for the pages or site.
     

Yes

     

      
      
No

There should be a feedback section for pages or site.
     

Yes

     

      
      
No

There should be page counters on all pages.
     

Yes

     

      
      
No

Pages should meet accessibility standards.
     

Yes

     

Most
      
No

Pages should reside on a fast server.
    

Fast

     

Med.

     

Slow

Pages should load quickly at 28.8 baud.
     

Fast

     

Med.
      
Slow

All links should be active.
     

All

          

      

     

Most

Sum of points in each column
     

     

     

      

     

     

Technical Details Points
     

       
 

Aesthetic Appropriateness

 

Points

(4)
(3)
(2)
(1)
(0)

There should be a consistent 'look and feel' within the site.
     

All
       
Many
       
Some

There should be school and division identification throughout the site.
     

All

     

Many

     

Some

The pages should be appropriately attractive.
     

All
      
Many

     

Some

Sum of points in each column
     

      

     

     

     

     

Aesthetic Appropriateness Points

      

Content

Points

(4)
(3)
(2)
(1)
(0)

Information should be provided for all types of current, prospective and alumni students, faculty and staff.
     

All

     

3 of 5
      
1 of 5

Care should be taken to not overemphasize any student sub-population.
     

All

     

Many
      
Some

Student affairs functional areas should be covered broadly.
     

All
      
Many
      
Some

Student affairs functional areas should be covered in depth.
     

Deep
      
Some
       
None

Academic material should be well integrated into the student affairs pages.
     

Fully
      
Some
      
None

Material must be current and accurate.
     

All
       
Many
       
Some

Department or division home page should list contact names, numbers and E-mail addresses for important people, as well as hours of operation.
     

All
      
Many
      
Some

Department or Division Mission is available.
     

Yes
       
      
      
No

Sum of points in each column
          

 
       
      
      
       

Content Points
     

      
      

Total Evaluation Score

      

References

Banning, J. H., Davies, T. G., & Quick, D. G., (2000) The campus web visit, Student Affairs Online,1(Fall). Available at http://www.studentaffairs.com/ejournal/Fall_2000/art3.html

Bloom, B. (1956). Taxonomy of educational objectives. New York, David McKay Company, Inc.

Center for Instructional Technology, Academic Technology and Networks(2000). Evaluating web sites for educational uses: Bibliography and checklist, University of North Carolina, Chapel Hill, NC. Available at: http://www.unc.edu/cit/guides/irg-49.html  

Chickering, A, & Rieser, L. (1993) Education and Identity (2nd Ed.) San Francisco: Jossey-Bass

Cornell University Libraries (2001) Evaluating Web Sites: Criteria and Tools, Available at: http://www.library.cornell.edu/okuref/research/webeval.html 

Flanders, V. (2001) Web Pages That Suck.com. Available at http://www.webpagesthatsuck.com/ 

Schrock, K., (1998) Evaluation of World Wide Web sites: An annotated bibliography, ERIC EDO-IR-98-02 http://ericir.syr.edu/ithome/digests/edoir9802.html

Wallace, H. (2000). Campus ecology theory and websites: One example of applying traditional student affairs theory to technology, Student Affairs Online, 1(Fall). Available at http://www.studentaffairs.com/ejournal/Fall_2000/art4.html