SURWEB has met and exceeded all objectives related to its original goal of improving education through the innovative uses of communication technologies. The SURWEB evaluation was structured around the original goals proposed by the project in 1995.  The project has addressed all of the original goals to:

  ·                    link resources to standards and encourage authentic assessment

·                    build local capacity for educational technology in a model that values local culture

·                    encourage resource-based teaching and learning through telecommunications

·                    develop resources that are student-centered, integrated, and encourage problem-solving  

·                    to encourage interactive communication with experts and peers in any geographic location 

·                    provide connectivity to enhance opportunities for information access  

·                    provide teacher and student training to strengthen the use of telecommunications in the K-12 classroom  

·                    mobilize local community agencies to use telecommunications to actively participate in local school improvement efforts  

·                    pilot and scale up the project to improve educational technology capacity across the state of Utah

The SURWEB evaluation was designed as a portfolio and organized to gather both qualitative and quantitative evidence between 1995-2000 about Connectivity and Access (Organizational & Consortium Building Outcomes); Developing Curriculum Resources; Professional Development Outcomes; Student Learning Outcomes; and Dissemination/Scaling Up Outcomes.  Complete details and full studies related to SURWEB outcomes are located in the SURWEB Summative Evaluation Report 1995-2000.  Some lessons learned from the evaluation are summarized below.

Connectivity and Access

Connectivity and access were measured by monitoring on-line traffic data, mapping changes in hardware and software; by three studies of teacher uses of technology; and by two studies of students’ uses of technology for learning.  Lessons learned are as follows:

·        Access to Internet computers increased dramatically from 1995-2000 by 66.3% while access to computers with no connectivity declined by 40.6%.  

·        Access to powerful multimedia computers increased to 77.5% in 2000.  

·        Over 88% of teachers spend more than 3 hours per week on-line in 2000, compared with the 19.2% who reported over 3 hours per week on-line in 1995.  

·        The percentages of teachers citing barriers to on-line resources have declined from 1995-2000 in every category.  

·        In 1997, teachers were more likely to believe that the number one barrier to access was staff training, followed by a host of technical issues. In 2000, teachers believe that the major barrier to access is the fact that computers are not located in the classroom.  

·        Very few students report that they have “never used” a computer (0.2% elementary and 2.4% middle school).  

·        Students are most likely to use a computer at home, followed by use in the classroom “by myself”.   

·        Computer labs are still necessary if teachers are to use them for group instruction and therefore need to approximate a one-to-one computer/student ratio.  More use of screen projection units in the classroom might mitigate the necessity of computer labs for some types of group instruction.

Developing Curriculum Resources

The design of SURWEB is related to the goal of school improvement through the use of customized, multimedia curriculum enrichment.   On-going site design is based on demonstrated responsiveness to user feedback gathered on-line, in professional development workshops and in focus groups.  Analysis of the site and synthesis of user feedback was presented in a report in 1999 that contributed to a complete renovation of the site.  In addition, SURWEB user traffic was analyzed to provide insight into the uses of various sections of the SURWEB database.  Lessons learned are as follows: 

·        SURWEB experienced over 14 million user hits from January-October 2000.  

·        SURWEB user hits increased 1300% between 1997 and 2000.  

·        SURWEB page views increased from 1.3 million in 1998 to 5.6 million in 2000.  

·        By the end of 2000, SURWEB will have approximately 200,000 users.  

·        SURWEB users average 24 minutes on-line in over 500 user sessions.  

·        The Utah Education Network server generates approximately 50% of the user traffic on SURWEB.   

·        SURWEB is a virtual archive of over 237 media collections with over 50,000 images available to users.  

·        SURWEB registered over 21,146 users who have created more than 20,922 media shows from 237 different collections of media objects representing over 50,000 multimedia source materials, mostly images 

·        SURWEB contains over 746 public Media Shows built by hundreds of educators and their students.  The “Personal/Private” category refers to creations that owners do not wish to publish and share publicly most of the 20,922 media shows are in this category.  

·        Educators have published 115 Learning Segments and contributed 162 tests to the SURWEB collection and the number is growing.  

·        According to a survey of over 800 students in 2000, SURWEB ranks fourth as the software application most used by students with 48.6% of elementary students and 69.6% of middle school students in the study who say they use it.  Thirty-one percent report using it “daily” or “weekly.”  

·        The use of SURWEB by teachers increased from 15% in 1997 to 77.9% in 2000, although it is important to keep in mind that the 2000 study was composed primarily of SURWEB-using teachers.  

·        The publishing of SURWEB Media Shows builds and sustains a community of learners, offers students a wider audience for publishing and deeps the database of learning products fresh and dynamic.  

·        The publishing of SURWEB Learning Segments ensure that SURWEB content is field-tested; provides an important quality assurance component; and serves as a grass-roots strategy to disseminate and promote the use of SURWEB.

Professional Development Outcomes 

Increase in SURWEB user traffic can be most attributed to professional development activities in the state of Utah. In addition to training teachers and students to use SURWEB, professional development workshops also provide: orientation and skill-building for a host of technology applications; strategies for technology integration across the curriculum; strategies for technology integration of images, sound and moving images in the text-based classroom; and opportunities to network and build a community of learners for SURWEB. Training activities were informed by the analysis of teacher exit surveys, teacher interviews, a feedback area on the SURWEB on-line site, and by workshop observation.  Lessons learned from the SURWEB professional development activities are as follows:

·        Between 1998-2000, one SURWEB professional development coordinator trained at least 14,080 teachers, students, and other educators in professional development workshops.  

·        Over 6866 teachers and 6,742 students have participated in SURWEB training in the state of Utah from 1998-2000.  

·        SURWEB conducted over 436 workshops from 1998-2000 in the state of Utah.  

·        Although SURWEB-using teachers report success in classroom activities that focus on student-centered, hands-on approaches, they report that they spend little time on these tasks.  

·        SURWEB teachers and their students are more likely to use the existing SURWEB archive to create Media Shows than they are to upload or link their own original, customized, multimedia sources.

Student Learning Outcomes

Because of the multiple variables associated with student achievement, learning outcomes are perhaps the most difficult to measure.  It took at least 2 years of the SURWEB project’s time to build the infrastructure that enabled teaching and learning outcomes to be measured.  In addition to site observation and interviews by the evaluator, four studies related to student learning with technology conclude that SURWEB contributes to student achievement.  Two of the studies explored students’ access and use of technologies, including SURWEB.  Two studies conducted at Southern Utah University were devoted entirely to the investigation of SURWEB’s effect on learning.  Lessons learned are as follows: 

·        Seventh grade SURWEB social studies students who create their own Media Shows and who use SURWEB Learning Segments demonstrate significantly higher scores on criterion-referenced, textbook-based tests than do students instructed using traditional methods.  The same students also show positive attitudes toward the SURWEB approach.  

·        SURWEB-users mean score on criterion-referenced, textbook-based tests were 3-25 points higher than were the scores of the control group of non-SURWEB-users.  

·        A study of sixth grade SURWEB-users concludes that students’ creative thinking abilities, problem-solving skills, and the ability to construct complex knowledge structures was enhanced by the use of SURWEB when used in constructivist, learn-centered classroom environments.  

·        High skill levels and access occur at a younger age, with elementary students reporting skill levels in the use of technology hardware and software that is comparable to those reported by high school students in 1999.  

·        In 2000, students report higher skills in the use of multimedia products and sophisticated software than they did in previous studies.  They report skill in programming and creation of digital products.  However, their use of software in school is still narrowly limited to three categories: computer games, word processing and paint/draw programs.  

·        The Internet is most often used by students for research, project-based learning, and to complete classroom assignments.   

·        In spite of high degrees of connectivity and skill, students’ use of email and the design of their own web pages at school is relatively low.  However, the use of networked interactive and programming applications increases with age.  

·        Over 64% of elementary and 76% of middle school students report that they use SURWEB to create Media Shows.  

·        31% of students use SURWEB daily or weekly.  

·        SURWEB-using students report more skill in the use of interactive multimedia applications.  They are more likely to use SURWEB to create Media Shows on the Web than they are to publish their own Web pages with commercial HTML-based software applications.  

Dissemination/Scaling Up Outcomes

Professional development has proven to be a powerful way to disseminate and scale the project.  Professional development activities related to SURWEB have spread through their integration into the general educational technology training opportunities provided by Utah state educational agencies.  All activities related to the building of the infrastructure, curriculum resources, and spread of SURWEB have demonstrated that partnerships and consortia are essential to scale up and dissemination of promising practices.  Initially, the Southeastern Utah Education Service Center partnered with local parks, museums and other agencies to develop resources for the site.  In 2000, SURWEB partnered with the Library of Congress to add images to the collection.  SURWEB also partnered with a total of 18 Challenge Grant recipients in the iMatrix program. The iMatrix program represents an innovative consortium strategy with high potential to sustain the SURWEB mission.

Prepared by  

Kathleen Tyner, Lead Evaluator

Media, Analysis & Practice

San Francisco, CA