EFFECTS OF USING INSTRUCTIONAL TECHNOLOGY
IN ELEMENTARY AND SECONDARY SCHOOLS:
WHAT CONTROLLED EVALUATION STUDIES SAY*

   

James A. Kulik
SRI International, 2003

Last updated October 9, 2006
The Best Evidence Encyclopedia

EDUCATOR’S SUMMARY

What are the impacts of the use of instructional technology on reading, writing, and math achievement in elementary and secondary schools? This review considers studies published since 1990 to find out. The types of technology applications reviewed are as follows.

Reading

  • Integrated Learning Systems (ILS) in Reading: Software programs that provide sequential, usually tutorial reading instruction while keeping extensive records of student progress. Examples: Jostens/Compass Learning and CCC SuccessMaker.
  • Writing to Read (WTR): A kindergarten-first grade program that taught students to read at least in part by teaching writing. This program is no longer disseminated.
  • Accelerated Reader: A supplementary program that helps students make book selections and then tests their understanding of what they have read.

Writing

  • Word Processing: Giving students opportunities to use word processors to improve their creative writing skills.
  • Computer Enrichment: Games, simulations, access to the Internet, and other supplementary applications.
  • Computer Writing Prompts: Word processing programs that provide hints to students on writing mechanics or ideas.
  • Integrated Learning Systems (ILS) in Mathematics: Software programs that provide sequential, usually tutorial mathematics instruction while keeping track of student progress.

Science

  • Computer Tutorials in Science: Tutorial programs that focus on specific science instruction.
  • Computer Simulations in Science: Programs in which students experiment with models of real-world phenomena, such as a frictionless world.
  • Microcomputer-Based Laboratories: Technology that uses electronic sensors to collect data on physical systems for use in laboratory experiments.
For general findings click here
For program ratings click here
For a description of review methods click here
For connections to related reviews click here

General Findings

A total of 61 studies across all subjects and grade levels met the inclusion criteria. Outcomes were summarized as effect sizes (ES), the difference between adjusted experimental and control means divided by the control group standard deviation. Overall conclusions were as follows.

Reading

  • Integrated Learning Systems in Reading: Very small effects (ES= +0.06 across 9 studies)
  • Writing to Read: Clear positive effects in kindergarten (ES= +0.84 across 2 studies) and Grade 1 (ES= +0.40 across 6 studies), and mixed effects beyond Grade 1 (ES= +0.25 across five studies).
  • Accelerated Reader: Mixed but mostly positive effects (ES= +0.43 across 3 experimental studies)

Writing

  • Word Processing: Mixed but mostly positive effects (ES= +0.30 across 4 studies)
  • Computer Enrichment: Mixed but mostly positive effects (ES= +0.34 across six studies)

Mathematics

  • Integrated Learning Systems in Mathematics: Clear positive effects (ES= +0.38 across 16 studies)

Science

  • Microcomputer-Based Laboratories: Mixed effects, no overall trend (ES= +0.01 across 8 very brief studies)
  • Computer Tutorials: Clear positive effects (ES= +0.59 across 6 brief studies)
  • Computer Simulations: Mixed but mostly positive effects (ES= +0.32 across 6 very brief studies)

Program Ratings

Elementary Reading

Moderate
Jostens

Zero
Waterford
Wasach
SuccessMaker
Elementary Mathematics
Moderate
Jostens
Limited
SuccessMaker
Milliken Math
Waterford
Zero
Wasach

Review Methods

Kulik carried out a broad search for technology programs at all grade levels and in all
subjects. He included studies that met the following criteria:

  • Schools using technology programs had to be compared to those receiving traditional instruction, with equivalent pretests
  • Outcome measures had to be measured the same way in experimental and control groups, and could not be measured in a way more likely to register gains in one group than in another
  • Studies had to be free from serious differential attrition (i.e., loss of more subjects from one group than from another)
  • The study report had to have appeared in 1990 or later

Kulik did not provide ratings of individual technology programs, but he did provide sufficient information to permit ratings to be made in two categories in which there were sufficient numbers of studies: integrated learning systems (plus Accelerated Reader) in elementary schools, and ILS in secondary schools. Currently available programs that had been studied in experiments of at least 12 weeks in duration were categorized using the rating system developed by Slavin & Lake (2006)*:

Programs were rated according to the overall strength of the evidence supporting their effects on achievement. “Effect size” (ES) is the proportion of a standard deviation by which a treatment group exceeds a control group. Large studies are those involving a total of at least 10 classes or 250 students. The categories are as follows:

Program Ratings
Strong Evidence of Effectiveness: At least one large or two small randomized studies with median ES= +0.20 or more.
Moderate Evidence of Effectiveness: At least two large or four small studies (randomized and matched) with median ES= +0.20 or more.
Limited Evidence of Effectiveness: At least one qualifying study with median ES=+0.10 or more.
No Impact: Studies show no significant differences
N
No Qualifying Studies: No studies met inclusion standards


*Slavin, R.E., & Lake, C. (2006). Effective programs in elementary mathematics: A best-evidence synthesis. Baltimore, MD: Johns Hopkins University, Center for Data-Driven Reform in Education.

Connections

For another review of research on computer-assisted instruction in elementary mathematics, see Slavin & Lake, 2006*.

*Full report available at http://www.sri.com/policy/csted/reports/sandt/it/Kulik_ITinK-12_Main_Report.pdf

 
     

 

 

Back to Homepage Elementary Math Word Doc Printer Friendly Word Doc Back To Homepage