2. Improving learning: Use technology to improve 'test' scores

Of the ten things I once believed (beliefs I now consider misleading or false), #2 is 'If you want to improve what people learn in a demonstrable way, use technology to improve test scores.'

For decades, skeptics about the value of each new technology have challenged its proponents to show that the use of that technology causes gains in test scores.

Accepting the terms of that question, the proponents of distance learning have boasted that their students score just as well on tests as students in comparable courses on campus: 'no significant difference'. And I remember feeling great when seeing Kulik's meta-analysis of research on computer-aided learning showing that, typically, students using computers learned about 1/3 faster than students who did not use computers.

At first, no one questioned the terms of the question itself: Does technology X (e.g., facilities on a campus) cause better learning than technology Y (e.g. some distance learning infrastructure)?

Do you see the fallacy? Well, consider this version of that same question about the learning impact of a more familiar technology: paper, "Let's measure educational achievement by two sets of courses. One set of courses will use paper. The other group of courses will have no paper. Will the paper-aided learners score higher on exams, on the average, than the paperless learners? How much higher?"

Silly questions. Although paper has valuable uses for learning, sheets of paper don't cause anyone to learn. (Try taping a sheet of paper on your head, and see how much you learn if you wear it there all day.) 'Tell us whether the paper is used for textbooks,' you might insist. Even then, you'd probably hesitate about predicting gains in test scores; you'd want to know how good the textbook was, and whether students actually read it or not. And you'd also have a right to ask how the paperless group was studying.

Well, paper is a technology. A textbook, campus, a computer, and the Web are all technologies, too. None of them 'cause' learning.

Technology is just a tool. Its value for learning lies in what teachers and students do, thanks to their use of that technology: their teaching/learning activities. How much learning results from making a technology available? That depends on the activity and on the circumstances.

I used to talk about two ways that teaching/learning activities could be enhanced by using the right technology:

1. "Help a popular teaching/learning activity occur better, more frequently, or with less effort (.g., using PowerPoint to improve the legibility of a faculty member's notes on the board)" and/or


2. Make a hitherto little-used activity so much easier or richer that the instructor or student changes the course activities themselves. For example, in the 1980s, distant learners rarely communicated with each other. Today, thanks to email, discussion boards, and chat rooms, discussion among distant learners is common, and research suggests that such discussion improves learning outcomes.

Which activities are most likely to improve outcomes? In 1986 Chickering and Gamson answered that question by describing 'seven principles of good practice.' A decade later, Chickering and I wrote a widely-read article in 1996 summarizing how each of those seven principles could be implemented with technology. In recent years, I've greatly expanded those seven sets of suggestions.

(Notice that all this still accepts the basic terms of the original question: 'When trying to demonstrably improve the value of what students learn, the goal should be to improve performance on traditional tests of learning outcomes. That's the only practical, politically feasible way to show that computer use can improve what students learn.'

Well, what do you think of my old belief?

• Have you seen any such gains in test scores resulting from the use of digital technologies such as computers, clickers, portfolios or the web itself? evidence of a lack of such gains? or even lower test scores when digital technologies are used in certain ways in courses?
  
• In your program, has any such evidence ever played a role in budgeting for technology or planning teaching improvement?
  
• Are there other ways in which technology use has improved what students learn in your program? If so, suppose someone challenged you to provide evidence that the student learning had improved and you couldn't cite improvements in test scores. What evidence would you gather instead?

PS. If you'd like to see a table with my ten old beliefs on the left, and my 10 new ones on the right, see: http://bit.ly/ten_things_table