Relationship between teaching approach, technology use, and assessment use

You may already know about the 1981 study by Schneider, Klemp and Kastendiek, "The Balancing Act." This well-designed study of faculty in bachelor's degree programs designed for 'returning adults' found that those faculty widely-regarded as excellent had a distinctively different approach to teaching than did their colleagues.

The 'excellent' faculty believed all their students were capable of becoming engaged. Throughout the course, the instructor would keep trying things, watching to see what happened,and modifying their approach-- sometimes for the whole class, sometimes for small groups or individuals -- trying to get everyone engaged and on the road to mastery.

Meanwhile, many of their faculty colleagues had quite different beliefs about the nature of their work as instructors. These 'normal faculty' (my term) believed that some students would do well, while many others would not. The instructor's responsibility was just to teach the content, and assess who was doing well and who was not. Differences in student input (talent, energy, etc.) would determine differences in student outcomes.

There's a chapter in the Flashlight Evaluation Handbook where I post ideas for dissertations and research projects (some of which might be good ideas for grant proposals). I especially hope our subscribing institutions and individual members find these ideas useful. I've just posted some ideas there about implications of the "Balancing Act" model for technology use and attitudes toward assessment. These ideas are explained in detail here.

To sum it up, after suggesting that someone create an easier way to measure faculty beliefs and behavior in ways that relate to the model suggested by "The Balancing Act," I suggest that the 'excellent' faculty are likely to give students technology-enabled assignments that reveal student differences, and help students excel in different ways.

In contrast, I predict that the 'normal' faculty will be found to use technology as a means to present the same instruction to everyone more effectively and efficiently (e.g., PowerPoint, video recordings of lectures, computer-aided instruction and assessment).

Even when the two types of faculty choose the same technology (e.g., student response systems such as clickers), I predict they will use that technology in different ways. The 'excellent' faculty will tend to use clickers in ways that reveal and challenge student thinking. The 'normal' faculty will use clickers to give quizzes, take attendance, and motivate all students to pay attention to readings and what's going on in class.

Similarly, the two groups of faculty will have different reactions to the term 'assessment.'

Let's imagine that, due in part to encouragement from accreditors, an institution begins creating an 'assessment initiative' designed to 'close the loop,' i.e., to assure that faculty and departments get feedback that they need to improve learning, and to document those improvements. The 'excellent' faculty are already accustomed using feedback to guide their own actions and work toward excellence for all students. So, I predict, 'excellent' faculty will initially assume that this assessment initiative is intended to empower them and their departments with information they can use collectively to improve.

In contrast, I predict, the 'normal' faculty at that institution will more often have been using assessment (quizzes, projects, final exams) to separate excellent students from average, mediocre and failing students. When students get back papers from these faculty, the papers will have a grade and perhaps some criticism, but less often will the paper have advice to help the student improve. Therefore, most of these faculty will assume that this assessment initiative is designed to 'grade' faculty or departments, in order to make personnel and budget decisions. Even if the assessment initiative is described as being for the benefit of their students and themselves, they will assume that there is nonetheless a hidden agenda for using assessment to illegitimately control them.

Corollary: there should be some correlation between how faculty who use technology actually use it (what technologies they choose; how they use what they choose) and how those same faculty view assessment initiatives. Faculty use of technology will be a predictor of faculty beliefs about assessment (in circumstances where faculty are technology users and are also confronted with pressure to participate in 'assessment.')

This won't always be true, however. When an institution actually does use assessment for control, or when trust levels are low due to prior betrayals or poor support, everyone has a right to assume that any new assessment initiative is an attempt to exert control over faculty and budgets.

1. What do you think?
2. Have you seen any faculty move from 'normal' to 'excellent' approaches over the years? or from 'excellent' to 'normal?' Under what circumstances?

TLT Vision or Strategic Plan for an Institution: Good Idea?

Steve Gilbert and I did some really exciting consulting for Howard University this past week, helping their TLT Roundtable develop a vision of teaching and learning with technology for the next few years, at the urgent request of Howard's administration for use in the upcoming budget cycle.

I was impressed with the planning work at Howard, which is just getting started. And I think a Roundtable is a good group to develop a vision that a community can embrace, and implement.

But I haven't heard about many multi-year plans to use technology to improve, alter, or
transform teaching and learning in departments, colleges or universities. I'm talking about plans to make relatively specific, programmatic improvements that imply relatively specific technology-related investments (e.g., improving undergraduate research by relying in part on more remote labs, simulations, and ePortfolios; internationalizing a program by relying in part on networking with people and resources in other countries; empowering faculty to improve their teaching through large-scale training in how to find and use free and open source tools). (Planning to improve technology capacity, e.g., changing learning management systems, or expanding enrollment by relying part on technology- these are more common topics for IT plans. But I'm interested in TLT plans, either standalone, or integrated into a larger academic plan or vision).

Budgets are tight, and many institutions feel pressed against the wall. Isn't that exactly the moment to plan how to make changes, year by year, to do these a bit differently? (In our work with Howard so far, we've already developed some new ideas and materials for doing just that.)

Enough preaching. To help us help you, and other institutions, here are two requests.

First, I've created this brief Flashlight Online survey that asks about whether or not your institution or department has any sort of vision or plan for using technology to support and improve teaching and learning over the next few years. (If you don't have a plan, the survey also asks whether that's intentional - 'we don't think planning is appropriate here'). Once you respond, you'll be able to see what others have said (and you can return to the URL as often as you like to see updates. We'll publicize these responses, and the plans you send us. So please tell us all what your institution has been doing. It will take you only 2-3 minutes.

Second, if your institution or department has a vision or plan for how to improve teaching and learning with technology in the next few years that you'd like to share with others, please send it to me, Steve Ehrmann.