C. Distance (and campus) learning reconceived (10 things I believe)

My conception of ‘distance learning’ has changed over the last thirty years. My old vision, still held by many folks, was explained in the previous post. To explain why and how I’ve changed my mind, let me tell you about five programs I’ve seen over the years.

As you think about each story, consider that a large fraction of your “distance learning” students may well also take courses on your campus. They’re actually not ‘distant.’ But online learning offers them flexibility to have a part-time or full-time job to cover educational costs, to study when and where they want, to carry heavier loads, and to graduate sooner.

1. Bridging distance to include new students, and teachers: About twenty years ago, my friend Prof Nick Eastmond of Utah State taught an educational technology course to distant learners using audioconferencing. Nick invited me to be a guest lecturer for an upcoming session. “You want me to come to Logan?” I asked. “No, just call this phone number at the beginning of the class hour,” he responded, “and talk with my students about evaluation.” I was startled by what I found when I ‘arrived.’ If the students had all been in a room on campus, listening to my disembodied voice coming from a speaker, I probably would have felt distant. But the students, Nick, and I were just talking together on the phone. It was quite intimate. He invited a different expert from around the country to talk with his students by phone every week.

2. Reaching different kinds of students so all students can learn better: In the early 1990s, Leslie Harris was teaching a composition course at a small college. Students would read articles and then write an essay, summarizing evidence and arguing their own points of view about an issue. His students were reading about family and cultures. But they knew relatively little about families or cultures outside their own small world. In addition, the students shared too many beliefs and preferences. In short, there was little room for a real exchange of facts and views.

So, with a friend teaching a similar course at a dissimilar college, Leslie scheduled a series of online chats. At designated times each week, students from the two colleges would enter the same chat room in order to discuss a topic drawn from shared course readings. The students would also collectively contribute to a listserv discussion between the two classes. These conversations could easily grow impassioned. I recall one student writing in the chat room, “I am punching you in the nose!” That provided plenty of energy and direction to the essays students wrote later.

Although difficult to implement, the idea worked well enough that Harris organized several such pairs of composition courses in the early 1990s. Differences among students can be an asset when those differences can lead to productive dialogue, helping students become more self-aware. Bridging distance can help faculty increase such differences by design.

3. Slowing down the conversation in order to improve learning: During that same period, Karen Smith tried a new technique in foreign language learning at the University of Arizona. About forty students in Spanish IV spent some course time using a computer discussion board instead of going to a language laboratory. They wrote to each other in Spanish about many topics, including some issues they chose for themselves (e.g., planning a class party; advising a fellow student who was depressed). Initially these students used the bulletin board by going to a computer lab at scheduled class times. Soon they realized that they could add entries whenever they had access to a computer and a modem. Their contributions to the online conversation were graded on their fluency, not their grammar.

Meanwhile, many other students were assigned to use word processing to write in Spanish instead. And a third large group continued to use the language laboratory. In other respects, all three groups were taught in the same way.

Outcomes of Karen’s experiment: I visited the course that semester, and it was the most enthusiastic group of students I’ve ever met, before or since. Several said, “This is the first time I’ve actually used Spanish!” Second, they liked the deliberate pace of conversation. They pointed out that face-to-face exchange allows little time for silent thought. However, when using the discussion board, these students had plenty of time to interpret what had just been ‘said,’ and then to thoughtfully compose their replies. Third, they could focus on what to say in Spanish without being distracted by worries about pronunciation.

Amazingly, oral examination revealed these students had learned to converse in Spanish better than students who had trained in the language laboratory. The ‘discussion board’ students had become already adept at thinking in Spanish so that was no problem for them during the oral exam; instead they could focus on pronunciation and speed, and they did well. Karen concluded that these students were attracted to invest more time and energy on the course, which would also help explain their superior performance.

Even for undergraduates, joining any discipline is akin to learning a second language and culture. It’s about learning to communicate with a community, learning its language and adopting its values. How do you teach your students to ‘speak’ math, political science, or art? How do they learn to talk together as they do the work of the discipline? Is face-to-face always ideal? Or, sometimes, would a slower tempo and a little distance help them learn new conversational skills? Could you use those technologies to give each student different conversational partners, such as more advanced students or experts? Do participants in such conversations also need specialized tools? For example, should a participant be able to call up a painting, X-ray, or clip of news footage, and then point to elements of that image or video as they discuss it?

4. Global diversity can be a defining strength for a program: My first three stories were from decades past – their lessons do not depend on glitzy new technology. My fourth story is from today, but it’s still not high tech: five leading institutions on four continents collaborate in offering an executive MBA in global management. The US partner in “OneMBA” is the University of North Carolina, Chapel Hill. Students enroll at one of the five institutions and come to campus monthly for meetings of half their courses. Meanwhile, the other half of the curriculum is taught online, each course led by a faculty team from the participating universities. Students from the five institutions work together in virtual teams. In addition, four times during their program, OneMBA students gather to learn and do research together. Significantly, when they meet in the US, they do not come to Chapel Hill; instead, they work together in Washington DC. The world is the laboratory for the students and faculty in OneMBA. By bridging time and space, they’ve created a program that could never have been offered on campus.

5. Including new learners and more diverse resources while saving money: Several institutions have recently developed laboratory equipment that can be used by undergraduates from a distance. Students can plan and carry out their experiments over the Web, and then download their data for analysis. They are running real, physical experiments, not simulations. Zhejiang University in China has pioneered the use of such online laboratories for distance learning in engineering.

MIT’s iLabs software enables institutions to network their online laboratory equipment together, supporting a multi-institution constellation of laboratories with shared tools for scheduling equipment use, storing student data, and other logistics.

However, iLabs has implications beyond reaching ‘distant learners.’ When using a traditional undergraduate laboratory, twenty or thirty students might share ten experimental stations, equipment used intensively for the hour or two of the laboratory session and then lying idle most of the week. That idle, costly, aging equipment has been one reason why undergraduate laboratories are so rare. In contrast, with iLabs, only one experimental station may be needed for a large group because students can take turns using that station, day and night. In fact, a single iLab may have so much capacity that students from other institutions can use the equipment as well. In fact, students in Africa have been using MIT laboratory, and vice versa, at hours when the host institution makes little use of its own equipment.

MIT’s reorganization of the concept of laboratory learning has implications:
• for access (learners half way around the world; learners on campus doing research at night),
• for the character and quality of education (with iLabs as with the web, the more institutions create iLabs, the greater the variety of laboratory equipment available to all participating institutions), and
• for costs (each station and its laboratory space can be used far more cost-effectively because they are used 24x7; even if institutions share the costs of expensive laboratory equipment, paying part of the cost is less than paying the full cost).

Summing up this essay: the terms “distance learning” and “online learning” suggest to many people that a) all the students are far away, b) the goal is to offer courses that are ‘comparable’ with campus-bound offerings.

But these stories all contradict that stereotype. They each improve upon what was previously possible when campus students were either in a classroom, or studying alone, and using only campus resources.

Second, in each case, the improvements in content, access (who can learn), and methods all resulted from the same technology-supported strategy. Eastmond’s use of audioconferencing enabled experts to bring in updated content, access for distant or busy learners, and tighter community, for example. Smith’s use of online discussion made education more accessible and also powered a far better, more engaging approach to learning Spanish.

Third, all but one of the stories involved hybrid strategy: distance and time were used to improve learning, but those elements were complemented with some kind of face-to-face interaction, sometimes nontraditional (e.g., OneMBA faculty and students doing field research together).

Your institution probably offers at least some online learning courses. It is certainly using technology in some ways to enrich education on campus. I suggest bringing those two efforts closer together. Faculty and support services should work together, using available technology, to help improve courses and programs in all three of these dimensions, simultaneously:
• What: How can we update and improve course content, even just a little?
• Who: How can we help even just a few additional people learn (including people whose differences might be assets for our program)? and
• How can each of these people learn a little more effectively than before?

Be pragmatic. When designing a program or improving a course, start with where you need to start (e.g. we want to serve this group of learners, or we want to make this change in content, or we want to make teaching and learning more effective). But then immediately consider how to use technology so that one change can achieve not only your initial aim but also net gains in the other two dimensions as well.

Things I hope you’ll comment about, below; to the right of 'posted by Steve Ehrmann,' click the word "Comments":
1. Does your institution already have any courses or academic programs whose technology-enabled design improves “what, who, and how” simultaneously?
2. If you're a leader at your institution, what kinds of changes in support services could make such a comprehensive approach work? What roles can evaluative feedback play in guiding such academic improvement so that it does succeed in all three dimensions?

PS If you like this series of posts, 'Ten Things I (no longer) Believe About Transforming Teaching and Learning with Technology' and think others should be debating their value and implications, please spread the word. If you got the word by Twitter, please retweet. Thanks!

3. Teach distant learners as much like f2f as possible (10 things I no longer believe)

Here's what I once believed about distance learning (e.g., telecourses, online learning):

Who learns: Distance learning was for isolated students who lived many miles from campus. Each student worked alone, connected by a slender technological bridge to (some of) the intellectual riches housed on that particular campus.

Teaching methods: The roots of distance teaching lay in old technologies:

• the book (distance learning began when a teacher first said to a student, ‘go away, read this book, think about it, come back, and we’ll talk) and
  
• the auditorium (‘go sit in the twentieth row and take notes while I talk’).

Because those presentational teaching methods were OK on campus, they were also OK for distance learning, we believed. Yes, computer courseware and online discussion provided some enrichment. But distant students learned mostly by reading text, watching lectures (on a screen), and doing homework, just as resident students mostly did. Perhaps that's why hundreds of studies could detect no statistically significant difference in student test scores. It's what students do that mostly determines what students learn. And, if you strip away the appearances, 'distant learners' and 'campus learners' were doing much the same thing. Threat to quality? Nonetheless, some people argued, these new students were isolated from each other, deprived of the faculty member’s personalized attention and unable to use the campus’s most prized resources (its library, laboratories, playing fields, and dormitory bull sessions, for example). Distance learning would necessarily be at best a little inferior, at worst a fraudulent education tempting needy students to take the easy way to an empty credential. Evaluation criteria for this use of technology:

1. Outcomes comparable to those from courses taught on campus?
   
2. Net gains in numbers of students enrolling and graduating?
   
3. Net economic gains? (additional tuition and fees paid by these extra learners; lower costs of teaching students off campus).

About this series of posts: One at a time, we’re discussing ten things I once believed about transforming teaching and learning with technology. The first five beliefs are strategies for using technology: 1. to attract resources, 2. to improve learning outcomes, 3. to increase the number of students enrolling and graduating (this post), 4. to increase revenue while cutting costs, and 5. to make work easier. I post one old belief every Monday. On Wednesday or Thursday of that week, I post what I now believe instead. Later this week, I’ll suggest that the label “distance learning” (or ‘online learning’ when used as a synonym) has become dangerously misleading: most of the students aren't distant. And trying to give them the same kind of education that the campus has provided wastes an exciting opportunity. To see a table that summarizes all ten old and new beliefs, with links to the posts that have appeared so far, see http://bit.ly/ten-things-table.

B. To improve what's learned, change goals. To assess progress, compare apples and oranges (10 things I believe)

 Major revision: December 6, 2009

Belief #2 (of my ten old beliefs) asserted that, to improve what students learn, use technology to change the teaching/learning activities (e.g., active learning, faculty-student contact and the rest of the seven principles of good practice.) And, I believed, the only practical, acceptable way to measure such changes was through changes in 'test' scores (i.e., whatever means faculty had traditionally used to estimate student learning in that program).

Belief #2 isn't false, but I now think it falls way short of describing how to improve what students learn, and how to evaluate how valuable that investment may have been.

So here's Belief B (my new beliefs are lettered A-J): The value of using technology in academic programs usually stems from changing what students learn, not just changing the activities by which they learned the old material.  Obviously new, technology-dependent content has appeared in almost every field (e.g, the widespread use of geographic information systems and other computer databases as tools for scholarly research). Less obvious, perhaps, are changes in traditional goals of liberal learning such as 'writing' or 'research skill.' This chunk of our web site sketches ways in which almost all the basic goals of college education could be reexamined.

Among the important changes in "what" should be learned: Technology uses in the world widen what each person can do, professionally, as a citizen and personally. Their need for wise judgment is therefore greater. And, as they age and the world changes, they will need to learn new skills more frequently. So the academic program should invest more time on student research, creative work, fieldwork, etc. One consequence of this emphasis: more variety of learning, within courses, within majors, and across the institution.

Obviously, when students have more options, there's more chance that they'll flounder or misuse the freedom that they're gradually being given. The faculty response to floundering ought to be to help students gradually take responsibility for their own learning, individually and in teams. (Similarly, faculty and staff will need to collaborate in order for programs to succeed. More on how to achieve such collaboration below).

How to evaluate possible gains from such changes in what students learn: If the goals and content have changed, you can't use scores on last year's test. It probably tested some things you no longer teach, and some of your exciting new stuff isn't on last year's test. What do you do?


My suggestion:

1. Each year, collect both the 'tests' (what students are asked to do in this degree program) and a random sample of their results (what students did on those 'tests') across the courses in the program. This collection of evidence would include exams, student projects, homework assignments, online discussions in courses, and student portfolios, for example. The evidence might come only from senior courses (to gauge program outcomes), or from courses across the curriculum (to help look at student learning as they progress through the program). 
   
2. Commission a set of stakeholders to serve as an evaluation team (e.g., representatives from faculty, benefactors, students, employers, graduate schools)
   
3. Ask your team to critique your evidence, year by year.
   
4. After examining how 'tests' and performance have changed, does your team favor the evolution in what students have been learning? What they see as the strengths and weaknesses of your evolving program? What advice do they offer?

We'll return to this kind of evaluation in section L as we discuss the 'unique uses' approach to assessment and evaluation.

What do you think? Has your program ever looked at evidence about the results of teaching improvement? What do you think of my suggestion? What do you think accreditors would think of this approach? PS. If you'd like to see a table with my ten old beliefs on the left, and my 10 new ones on the right, see: http://bit.ly/ten_things_table Next week: old and new beliefs about distance learning.

2. Improving learning: Use technology to improve 'test' scores

Of the ten things I once believed (beliefs I now consider misleading or false), #2 is 'If you want to improve what people learn in a demonstrable way, use technology to improve test scores.'

For decades, skeptics about the value of each new technology have challenged its proponents to show that the use of that technology causes gains in test scores.

Accepting the terms of that question, the proponents of distance learning have boasted that their students score just as well on tests as students in comparable courses on campus: 'no significant difference'. And I remember feeling great when seeing Kulik's meta-analysis of research on computer-aided learning showing that, typically, students using computers learned about 1/3 faster than students who did not use computers.

At first, no one questioned the terms of the question itself: Does technology X (e.g., facilities on a campus) cause better learning than technology Y (e.g. some distance learning infrastructure)?

Do you see the fallacy? Well, consider this version of that same question about the learning impact of a more familiar technology: paper, "Let's measure educational achievement by two sets of courses. One set of courses will use paper. The other group of courses will have no paper. Will the paper-aided learners score higher on exams, on the average, than the paperless learners? How much higher?"

Silly questions. Although paper has valuable uses for learning, sheets of paper don't cause anyone to learn. (Try taping a sheet of paper on your head, and see how much you learn if you wear it there all day.) 'Tell us whether the paper is used for textbooks,' you might insist. Even then, you'd probably hesitate about predicting gains in test scores; you'd want to know how good the textbook was, and whether students actually read it or not. And you'd also have a right to ask how the paperless group was studying.

Well, paper is a technology. A textbook, campus, a computer, and the Web are all technologies, too. None of them 'cause' learning.

Technology is just a tool. Its value for learning lies in what teachers and students do, thanks to their use of that technology: their teaching/learning activities. How much learning results from making a technology available? That depends on the activity and on the circumstances.

I used to talk about two ways that teaching/learning activities could be enhanced by using the right technology:

1. "Help a popular teaching/learning activity occur better, more frequently, or with less effort (.g., using PowerPoint to improve the legibility of a faculty member's notes on the board)" and/or


2. Make a hitherto little-used activity so much easier or richer that the instructor or student changes the course activities themselves. For example, in the 1980s, distant learners rarely communicated with each other. Today, thanks to email, discussion boards, and chat rooms, discussion among distant learners is common, and research suggests that such discussion improves learning outcomes.

Which activities are most likely to improve outcomes? In 1986 Chickering and Gamson answered that question by describing 'seven principles of good practice.' A decade later, Chickering and I wrote a widely-read article in 1996 summarizing how each of those seven principles could be implemented with technology. In recent years, I've greatly expanded those seven sets of suggestions.

(Notice that all this still accepts the basic terms of the original question: 'When trying to demonstrably improve the value of what students learn, the goal should be to improve performance on traditional tests of learning outcomes. That's the only practical, politically feasible way to show that computer use can improve what students learn.'

Well, what do you think of my old belief?

• Have you seen any such gains in test scores resulting from the use of digital technologies such as computers, clickers, portfolios or the web itself? evidence of a lack of such gains? or even lower test scores when digital technologies are used in certain ways in courses?
  
• In your program, has any such evidence ever played a role in budgeting for technology or planning teaching improvement?
  
• Are there other ways in which technology use has improved what students learn in your program? If so, suppose someone challenged you to provide evidence that the student learning had improved and you couldn't cite improvements in test scores. What evidence would you gather instead?

PS. If you'd like to see a table with my ten old beliefs on the left, and my 10 new ones on the right, see: http://bit.ly/ten_things_table

A. How to use TLT to Attract People & Resources (10 Things I Believe)`

To attract attention, people and money to your program (and to do some other good things), use technology to support important changes in how students learn, what they learn, who can learn, and what it costs for them to learn. This series of posts is about how to do that successfully. Each week, I'll suggest a commonsense suggestion now seems misleading or even false to me; the earlier post this week was about attracting people and resources by spending big bucks on a hot new technology. The second post each week suggests an alternative strategy that is more likely to achieve long term success. Today's counter-suggestion: to attract resources, use technology to help support an emerging academic strength of your program. This strength may depend crucially on some use of technology but will usually not be defined by that technology.

For example, in 1973, Alverno College was a small, obscure Catholic women's college in Milwaukee. That year they began to develop an abilities-focused curriculum. In the decades since, Alverno has become a global leader in this arena. That's one reason the College was recently named one of the nation's top ten institutions for teacher preparation by the George Lucas Educational Foundation and why so many educators and policy makers visit Alverno each year. In early years of this work, Alverno pioneered the use of video recording for assessing student performance (e.g., skills of group work). In recent years, they have developed a distinctively useful electronic portfolio system for their students. They used technology one of many tools to develop their academic strength.

Another example: Over the last decade, Worcester Polytechnic Institute (WPI) has become well known for its popular study abroad program. Thanks in part due adroit use of common technology, most WPI juniors now work on team projects while studying abroad for a term. It took many years for WPI to build the relationships, skill sets, and reputation needed for the program to expand to its current scale. But, for that reason, their lead over competing institutions in this area can't be erased just because another university has bought into a new technology (or built an expensive building, or hired a high profile professor). And WPI's home page today reports that, according to Payscale.com, WPI graduates are in the top 10 nationally for starting salaries.

A third example: In 1969, MIT became one of the first institutions in the country to create an Undergraduate Research Opportunities Program. Computing and the Internet have vastly expanded the variety and importance of these undergraduate projects. Over the years a growing fraction MIT's staff and students have become participants, and UROP has gradually become a major asset to MIT's recruitment.

Technology isn't the star at any of these programs. But each institution has made pragmatic use of technology to continually develop the programmatic strength that has made them so visible and attractive over recent decades.

So focus on how students learn as you think about using technology as a lever to make your program more attractive to students, potential staff, benefactors and others. I stress 'how students learn' because we're talking technology, and technology is a tool for doing things. So what do you want your graduates, students, and faculty to become known for doing? Undergraduate research? 21st century communications skills? teamwork? their response to climate change? ethical behavior? community engagement? Every new year new academic options are opened, and others are enriched, by the increasing range of inexpensive, flexible technologies.

Achieving this kind of academic success will usually a range of improvements across your program: staff skills, curriculum, library and other support resources, relationships outside the institution, and more. It can take many years before your success is real, visible, and widely valued by the world. By that time, this year's hot new technology will have long ago become irrelevant, so it's usually a bad bet to start your planning with such a technology.

Instead, ask yourselves, 'How can we use today's (or yesterday's) technology to take the next step toward distinctive strength for our program?" Consider technology that is easy to learn, inexpensive to maintain, and flexible for use in changing circumstances. Rely only on technologies that are likely to be painless to replace and/or extremely long-lived; if the vendor goes out of business, would your staff or faculty need to develop new materials or skills?

Measure of success for this strategy: is the academic program developing sufficient visibility and reputation to attract people and resources (relative to competing programs), over many years? (But don't expect your admirers and supporters to know exactly what technologies you're currently using!)

QUESTION: Have you seen degree programs or institutions that have used technologies as key ingredients to enhance distinctive educational strengths (what their students do as they learn, and what their students can do after graduation)? What lessons do you learn from how they achieved that technology-enabled success?

PS Next week: what I once believed, and what I now believe, about how to use technology to improve learning outcomes, and what kind of evaluation is most likely to reveal the value (or lack of value) of those changes in learning.

1. Be the First to Buy Hot New Technology (#1 of Ten Things I No Longer Believe)

(Revised November 22, 2009)

This series of posts begins with five goals for programmatic change, each of which can be advanced by some use of technology (e.g., digital technologies, new buildings, academic resources -- in this series we focus mainly on the digital technologies but the logic will often apply to other facilities as well.) Those five goals are using technology to:

1. Attract outside attention, good people (students and faculty), and money to the program
   
2. Improve learning outcomes (results)
3. Enroll more (and more kinds of) students
4. Save money
5. Save staff time
   

We begin with using technology to attract attention, good people and money to your program. The old strategy for doing so: Be the first. Your program should leap to spend big bucks on the hot new technology that everyone has just started talking about. Be #1 before your competitors can claim that honor. Use the purchase to identify your program with that technology in order to make your program more visible and thereby attract good people (students and staff) and gain financially (grants, gifts, discounts from the vendor of that technology).

For example, promise that every new student or staff member will get popular technology X, or have access to service Y. Or, if technology Z is in hand, publicize how many of your people or courses use it. Cite the amount of money spent on the technology to dramatize your program's commitment. Use the name of the technology or its vendor in conjunction with the name of your program in recruitment and grant proposals (e.g., "We are the Z University").

Obviously, programs also hope to use their technology somehow for educational gains. But what distinguishes this strategy from the four I'll describe in coming weeks is the hope that the acquisition of the technology will itself lead to visibility and thereby to material gains.

More often than not, this strategy seems to be associated with an educational theory that might be labeled, "Technology is magic." Advocates might simply assert, "This technology is so exciting in its potential and is easy enough to use that faculty and students are bound to embrace it and do any number of good things with it!!!"

HOW TO EVALUATE THIS STRATEGY'S SUCCESS: Did your program actually achieve lasting gains in visibility and thereby material gains, relative to its competitors? (Each of these posts will include brief notes about how to evaluate its strategy.) That's strategy #1.

Think back five or more years ago; do you know of an academic program that has followed this strategy? Did they achieve the results they sought? To write about it (and feel free to leave the program anonymous), click the 'comment' button below and respond to any of the following questions:

• Were there gains in visibility, staffing or money? Did those gains diminish quickly as some competitors waited for prices to drop and then bought the same technology? Did other competitors wait until the buzz about technology X faded, and then buy hot new technology Y?
  
• Worse, were their efforts to use technology X for programmatic improvement sabotaged by technological change itself? For example, if hot new technology X was great for visualization, two years later did the program's innovative staff shift focus to a newer technology Y, a technology that promised a very different educational improvement? Did each of these promises whip by so quickly, each new one distracting attention from the last new one, so that none were ever fully realized?
  
• Once the program bought into the new technology, was it difficult to find enough qualified support staff who also understood education well enough to help staff learn to use it effectively for teaching?

In most programs I've seen over the last thirty years, these big investments in a hot new technology often failed to produce the material or educational gains that been the topic of such much hype a few years earlier. My next post in this series suggests an alternative way to use technology to increase a program's visibility and thereby to attract people and resources: a strategy much more likely to produce lasting gains.

Ten Things I (no longer) Believe About Transforming Teaching and Learning with Technology: Introduction

(revised November 23, 2009)

Working on a curriculum committee? planning a new academic building? are you a chief information officer? director of a teaching center? Welcome to this discussion!

I've been writing a series of posts about how to improve (or even transform) college education.  I'm 60 now, and I've begun to realize that I've changed my mind about a lot of things I used to believe in that area.

I first heard about an imminent technology revolution in higher education when I was a senior in high school.  Initially the change was to affect how people learned and what they learned (e.g., programming). Later the hopes also grew to include who could learn (e.g., distance learning) and a reduction in the costs of instruction.

I heard more about this coming transformation when I was in college, in graduate school, as a program officer for the Fund for the Improvement of Postsecondary Education (1978-85), as a senior program officer with the Annenberg/CPB Projects (1985-96), at the American Association for Higher Education (1996-97), and since Steve Gilbert and I founded the Teaching, Learning, and Technology Group (1998- ), a non-profit that supports over 100 colleges and universities around the world.

The buzz continues, continually changing its details. The catalyst would be mainframe engineering analysis and design programs. It would be simulations, computer-aided instruction, microcomputers, computer-aided design, videodisc, email, distributed laboratories, hypertext, Gopher servers, math tools, the Web, asynchronous learning networks, national learning infrastructure, streaming video, ePortfolios, eLearning, blended courses, course redesign, iPods, clickers, lecture capture, smart phones, digital cameras, social networking,...

Each new technology changed important elements of the direction -- students learning to program in BASIC on microcomputers would learn to think logically and creatively for themselves, videodisc would make education more visual and interactive, HyperCard would trigger a wave of hypertextual, interdisciplinary thinking...

There are common threads to this rapidly flickering vision, ideas that I've heard repeatedly over the decades:

• Education will shift the student's role from passive to active. 
• Education will become more self-paced. 
• Many new kinds of students will be involved. 
• As more of the explaining function of education becomes embodied in technological materials, the faculty role will shift from always being the 'sage on the stage' toward spending more of their time coaching and managing the learning process("a guide on the side"). 
• Although the upfront costs would be large, there will be cost savings, too. 
• And the exponential improvement in the power and efficiency of computer chips will help assure that the wave of change that we could already see would soon accelerate. 

Those are frequently repeated hopes.

THE BAD NEWS

Since 1966, most predictions I've heard for technology-driven, paradigm-shifting change in the "how" of learning have not come true - not pervasively, not yet.

For example, most of today's students still seem to do most of their learning through what instructors and textbooks explain to them, for example. The students' job is still mainly to take notes, preparing to verify on the test that they have understood what they have been told. (Do you agree that this was, and is, the norm in colleges? If not, what was the norm 20-40 years ago? has the norm changed?). I think the norm has changed a little, but not in the ways most people predicted years ago.

Please add your own observations (click the 'comment' button below.)

What claims for technology-driven change in learning did you hear, 5-10 years ago or more?

What did you think that academics would need to do in order to achieve that improvement in learning on a programmatic scale?

Have any of those hoped-for improvements in learning actually happened across your program?
Which predicted changes in learning have not yet occurred on that scale? Why not?

• Were those goals for changes in learning actually unattainable or over-valued?
• Were there problems with the proposed strategy for achieving those changes?
• Was the new technology of the day inadequate to the task?

Link to: The first thing I no longer believe: to attract attention, people and resources, be the first to buy into the hottest new technology.

Priorities for TLTG Online Symposium 2009 - Fac/Pro Dev/Support

Priorities for Faculty/Professional Development/Support Online Symposium

August, Sept, 2009, the TLT Group, Inc.
Frugal Innovations: Faculty Roles and Programmatic Support
What should we focus on first, next?

---

Include Exclude 1st results from similar online poll 8/4

This Web Page: http://preview.tinyurl.com/TLTG-FrugalPriorities09

---

---

Try to Include topics, issues, questions, activities, challenges, strategies - In what order?

We seek participants' input for determining the
order in which we address some of the following in this 2009 Symposium. We will also welcome
polite responses to "How could you have been so stupid as to omit
XXXYYYZZZZZ, which should be one of the top priorities for this
series?"

• Cope with $ Cuts
  How to cope with current budget cuts just for now? or forever? Planning/hoping for return to "normalcy"?

• Info for Decisions
  What kinds of information or data will really effect related decisions (by whom?) in the next 12 months?

• Nanovation, Extermissions, & Milli-Everetts
  See also: "Milli-Everetts - Smallest Essential Steps for Incremental Exponential Education Revolution: Nanovation?"
  Nanovation = Intentional use of new resources for "sharing forward" improvements
  and innovations in ways that multiply impact (like "viral marketing,"
  - morally and intellectually responsible version of "Ponzi
  scheme"/Pyramid/Chain Letter; Extermission = Intentional, informal outreach - used during TLT Group's online
  sessions about Twitter in July, 2009.
  Milli-Everett = The smallest effort required to enable and encourage at least one teacher to at least:
  

• Make one improvement in one course,
  
• Get a little feedback and improve that improvement,
  
• Help two at least colleagues each make similar improvements - and each help at least two more colleagues ....

• Mid-Level Roles
  Role of mid-level academic professionals (dept
  chairs, division leaders, deans, …)in success/failure of dissemination, innovation, ...
• Gmail Decision
  Tools that appear to have great potential for
  teaching/learning and become even more accessible when all students and
  others have Gmail accounts - implications for budget, policy, operations, teaching/learning
  
• Using Un-Owned Resources
  Web 2.0 -
  the difference between "free beer and a free puppy"; who is
  responsible for what kind of support and usage policies, standards when
  the college or university cannot own or control the resource? - implications for budget, policy, operations, teaching/learning
• Handheld Devices
  Love em or leave em? Use in courses or forbid in classroom? Clickers, PDAs, smart phones, netbooks, audio & video recorders - - implications for budget, policy, operations, teaching/learning
• Back-Channel Communication
  Constructive "back-channel" communication - anonymous or not,
  sanctioned or not!
  
• Brief Hybrids
  Taking advantage of these compact combinations of plans, media, activities, references, to make small low-risk, low-cost course improvements: Dipping a toe in the shallow, quiet end of the raging technology sea!
• Textbooks: Evolution or Extinction?
  Emerging alternatives and variations. How
  changes in technology are both impeding and facilitating the
  opportunities for students to take their own notes and engage actively
  with readings and other learning resources.
  The myth of standardized resources for independent learning.

• Fundamental Paradox of Faculty Development
  Recognize and try to reconcile these two approaches:
  A. Rational: emphasizes data, assessment, "scalable" strategies
  B. Personal: emphasizes trust, individual differences, relationships and growth
  
  Address these provocative-but-useful questions:
  Why is Faculty Development a low institutional priority?
  
  
  

  Why are so many
  faculty members still, in 2009, ''assessment-phobic'' and ''rubric-ignorant''? [Maybe we need to
  find Latin or Greek terms for ''assessment-phobic'' and ''rubric-ignorant' -
  ha! ha??]

  
  
  What could be done to avoid or reduce disproportionately large budget cuts for faculty development? So that when we return to better economic times, a larger portion of institutional resources will be committed to faculty development?
  
• Collection & Dissemination Challenge
  Agree/disagree/deal with: Most conscientious and valuable efforts to collect, assemble, publish, and get others to use valuable practices, policies, and ideas for improving teaching and learning in higher education fail to reach more than a few percent of all faculty - ever, certainly not in a year or two. Its getting even more difficult to get anyone to pay attention to any message or information - even about resources they need and are already entitled to!
  How can we enable and encourage effective exchange of information and innovations among Symposium participants, their own Institutions, even more widely!
  
• Faculty Learning Communities - Communities of Practice
  
  

• Even More Important!
  "How could you have been so stupid as to omit
  XXXYYYZZZZZ, which is the one that REALLY important to me? It should be one of the top priorities for this
  series!"
  

Return to Top

---

Try to Exclude the Seductively Plausible and Unrealistically Contingent:
topics, issues, activities, challenges, strategies

We'll try to exclude from our discussions topics, etc. that are

Seductively Plausible but we already know they just don't work or they're unlikely to be
accomplished in a short time with scarce resources

Unrealistically Contingent on
major changes in campus culture that are still not within reach (e.g., changing the promotion/tenure system)

Return to Top

A favorite science fiction novel

Leslie and I had a wonderful time at the Gilberts on Saturday, helping Sally celebrate her birthday. When she was telling us about the time she and Steve had just spent in Stockholm, and cautioning against visiting during the winter when it's so cold and dark, I thought one of my favorite science fiction novels, about a planet where the winters are so frozen and so long that all living things go into hibernation for years:


Vernor Vinge wrote A Deepness in the Sky. I've read it 2-3 times. It's a gripping story and some of the most human characters look like spiders on the outside: members of the intelligent race hibernating on the frozen planet as two competing human expeditions race to be the first to contact them. Everytime I think about really frozen winters, and how life can slow and change then, I think of this wonderful story.

President's Address at Notre Dame

If you have 40 minutes or so at some point (muted laughter), here’s the URL for the President’s commencement address at Notre Dame this afternoon:
http://tinyurl.com/qojd2k

Soundbites on TV can't capture the grace, intelligence and feeling of this amazing commencement address. His words are good humored, thoughtful, and utterly determined. One of his final sentences ("Remember that, in the end, we are all …") sent chills up my spine, a climax that I think many of these Notre Dame students will remember for a long time. The whole speech has subtly built to a point where that line, with its Biblical reference, packs an amazing punch, especially for these graduating students. It's a wonderful example of a powerful, resonant speech.

How to write a good survey

People frequently send me draft surveys for feedback, needs assessment, evaluation, etc. and ask for help in improving them. I'm happy to help. But I almost always need to ask them this:

"Before I can do anything, I need to know what choices you hope to illuminate with people's responses to your form. Such a choice might be 'what, if anything, should we do about X?' or 'should we do A or should we do B?'

If the form is well-constructed, once the answers are in, authors should be able to explain to respondents how their answers to EVERY question helped crystallize some action. Ideally, an author should never need to confess, 'We only asked question 7 because we were curious; we had no plans to actually use your answers.'

Without a picture of the survey's purpose, it's impossible to see what's missing, or to assess whether the draft questions are an efficient way to help make those choices.

Resources for workshops on assessment and evaluation

1. Do something.
2. Pause.
3. Gather some more information.
4. Then decide what to do next.

Sounds easy. A lot of what we do at the TLT Group is helping academics with #3, whether you call it "assessment," "evaluation," "scholarship of teaching and learning," "accreditation self-studies," or something else. We provide Flashlight Online so people can gather their own information. We do external evaluations. Of equal importance, we help educate faculty and other staff so that they can ask the right questions for themselves: questions whose answers are likely to be useful in deciding what to do next.

We continually revise and expand our educational resources, including workshop materials and the Flashlight Evaluation Handbook.

For example, we just added some new material to our list of 'frequently made objections' to assessment. If you're doing an evaluation, or training others, and get some push back, it's time for you to do steps 1-4 above: why are people really objecting? This web page should help you ask them the right questions.

And, if you run workshops or give talks about evaluation, a little humor can help lighten the mood while making a point. Here are some cartoons and stories you might find laughable.

We assembled these resources for our subscribing institutions. If you find them useful and your institution isn't supporting the TLT Group's work, perhaps you could pursue a subscription.

Challenging faculty assumptions (and your own)

I was recently asked for ideas for a workshop for faculty who believe strongly in the power of lectures and textbooks alone to educate their students. This is what I wrote in reply:

"You might try showing all of (or clips from) "Minds of our Own," an amazing video series (3 hours in all) from the Harvard-Smithsonian Center (producers of the earlier, brilliant "A Private Universe." Both MOOO and PU are available in streaming form, free, at http://www.learner.org. Or you can buy the DVDs, I believe.

"In these videos, there are many sequences of graduating Harvard and MIT seniors, in graduation robes (and some of them graduating with majors in science or engineering) confidently misunderstanding fundamental ideas in science that are taught in junior high. The sequence on using a battery and a piece of wire to light a light bulb is particularly rich. It begins by showing the misconceptions of graduating seniors (one mechanical engineering grad from MIT who thinks she can light the bulb and then fails laughs and attributes the problem to "operator error." The scene then shifts to a public school. A bright girl is defeated by her misconceptions, even after a very clear lecture and what looks like a very good, thorough lab on batteries and bulbs. In fact, even when an interviewer later puts her in a situation when she can reason her way to the correct answer (which she then draws on an easel), she refuses to believe her reasoning over her preconception. Not until her easel solution lights the bulb does she see it. And hen she immediately flips to recommending that other students be taught in a way that we recognize as the way that she was taught in the first place: the way that failed.

"Several faculty have told me over the years that they have shown clips like these to their students (in one case, a course in classical music) and found that they triggered valuable discussions about the nature of teaching and learning. And I use them in workshops with faculty.

"Having said that, note that any teacher (including me) has strong preconceptions about how things work in a course or workshops, just like that girl had her preconceptions about how to light a bulb. Watch the whole three hours, and "A Private Universe" (22 minutes), thinking about those faculty you want to teach as the 'students.' The single most important lesson of both videos: if you don't know what your students already believe, and why they believe it, you're already 3/4 of the way down the toilet. And your 'students' almost certainly don't all believe the same things."


What do you think? Would you agree to lead such a workshop to help faculty become conscious of, and question, assumptions that have provided a foundation for their teaching? How would you organize it?

Relationship between teaching approach, technology use, and assessment use

You may already know about the 1981 study by Schneider, Klemp and Kastendiek, "The Balancing Act." This well-designed study of faculty in bachelor's degree programs designed for 'returning adults' found that those faculty widely-regarded as excellent had a distinctively different approach to teaching than did their colleagues.

The 'excellent' faculty believed all their students were capable of becoming engaged. Throughout the course, the instructor would keep trying things, watching to see what happened,and modifying their approach-- sometimes for the whole class, sometimes for small groups or individuals -- trying to get everyone engaged and on the road to mastery.

Meanwhile, many of their faculty colleagues had quite different beliefs about the nature of their work as instructors. These 'normal faculty' (my term) believed that some students would do well, while many others would not. The instructor's responsibility was just to teach the content, and assess who was doing well and who was not. Differences in student input (talent, energy, etc.) would determine differences in student outcomes.

There's a chapter in the Flashlight Evaluation Handbook where I post ideas for dissertations and research projects (some of which might be good ideas for grant proposals). I especially hope our subscribing institutions and individual members find these ideas useful. I've just posted some ideas there about implications of the "Balancing Act" model for technology use and attitudes toward assessment. These ideas are explained in detail here.

To sum it up, after suggesting that someone create an easier way to measure faculty beliefs and behavior in ways that relate to the model suggested by "The Balancing Act," I suggest that the 'excellent' faculty are likely to give students technology-enabled assignments that reveal student differences, and help students excel in different ways.

In contrast, I predict that the 'normal' faculty will be found to use technology as a means to present the same instruction to everyone more effectively and efficiently (e.g., PowerPoint, video recordings of lectures, computer-aided instruction and assessment).

Even when the two types of faculty choose the same technology (e.g., student response systems such as clickers), I predict they will use that technology in different ways. The 'excellent' faculty will tend to use clickers in ways that reveal and challenge student thinking. The 'normal' faculty will use clickers to give quizzes, take attendance, and motivate all students to pay attention to readings and what's going on in class.

Similarly, the two groups of faculty will have different reactions to the term 'assessment.'

Let's imagine that, due in part to encouragement from accreditors, an institution begins creating an 'assessment initiative' designed to 'close the loop,' i.e., to assure that faculty and departments get feedback that they need to improve learning, and to document those improvements. The 'excellent' faculty are already accustomed using feedback to guide their own actions and work toward excellence for all students. So, I predict, 'excellent' faculty will initially assume that this assessment initiative is intended to empower them and their departments with information they can use collectively to improve.

In contrast, I predict, the 'normal' faculty at that institution will more often have been using assessment (quizzes, projects, final exams) to separate excellent students from average, mediocre and failing students. When students get back papers from these faculty, the papers will have a grade and perhaps some criticism, but less often will the paper have advice to help the student improve. Therefore, most of these faculty will assume that this assessment initiative is designed to 'grade' faculty or departments, in order to make personnel and budget decisions. Even if the assessment initiative is described as being for the benefit of their students and themselves, they will assume that there is nonetheless a hidden agenda for using assessment to illegitimately control them.

Corollary: there should be some correlation between how faculty who use technology actually use it (what technologies they choose; how they use what they choose) and how those same faculty view assessment initiatives. Faculty use of technology will be a predictor of faculty beliefs about assessment (in circumstances where faculty are technology users and are also confronted with pressure to participate in 'assessment.')

This won't always be true, however. When an institution actually does use assessment for control, or when trust levels are low due to prior betrayals or poor support, everyone has a right to assume that any new assessment initiative is an attempt to exert control over faculty and budgets.

1. What do you think?
2. Have you seen any faculty move from 'normal' to 'excellent' approaches over the years? or from 'excellent' to 'normal?' Under what circumstances?

TLT Vision or Strategic Plan for an Institution: Good Idea?

Steve Gilbert and I did some really exciting consulting for Howard University this past week, helping their TLT Roundtable develop a vision of teaching and learning with technology for the next few years, at the urgent request of Howard's administration for use in the upcoming budget cycle.

I was impressed with the planning work at Howard, which is just getting started. And I think a Roundtable is a good group to develop a vision that a community can embrace, and implement.

But I haven't heard about many multi-year plans to use technology to improve, alter, or
transform teaching and learning in departments, colleges or universities. I'm talking about plans to make relatively specific, programmatic improvements that imply relatively specific technology-related investments (e.g., improving undergraduate research by relying in part on more remote labs, simulations, and ePortfolios; internationalizing a program by relying in part on networking with people and resources in other countries; empowering faculty to improve their teaching through large-scale training in how to find and use free and open source tools). (Planning to improve technology capacity, e.g., changing learning management systems, or expanding enrollment by relying part on technology- these are more common topics for IT plans. But I'm interested in TLT plans, either standalone, or integrated into a larger academic plan or vision).

Budgets are tight, and many institutions feel pressed against the wall. Isn't that exactly the moment to plan how to make changes, year by year, to do these a bit differently? (In our work with Howard so far, we've already developed some new ideas and materials for doing just that.)

Enough preaching. To help us help you, and other institutions, here are two requests.

First, I've created this brief Flashlight Online survey that asks about whether or not your institution or department has any sort of vision or plan for using technology to support and improve teaching and learning over the next few years. (If you don't have a plan, the survey also asks whether that's intentional - 'we don't think planning is appropriate here'). Once you respond, you'll be able to see what others have said (and you can return to the URL as often as you like to see updates. We'll publicize these responses, and the plans you send us. So please tell us all what your institution has been doing. It will take you only 2-3 minutes.

Second, if your institution or department has a vision or plan for how to improve teaching and learning with technology in the next few years that you'd like to share with others, please send it to me, Steve Ehrmann.

Worldware: To the Cloud and Beyond

"Worldware" is a concept I developed in the 1990s, as part of study that a group of us did under the aegis of Educom (now EDUCAUSE). The question we were addressing: what kinds of software were having a noticeable impact on higher learning. We undertook this study of "Valuable, Viable Software" because of increasing anxiety that, despite a couple of decades of investment in courseware development, computer software might not be making much difference in teaching and learning. Our "aha!" moment came when we realized that technology developed for non-instructional purposes ("worldware") was sometimes having quite an impact: personal computers, word processors, research software, the Internet, ... There were several reasons for worldware's surprising value and viability, including

• the amortization of development, upgrade, marketing and support costs across a large market,
• the fact that faculty and at least some students were already using it and thought it valuable to learn more about, and
• the comparative likelihood that, if the original vendor or developer disappeared, curricular materials based on that worldware would not need to be discarded; some other provider would be able to use the files.

Recently, "Academic Technology" invited me to engage in a panel discussion to explore and extend this idea of worldware in the age of data clouds and social networking. I think the ideas extended in some rather nice ways. Judge for yourself.

Evaluation 101

I just emailed one of our subscribers about an evaluation of learning spaces that they're beginning to plan. We've got materials for the kinds of inquiry (focus groups, surveys, trips) that can guide planning. I also suggested he look at "The Flashlight Approach." I concluded with an old thought, which I think is framed more clearly than I usually say it!

"I always start designing a study by trying to identify the most important choices that people need to make, choices about which they are still reasonably uncertain. And I mean “reasonably” in both senses of the term:

1. In the minds of these folks, the uncertainty about what to do is significant, and
   
2. the uncertainty can be reduced by looking into evidence. Before the study, they may be leaning one way or another, but they could still be convinced to change their minds and make a different choice if they were to see evidence that my study would be capable of producing.
   

Grant-writing advice(2): Grant proposals as an example of teaching over distance

As the author of a grant proposal, is this the process you're hoping will happen?
1. Two or more people working for the funding program separately read your proposal
2. A day or more after reading it, the group discusses a big stack of proposals. Buried in that stack is yours. One of them says to the other(s), "Which proposal are we going to talk about now?"
3. Someone replies, "Oh this is the one that (and they summarize it in only 2-3 sentences, concluding with what they like about it, compared with the others in the stack.)
4. The first person replies, "Oh yes, I remember that one now! Here's what I think."
5. Hopefully the discussion advances your proposal toward funding.

Focus on step #3. How likely is that to happen? Is what you're proposing so clear and compelling that a day later, your 'distant learner' will remember what they liked about it?

Try an experiment. Give your draft proposal to a stranger who knows nothing about it (tell them nothing except that this is a grant proposal), have them read it, give them a day, and then ask them to summarize briefly what you've asked for and what they think of the proposal. If they can't do it to your satisfaction, consider rewriting the proposal.

If you like this idea, don't do what most people do -- don't let the writing process lag so near to the deadline that there's no time to get this kind of review and then (if need be) reorganize and rewrite the text. Hint: if you have time for nothing else, focus on the title and the abstract.

If you like this note and would like to see more on how to write successful grant proposals, please let me know!


If you missed Advice #1, here it is in a nutshell: Don't use the saying I invented, "So far as I know, no one has ever done this"),

We're experimenting with a Ning site. My post on Ning makes the argument that 'putting a course online' is a goal that's likely to lead to problems and that, instead, we should think about redesigning courses so that, in some ways at least, they're dramatically better than the same topic taught in a traditional campus-bound way. Take a look and try using Ning to engage in a conversation. And let us know what you think of Ning, versus this blog. Thanks!

Can "meaningful" higher education embrace technology AND personal interaction?

"In an age when higher education is threatened with a relentless technology that threatens to dispense with human beings altogether, Professor Van Doren exemplified a tradition of inquiry that celebrates personal interaction as the path to a meaningful education — one shaped by spontaneity, emotion and, yes, reverence." … "Mark Van Doren believed that young people were intuitively capable of grasping even the most complex literature."

Above from: "Notebook: Kerouac Got an A," by Adam Van Doren, NY Times, "Education Life," July 27, 2008, p. 38. "A grandson remembers the professor who inspired a generation of literary giants."

For more excerpts and links to full text, see the following and below.
"Seminar over, the students would follow their professor out of the classroom, conversing as he crossed the campus. Down into the subway they would go, even onto the train, not stopping until they reached the entrance to his townhouse in Greenwich Village. Finally, their professor, genially and perhaps reluctantly, would say, “Well, that’s it, boys, I’ve got to call it a day,” and then he would close the door, leaving those impressionable young men dazed on the sidewalk, wondering how they had landed so far from Morningside Heights." …

"My grandfather maintained bonds with students that lasted long after they graduated, and they in turn revered him." …

“Mark’s questions were very good and if you tried to answer them intelligently,” Merton wrote, “you found yourself saying excellent things that you did not know you knew, and that you had not, in fact, known before. He had ‘educed’ them from you by this question.” …

"The professor kept corresponding with these college youths years later, suggesting career paths, critiquing their manuscripts, promoting their work — even writing poems about them. “Death of a Monk (T. M.)” was written shortly after Merton’s death. He contacted publishers about promising students, encouraging Kerouac to publish “The Town and the City.” Kerouac quit the football team after getting an A in “Shakespeare.” (It should be noted that though the ledgers show my grandfather was a tough grader, those who would go on to make a literary impression on the world also did so in his classroom.)"

FOR FULL TEXT:

Permalink as of 20080727

Technique for breaking a workshop or class into small groups

Problem: you're running a large meeting or workshop, or teaching a large course. You break people into parallel groups, each working on the same task. Someone keeps notes for each group, on a pad or on an easel. How can you facilitate all those groups at once? After some minutes, it's time for 'reporting out.' How can you keep the reporting from taking too long?

Here's one way to do it. The University of Queensland in Australia (UQ) tried this approach for an ePortfolio workshop in mid-May. (We were helping them with planning and running the workshop.) They planned to divide 70 people into about 5 working groups, each sharing experiences on the same topics. (And, to complicate things slightly, I was assigned to pull the threads together but I was in the US.)

Know about Google Spreadsheets? It's a free web service that allows you to create spreadsheets just using a web browser. Better yet, more than one person can see, and write on, the spreadsheet at the same time.

So the UQ team created a Google spreadsheet with 5 identical worksheets, one for each group. Each group had a facilitator and an 'eScribe' with a laptop, who had already briefed. When the groups started their work, the eScribes already had open spreadsheets with the topics already written down the left hand column of their worksheet. (Each small group was assigned a differetn worksheet - a different tab - in the same spreadsheet; when the groups started, these served as templates for taking notes.)

I had been asked to facilitate all these groups. As it happened, I could not be in Brisbane that day. In fact, I was doing some work at the University of Alabama in Tuscaloosa. At the time of the workshop in Brisbane, I was sitting alone in a classroom with laptop. I could flick from one worksheet to another almost instantly by clicking on the tabs at the bottom of the page, watching the eScribes' notes appear cell by cell.

Each time I saw a note that I wanted that group to "report out," I highlighted that cell in yellow. And they could instantly see my highlights appear! By highlighting only 2-3 cells per group (and making sure there were no duplicates across groups), we could assure brief reports that would probably interest the other groups. After each reporter concluded (just a minute or two per team), I got on speaker (via VoIP), summarizing to the whole workshpo what I'd learned from reading all the notes from all the groups.

Advantages over traditional techniques: because the comments were typed online rather than handwritten, I didn't have to decipher their writing. I didn't have to spend time walking from group to group , and disrupting them as I walked. Instead, I was clicking from one group to the next about every 15 seconds, watching their notes appear. If I had a question or noticed a problem, I could use Skype or a chat window to communicate with Brisbane. If necessary someone could walk over to that small group and relay my question.

I'll never do breakouts the same way again, even in person. Try it! And thanks to the folks at UQ for coming up with this technique!

Strategic thinking for programs

A conversation with Mace Mentch of Case Western Reserve shed some fresh light on my recent blog post on a multi-faceted strategy for supporting improvement of selected teaching/learning activities.

Mace and I talked about how it's becoming somewhat more common to talk about the goals of academic programs in terms of learning outcomes. Because these learning outcomes are often defined as skills (e.g., design skills), a logical next step is to ask how often students are practicing, and receiving coaching, in skills. In effect, attention shifts from the outome-as-noun to the outcome-as-verb.

It's a shift people have talked about for decades:

• From teaching as explaining, with IT providing an explaining tool for faculty, toward a second, and perhaps greater emphasis on
• Learning as practicing, with coaching from experts, peers and clients, with IT providing tools for practicing, coaching and other forms of communication.

Big switch, and one that's not possible unless students have, and can use, appropriate technology for that discipline.

I'm not saying anything new when I point out that this sometimes enables faculty to exploit parallels between their own (technology-enabled) research and the learning of students, using comparable tools to learn by working on comparable problems. We studied several examples of that in our study of the iCampus program at MIT.

So here it is in a nutshell:
1. Identify one or more skills of graduates that some faculty want to improve (poor to good; great to world class) (e.g., writing, design, research, composition, academic argument, performance, ...)
2. To develop those skills over 3-4 years, what activities do students need to practice more, have coached better etc.? What tools and resources do they need in order to practice?
3. Develop accessible, modular resources and services that faculty can use to learn additional ways to:

• develop assignments and teaching skills to help students develop the target skills.
  
• assess how well those assignments are doing, in order to improve them;
• develop wisdom for dealing with the problems they're likely to encounter when their courses spend more time on helping students practice and improve these skills.

That set of bullets was the topic of the last post.

Many institutions, like Case, are making progress on item #2 in the list above: faculty are developing or getting access to digital libraries, easy-to-use software tools and other technologies needed in order for students to practice and develop sophisticated academic skills.

It's time to attend to #3: the many small steps that faculty will need to take in order to successfully build their courses more around the practice, coaching, and assessment of such skills. That's an especially exciting challenge with large courses!

Watch the doughnut, not the hole

The TLT Group's work has several hallmarks: taking low threshold steps forward; collaboration; and making a pluralistic, constructive approach to assessment. But arguably the most important, most counter-intuitive, and most pervasive theme of our work over the last decade has been our focus on 'activities.'

In contrast to our approach, the typical technology initiative focuses on -- surprise! -- technology. Buy new, fast, and inexpensive technology. Help faculty learn how to like and use that technology. Evaluate that technology (occasionally). And, all too soon, regret that the technology is now so old, slow, and expensive, and replace it with the technology that is newer, faster, and less expensive.

In contrast, The TLT Group usually focuses on what faculty, students and staff are likely to do with technology: how technology can potentially enable them to alter specific patterns of teaching/learning activity, e.g.,

• discussion,
  
• faculty-student contact,
  
• peer instruction,
  
• thinking through complex problems,
  
• calling up visual images and pointing to elements of those images while talking about them,
  
• integrative thinking,
  
• active learning,
  
• teaching in ways that take instructional advantage of diversity, ...

Our approaches to faculty support and course improvement, to cost modeling and time-saving, and to formative evaluation all focus on helping educators and institutions improve teaching/learning activities over time: small steps and, ultimately, larger changes.

Focusing on activities has several advantages:

• Activities drive outcomes, including learning, retention and costs. so the most direct way to change an outcome is usually to alter activities.
  
• Technology is never the only ingredient for improving outcomes; by focusing on the activity, we tend to notice the non-technological changes that also need to be made (e.g., reward systems, policies, partnerships, ...)
  
• Activities change slowly, in part because they are influenced by more than just the technology. It often takes several generational changes in technology before an activity and its outcome can be altered significantly. So plans to change outcomes need to be on a longer time scale; we help programs budget time and money in ways more likely to pay off in real improvement in learning.
  

Many of our materials and services focus on activities:

• Faculty support and course improvement (e.g., ideas for TLT sorted by Chickering and Gamson's seven principles of good practice; assessment workshop materials organized by activities in a course)
  
• Formative evaluation (evaluation questions sorted by the seven principles and other crucial teaching/learning activities, as well as technologies used for those specific activities)
  
• Learning space design and evaluation
• TLT case studies for faculty workshops, each focused on problems that can arise when faculty change course activities
• Our materials about content change are also organized around teaching/learning activities such as (digital) writing, research by students, and learning about other cultures.
  

Here's the point. In the past, our development of materials for professional and faculty development, evaluation, learning spaces, etc. has been opportunistic. While developing materials about the seven principles in one arena we might be focusing on diversity or visualization in another.

In the next stage of development of materials and services we are consider picking one or more activities and then developing a suite of faculty development workshops, evaluation templates, learning space examples, etc. needed to improve that activity. And, in true TLT Group fashion, we'd look at this 'warts and all,' studying the dangerous discussions and dilemmas that can arise in the process of making such changes.

Does that make sense? If so, which activities would be most important for The TLT Group to support at your own program or institution? if we wanted grant support for a consortial approach to such development, which activities might draw funder support?

• Teaching diverse classes and workshops?
  
• Using evidence to improve practice, in courses and services? ("culture of evidence")
  
• Digital writing Across the Curriculum?
  
• Learning communities?
  
• The 'seven principles?'
• something else?
  

Where should we work first in 2008-2009?

Organizing workshops for participants who differ in needs, excitement, ...

The TLT Group is well-known for its faculty and staff workshops. Here's one technique we've used to organize workshops for faculty who differ in many ways: in skills, in degree of excitement, and in reasons for attending.

Several years ago, the Butler University College of Pharmacy and Health Sciences decided to plan a laptop requirement for its entering students. In the 18 months leading up to the entry of the first laptop cohort, faculty would redesign their courses in order to take advantage of the new technology. The University subscribed to The TLT Group to get some help for those 50 or so faculty. We planned a day-long workshop for Butler staff. As part of that workshop, we wanted faculty to learn from the findings and experience of an expert at another university.

Usually, a workshop has to pick one of two strategies to help participants learn from an expert. The expensive option is to bring the expert to campus to be a featured speaker in the workshop. The inexpensive alternative is to have participants read an article written by the expert.

We used a technique that combines virtues of those two approaches, while also doing better than either at meeting the needs of a diverse group of faculty.

1. Before arriving at the workshop, participating faculty read an article by the expert.
2. At the workshop, we spent about 10 minutes planning questions to ask the expert. As they read this long and provocativ research report, our faculty had noticed different things, liked different things, and objected to different things. So there were lots of questions and comments. (At the time we had picked the article, we had asked the author if he would be willing to take our call at the time of the workshop so that our faculty could ask him questions. He was delighted to do so.)
3. Then we called him and began with the questions we'd planned, as faculty handed a microphone from one questioner to the next. We could hear our expert over a speaker. He didn't give a talk; he just responded to our questions and comments about his article. We actually spent far more time quizzing the expert then if he had given a talk.

Cost: just the price of the phone call.

Time-saving: People were able to read the article (which was quite long- 15 or 20 pages) in far less time than a speech would have taken. Even with the time for discussion, it was a great time-saver. : each person could focus just on those points that had excited or provoked them the most.

Impact of Class Size on Learning Outcomes?

Someone posted that question on the Professional and Organizational Development (POD) listserv this morning. I thought you might like to know how I responded:

There is no general relationship between class size and learning outcomes. (Think of a self-help book that sells millions of copies, or a PBS program on the civil war, if you'd like an analogy. It's possible for people to learn relatively sophisticated lessons even when "class sizes" are enormous. And the learning outcomes of presentation, whether delivered by book, video or by uninterrupted lecturing, aren't affected much by class size).

The research only gets interesting when you begin by specifying some teaching/learning activity and then asking about

1. the impact of class size on its feasibility, and
2. the outcome when that pedagogy is in successful use.

I call these three elements of resource/technology (class size), teaching/learning activity, and outcomes a "triad." The resource or technology offers choices. The activity is what people actually did with the choices. And the outcomes result from those activities. (That simple grammar suggests some powerful questions for the scholarship of teaching and learning in any course or set of courses.)

Some important teaching/learning activities do far better in small classes, and some learning outcomes are much easier to achieve with those activities. (e.g., seminar style discussion; activities that require personal coaching by an expert) But the threshold (the class size where the pedagogy breaks down) varies: I used to work at Evergreen where seminars of 20 students worked pretty well. But a master class in violin would certainly work much better with a 1:1 ratio than with 20:1.

Of course, if you summarize studies of class size that include all sorts of pedagogies and all sorts of outcomes, some of which involve lots of interaction, you'll find a small relationship between class size and outcomes. Which is what Pascarella and Terenzini report in their synthesis of research on learning, How College Affects Students.

Your question is an example of a larger class of questions about the relationships of inputs or technologies to learning outcomes (e.g., how does the size of the library affect learning outcomes? distance learning v. campus classrooms? highly paid profs v. low paid profs?) The role of resources in learning, even a scarce resource such as the attention a professor might pay to a student, is to create possibilities for action.

The power comes from the teaching/learning activities - what faculty and students actually do in those classes. That's why you'll see relatively significant findings on studies of learning where the independent variable is a powerful pedagogy, but not where the focus is a resource or a technology. For examples of such powerful pedagogies, take a look at the P&T book above or at the 2007 annual report from the National Survey of Student Engagement or reports from the Association of American Colleges and Universities.

Steve Ehrmann

Introducing The TLT Group: Why Subscribe?

As you probably know, Steve Gilbert has been doing pioneering work on brief hybrid workshops - workshops that:

• Are only 5-15 minutes long (short enough that people will be able to find a moment to do the workshop, even on a busy day);
  
• Involve a little learning by doing and interpersonal interaction; and
  
• Are well enough supported with eClips and other materials so that peers can use them to help peers to learn something useful.

The other day Steve challenged me to create some brief hybrid workshops about The TLT Group. The notion is that these materials would be used by someone who would like to tell colleagues about The TLT Group so that they could discuss whether or not their institution should subscribe. Here's my first attempt at such a workshop: http://www.tltgroup.org/about/intro.htm

I've tried it yesterday while visiting the University of Alabama (where we just did a great workshop, one of the goals of which was to spread the word about all the different benefits of their subscription). Ideally people at our workshop can now use these materials if they want to tell other people (who didn't attend) about what their subscription offers.

What do you think? Would you use these materials to help your colleagues learn about The TLT Group? If the workshop design or materials needs to be tweaked before you'd use it, what would you like us to change?

Thanks for your help!

Faculty Learning Communities: A New Strategy

Imagine a faculty learning community - a community of inquiry and evidence - that is doing collaborative research in order to improve its members' success in 'teaching online' (or teaching with ePortfolios, or teaching with student response systems, or fostering digital writing, or teaching in a computer classroom, or any other technique or technology you choose). The TLT Group and Washington State are now beta testing a new kind of survey tool that opens up new avenues of research for such communities. The tool is Flashlight Online 2.0 and the strategy it supports is called a 'matrix survey.'

Let's imagine that the faculty learning community is going to do research together on its members' uses of ePortfolios, in order to discover ways in which its members could improve learning in their courses.

Up until now, if the community were interested in using a survey or interview protocol, members might have assumed that all students in all classes would need to be asked the same questions, so that faculty could pool their data:

• "How satisfied are you with the use of ePortfolios in this course?"
  
• "Is the ePortfolio easy to use?"
  
• "How much is the ePortfolio helping you learn?"

These are 'lowest common denominator' questions. After the community gets answers to questions, would they really understand student learning any better? would they have learned something useful for improving learning? Probably not. The questions are so sterile because they have to be conceptually broad enough to cover the wide variety of uses of ePortfolios across courses.

And there are many activities for which ePortfolios can be used, but only a few are likely to be important in any one course. Here's a list that Susan Kahn (IUPUI) and I developed:

1. Reflection as a means of deepening learning
2. Integrate/synthesize prior learning and course learning
3. Student academic self-assessment, guidance (within degree program)
4. Building a sense of professional identity
5. Personal/developmental: Each student develops/describes own goals & abilities
6. Audiences and assessors for the student’- more, better
7. Learning communities, support of
8. Department reframes major in terms of competences across courses
9. Faculty share practices, perspectives
10. Job and school applications
11. External accountability

An activity that specific is much easier to study: do the students understand the goal? what are the software's strengths and weaknesses when used for that purpose? what factors are helping and hindering the student in doing this particular thing? and so on.

Research would be much easier for a community if all its members used ePortfolios the same way. But the needs of real faculty learning communities is seldom this neat. Different faculty are likely to use ePortfolios in different ways. And that's where the matrix survey can play such an important role in supporting more focused, flexible research.

Imagine members of the community working together to develop a useful set of student feedback questions for each of those eleven activities above (assuming each of those activities was a goal for at least one of the participating faculty).

Flashlight Online 2.0 enables each faculty member to fill in a menu specifying which of those eleven activities are important for her course, automatically a student response form that contains only questions about those particular activities.

Once the students have responded:
a) the individual faculty member can get a report on how her students have responded; and
b) the faculty learning community can get a report on how all their students have responded, activity by activity, for all eleven activities. Imagine that there are 25 courses being studied. The report on activity 1 might come from 17 of those courses, on activity B from 4 of those courses, and so on. The community could use this pool of data to study each activity, searching for insights from the larger pool of data. I emphasize activity F in my course and saw something that surprised me from my own students: is that typical for courses that emphasize activity F? or unusual?

Such a survey is not just a one-time thing. The faculty learning community could use its matrix survey term after term, even if different members of the community were on different academic schedules (or even at different institutions), accumulating data.

To read more about matrix surveys, with examples of such survey questions for different uses of ePortfolios, click here.

To comment on this idea or ask questions, please post a comment below.

And, most importantly, if you are interested in working with us on developing such a matrix survey strategy for a faculty learning community (or any other purpose), please contact me.

Fostering Incremental, Cumulative Improvement

For over a decade, Steve Gilbert and I have been working on ways to help institutions foster incremental, cumulative improvements in teaching and learning with technology.

It's easier to attract attention to the opposite: big grants and top-down redesign. What's wrong with that? Nothing, but that 'big push' approach is quite limited, especially in times of tight budgets:

• It's hard, and rare, to get such chunks of time and money
• What do you do a few years later, once the initial capital has been expended? does the innovation that stagnate, regress, or what?
• Most change in higher education can't be achieved through such an approach. Most faculty, most courses, and most student learning experiences would remain untouched.
  

To foster an incremental, cumulative improvement in teaching and learning (with technology) here are some ideas, materials and tools that The TLT Group has developed for its subscribers.

As you'll see, many of these items have a common working assumption: most faculty are interested in improving their teaching, and the learning of their students, to the extent that each such improvement can be achieved without undue time or risk.

• Identify "low threshold" ideas for improving teaching and learning, and circulate them (e.g. via frequent, brief, read-at-a-glance e-mails.)
  
• Offer brief, hybrid workshops (BHWs) to faculty - each just 5-15 minutes -- to help them learn low threshold ways of improving their teaching. These workshops should be well-documented (e.g. with brief online tutorials, brief testimonials from faculty who have used the ideas in the past). If faculty participate in such a workshop (e.g., as an agenda item in a departmental faculty meeting; during a brown bag lunch), try the idea in their own courses, and like the results, they should be able to use the same workshop materials to help other interested colleagues to adapt the same techniques. In other words, a good BHW can be offered peer-to-peer, without always requiring a highly skilled leader.
• Train students to help faculty improve teaching, especially teaching and learning with technology. For years, The TLT Group has offered subscribers help in upgrading their programs for student technology assistants so that the students can provide this kind of aid to faculty.
• Help faculty use student feedback in order to figure out how to improve their teaching. The TLT/Flashlight program to develop such brief hybrid workshops is called "Asking the Right Questions" (ARQ).
  
• Support faculty who act as 'compassionate pioneers' (e.g., helping their colleagues use ideas that the first faculty member has already tried)
• Stretch and coordinate the use of scarce faculty support resources by developing a virtual Teaching, Learning, and Technology Center. In strategy for encouraging incremental, cumulative improvement, faculty have to do much of the work: identifying a need and making an improvement. But that doesn't mean they're completely unaided. So how can the institution speed help to all interested faculty when, at almost every institution, support resources are splintered among faculty development, information technology, the library, distance learning, individual departments, and other units? By having as many of those support staff as possible work (in many ways) as though they were one unit. Such a virtual TLT Center can have a single help line, a single calendar, co-planning of events, shared development of its (their) own staff, etc.
• The study done by TLT/Flashlight of the MIT/Microsoft iCampus program revealed more ways to support incremental, incremental change. For example, it is far easier for faculty member B to understand, appreciate, and implement a teaching idea from faculty member A if A & B teach the same course, perhaps with the same text, perhaps with the same academic calendar, with a similar approach, and similar concerns. In that case, B can 'grok' what A is saying almost instantly, decide quickly whether to try the idea, and (the first time) try it by copying what A did. It's a relatively low threshold process. So the trick is to find a way to pair A and B. Interestingly, the larger the universe of faculty (e.g., a big system of institutions, a statewide effort, a national professional association), the easier it is to create such pairs and small groups of like-minded faculty.

Everything I've written so far focuses on how to invest in and support incremental improvement. What about making enough such small changes, and congruent changes, so that the results are 'cumulative?" Over time, we'd like the results of all these tiny changes become apparent to instructors, and also to students, departments, alumni, accreditors, etc.

At most institutions, the job of larger entities (departments, professional associations, institutions, libraries) is to work to support all kinds of improvement (and thereby offend nobody). The job of the faculty member is 'merely' to choose.

There are real problems with that working model. Few of those larger entities can afford to support all conceivable incremental improvements. And few faculty have the time to scan all those opportunities, and then choose one.

Doesn't it make more sense for those larger entities (departments, associations, and institutions) to choose some directions for improvement, and focus on incremental improvements in those directions.

I'm indebted to Chris Alexander of Berkeley for that insight. Decades ago, in his book The Oregon Experiment, Alexander rejected the idea of campus master plans and instead argued for focused support of incremental change in the university's physical infrastructure. The goals would be agreed on and adjusted through community governance.

As it happens, The TLT Group also helps institutions organize Teaching, Learning, and Technology Roundtables. One role that such TLTRs can play is to pick some goals for incremental instructional improvement, and then help the institution maintain its focus for enough years so that the results become visible, valued, and rewarded. What kinds of goals are worth the candle? How about "learning community," "undergraduate research," "writing across the curriculum," or "building a culture of evidence"? The important thing, for departments as well as institutions, is to pick a few such goals of real importance, where systematic support of incremental, cumulative improvement can really pay off for students, faculty, the department, and the institution.

Your comments? What strategies have I missed? What resources?