pedagogy

by Clint Lalonde on Teaching & Learning, Textbooks

Fleshing out the pedagogical features of textbooks

In a post from last week I wrote about some of the research I’ve begun doing around the pedagogical features of a textbook as I try to identify the features of textbooks that we need to make sure we include as we begin to construct open textbooks.

In my initial scan, I’ve found a few interesting papers & studies looking at the effectiveness of pedagogical aids in textbooks. This morning I read two papers from Regan Gurung at the University of Wisconsin (Pedagogical Aids and Student Performance published in 2003 & Pedagogical Aids: Learning Enhancers or Dangerous Detours? from 2004) and one earlier paper from 1996 from Santa Clara University (Wayne Weiten, Rosanna Guadagno & Cynthia Beck) titled Student’s Perceptions of Textbook Pedagogical Aids.

These 3 papers are specific to Psychology textbooks and are primarily built around student perceptions of the pedagogical aids in the books & whether or not students used them.

Student perceptions are important, especially if they do not use a pedagogical aide since an “unused pedagogical aide cannot facilitate learning” (Weiten, Guadango & Beck 1996), but perception is just one factor I want to look at & Gurung’s research digs a bit deeper than student perceptions to see if there is a connection between student use of pedagogical aids and better exam performance.

Weiten, Guadango & Beck surveyed 134 students asking them how familiar they were with the different pedagogical aids in their textbook, the probability of use and their perceived value of each aid. From their research, Weiten, Guadango & Beck showed that the top 3 pedagogical aids students used in their textbooks were bold-faced technical terms, chapter & section summaries & glossaries.

Mean Ratings of Pedagogical Aids (Weiten, Guadango & Beck, 1996)

An interesting takeaway from their research (although it is over 20 years old now) is that at the time “virtually no research has assessed the usefulness of the numerous pedagogical aids that are now standard far in psychology texts”. Meaning that, in the views of these researchers, the features of a textbook that have been put in place to help student learn weren’t put there because they have been shown to help student learn.

Again, the caveat that I am looking at research from 20 years ago, but so far my scan has shown something similar – there is not a huge amount of empirical  research on whether these features of a textbook actually help student learn. In fact, some of the research from Gurung hints at something quite the opposite; that there may be some textbook features in use that we take for granted that may actually hurt student performance.

Do they help or hinder?

In Gurung’s 2003 research Pedagogical Aids and Student Performance,  Gurung surveyed more than 200 undergraduate students and asked them to rate the usefulness of 10 pedagogical aids and instructional techniques (Gurung’s research wasn’t specific to textbook aids, but included a number of textbook specific aids like outlines, chapter summaries & reviews, boldfaced & italicized terms, key terms & practice questions found in a textbook).

Looking at the types of aids mentioned in the research that are textbook specific (ie eliminating items like paper assignments and research participation) and the results showed that the top textbook 3 aids used by students were boldface terms, italicized terms and practice questions (with chapter summaries & reviews a very close 4th). In terms of helpfulness, students rated boldfaced (92%) and italicized (81%) terms as the most useful pedagogical aid, followed by practice tests questions (77%), and chapter summaries & reviews (73%) all as being moderately to extremely helpful.

Reported Use & Helpfulness of Pedagogical aids (Gurung, 2003)

Reported Use & Helpfulness of Pedagogical aids (Gurung, 2003)

When Gurung compared the reported use and helpfulness of the textbook specific aids and student performance based on their test scores he determined that “correlation analysis did not show any positive relations between the reported use of a pedagogical aids and learning as measured by exam performance” and that textbook authors, “…should not feel pressured to load their books with such aids.” Gurung also notes that the lack of effectiveness of textbook pedagogical aids isn’t an isolated finding & quotes research from 2001 by Blach (guess what is going high on my list for further reading).

Can pedagogical aids actually hurt learning?

One of the really interesting findings from Gurungs 2003 paper was that there was one correlation between a pedagogical aid and exam outcomes was “significant” and that had to do with key terms. Students who rated key terms as being helpful had lower test scores than those who did not use key terms. However, Gurung does note that “the correlational nature of the data does not allow for a true test of this question (can a pedagogical aid hurt exam performance)” and there are a few significant limitations to the research, including not accounting for student performance, ability or effort, nor the amount of time the student spent studying. Also important to note that Gurung only looks at one outcome; exam performance.

Still, it isn’t hard to see how a pedagogical aid could negatively affect student performance if the student tries to get by on the built in aids as an alternative to doing the actual reading. If a student sees the aid as a shortcut to doing the actual reading, then it isn’t hard to imagine that these tools could affect student learning. A scenario where a student is crunched for time and instead of doing the reading for the course instead relies on the chapter summaries to give them all the information could be fairly common.

Gurung followed up his 2003 research with a 2004 study that supports the ineffectiveness of the pedagogical aids we seem to take for granted. In his paper Pedagogical Aids: Learning Enhancers or Dangerous Detours? Gurung assessed 240 introductory psychology undergraduates (again looking at test scores) and showed that the reported use of aids “did not positively relate to student performance on any exams” and again showed that key terms might hurt test performance. In this research, Gurung did try to account for the 2 limitations he noted in his first, namely student ability & time studying.

My takeaway

I’m still early in my research so it is hard to draw any definite conclusions yet. But articles like these help me flesh out pedagogical features of our textbooks. For example, all the articles note that students use bold and italicized text (whether it is actually increases their learning is another matter all together). But knowing that those features will actually be used by students helps to guide our advice to open textbook authors. When you make a textbook, concentrate on the way you use bold and italicized text because students will be looking for that to help them understand the content.

This is also helping curb my assumptions that just because something appears in a lot of textbooks doesn’t mean it is either a good nor a proven aid to learning, or that students will use the aid in the way it is intended. What we may be doing when we add features that we think students will use to connect deeply with the material may, in fact, be convenient devices students use to shortcut their learning. I’ll be interested to see if this issue of pedagogical feature as shortcut instead of pathway to deeper understanding comes up more in the literature.

References

Weiten, W., Guadagno, R. E., & Beck, C. A. (1996). Student’s Perceptions of Textbook Pedagogical Aids. Teaching of Psychology, 23(2), 105-107. doi:10.1207/s15328023top2302_8

Gurung, R. A. R. (2004). Pedagogical Aids: Learning Enhancers or Dangerous Detours? Teaching of Psychology, 31(3), 164-166. doi:10.1207/s15328023top3103_1

Gurung, R. A. R. (2003). Pedagogical Aids and Student Performance. Teaching of Psychology, 30(2), 92-95. doi:10.1207/S15328023TOP3002_01

 
by Clint Lalonde on Moodle, Teaching & Learning

Pedagogy drives technology drives pedagogy

We are in the process of switching to Moodle 2.1 from Moodle 1.9. We’ve been planning this switch for a year but, like many tech projects, it doesn’t matter how much planning and testing you do, the real test happens when users start rolling in.

We’re at that point right now. People are starting to use the system. The most painful point. The transition.

I won’t get into details about the inner workings of Moodle, but those who know the 2.x version compared to the 1.9 version know that there has been a major overhaul of how the file system works. Gone is the file storage  area – the place where people dumped all their course files. Instead we have a new file repository system.

From the reading I have done and the people I have talked to, this change has been one of the most contentious changes in Moodle, and we are struggling with how to support it as it means a big shift in how people organize their stuff. It forces people to make a conceptual shift in that their content is now somewhat disaggregated from their course. Dispersed, distributed and decentralized. Not contained within neat little folders. Not easily accessible in a single place. Living….somewhere?

It is forcing people to think about their content in a different way, and it is changing their workflow at the most basic level.

How do I organize my stuff?

How do I delete my stuff?

Where is my stuff? WHERE IS MY STUFF?

It makes sense to me why Moodle is moving to the new repository system, but I can see the technical reasons. That (usually) doesn’t fly with users, and the new system is stressing people out.

One thing I never considered until I read Mark Dreschler’s post, however, is that the pedagogical framework of social constructivism that underpins Moodle means having a powerful file management system could be a rather low priority for Moodle developers because social constructivism moves the focus of a learning experience away from content as the cornerstone and refocuses the experience on the construction of knowledge among participants.

I never really thought about this until Martin’s discussion with the group yesterday, but, and I’ll say it loud and clear now – Moodle is not meant to be a file repository. When I look back at Martin’s original pedagogical drivers of social constructionism then it makes perfect sense that storing files should be low on the list of priorities. Learning in a social constructionist world isn’t about downloading and reading files, its about collaboratively constructing them with others – a critical distinction.

Learning in a social constructionist world isn’t about downloading and reading files, its about collaboratively constructing them with others

In this specific case, the pedagogical model drives the technological development.

Now, that is all wonderful IF you use it in a homogenous environment where all users are on board and working from the same pedagogical model. Great. However, stray from that model and you find yourself working against the technology; fighting, wrestling and wringing it into submission to do what you want to do with it. Or, you are forced to alter your own pedagogical model to make it fit with the technology.

In this case, it’s hard to argue that the instructivist “here are my notes and PowerPoint slides” model is superior to the social constructivism Moodle model, but still; it’s a pedagogical choice being enforced on a user by technology. People don’t like that. They fight back and get defensive when a machine forces them to do something they don’t want to do. It’s technology driving pedagogy.

And this is the inherent problem (feature?) of ANY LMS. It is not neutral. It WILL impose its way on you.

In the case of Moodle, the pedagogy is explicit. Indeed, I think this is one of the reasons why Moodle is a popular choice – it is built around an explicit pedagogy, which appeals to many educators. The foundation is educational, not technological. But, just because it is explicit (and, let’s face it, a pretty good model) doesn’t mean the pain of fitting into that model is any less.

Right now, I am not sure how we are going to deal with the Moodle file issue. Secretly, deep down, part of me smiles just a little to think that the system is actually making it more difficult to stuff a course full of Word and PDF documents; that using the LMS as a content repository is just a little bit tougher to do. But that fades quickly when I realize that this is causing stress and friction for the people I support.

It is also difficult to use moments like this as leverage into a conversation about whether uploading a whack of files into an LMS is the best way to encourage learning when faculty have students breathing down their neck for the latest PowerPoint presentation. But we’ll try. And it won’t be the last time we make choices on how we do things in order to fit the pedagogy imposed on us by our technology.

As Neil Postman says in Technopoly: The surrender of culture to technology:

Every technology is both a burden and a blessing; not either-or, but this-and-that.

This is the moment I am living right now.