In a post from last week I wrote about some of the research I’ve begun doing around the pedagogical features of a textbook as I try to identify the features of textbooks that we need to make sure we include as we begin to construct open textbooks.

In my initial scan, I’ve found a few interesting papers & studies looking at the effectiveness of pedagogical aids in textbooks. This morning I read two papers from Regan Gurung at the University of Wisconsin (Pedagogical Aids and Student Performance published in 2003 & Pedagogical Aids: Learning Enhancers or Dangerous Detours? from 2004) and one earlier paper from 1996 from Santa Clara University (Wayne Weiten, Rosanna Guadagno & Cynthia Beck) titled Student’s Perceptions of Textbook Pedagogical Aids.

These 3 papers are specific to Psychology textbooks and are primarily built around student perceptions of the pedagogical aids in the books & whether or not students used them.

Student perceptions are important, especially if they do not use a pedagogical aide since an “unused pedagogical aide cannot facilitate learning” (Weiten, Guadango & Beck 1996), but perception is just one factor I want to look at & Gurung’s research digs a bit deeper than student perceptions to see if there is a connection between student use of pedagogical aids and better exam performance.

Weiten, Guadango & Beck surveyed 134 students asking them how familiar they were with the different pedagogical aids in their textbook, the probability of use and their perceived value of each aid. From their research, Weiten, Guadango & Beck showed that the top 3 pedagogical aids students used in their textbooks were bold-faced technical terms, chapter & section summaries & glossaries.

Mean Ratings of Pedagogical Aids (Weiten, Guadango & Beck, 1996)

An interesting takeaway from their research (although it is over 20 years old now) is that at the time “virtually no research has assessed the usefulness of the numerous pedagogical aids that are now standard far in psychology texts”. Meaning that, in the views of these researchers, the features of a textbook that have been put in place to help student learn weren’t put there because they have been shown to help student learn.

Again, the caveat that I am looking at research from 20 years ago, but so far my scan has shown something similar – there is not a huge amount of empirical  research on whether these features of a textbook actually help student learn. In fact, some of the research from Gurung hints at something quite the opposite; that there may be some textbook features in use that we take for granted that may actually hurt student performance.

Do they help or hinder?

In Gurung’s 2003 research Pedagogical Aids and Student Performance,  Gurung surveyed more than 200 undergraduate students and asked them to rate the usefulness of 10 pedagogical aids and instructional techniques (Gurung’s research wasn’t specific to textbook aids, but included a number of textbook specific aids like outlines, chapter summaries & reviews, boldfaced & italicized terms, key terms & practice questions found in a textbook).

Looking at the types of aids mentioned in the research that are textbook specific (ie eliminating items like paper assignments and research participation) and the results showed that the top textbook 3 aids used by students were boldface terms, italicized terms and practice questions (with chapter summaries & reviews a very close 4th). In terms of helpfulness, students rated boldfaced (92%) and italicized (81%) terms as the most useful pedagogical aid, followed by practice tests questions (77%), and chapter summaries & reviews (73%) all as being moderately to extremely helpful.

Reported Use & Helpfulness of Pedagogical aids (Gurung, 2003)

When Gurung compared the reported use and helpfulness of the textbook specific aids and student performance based on their test scores he determined that “correlation analysis did not show any positive relations between the reported use of a pedagogical aids and learning as measured by exam performance” and that textbook authors, “…should not feel pressured to load their books with such aids.” Gurung also notes that the lack of effectiveness of textbook pedagogical aids isn’t an isolated finding & quotes research from 2001 by Blach (guess what is going high on my list for further reading).

Can pedagogical aids actually hurt learning?

One of the really interesting findings from Gurungs 2003 paper was that there was one correlation between a pedagogical aid and exam outcomes was “significant” and that had to do with key terms. Students who rated key terms as being helpful had lower test scores than those who did not use key terms. However, Gurung does note that “the correlational nature of the data does not allow for a true test of this question (can a pedagogical aid hurt exam performance)” and there are a few significant limitations to the research, including not accounting for student performance, ability or effort, nor the amount of time the student spent studying. Also important to note that Gurung only looks at one outcome; exam performance.

Still, it isn’t hard to see how a pedagogical aid could negatively affect student performance if the student tries to get by on the built in aids as an alternative to doing the actual reading. If a student sees the aid as a shortcut to doing the actual reading, then it isn’t hard to imagine that these tools could affect student learning. A scenario where a student is crunched for time and instead of doing the reading for the course instead relies on the chapter summaries to give them all the information could be fairly common.

Gurung followed up his 2003 research with a 2004 study that supports the ineffectiveness of the pedagogical aids we seem to take for granted. In his paper Pedagogical Aids: Learning Enhancers or Dangerous Detours? Gurung assessed 240 introductory psychology undergraduates (again looking at test scores) and showed that the reported use of aids “did not positively relate to student performance on any exams” and again showed that key terms might hurt test performance. In this research, Gurung did try to account for the 2 limitations he noted in his first, namely student ability & time studying.

My takeaway

I’m still early in my research so it is hard to draw any definite conclusions yet. But articles like these help me flesh out pedagogical features of our textbooks. For example, all the articles note that students use bold and italicized text (whether it is actually increases their learning is another matter all together). But knowing that those features will actually be used by students helps to guide our advice to open textbook authors. When you make a textbook, concentrate on the way you use bold and italicized text because students will be looking for that to help them understand the content.

This is also helping curb my assumptions that just because something appears in a lot of textbooks doesn’t mean it is either a good nor a proven aid to learning, or that students will use the aid in the way it is intended. What we may be doing when we add features that we think students will use to connect deeply with the material may, in fact, be convenient devices students use to shortcut their learning. I’ll be interested to see if this issue of pedagogical feature as shortcut instead of pathway to deeper understanding comes up more in the literature.

References

Weiten, W., Guadagno, R. E., & Beck, C. A. (1996). Student’s Perceptions of Textbook Pedagogical Aids. Teaching of Psychology, 23(2), 105-107. doi:10.1207/s15328023top2302_8

Gurung, R. A. R. (2004). Pedagogical Aids: Learning Enhancers or Dangerous Detours? Teaching of Psychology, 31(3), 164-166. doi:10.1207/s15328023top3103_1

Gurung, R. A. R. (2003). Pedagogical Aids and Student Performance. Teaching of Psychology, 30(2), 92-95. doi:10.1207/S15328023TOP3002_01