4.1 Evaluating an Open Textbook
Evaluating an open textbook is similar to evaluating any textbook. To begin, let's look at the qualities that make a good textbook., whether it's open or not.
What are the Qualities of a Textbook?
You have probably reviewed many textbooks in your career and have your own set of evaluation guidelines. Before we begin looking at the criteria that various open textbook projects are using to evaluate the textbooks in their collection, let's review some of the qualities and structures that make up a textbook.
Pedagogical Aids for Textbooks
When it comes to textbooks, there is no doubt that the content of the book is important. You want to be sure that the content in the textbook aligns with the learning outcomes of your course and that the content is written at the right level for your learners.
However, a good textbook is more than a collection of facts. Textbooks also contain a number pedagogical aids which are features or attributes of the book that help a learner understand the content being presented.
There are a number of common pedagogical aids in textbooks, including:
- Chapter objectives
- Chapter learning outcomes
- Chapter outline
- Chapter summary or review
- Headings and subheadings
- Bold and italicized text
- Table of contents
- Table of figures
- Focus questions and practice questions
- Case studies, vignettes and examples of best practices
- Glossary and key terms
- Demonstrations and simulations
- Maps and timelines
- Illustrations, including photos, charts, diagrams and figures
- Pronunciation guide
Usefulness of pedagogical aides in textbooks
Of these pedagogical aides, which are the ones that students actually use and find useful? Weiten, Guadango & Beck (1996) surveyed 134 students asking them how familiar they were with the different pedagogical aids in their textbook. They also asked the students what the probability they would use each pedagogical aide, and their perceived value of each aid. From their research, Weiten, Guadango & Beck showed that the top three pedagogical aids that students used in textbooks were bold-faced technical terms, chapter and section summaries and glossaries.
In his 2003 research Pedagogical Aids and Student Performance, Gurung surveyed more than 200 undergraduate students and asked them to rate the usefulness of 10 pedagogical aids and instructional techniques. Gurung’s research wasn’t specific to textbook aids, but did include a number of textbook specific aids like outlines, chapter summaries and reviews, boldfaced and italicized terms, and key terms and practice questions found in a textbook. When you eliminate the non-textbook related aides from Gurung's research, we find that the top textbook aids used by students were boldface terms, italicized terms and practice questions. Chapter summaries and reviews was a very close fourth.
In terms of helpfulness, students rated boldfaced (92%) and italicized (81%) terms as the most useful pedagogical aid, followed by practice tests questions (77%), and chapter summaries and reviews (73%) all as being moderately to extremely helpful.
So, as we can see, it is often the simplest aides, like bold and italicized terms, chapter summaries and outlines that are most often used by students.
Another consideration when reviewing a textbook is to take a close look at the way the content is structured. A good textbook should follow some basic rules of structure.
Open Textbook Specific Considerations
So far we have been looking at textbooks in general and not specifically open textbooks. For the most part, the criteria you would use to judge the effectiveness of an open textbook would be the same as a textbook in general. However, there are some unique criteria that you might want to consider for open textbooks that may not apply to a closed textbook. Some of the additional items you may want to consider when evaluating an open textbook are:
- Can I modify or edit the book to meet my needs and, if so, does the license allow me edit or modify the book?
- Is the open textbook available in a technical format that I can edit, if I want to modify it?
- Are there ancillary resources available for the textbook? For some faculty, ancillary resources are important components of a textbook adoption. While some open textbooks do have ancillary resources associated with them, many do not. This may or may not be an issue for you.
- Is there a community to support the open textbook? Often, these communities can help provide support for you in your adoption. If you have a problem, or concern, or are just looking to connect with others who are using the same resource, a well developed community can help provide support for your adoption. For example, Stitz-Zeager's open pre-calculus textbook website has a community with discussion forums and blogs that can help provide some support for your adoption. The Writing Commons is another example of an open textbook with a community associated with it.
Open Textbook Review Criteria
Increasingly, open textbook projects are placing a greater emphasis on having peer reviewed materials in their collection to help faculty with adoption and address concerns some have about the quality of open textbooks. A number of open textbook projects have created criteria for evaluating and reviewing open textbooks.
After reviewing the qualities of a good textbook and the criteria others use to review textbooks, consider what criteria are most important to you when choosing learning resources for your class.
- Do you have an articulated set of criteria you use when picking a textbook? What criteria do you use when choosing a textbook?
- What are other important criteria that are missing from the list?
- What do you think are the most important qualities of a textbook? Why?
- Thinking about textbooks you have used in the past, what do you like/don’t like about them?
- Do you think the criteria, considerations and qualities are different for printed books than for digital books?
Weiten, W., Guadagno, R. E., & Beck, C. A. (1996). Student’s Perceptions of Textbook Pedagogical Aids. Teaching of Psychology, 23(2), 105-107. doi:10.1207/s15328023top2302_8
Gurung, R. A. R. (2003). Pedagogical Aids and Student Performance. Teaching of Psychology, 30(2), 92-95. doi:10.1207/S15328023TOP3002_01