BIE « site menu
Tracker Pixel for Entry


by Jay McTighe
Understanding by Design

Top 10 related resources »

Topic tags:

share

April 3, 2017
Beware of the Test Prep Trap

by Jay McTighe
Understanding by Design

Originally posted at Newsela.com.


Editor’s Note: Many teachers say that the need to prepare students for standardized tests is a significant barrier to using Project Based Learning. In this guest post, the co-founder of Understanding by Design explains how to rethink “test prep” to make room for deeper learning such as PBL. Part two of this article will be posted on Wed. April 5, 2017, at the end of which all citations will be listed.

 

In this era of accountability, educators throughout the nation are under pressure. Administrators are held accountable for student achievement in their schools as gauged by standardized tests. Increasingly, teachers’ evaluations include a percentage based on the results of test scores (at least in the tested grades and subjects). In some states, a school can be “reconstituted” if standardized assessment results do not improve over time. And in many communities, the test scores for a district or a school affect real estate values within their boundaries. Not surprisingly, these factors lead teachers and administrators to pay close attention to the results of external tests and strive to improve them. One consequence of this high-stakes accountability system is the increased use of “test prep” in the classroom; i.e., where teachers spend time focusing primarily on the tested content while giving students lots of practice with the test format (primarily multiple choice).

While mindful of the pressures associated with the high-stakes of accountability testing that lead to test preparation actions, excessive test prep can narrow the curriculum, undermine meaningful learning, and negatively affect student interest and motivation. At best, test prep can yield modest, short-term gains in test scores, especially if students are unfamiliar with standardized test formats and protocols.  However, I contend that the practice itself, while well intentioned, is grounded in misconceptions that may, ultimately, undermine the learning that students need to perform well on standardized tests. Let’s explore these points further.

What is Test Prep?
The practice of test prep in the U.S. has several distinguishing characteristics. Students typically engage in exercises and worksheets that mimic the format of standardized tests. Since accountability tests are generally constructed around sets of selected-response items, test prep involves lots of practice on decontextualized, multiple-choice questions. Sometimes, test prep includes timed, on-demand, assessments to simulate test-day conditions. In states that employ computer-based testing, students are often given opportunities to practice using a laptop or tablet device. Many schools and districts have institutionalized test prep by mandating the use of interim or benchmark assessments modeled after their state tests. Not surprisingly, we have witnessed the growth of an entire cottage industry of commercial “test prep” materials to address this perceived need.

A second characteristic of test prep relates to the content that is practiced. Typically, teachers are exhorted to focus only on tested knowledge and skills. The logic is understandable; i.e., since we are being held accountable for student achievement on standards A, B, and C, then we don’t have to worry about standards X, Y, and Z since they are not tested. Yet this logic has problematic consequences for learning. For example, in English/Language Arts, most standardized tests do not assess Listening, Speaking, or extended writing even though they are listed in all E/LA standards. Accordingly, teachers rarely spend any “test prep” time on listening and speaking skills or essay writing, even though these skills are fundamental to literacy development.

A third aspect of test prep involves the explicit teaching of test taking strategies and common “trigger” words used in test prompts. Examples of trigger words include compare, critical, distinguish, differentiate, key, major, significant, solve. Here are common test-taking strategies that are taught:

  • Read the question completely before you look at the answer options.
  • Read all the choices before choosing your answer.
  • Cross out any choices that you know are wrong.
  • Answer the questions that you know first; then tackle the remaining questions.
  • Usually the correct answer is the choice with the most information.
  • Always take an educated guess if you do not know the answer.
  • In a question with an “All of the above” choice, if you see that at least two correct statements, then “All of the above” is probably the answer.
  • Be aware of the time and pace yourself. Don’t spend too much time on any one item.
  • If time is about to expire, just select an answer for as many of the items as possible, even if you haven’t had time to read the questions.

It is not uncommon to see such trigger words and test-taking tips posted on classroom walls prior to test days as reminders to students.
Of course, it makes sense to familiarize students with standardized test formats since it is a genre that they will see throughout their school lives. Similarly, there is nothing wrong with imparting test taking strategies in advance of a high stakes assessment. However, such actions can be accomplished quickly and should not divert valuable instructional time from more substantive learning.

Test Prep Practice is Rooted in Misconceptions
The logic of test prep is plausible and rooted in experience from other domains. For example, if you want to improve your performance in dribbling a basketball or piano playing, then you must practice those activities. Shouldn’t the same apply to test taking?  Perhaps, but I contend that excessive test prep practices reflect two fundamental misconceptions that deserve to be critically examined:

Misconception #1 – The best (and only) way to improve test scores is to practice the test.
While this statement seems to makes sense on the surface, Wiggins and McTighe (2001) offer the following analogy to expose this misconception:

“To begin to uncover the flaw in this reasoning, consider an analogy. Once per year, we go to the doctor for a physical exam. No one particularly relishes the thought of such an exam, but we go with the understanding that it is in our long-term interest to get an objective (yet superficial) measure of our health. The doctor performs a few tests in a short span of time (e.g. blood pressure, pulse, temperature, blood work for cholesterol, etc.). The ‘physical’ is a small sample of tests, yielding a few useful indicators of one’s health status. Its validity and value stem from the fact that the results suggest our state of health, not because the physical defines healthfulness.

 

Now suppose we are terribly concerned about the final numbers (weight, blood pressure, cholesterol, etc.) and that these ‘scores’ ultimately link to our personal health insurance costs. What we might do, in our panicky state prior to each annual physical, would be to ‘practice’ for the test – focus all our energy on doing well on the physical exam (as opposed to what its indicators suggest). If our doctor knew of our actions, her response would surely be: ‘Whoa! You’re confused: you have mixed up causality and correlation here. The best way to prepare for your physical exam is to live a healthful life on a regular basis –exercising, watching weight, lowering intake of fats, eating more fiber, getting sufficient sleep, avoiding tobacco, etc.’


It would be thought silly to practice the physical exam as a way to improve one’s health. But this confusion is precisely what we see in schools all over North America. Local educators, fearful of results, focus on the indicators, not their causes. The format of the test misleads us, in other words.”

Misconception #2 – Standardized test items involve primarily recall and recognition, and thus drill and practice will be the most effective method to prepare students for them.

Given the predominant use of the multiple-choice format, there may be the assumption that these items primarily test factual knowledge, basic skills and “low-level” thinking. After all, there is a “right” answer, and all a student has to do is select that answer from a set of given alternatives. What follows from this assumption is the belief that covering lots of factual information and drilling and practicing multiple-choice items will provide adequate preparation for the accountability tests. Moreover, one might conclude that there is no need for, nor benefit from, in-depth learning involving extended thinking or the use of more authentic assessments.

Grant Wiggins (2013) points out the flaw in this reasoning: “Even though the test format requires a selected response, it does not mean that the tested knowledge is necessarily simple. The [format] deceives you into thinking that since you are mimicking the format of the test, you are therefore mimicking the rigor of the test. But data show the opposite conclusively: local tests are often less rigorous than state and national tests even when they mimic the format.”

Item analyses of consortium and state test results validate Grant’s point. Indeed, the pattern is remarkably consistent – the most widely missed items on standardized tests are not those assessing simple recall of factual knowledge or basic skills as referenced at Level 1 on the Depth of Knowledge scale (Webb, 2006). Instead, they require inference and interpretation in reading, analysis and reasoning in mathematics and science. While knowledge and skills are needed, the more difficult items demand “higher-order” thinking and involve transfer at Depth of Knowledge (DOK) Levels 2 and 3. Such items often include distractors that present typical misconceptions, common errors, and flawed reasoning that will trip up test takers who only have learned by rote. Accordingly, low-level, drill and practice is not the optimal instructional method for improving test scores.

Too often, the information revealed by test prep exercises identifies whether students have chosen the “correct” answer rather than helping teachers determine if they have a conceptual understanding of the underlying concepts and skills and can apply (transfer) those.


The second part of this article will be posted on Wed. April 5, 2017. The citations above will be listed at the end of that post.

Comments or questions about test prep in the context of PBL? Please leave them below.


 Comments

[Leave a new comment]