> ## Documentation Index
> Fetch the complete documentation index at: https://meta.niceshare.site/llms.txt
> Use this file to discover all available pages before exploring further.
# Higher-Order Thinking
> Higher-Order Thinking works a question whose path is not already given, then leaves a move another person can check. Learn the sources and the limits.
**Category**: Thinking
**Type**: Practice Habit
**Origin**: Lauren B. Resnick, *Education and Learning to Think* (National Academy Press, 1987); Arthur Lewis and David Smith, *Theory Into Practice* (1993); the school ladder is often taken from Bloom and colleagues (David McKay, 1956) and from Anderson and Krathwohl (Longman, 2001)
**Also known as**: Working a question whose path is not given
**Quick Answer** — **Higher-Order Thinking** is the habit of working a question when no procedure yet gives the path, and of leaving the move another person can check. Lauren Resnick's 1987 list calls that work nonalgorithmic. The path of action is not fully specified in advance. Arthur Lewis and David Smith, in 1993, name the job: decide what to believe or what to do, make something, predict, or solve a problem that has no routine path.
## What is Higher-Order Thinking?
Higher-Order Thinking is the habit of working a question that a path already written down cannot finish, and of showing the move that produced the answer.
> Higher order thinking occurs when a person takes new information and information stored in memory and interrelates and/or rearranges and extends this information to achieve a purpose or find possible answers in perplexing situations.
That sentence is the definition Arthur Lewis and David Smith offered in 1993. The everyday picture is a closed street, and a marked detour is a procedure. [Algorithmic Thinking](/thinking/algorithmic-thinking) is that card: the steps are definite, and they stop. If no card says which turn to take, you weigh routes that cost different things, and you leave a reason another person can check. That weighing is this habit.
[Critical Thinking](/thinking/critical-thinking) decides what to believe, and that decision is one purpose on the 1993 list. A prediction, a thing you make, and a problem with no routine path sit on the same list. [Analytical Thinking](/thinking/analytical-thinking) takes a whole apart, and that move can serve the habit. The habit also names the purpose and leaves a check. [Creative Thinking](/thinking/creative-thinking) produces a new combination, and making is one purpose on the list. A combination whose steps were already on a card is a different habit.
Resnick's first feature is the practical test: the path of action is not fully specified in advance. If a second person can only copy your steps, the card is a procedure. If the next case has no such card, this habit is the one that starts.
### Higher-Order Thinking in 3 Depths
* **Beginner**: The cue is a question a worked example would finish. If the example covers it, you are still on a written path. If it does not, write what you are trying to produce, and one reason the first path fails.
* **Practitioner**: Name the purpose before the answer. Use Lewis and Smith's list: believe, do, make, predict, or a problem with no routine path. Then write the move. Say what you joined, what you left out, and what would change the answer.
* **Advanced**: A poster that places remember, understand, and apply under analyze, evaluate, and create is a later teaching chart. In the 2001 revision, Apply still includes carrying out a given procedure. Resnick's test is whether the path was specified in advance. A larger gain on a basic-skills test, after a demanding assignment, is not by itself a score for this habit.
## Origin
The work of judging, making, and solving is old. The school name, and the ladder people pin to it, arrived later, and they do not mark the same boundary.
In 1956 a committee edited by Benjamin S. Bloom published *Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook I: Cognitive Domain* (David McKay, New York). Max D. Engelhart, Edward J. Furst, Walker H. Hill, and David R. Krathwohl worked on it with him. The cognitive categories run Knowledge, Comprehension, Application, Analysis, Synthesis, and Evaluation. David Krathwohl's 2002 overview says the original list was ordered from simple to complex and from concrete to abstract, and that it assumed a cumulative hierarchy. Mastery of a simpler category was treated as prerequisite to the next. The handbook classifies goals for tests and curricula, so it is a ladder of objectives. This page does not treat that handbook as the document that coined the school name.
Lauren B. Resnick, of the University of Pittsburgh, wrote *Education and Learning to Think* (National Academy Press, Washington, D.C., 1987). She says thinking skills resist the precise definitions schools like to set as objectives. She still offers a working list, so a reader can recognize the work when it happens. The path is not fully specified in advance. The whole path is often not visible from one vantage point. There is often more than one solution, and each has costs and benefits. The work uses nuanced judgment, and more than one criterion, and those criteria sometimes conflict. Not everything that bears on the task is known. The thinker regulates the process. Resnick says we do not recognize the work when someone else calls the plays at every step. The thinker imposes meaning and finds structure in apparent disorder. The work is effortful. There is considerable mental work in the elaborations and judgments. She also draws a line about who the work is for. Putting thinking and problem solving into someone's curriculum was not new. Putting it into everyone's curriculum was the new demand. The list describes the work. It is not a trial, and it does not report a gain in learning.
Fred M. Newmann put a short test under the same name in 1991, in *Theory and Research in Social Education*, volume 19, pages 324–340. He calls higher order thinking challenge and expanded use of the mind. He calls the contrasting work routine, mechanistic application, and limited use of the mind.
> Challenge or expanded use of mind occurs when a person must interpret, analyze, or manipulate information, because a question to be answered or a problem to be solved cannot be resolved through the routine application of previously learned knowledge.
His examples of the routine end are concrete. List information you memorized. Insert numbers into a formula you already learned. Apply the rules for a footnote. A hard problem can still have one correct answer, or several answers that conflict. The test is whether routine application finishes it.
Lewis and Smith wrote the 1993 definition in *Theory Into Practice*, volume 32, pages 131–137, in an issue called "Teaching for Higher Order Thinking." Lewis was professor emeritus, and Smith was dean, in the College of Education at the University of Florida. They quote Larry Cuban's 1984 remark that the pile of labels was a "conceptual swamp." The article they cite is in *Review of Educational Research*, and the phrase sits on page 676. They wanted one term wide enough for problem solving, critical thinking, creative thinking, and decision making. The purposes they list are deciding what to believe, deciding what to do, creating a new idea, object, or artistic expression, making a prediction, and solving a nonroutine problem. They say the definition uses earlier psychology, including Frederic Bartlett's idea of filling a gap. They are offering a definition, and they are not reporting a measured gain.
Lorin W. Anderson and David R. Krathwohl edited the revision, *A Taxonomy for Learning, Teaching, and Assessing* (Longman, 2001). Krathwohl's overview is in *Theory Into Practice*, volume 41, pages 212–218 (2002). The process names become Remember, Understand, Apply, Analyze, Evaluate, and Create. Synthesis is renamed Create and placed after Evaluate. A second dimension names the kind of knowledge: factual, conceptual, procedural, and metacognitive. [Metacognition](/thinking/metacognition) on this site is Flavell's name for thinking about one's own thinking. In the 2001 table, metacognitive knowledge is one kind of knowledge, and it can cross any of the six processes. It is not a top rung. Apply, in that overview, is carrying out or using a procedure in a given situation. Executing a procedure that was already given can be routine. Susan M. Brookhart, in *How to Assess Higher-Order Thinking Skills in Your Classroom* (ASCD, Alexandria, Virginia, 2010), sorts the school definitions into three groups: transfer, critical thinking, and problem solving. She takes transfer from the 2001 book: students remember, and they also make sense of what they learned and use it. She treats a problem as a goal whose path the person does not already recognize. The book is a guide to assessment, and it is not a trial of this habit.
## Key Points
Higher-Order Thinking pays when a copied path would answer a different question. It stalls when the purpose is still unnamed, or when a finished procedure is treated as if the path were still open.
Ask what a second person would do with only the card you already have. If that card names the input, the steps, and the stop, the work ahead is to run it. Resnick's line is that the path of action is not fully specified in advance. Newmann's line is that routine application of what you already learned does not finish the question. A closed street with a posted detour fails this test. A closed street with no detour passes it.
Lewis and Smith separate five purposes: what to believe, what to do, what to make, what to predict, and a nonroutine problem. Write one of them down before you collect reasons. "Think harder" does not say which purpose the answer has to serve. A hiring note that never says whether it is choosing a person or predicting a first-year result will mix two questions.
Newmann's challenge is to interpret, analyze, or manipulate information. The Chicago writing standard below asks students to interpret, analyze, synthesize, or evaluate, rather than only to reproduce. Write what you joined, what you ruled out, and which conflicting criterion you let win. An answer with no move cannot be checked by someone who was not there. A polished paragraph can still be a reproduction.
Lewis and Smith start from new information and from information stored in memory. The 1956 handbook treated simpler objectives as prerequisite to later ones. A question about a budget still needs the arithmetic. A question about a source still needs what the source says. Dropping that layer does not make the work higher. It makes the move impossible to check. The habit begins when those pieces are in hand and the path that uses them is not yet written.
## Applications
Use Higher-Order Thinking when the next case has no card that already finishes it. Name the purpose, write the move, and keep the routine piece the question still needs.
Take a review item you can finish by matching a worked example. Change one condition so the example's path no longer fits. Write the purpose, the move, and the fact that would change the answer. If the classmate can still finish it by copying the example, the path was already written.
For a work decision with more than one defensible end, list the criteria that pull apart. Say which one you let win, and what new fact would reverse it. Resnick's list includes multiple criteria that sometimes conflict. A recommendation that hides the losing criterion cannot be checked next month.
For a household decision another adult will have to live with, write the purpose and one observation that would make you choose differently. A note that says only "we talked it through" does not leave the move. The next person should be able to see what you weighed.
At a clinic desk, a school office, or a public meeting, some decisions already have a written rule. Run that rule as a procedure. When a case falls outside the rule, say so, and write the purpose of the exception and the move. Copying the rule past its own line is not this habit. Neither is an exception with no reason attached.
## Case Study
In January 2001, Fred M. Newmann, Anthony S. Bryk, and Jenny K. Nagaoka published *Authentic Intellectual Work and Standardized Tests: Conflict or Coexistence?* with the Consortium on Chicago School Research. The Chicago Annenberg Challenge supplied the primary funds. They asked what happened to basic-skills scores when teachers in Chicago assigned work that demanded more than reproduction. The report compares classrooms. It does not assign this habit to one group and withhold it from another.
The score they used is not a score of Higher-Order Thinking alone. They adopted a framework called authentic intellectual work, from earlier research at the Center on Organization and Restructuring of Schools. Three criteria make the framework. Students construct knowledge, through disciplined inquiry, and the product has value beyond success in school. Construction of knowledge, in their words, means organizing, interpreting, evaluating, or synthesizing prior knowledge to solve problems that routine use of earlier information or skills will not solve. That clause is the neighbor of this habit. The other two criteria are additional. On the writing rubric, standard 1 asks students to interpret, analyze, synthesize, or evaluate information, rather than merely to reproduce it. Standard 2 asks them to draw a conclusion, a generalization, or an argument, and to support it in extended writing. The mathematics standards ask students to organize and interpret information, to explain a solution path, and to address a problem like one they meet outside school. The measure adds the standards. A Many-Facet Rasch model adjusts for how hard each standard is and how strict each scorer is. The authors then place the result on a scale from 0 to 10.
The assignments came from writing and mathematics in grades 3, 6, and 8. The first three years of the study looked at about 1,800 third graders, about 1,700 sixth graders, and about 1,400 eighth graders, in the classrooms of teachers who took part. Field researchers collected assignments in 19 elementary schools from 1997 through 1999. Teachers were asked for work they saw as typical and work they saw as challenging. In a full year the request was four typical assignments and two challenging ones. In 1997 the collection covered the spring semester only, and the request was two of each. At least 60 percent of the assignments were scored a second time, by a different teacher, without the first score in view. Expert raters later found that scores had drifted upward from 1997 to 1998. The authors adjusted the earlier years onto the 1999 scale.
Two tests supplied the outcomes. The Iowa Tests of Basic Skills is the nationally normed test the Chicago Public Schools gave each May, in reading and mathematics. The Illinois Goals Assessment Program is the state test in reading, mathematics, and writing, given in grades 3, 6, and 8. Only the 1997 and 1998 IGAP results enter this report. In 1999 Illinois introduced a new assessment. For reading and writing outcomes, the authors used the writing assignments. For mathematics outcomes, they used the mathematics assignments. The models control for race, gender, socioeconomic status, and prior achievement.
On the ITBS, one-year learning gains in classrooms with high-quality assignments were 20 percent greater than the national average. In classrooms where the assignments demanded less, students gained 25 percent less than the national average in reading and 22 percent less in mathematics. Those sentences compare classrooms with the national average gain. They do not compare a class that was taught this habit with a class that was not. The 20 percent figure is not split by subject in that sentence. The 25 percent and the 22 percent belong to the lower-demand classrooms, and they are split by subject.
The IGAP was not given every year, so the authors did not compute a yearly gain on it. They computed an adjusted classroom mean, using demographics and the prior year's ITBS. To show the size, they describe two students who share a background, a school, and a prior ITBS score. In the model, student A sits in classrooms whose writing assignments and mathematics assignments both score high for authentic intellectual work. Student B sits in classrooms where both score low. The model predicts that the next spring A would score 32 points higher on IGAP reading, 48 points higher on IGAP mathematics, and 2.3 points higher on the IGAP writing rubric. The report turns those gaps into effect sizes of 0.43, 0.64, and 0.52. A and B are a contrast inside the model. They are not two named children the authors followed.
A further cut asks whether students who arrived ahead, and students who arrived behind, show the same pattern. For mathematics, the one-year ITBS gains of students who started lower were 29 percent larger in the high-quality-assignment classrooms than the gains of similar students in the low-quality-assignment classrooms. For students who started higher, the incremental gain tied to the high-quality assignments was 17 percent. For reading, the report's comparable figures are 42 percent for students who started higher and 28 percent for students who started lower. The authors' reading is that the assignments enriched instruction for both groups in this comparison. The percentages are not a single gain for every student.
They also asked whether the harder assignments simply followed classrooms that were already advantaged. Across 437 classes in those 19 schools, the correlations between assignment quality and prior achievement, racial composition, and socioeconomic composition were weak, and they were not statistically significant. The authors read assignment quality as tied more closely to teachers' choices than to those traits. That is a reading of those correlations. It is not an experiment that assigns the tasks.
The lesson is narrow. Classrooms whose collected assignments scored higher on a three-part rubric also showed larger basic-skills gains, after the controls above. One part of that rubric asks for interpretation rather than reproduction. That part is the neighbor of this habit. The gain belongs to the whole rubric, in a comparison of classrooms that were not randomly assigned. [Statistical Thinking](/thinking/statistical-thinking) keeps the national-average comparison, the model contrast of 32, 48, and 2.3, and the within-classroom percentages from collapsing into one effect of the habit.
## Boundaries and Failure Modes
Higher-Order Thinking stops being the right name in three places. The path may already be written. The Chicago percentages may be treated as a property of the habit. Or a poster of six verbs may be treated as the habit itself.
The first boundary is a finished procedure. Resnick does not count the work when someone else calls the plays at every step. Newmann's routine end is a memorized list, a learned formula, or a footnote rule. [Algorithmic Thinking](/thinking/algorithmic-thinking) is the habit that writes those steps so a second person can run them and stop. Use that habit when the card exists. This habit starts when the next case has no such card. Writing "think harder" on the card does not create a path, and it does not leave a move.
The second boundary is the Chicago report. Do not cite 20 percent, 25 percent, or 22 percent as the effect of Higher-Order Thinking, of a workshop, or of one rewritten question at home. The 20 percent compares high-scoring assignments with the national average gain on the ITBS. The 25 percent and the 22 percent describe lower-demand classrooms, in reading and in mathematics. Do not cite 32 points, 48 points, or 2.3 points as the gap between two real classmates. Those figures are the model's prediction for a constructed pair, with effect sizes 0.43, 0.64, and 0.52. Do not cite 29 percent, 17 percent, 42 percent, or 28 percent as what any student gains from this habit. They compare one-year ITBS gains for students with different starting points, inside the same observational design. The rubric scores three criteria together. Students were not placed at random. Teachers chose which assignments to hand in. The authors conclude that such assignments improve scores on conventional tests. That verb is their conclusion about this comparison.
The common misuse is the poster. A chart that labels Remember, Understand, and Apply as low, and Analyze, Evaluate, and Create as high, is a later teaching aid. The 2001 overview still defines Apply as carrying out or using a procedure in a given situation. Resnick's test is whether that procedure was already specified. Lewis and Smith start from information you have and information you just met. Dropping the routine layer does not raise the work. Anat Zohar and Yehudit J. Dori, in the *Journal of the Learning Sciences*, volume 12, pages 145–181 (2003), looked at four projects that set out to teach higher-order thinking in science classrooms. The students were in grades 7 through 12. Zohar was at the Hebrew University, and Dori was at the Technion. By the end, students with high academic achievement had higher thinking scores than students with low academic achievement. Both groups made considerable progress against their own initial scores. In one of the four studies, the net gain of the low-achieving group was significantly higher than the net gain of the high-achieving group. The paper does not give one effect size for all four projects. That pattern is not the Chicago ITBS result. Neither study shows that a hard question can replace the routine skill the question still uses.
## Common Misconceptions
Three confusions travel with the school name. The habit gets mistaken for the top of a poster, for a privilege of students who already score high, and for the Chicago percentages.
Remember, Understand, Apply, Analyze, Evaluate, and Create are the 2001 process names. Krathwohl's overview presents them as an ordered revision of the 1956 goals. It does not present a separate object called Higher-Order Thinking. Apply still includes using a procedure that the situation already gives. Resnick's test, and Newmann's test, ask whether a written path finishes the question. A task can use the verb "analyze" and still be a reproduction. A task can sit lower on the poster and still be open, if no procedure yet gives the path.
Resnick's 1987 contrast is someone's curriculum against everyone's. The Chicago report finds weak, nonsignificant correlations between assignment quality and prior achievement, across 437 classes. In that comparison, students who started lower and students who started higher both showed larger ITBS gains in the high-quality-assignment classrooms. The mathematics gap in favor of students who started lower is 29 percent against 17 percent. The reading gap runs the other way, 42 percent against 28 percent. Zohar and Dori report progress for both achievement groups on thinking scores, and a larger net gain for the low-achieving group in one of four studies. None of these results is a license to drop the knowledge the question uses. They do cut against the idea that the work belongs only to students already ahead.
The 20 percent is one sentence in the 2001 report: ITBS learning gains in classrooms with high-quality assignments were 20 percent greater than the national average. The assignments were scored for authentic intellectual work. Construction of knowledge is one of three criteria. Disciplined inquiry and value beyond school are the other two. The design controls for race, gender, socioeconomic status, and prior achievement. It does not randomly assign the work. The 32-point, 48-point, and 2.3-point IGAP contrasts are predictions for a modeled pair of students. A larger basic-skills gain is not a direct score of this habit.
## Related Concepts
Higher-Order Thinking sits next to habits that judge a claim, take a whole apart, make a new combination, watch one's own thinking, or write a path that already stops. The links below are neighbors. They are not other names for this habit.
Decides what to believe. That decision is one purpose on the 1993 list. A prediction or a made thing can be this habit without that decision.
Takes a whole apart and states how the parts depend on each other. The move can serve this habit. The habit also names a purpose and leaves a check.
Produces a new combination. Making is one purpose Lewis and Smith name. A combination whose steps were already written is a different habit.
Flavell's name for thinking about one's own thinking. In the 2001 table, metacognitive knowledge is a kind of knowledge, not the top process.
Writes a finite, definite procedure that stops, so a second person can run it. Resnick's test says this habit starts when that path is not yet specified.
Examines your own action while the work is still open, and again after. Self-regulation is one of Resnick's features. It is not the whole habit.
## One-Line Takeaway
When no card yet finishes the question, name the purpose, write the move another person can check, and keep the routine skill the question still needs.
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.