Think Alouds

Coding Guide

Cognitive Operation

Examples

Code

Core Cognitive Skills

Questioning

  • Who? What? When? Where? Why? So What?

Q

Connecting

  • Corroborates testimony
  • Compares/contrasts
  • Detects change/continuity over time

C

Sourcing

  • Distinguishes primary and secondary sources
  • Notes date of the document
  • Considers the personal bias of the author
  • Identifies an assumption
  • Considers how well situated the author was to report on the events in question
  • Considers what a document is useful for

S

Inferencing
  • Induction
  • Deduction
  • Forms a general thesis

I,
Thesis

Considering multiple perspectives
  • “Another way to look at this would be…”
  • Considers objections to inferences

MP

Recognizing limits to knowledge
  • Holds hypotheses as tentative (“probably” “I can’t be sure,” etc.)
  • Recognizes need for more information (“I’d like to see some documents from the Indians’ point of view.”)

H

Other Cognitive Skills

Empathy
  • A willingness to understand points of view that seem odd or off-putting at first

Empathy

Strategizing
  • “I think I’ll read the dictionary article first, to get an overview. Then read the others.”

St

Narration
  • Constructs a narrative account of events

N

Revision

  • Corrects a previous inference
  • “Looks like I was wrong to think that…”

R

Causation
  • “Another factor that may have caused Washington to retreat was…”

CN

Other Cognitive Operations

Elaboration
  • Describes prior knowledge (“Little Big Horn is in Montana, I think.”)
  • Describes a personal response (“This makes me so angry!”)
  • Autobiographical recollection (“I saw a movie about this.”)

E

Comprehension Monitoring
  • Wonders aloud (“huh? Why would Benteen do this?”)
  • Recognizes successful comprehension (“Oh, now I get it.”)
  • Aware of non-comprehension (“I don’t know what a regiment is.”)
  • Paraphrases, summarizes
  • Fix-up strategies: e.g., re-reading
  • Verbalizes a plan that reflects knowledge of the structure of texts (“well, maybe this will be explained later in the text.”

Comp,
Non-Comp

Text Description
  • Comments on style (“This is kind of wordy.”)

TD

Uncategorizable/Irrelevant
  • “It’s hot in here.”
  • “This is really hard!”

X

How we did it:

We (my coding partner and I) came up with the guide in a process that looked like this:

  1. I did some reading on think alouds.
  2. I made a preliminary coding guide that identified maybe 8 cognitive processes.
  3. We read the Think Alouds transcripts and found that students were doing things we didn't have codes for.
  4. We tentatively labeled these unexpected processes.
  5. I did some more reading to compare our labels with ones other people used in the literature.
  6. We made a final coding guide.

The six skills I was teaching in the course and was particularly interested in looking for in the TAs are the first six skills on the guide.

In the first study, my partner was a respected (i.e., well-published) professor from another institution who teaches introductory history courses in traditional ways but is open-minded and interested in teaching innovations. I paid him $500 for a couple days of work. We decided on degrees of proficiency in a process that looked like the following.

  1. We agreed that “1” would indicate the proficiency of someone who knows almost nothing about historical thinking, who shows no evidence of the skill, or severely misunderstands the skill and how to do it; and “5” that of an average expert historian, i.e., ourselves.
  2. We read half the transcripts (3) to get a feel for what students were doing, and talked about our impressions.
  3. We made a rubric to fill in the gaps between 1 and 5.
  4. We proceeded to read the transcripts, independently ranking the students' performances accoring to our rubric.

Later, when we compared our rankings, if we differed on a ranking, then we talked the ranking through with evidence from the transcripts, until we agreed on a 1–5 ranking for the particular performance in question.

With these decisions made, in a later study to replicate the first one, I used a very sharp student as my partner. I found her to be tougher with rankings than my earlier colleague.

Think Alouds document list >