The Literary Canon on Jeopardy

Erik Fredner

Oregon State University

2025-07-16

Follow along:

What is Jeopardy?

  • The most popular American quiz show on TV
  • On air since 1984
  • Approximately 9 million viewers per week1
  • Audience is surprisingly diverse2

What does Jeopardy have to do with DH?

  • Questions are often about the humanities.
    • About 1 in 5 clues has a literary reference.
  • Subjects of quiz questions with literary references reveal popular knowledge of literature, which is both related to and distinct from the academic literary canon.
  • Relative difficulty of clues encodes writers’ subjective probabilities that contestants will know a given author or work.

How Jeopardy is played

  • A contestant selects a category and a value within that category. Each value is associated with a clue.
  • The higher the value, the more difficult the clue.
  • After the clue is revealed, any contestant may respond.
  • The first correct response adds the clue’s value to that player’s total. Incorrect responses subtract the clue’s value.
  • Special clues allow players to wager any amount of their winnings.

Example

  • CATEGORY: Classic Literature
  • VALUE: $800
  • CLUE: Chapter 1 of this book describes “a cyclone cellar, where the family could go in case one of those… whirlwinds arose”
  • RESPONSE: The Wonderful Wizard of Oz1

Data

  • The editor of the Jeopardy fan site J! Archive gave me a copy of their database for this research.
  • J! Archive created data for 548,000+ clues.

Process

  1. Use a large language model (LLM) to do binary classification for all clues to determine whether they 1) reference literature and 2) are about literature. Validate outputs.
  2. For clues that reference literature, use LLMs to extract normalized authors and works referenced in the clue. Validate outputs.
  3. Associate normalized references with clue metadata for analysis.

“Reference” vs. “about” literature

  • Does the clue reference literature?
    • e.g., “this director of The Lord of the Rings (2001)…”
    • References Tolkien’s fantasy series.
    • But the question is about the film.
  • Is the clue about literature?
    • e.g., “this author of The Lord of the Rings (1954)…”

Binary classification evaluation

  • Gold-standard data for 1,250 clues
  • N gives ±2% margin of error for 550k questions
  • o3 gives the best result

Clue references literature

Model Precision Recall F1 N
o4-mini 0.925 0.877 0.9 1250
gpt-4.1 0.85 0.944 0.895 1250
o3 0.842 0.952 0.894 1250
gpt-4.1-mini 0.769 0.937 0.844 1250
gpt-4.1-nano 0.593 0.821 0.689 1250

Clue is about literature

Model Precision Recall F1 N
o3 0.848 0.932 0.888 1250
gpt-4.1 0.803 0.963 0.876 1250
o4-mini 0.919 0.827 0.871 1250
gpt-4.1-mini 0.707 0.948 0.81 1250
gpt-4.1-nano 0.737 0.791 0.763 1250

Extracting structured data (simple)

Only for questions that reference literature, use an LLM to output Pydantic-validated JSON.

{
  "authors": [
    {
      "id": "a1",
      "name": "L. Frank Baum",
      "birth": 1856,
      "death": 1919
    }
  ],
  "works": [
    {
      "id": "w1",
      "title": "The Wonderful Wizard of Oz",
      "year": 1900,
      "author_ids": [
        "a1"
      ]
    }
  ]
}

Provisional results

Percentage of Jeopardy questions about literature over time.

Most referenced authors

writer count
William Shakespeare (1564-1616) 5285
Charles Dickens (1812-1870) 1188
Mark Twain (1835-1910) 879
Edgar Allan Poe (1809-1849) 711
Homer 665
Lewis Carroll (1832-1898) 633
Ernest Hemingway (1899-1961) 612
John Steinbeck (1902-1968) 499
Stephen King (1947- ) 498
Herman Melville (1819-1891) 478
Arthur Conan Doyle (1859-1930) 457
Tennessee Williams (1911-1983) 452
George Bernard Shaw (1856-1950) 437
L. Frank Baum (1856-1919) 424
Henry Wadsworth Longfellow (1807-1882) 424
Rudyard Kipling (1865-1936) 414
J.R.R. Tolkien (1892-1973) 414
Robert Louis Stevenson (1850-1894) 401
Nathaniel Hawthorne (1804-1864) 400
Victor Hugo (1802-1885) 396
Jane Austen (1775-1817) 395
F. Scott Fitzgerald (1896-1940) 385
J.K. Rowling (1965- ) 380
Dr. Seuss (1904-1991) 377
Oscar Wilde (1854-1900) 374
Margaret Mitchell (1900-1949) 371
George Orwell (1903-1950) 347
Agatha Christie (1890-1976) 345
Brothers Grimm (Jacob Grimm: 1785-1863, Wilhelm Grimm: 1786-1859) 329
James Joyce (1882-1941) 323
Alexandre Dumas (1802-1870) 322
Miguel de Cervantes (1547-1616) 321
Leo Tolstoy (1828-1910) 315
T.S. Eliot (1888-1965) 305
William Faulkner (1897-1962) 303
Alfred, Lord Tennyson (1809-1892) 302
Washington Irving (1783-1859) 294
Hans Christian Andersen (1805-1875) 286
Robert Frost (1874-1963) 283
Jules Verne (1828-1905) 281
Louisa May Alcott (1832-1888) 271
Arthur Miller (1915-2005) 267
Walt Whitman (1819-1892) 266
Geoffrey Chaucer (c. 1343-1400) 259
James Fenimore Cooper (1789-1851) 257
John Milton (1608-1674) 256
Dante Alighieri (1265-1321) 256
Jonathan Swift (1667-1745) 246
Eugene O’Neill (1888-1953) 243
J.M. Barrie (1860-1937) 243

How Jeopardy authors conform to literary canonicity

  • Shakespeare is Shakespeare
  • Nineteenth-century novelists rank highly
  • Authors from before the 19th century must be major: Homer, Cervantes, Chaucer, Milton, Dante, etc.
  • Few contemporary authors. Most have already “passed the test of time.”
  • Top 50 authors’ identities:
    • Entirely western
    • Entirely white, except Alexandre Dumas
    • Mostly Anglophone
    • Mostly male (5 of the top 50 are women)

Clue face value != clue difficulty

Round Row DD Med. Value p(correct) Questions
DJ 5 True $2,000 58.5% 4,311
J 5 True $1,000 62.5% 2,919
FJ FJ False N/A 63.0% 9,223
DJ 4 True $2,000 63.7% 6,874
DJ 3 True $2,000 66.2% 5,071
J 4 True $1,000 66.3% 3,241
J 1 True $550 66.7% 6
J 3 True $1,000 70.1% 2,247
DJ 2 True $2,000 71.8% 1,842
J 2 True $1,000 73.7% 700
DJ 1 True $1,400 79.4% 34
DJ 5 False $2,000 82.9% 37,620
J 5 False $1,000 86.6% 42,792
DJ 4 False $1,600 86.9% 40,019
DJ 3 False $1,200 89.6% 44,382
J 4 False $800 89.9% 46,188
J 3 False $600 91.6% 49,069
DJ 2 False $800 92.0% 49,273
J 2 False $400 93.2% 51,865
DJ 1 False $400 94.7% 52,443
J 1 False $200 95.5% 53,580

Hardest authors by clue position

Rank Author Avg Difficulty Z-Score Clue Count
1 Robert Graves 41.82 2.281 58
2 Nikolai Gogol 41.66 2.252 43
3 Thomas Pynchon 40.91 2.114 48
4 Aristophanes 40.17 1.978 79
5 Petrarch 39.95 1.938 46
6 August Strindberg 39.87 1.923 53
7 H.P. Lovecraft 39.64 1.881 41
8 Robert Penn Warren 39.6 1.873 68
9 Lillian Hellman 39.46 1.848 82
10 Edgar Lee Masters 39.23 1.805 54
11 Daphne du Maurier 38.96 1.756 99
12 Bret Harte 38.92 1.748 44
13 Umberto Eco 38.73 1.713 53
14 Wilkie Collins 38.55 1.68 53
15 Alexander Hamilton 38.35 1.644 44
16 Stephen Vincent Benét 37.98 1.576 57
17 Saul Bellow 37.8 1.543 76
18 George Sand 37.57 1.5 62
19 Albert Camus 37.51 1.489 112
20 James M. Cain 37.47 1.482 44
21 John Donne 37.46 1.48 115
22 Alexander Pushkin 37.41 1.471 82
23 Aeschylus 37.4 1.469 94
24 William Gibson 37.28 1.447 70
25 William Butler Yeats 37.21 1.434 92

This only counts authors who have appeared in 40 or more clues, i.e., more than once per season on average.

Easiest authors by clue position

Rank Author Avg Difficulty Z-Score Clue Count
357 Carolyn Keene 20.81 -1.579 46
358 L. Frank Baum 20.75 -1.59 422
359 Ian Fleming 20.73 -1.594 185
360 Thomas Jefferson 20.66 -1.607 65
361 Miguel de Cervantes 20.42 -1.651 321
362 Thomas Harris 20.35 -1.664 107
363 Brothers Grimm 19.95 -1.737 328
364 Daniel Defoe 19.72 -1.779 232
365 Stephenie Meyer 19.59 -1.803 94
366 Harriet Beecher Stowe 19.54 -1.812 219
367 Helen Keller 19.54 -1.812 63
368 J.R.R. Tolkien 19.45 -1.829 414
369 Tim Rice 19.36 -1.845 41
370 Dr. Seuss 19.36 -1.845 376
371 George R.R. Martin 19.32 -1.853 97
372 Louisa May Alcott 19.31 -1.855 271
373 Anne Frank 19.3 -1.856 92
374 P.L. Travers 19.09 -1.895 52
375 Peter Benchley 18.86 -1.937 51
376 Margaret Mitchell 18.6 -1.985 370
377 Erle Stanley Gardner 18.38 -2.026 47
378 J.K. Rowling 17.91 -2.112 378
379 Suzanne Collins 17.79 -2.134 82
380 Charles M. Schulz 17.36 -2.213 69
381 Aesop 15.71 -2.516 169

Interpretation of relative difficulty

  • Authors in harder clues are more canonical.
  • Authors in easier clues are more associated with popular culture and literature for young people.

Literature on Jeopardy

  • Jeopardy rewards a (largely) Anglophone literature that reproduces the traditional literary canon in both form (a hierarchical list) and content (familiar names).
  • But its view of literature does not reproduce the academic canon directly. It mixes the popular and the prestigious.
  • It reinforces the high vs. low culture distinction through the difficulty of its questions.
    • High culture more valuable than low culture.
    • High culture more valuable because it is rarer.

Next steps?

  • Improve classification and extraction steps
  • Conduct similar analyses with works
  • Reconcile entities by referencing external sources
    • Authors to VIAF
    • Works to MLA International Bibliography entities and tags

Thank you!

Appendix

Jeopardy and AI

In 2011, IBM’s Watson defeated Ken Jennings and Brad Rutter.1

From left to right: Jennings, Watson, Rutter.

Popularity and prestige

French literary field, 1850-1900.1

Jeopardy per Bourdieu?

A more complex clue

THE OLD TESTAMENT

In Genesis 21, Abraham banishes Hagar & this son of theirs to the desert; call him…

Ishmael

Ways in:

  1. Genesis
  2. Moby-Dick (“Call me Ishmael.”)

Extracting structured data (complex)

{
  "authors": [
    {
      "id": "a1",
      "name": "Herman Melville",
      "birth": 1819,
      "death": 1891
    }
  ],
  "works": [
    {
      "id": "w1",
      "title": "Book of Genesis",
      "container_id": "w2"
    },
    {
      "id": "w2",
      "title": "Bible"
    },
    {
      "id": "w3",
      "title": "Moby-Dick",
      "year": 1851,
      "author_ids": [
        "a1"
      ]
    }
  ]
}

How Jeopardy authors break from literary canonicity

  • Playwrights other than Shakespeare (e.g., Williams, Shaw)
  • Genre fiction (e.g., King, Christie)
  • Children’s literature (e.g., Carroll, Baum)
  • Works with film and TV adaptations (e.g., Tolkien, Mitchell)

Demoted

  • Poetry
  • Antiquity (other than Homer)
  • The Renaissance
  • World literature
  • And much more…

Works Cited

Barré, Jean, Jean-Baptiste Camps, and Thierry Poibeau. “Operationalizing Canonicity: A Quantitative Study of French 19th and 20th Century Literature.” Journal of Cultural Analytics 8, no. 3 (October 2023). https://doi.org/10.22148/001c.88113.
Baum, L. Frank. The Wonderful Wizard of Oz. Chicago: G.M. Hill Co., 1900.
Bourdieu, Pierre. The Field of Cultural Production: Essays on Art and Literature. Translated by Randal Johnson. New York: Columbia University Press, 1993.
Brode, Noah. “The AverageJeopardy!’ Watcher Isn’t Who You Think They Are.” CivicScience, March 2022.
Deijl, Lucas van der, Antal van den Bosch, and Roel Smeets. “The Canon of Dutch Literature According to Google.” Journal of Cultural Analytics 4, no. 2 (September 2019). https://doi.org/10.22148/16.046.
Feldkamp, Pascale, Yuri Bizzoni, Mads Rosendahl Thomsen, and Kristoffer L. Nielbo. “Measuring Literary Quality. Proxies and Perspectives.” Journal of Computational Literary Studies 3, no. 1 (October 2024). https://doi.org/10.48694/jcls.3908.
Fredner, Erik, and J. D. Porter. “Counting on The Norton Anthology of American Literature.” PMLA 139, no. 1 (January 2024): 50–65. https://doi.org/10.1632/S0030812923001189.
Karmanov, Fedor, and Joshua Kotin. “A Counterfactual Canon.” Journal of Cultural Analytics 9, no. 2 (May 2024). https://doi.org/10.22148/001c.116915.
Markoff, John. “Computer Wins on Jeopardy!’: Trivial, It’s Not.” The New York Times, February 2011.
Porter, J. D. “Popularity/Prestige.” Pamphlets of the Stanford Literary Lab, no. 17 (September 2018).
Roca, Teresa. “Jeopardy! Beats Wheel of Fortune in Ratings by 500k Viewers.” The US Sun, June 2025.
Rutherford, Markella, and Peggy Levitt. “Who’s on the Syllabus?: World Literature According to the US Pedagogical Canon.” Journal of World Literature 5, no. 4 (November 2020): 606–29. https://doi.org/10.1163/24056480-00504009.
Wenz, John. “What Goes Into a Jeopardy! Clue.” Popular Mechanics. https://www.popularmechanics.com/culture/tv/a19435419/how-to-write-a-jeopardy-clue/, March 2018.