AI DM Evaluation Activity

~20-30 minutes

Learning Targets

I can evaluate AI DM techniques for supporting player agency.

I can identify how different DM approaches affect player experience.

Context

In the lesson, you learned about player agency and what makes DM design responsive. Now you'll compare AI-generated campaign content to human-crafted content and evaluate the difference.

Directions

Step 1: Read Both Quest Hooks

Read the two quest hooks below. One was written by a human DM for a specific group of players. The other was generated by AI. Don't look at the labels yet—analyze both without knowing which is which.

Quest Hook A

The party arrives at a small fishing village where the catch has been mysteriously poor for weeks. The village elder, Maren—a woman who walks with a carved driftwood cane and speaks in proverbs her grandmother taught her—asks the party for help. She suspects the lake spirit has been offended, but she's also noticed that a mining operation upriver started around the same time the fish disappeared. She doesn't trust outsiders easily, but her granddaughter Lira vouches for one of the party members who once helped Lira during a storm. Maren's real concern isn't the fish—it's that the village is splitting over whether to confront the miners or appease the spirit, and she fears the conflict will tear her community apart.

Quest Hook B

The party arrives at a remote village plagued by a mysterious curse. The village elder, a wise old woman named Elara, asks the heroes to investigate strange occurrences in the nearby forest. Villagers have reported eerie lights, missing livestock, and whispers in the night. The elder warns that an ancient evil may have awakened and offers a reward of gold and a magical artifact for whoever can lift the curse. The adventurers must decide whether to brave the haunted forest or seek answers in the village's forgotten library.

Step 2: Analyze Both Hooks (Without Labels)

For each quest hook, answer these questions:

  • (a) Does this hook create a specific, unique situation or a generic one? Look for details that feel particular and surprising versus details that could describe any fantasy village.
  • (b) Are the NPCs specific people with real motivations, or are they stock characters? Would this NPC make different choices than any other wise elder or village leader? Do they have wants and fears that drive their actions?
  • (c) Does the hook pass the Meaningful Choice Test? Would your party have real agency here, or are they on a railroad toward a predetermined solution?
  • (d) Would you be excited to play this quest? Why or why not? Which hook makes you want to log in to the session?

Step 3: Reveal and Explain

Quest Hook A was written by a human DM. Quest Hook B was generated by AI.

Based on your analysis, explain in 4-5 sentences what specific qualities the human-written hook has that the AI-generated one lacks. Point to concrete examples from the text.

Step 4: Write Your Own Quest Hook

Write your own quest hook (5-7 sentences) that demonstrates the qualities you identified in Step 3. Make it specific, personal, and built for player agency. This is your chance to show what you've learned about responsive DM design.

Sentence Frames

Use these frames to organize your thinking:

"Quest Hook ___ feels more specific because ___."
"The NPC in Hook ___ has real motivation because ___."
"AI-generated content tends to ___, while human-crafted content ___."
"One quality that makes human DM work unique is ___."

AI Guidance

For this activity, you ARE analyzing AI-generated content—that's the point. But do not use AI to write your analysis or your original quest hook. Your critical thinking about AI's limitations is the skill being assessed.

After writing your own quest hook, you may use AI for grammar only.

Process Evidence

Submit your analysis notes alongside your completed evaluation. Show your thinking as you answered the four analysis questions for each hook.

What's Next: Continue to Lesson 5.6: Session Prep