Activity 4.8a: AI Dialogue Evaluation Activity

30–40 minutes

Learning Targets

I can evaluate AI-generated writing and explain how human creativity produces more authentic dialogue.

Your Task

Throughout this unit, you've worked on developing authentic character voices, writing dialogue with subtext, and crafting speech patterns that respect cultural identities. Now you'll put those skills to the test by comparing human-written dialogue to AI-generated dialogue. This activity builds your critical evaluation skills—an essential part of working responsibly with AI as a writer.

Directions

  1. Read both scenes carefully. Below are two dialogue scenes between the same two D&D characters (a suspicious ranger named Kira and a smooth-talking merchant). One was written by a student. The other was generated by AI. Both are approximately the same length.
  2. For EACH scene, analyze using the questions below. Don't skip any scene—you need to evaluate both.
  3. Identify which is human-written and which is AI-generated. Use specific evidence from the text.
  4. Write your conclusion (4–6 sentences) reflecting on what the comparison reveals about AI's strengths and limitations in dialogue writing.

The Two Scenes

SCENE A
"Three gold for a healing potion?" Kira's hand didn't move toward her coin purse.
"Supply and demand, my friend. The roads aren't safe, and I take considerable risk—"
"You take considerable liberties with the word 'considerable.'"
The merchant's smile tightened. Just a fraction. "You wound me."
"Not yet." She picked up the potion, turned it in the firelight. The liquid was cloudy. "How old is this?"
"Fresh as morning dew."
"Morning dew doesn't settle."
SCENE B
"How much for the healing potion?" Kira asked the merchant.
"Three gold pieces," the merchant replied with a smile. "It's a fair price given the current dangers on the roads."
"That seems expensive," Kira said, crossing her arms. "I've seen them for less in other towns."
"Ah, but those other towns don't have the quality I offer," the merchant said smoothly. "My potions are top-notch, I assure you."
"I'm not sure I trust that," Kira said suspiciously.
"You can trust me completely," the merchant responded. "I have a reputation to maintain."

Analysis Questions

For EACH scene, answer all four questions:

  1. Voice distinction: How distinct are the two characters' voices? Can you tell who's speaking without the dialogue tags? What specific words, sentence structures, or patterns make one character sound different from the other?
  2. Subtext: Is there subtext—meaning beneath the surface? What's actually being said vs. what's really meant? What does each character want, and how do they try to get it?
  3. Nonverbal communication: How do physical details (actions, gestures, facial expressions, setting details) enhance—or fail to enhance—the dialogue? What does the body language tell you that the words don't?
  4. Dialogue craft: Does every line pass the "So What?" test? Are there pitfalls like on-the-nose dialogue ("As You Know, Bob" moments), exposition being told rather than shown, or unnecessary explanation? Does the dialogue move the scene forward, reveal character, or do both?

Comparison and Identification

  1. Based on your analysis, identify which scene you believe was written by a human and which by AI. Explain your reasoning using specific evidence from both scenes. What textual clues led you to your conclusion?

Sentence Frames for Your Analysis

"Scene ___ has stronger subtext because ___. For example, the line '___' suggests ___ beneath the surface."
"In Scene ___, the characters sound more/less distinct because ___."
"The nonverbal detail of '___' in Scene ___ adds meaning by showing ___."
"Scene ___ struggles with ___ because ___."
"I believe Scene ___ was written by a human because ___, while Scene ___ shows AI characteristics like ___."

Final Reflection

Write a 4–6 sentence conclusion: What does this comparison reveal about the strengths and limitations of AI when it comes to dialogue writing? What can human writers do that AI struggles with? What patterns did you notice in the two scenes?

AI Guidance for This Activity

This is one activity where thinking ABOUT AI is the point. But do not use AI to write your analysis. Your ability to evaluate dialogue quality is what's being assessed. Use your own critical thinking skills to examine these scenes, compare them, and draw your own conclusions. That's where the learning happens.

What to Submit

Submit your annotated notes from the analysis questions above, your identification of which scene is which (with reasoning), and your final reflection. Your notes should be thorough enough that another reader could understand your thinking process.

What's Next

Assignment 4.9: Culminating Multi-Voice Scene — You'll draw on all the dialogue, character voice, and speech pattern techniques from this unit to write your own multi-character scene with distinct, authentic voices.