Slides: Critical Play with GenAI Workshop Shared Google Document: March 2025 Critical Play with GenAI: Evaluating AI Through Disciplinary Expertise - Google Docs UDL Guidelines in Plaintext: Universal Design for Learning Guidelines - plaintext.md · GitHub
This hands-on workshop invites participants to explore generative AI through the lens of their disciplinary expertise. I believe that GenAI is most (maybe only) interesting and useful for experts working within their areas of expertise. To get valuable responses, you need to understand the specific issues and language of your field. More importantly, your expertise allows you to use your judgment to evaluate when AI is giving you something useful or not.
As faculty and staff at a university, I think it is especially important that disciplinary expertise is central to our approach to generative AI.
We're also going to focus on how AI might be used to enhance what we create, not on what AI can create independently. The distinction is subtle but important - we're not interested in having AI generate content from minimal prompts, but rather in using AI to improve and extend the work we're already doing as experts.
The workshop also follows a makerspace pedagogy since that is where I work and what I do, so the focus will be on learning through doing and reflecting.
This isn't a computer science class. Whether you've used AI tools extensively or are trying them for the first time today, what matters is your disciplinary knowledge. We're here to explore how AI intersects with your expertise, not to judge technical proficiency.
Finally, there are no manuals for AI, so instead of worrying about the "correct" way to use AI, we'll emphasize playful experimentation, critical evaluation, and collaborative learning.
Instead of thinking of working with AI as something like computer programming, think about it as having a dialogue with another person. There are problems with this framing, but it also helps level the playing field.
One of the reasons I continue to do these workshops is that I want to find ways to engage with this technology that are critical, exploratory, and interesting. I am still discovering what that means and whether it is even possible.
Not everything we do today is going to work particularly well. It's impossible to completely plan how generative AI models will respond to a given prompt, and even if you did, that wouldn't reflect the reality you all experience once you leave the workshop.
By the end of this workshop, participants will be able to:
- Demonstrate different techniques for prompting GenAI models
- Apply personas and use context to AI prompts
- Critically evaluate AI strengths and limitations based on their disciplinary expertise
- AI Realism.
- Expertise is key to evaluating and using AI
- AI is potentially hostile to workers
- Accountability
- Against foreclosure of debate
- Cannot lose sight of risks, biases, power, and impact
- I do think its revolutionary, not sure yet if it is good.
- Laptop/device with internet access for each participant
- Access to an AI tool (ChatGPT, Claude, etc.)
- Workshop resource page with shared document link
- Welcome and brief introductions (10 minutes)
- Workshop purpose and objectives
- Timer-guided round-robin: "Your name, role, and one thing you'd like to learn today" (1 minute per person)
- Workshop meta-example: Exploring the workshop prompt (15 minutes)
- Demonstrate the prompt used to create this workshop
- Collaboratively discuss strategies visible in this prompt
- Highlight key prompting principles and techniques
- Demonstrate rapid iteration with 2-3 quick examples showing how to quickly try different approaches rather than spending time perfecting a single prompt
- Brief overview of prompting strategies (5 minutes)
- Introduce the strategy handout and shared document link
- Emphasize the importance of quick, iterative prompting over perfectionism
- Quick examples of each approach
- Individual work: Adapt to your discipline (15 minutes)
- Take the workshop prompt (see online resource) and adapt it to something you teach or do for work
- Apply prompting strategies from the handout
- Try at least 3 different prompt iterations within the 15 minutes
- Note what was interesting, weird, and what the AI did right and wrong from your expert perspective
- Copy your most effective prompt and interesting outcome to the shared document
- Individual work: Enhancing content with UDL principles (15 minutes)
- Use AI to enhance your existing content by identifying opportunities to make it more inclusive and accessible
- Prompt AI to evaluate your content based on UDL principles , including by pasting the content into the prompt
- Ask AI for specific enhancements that align with UDL principles
- Try at least 3 different prompt iterations within the 15 minutes
- Copy your most interesting UDL prompts/responses to the shared document
- Group review of shared examples (10 minutes)
- Facilitator highlights interesting examples from the shared document
- Brief discussion of effective approaches and interesting outcomes
- Identification of patterns and strategies
- Partner work: Swap and iterate from another disciplinary perspective (15 minutes)
- Exchange the discipline-specific prompts you developed before the break with another participant (copy and paste from shared document is easiest, or swap computers)
- Start a new conversation using their initial prompt
- Start trying to develop a discipline-specific workshop for their discipline using UDL principles.
- Apply at least 3 iterative prompts
- Prompt exchange review (10 minutes)
- Work with partner to review what you each created
- Discuss how the AI responded differently to your discipline when prompted by someone else
- How does working in an unfamiliar discipline change how you interact with AI?
- What disciplinary assumptions become visible when working outside your field?
- How does your own expertise influence how you approach another discipline?
- How would a student in your class do at this?
- Group works together to gather:
- Observations about AI's strengths and limitations
- Role of expertise in using GenAI
This is the initial prompt I used for the first round of these workshops:
I am creating an active learning workshop for university faculty, staff, and students on how to use artificial intelligence, and specifically text generation or GPT models. Attendees might come from across the university, work in teaching, administrative, and/or instructional support roles. Since we can't know ahead of time, the session must start with a roundtable where people introduce themselves and what they do at the university and talk for 1 minute at most about their experience with AI. The learning objectives are: 1. Demonstrate different techniques for prompting GenAI models, including brainstorming and generating content, getting feedback from various perspectives, and transforming content from one format to another. 2.Analyze and explain how different prompting techniques affect AI-generated outputs. 3. Critically evaluate the strengths and limitations of AI models for different types of content and tasks. 4. Critically evaluate the role of expertise in the use of GenAI models, including for prompting and evaluating content. The rest of the session will involve participants using this prompt to build the workshop they are in. It must include activities for each of the techniques in the learning objectives, and discussions of the role of expertise, strengths and limitations for different types of content and tasks. Generate a draft outline of the workshop.
An iteration will likely be needed:
a couple changes. the workshop will have 6 participants and will be 2 hours long. the workshop will involve the group exploring models through the application of their disciplinary expertise and judgement. we will start with a demonstration of this prompt and some tips and strategies, and then as individuals participants will spend 15 minutes taking the promts we are using here and using them to create workshops for their own disciplinary contexts. they will then spend 15 minutes exploring asking models to apply Universal Design for Learning principles to their workshop. They will then have time for a group review before a break. After the break they will explore the role of disciplinary knowledge by swapping their pre-break disciplinary specific prompts with a partner, and working with them for 10 minutes, before spending 10 minutes reflecting with their partner on what they noticed working with AI from a different disciplinary perspective. There will then be 15 minutes for group discussion and relfection and a wrap up.
- Clear Context and Purpose: Identifies audience, setting, and pedagogical approach
- Specific Parameters: Defined constraints, objectives, and available tools
- Detailed and Specific: Specifies exactly what's needed in the response
- The prompt is not overly formal and isn't always linear
- Not a lot of worrying about spelling or grammar.
- Very conversational.
- ✓ DO: Provide rich context, unique examples, and specific details
- ✗ DON'T: Use vague, generic requests that could apply to anyone
- ✓ DO: Apply your disciplinary knowledge to evaluate and refine AI outputs
- ✗ DON'T: Expect the AI to have expert knowledge in your field
- ✓ DO: Give AI a specific role, audience, and purpose
- ✓ DO: Give it specific directions for what you want it to produce
- ✓ DO: Give it examples of what you want it to produce
- ✗ DON'T: Treat AI as a search engine with simple commands
- ✓ DO: Try multiple short prompts rather than perfecting a single one
- ✗ DON'T: Spend more than 1 minute writing any single prompt
- ✓ DO: Use each response to inform your next prompt
- ✗ DON'T: Expect perfect results on the first attempt
- ✓ DO: Carefully evaluate AI outputs based on your expertise
- ✗ DON'T: Accept AI-generated content without critical assessment
Description: Assign the AI a specific identity or professional role
Act as an experienced instructional designer with expertise in
active learning. Review my lesson plan and suggest improvements
that would increase student engagement.
Evaluate this policy from the perspective of: 1) a first-year
student, 2) a faculty member with a heavy teaching load, and
3) an administrator concerned with resource allocation.
Description: Ask AI to break down its reasoning step-by-step
Analyze the strengths and weaknesses of this research methodology.
Walk through your analysis step by step, considering validity,
reliability, and potential biases.
Description: Provide detailed background to improve relevance
I'm designing an undergraduate course for third-year biology
students who have completed basic genetics but haven't yet
specialized. Many struggle with connecting molecular processes
to larger biological systems. With this context in mind,
suggest three active learning activities for teaching epigenetics.
Description: Give AI context in the form of documents, etc
Here are the UDL guidelines. Apply them to this workshop and give me recommended modifications in bullet-point form with examples.
(pasted guidelines)
Description: Clearly define the desired output format
Create a one-page lesson plan with the following sections:
1. Learning objectives (3-4 bullet points)
2. Pre-class preparation (100 words)
3. In-class activities (150 words)
4. Assessment strategy (100 words)
Format everything in markdown with clear headings.
Use these to engage in a back-and-forth conversation with AI:
- "I don't think that's quite right because…"
- "Can you explain your reasoning about…"
- "That's interesting, but in my field we would approach this by…"
- "Let me give you more context about my discipline…"
- "That's a good start. Could you specifically address…"