In what ways can AI be a teacher’s assistant? How do you keep a “human touch” in the AI assessment process? How can AI empower students as they request assessment feedback? Josh Ecker and Erin Cummings from the Ed Tech and Learning Design Team at Chester County Intermediate Unit in Pennsylvania examine these and additional questions with Steve.
Visit Chester County’s AI Support site here.
Email Josh Ecker: joshe@cciu.org
Email Erin Cummings: erinc@cciu.org
Subscribe to the Steve Barkley Ponders Out Loud podcast on iTunes or visit BarkleyPD.com to find new episodes!
Podcast Transcript:
[00:00:00.210] – Steve [Intro]
Hello, and welcome to the Teacher Edition of Steve Barkley Ponders Out Loud. Teaching is a complex and rewarding journey, and I’m always eager to explore the many ways teachers engage students in their learning. This podcast is dedicated to supporting you, the teacher, as you inspire and guide your students. Thanks for joining me. I’m excited you’re here.
[00:00:30.040] – Steve
AI Powered Assessment: Moving Beyond Traditional Testing. Joining the podcast today are Josh Ecker, Learning Design Coordinator, and Erin Cummings, Learning Innovation Specialist and gifted liaison at Chester County Intermediate Unit in Pennsylvania. They both provide specialized artificial intelligence consulting services for K-12 schools, focusing on integrating cutting-edge AI technology and the classrooms, enhancing learning experiences for students. Erin and Josh, thanks so much for joining us.
[00:01:07.610] – Erin
Thank you for having us.
[00:01:09.360] – Josh
Yeah, thank you.
[00:01:10.860] – Steve
As I explore the impact of AI, both from a teaching lens and a learning lens, the word empower continually appears. So I’m wondering if we started this with, how do you see AI empowering teachers?
[00:01:27.880] – Erin
I see AI empowering teachers in a variety of ways. A simplified way of empowering is helping with all the busy work that teachers have to do or the creation of worksheets, that type of thing. It allows those teachers it empowers them to spend more time on what is really meaningful and important in the classroom, thinking about perhaps creating that vision and the goals of their class instead of doing day-to-day tasks that take a while. We might have a teacher who is trying to come up with a hundred different warm-up questions to have throughout the course of the year, and they’re spending hours on this. Instead, they can use an AI tool to help them create this and spend maybe 5, 10 minutes, maybe half an hour at most, and have so much more time to really create more empowering lessons and more meaningful lessons for their students.
[00:02:26.530] – Steve
Would you go a little further with when you pull the word meaningful in there?
[00:02:32.790] – Erin
Sometimes there are tasks that have to be done, such as some districts might require warmups to be done every day. In the example I gave, we might have teachers who are spending hours trying to come up with these warmups or bell ringer type questions. Instead of looking at an assignment that they could be doing that would be more meaningful to the student, that would make them more interested in the content or be able to relate to the content more. I can remember as a teacher having to spend time for bell ringers and exit tickets, and then it comes time to the assignment that I actually want to create. And as I was teaching math in a special education classroom, and it’s like, well, do I just create a simple worksheet or can I create an assignment where my students will actually engage more with the content and it would be more meaningful to them? They would understand the content, what I’m teaching more. I would rather spend the time on that than those menial tasks or writing an objective up on the board. I’d rather be able to create an assignment where my student is, we’re working on budgeting and they’re able to create a budget for living in an apartment and all of the things that they need to include in that, which makes them more invested in their learning.
[00:03:56.140] – Steve
Can I stick the word relational in there or relationship?
[00:04:00.900] – Josh
Yeah.
[00:04:02.360] – Steve
Knowing the student more.
[00:04:04.540] – Erin
Absolutely.
[00:04:06.970] – Steve
I’m doing a trigger as I was listening to you, to my past, working with the dean of education at a school a whole lot of years ago when technology was first coming into the classroom. And he talked about the teachers who were afraid of technology was going to take over their job a role. And his response was, if you thought of the technology as a co-person in the classroom, what’s the technology good at doing? Whatever that is, you probably don’t want to compete with that. So what are you a whole lot better doing? And that’s what I was hearing in your description of meaningful, is that the teacher has the time to invest in knowing the student making that happen for the student.
[00:05:02.420] – Erin
Oh, absolutely. I found using technology to help me with those – not meaningless, but those simple tasks, the menial tasks, allowed me to create lessons and assignments and reach my students in ways that weren’t possible. I was able to do a station rotation and be able to create stations that made sense to my students, that they could do independently, that they wanted to actually do instead of, well, I didn’t have time, so here’s a worksheet of 10 questions.
[00:05:37.800] – Steve
I want to be looking at assessment today, and I know that when I visited your site and one of the sentences I saw in there was looking at AI as a teacher assistant. Is there a way that you describe that AI can provide that assistant role, especially as it connects to assessment?
[00:06:05.770] – Josh
Yeah, absolutely. So really, when we say, what can AI do? If it’s anything related to generating text, it can do it. Now, it can do it to varying degrees of success, depending on the task. But when we talk about it being an assistant, it really can help teachers with anything they’re doing that is ending up in a text-based document. Now, that’s really broad, really general, focusing specifically on assessment, it can create assessments. It can evaluate your assessments and provide feedback. It can provide student feedback on their work. Now, there are, again, varying degrees of success with each of those tasks. And the more initial responsibility you give to AI, the more important it is for you to review and revise and make sure it’s accurate and fair. So I would say my starting point at using AI as a teaching assistant would be to have it evaluate my assessments. And then from there, maybe create some assessment questions. Maybe those are open-ended, maybe they’re multiple choice. When we talk about menial tasks that take away from meaningful relationship building, think about building out multiple choice questions with the wrong answers. I mean, it’s not our time best spent as teachers.
[00:07:28.820] – Josh
So AI I can help create questions like that, open-ended questions. And then when we talk about providing feedback on student work, being an assistant to the teacher in that way, that is the area that I think requires the most delicate hand by the teachers because that feedback is seen by students. It needs to be accurate because that feedback process, that interaction with the teacher and student is one of the most important parts of the learning experience. So for teachers who want to use AI to provide student feedback, I would still ensure that I, as a teacher, was in the middle there. But it can do that. I mean, it can do all of these things. And over time, it can do them better and better. So that’s exciting and gives opportunities for teachers to continue to engage in that more meaningful work.
[00:08:21.510] – Steve
So how do we keep the human touch built into that?
[00:08:25.600] – Erin
One of the things that we emphasize a lot in our trainings is is this 80/20 approach. 80% of what you’re using AI for, or perhaps it’s creating that assessment, is AI. But 20% still needs to be the educator, the human. AI is not perfect. It has increased its intelligence over the course of the past year, even over the course of the past month. But it’s still not us. It is something that we really want to emphasize that teaching ultimately still needs to be done by the teacher. We can’t rely on AI 100%. Perhaps we have an example of a teacher uses AI to create an assessment 100%. The student uses AI to complete the assessment 100%. The teacher uses AI to provide feedback 100%. And then the student uses that feedback from AI into AI to then revise the assessment. Where is the learning here?
[00:09:38.680] – Steve
That’s sounding like a science fiction movie to me. [laughter]
[00:09:41.470] – Erin
[laughter] It really does. I see it often where teachers want to use it, and they use it, and it comes back with something awesome. They’re like, okay, I’m good. It’s like, no, we need to stop, and we need to look through this, because when you look through it, There will be mistakes. There will be things that are not correct. An example is one of the tools that I’ve used will help create questions from a YouTube video as long as closed captioning is enabled. Great. Let’s do this. I put the video link in, it shoots back 10 questions, I copy and paste, and I give it to my students. Unfortunately, perhaps that closed captioning wasn’t accurate, and it changed one of the words because we know closed captioning is not always 100%, so now the question is inaccurate, the answers are inaccurate, and we’re not doing right by our students. So we want to make sure that we’re still reviewing those documents that are coming out. And Josh and I and our colleagues really emphasize that whenever we are with educators. I was in a workshop with ISTI on Tuesday, and the same thing. They are emphasizing making sure that you are continuing to go back and look at what what AI is giving you because it is important.
[00:11:03.530] – Erin
AI is artificial. It is not a human. It’s not going to create that connection that our students need.
[00:11:11.440] – Josh
I would add to that, too, that from my own just personal experience using AI, it is really helpful to, as the human in the loop, be there at the start. To provide or really vision out the overall structure of whatever it is you’re asking AI to do. So for an assessment, that might mean rather than prompting an AI tool saying, I need 10 multiple choice questions on this topic, go – it might be a little bit more involved up front and saying, I’m a teacher who teaches this, really give them the additional context and then saying, The unit we’re working on covers these topics. I want two questions on this, two on this, and really going into a higher level of depth and as the human, owning that overall vision or structure. Because it’s really hard, I found, to go back once you’ve delegated the big picture responsibilities to AI, to as the human in the loop, understand why is it doing this? Where did this come from? So to really break it down more and more, you can still have it accomplish really big tasks, but maybe in smaller chunks rather than all at once.
[00:12:24.250] – Steve
So I’m wondering about, are there tools out there that students can use with the student making the choice to do it in order to get feedback to advance their learning? Maybe feedback that a student gets before they take the next step to their teacher or the teacher is not available and I think I’m doing this right, but I’d like feedback. Are we at that spot yet?
[00:12:54.960] – Josh
Yes. And there are actually a decent number of AI tools that provide free accounts, free individual accounts. Their freemium model is to provide free individual accounts and then show the values. So that way, schools take on school-wide licenses and things like that. There are, in terms of specific tools, when we’re thinking about the large language models themselves, there’s ChatGPT, Claude, Gemini. Those would be my top three personally. And then there’s also Copilot and Perplexity. And then in terms of school-specific tools that students can access, there’s Magic School, School AI, ConMigo. And one small caveat to all this is that with how quickly the technology changes, the specific tools we’re putting out there could be irrelevant in six months. There are some we can predict will be around for the long haul, but you never know. So all of this is to say that these are the tools as of today, as of recording, that are popular and common in that we’ve seen. But the main thing is to understand the overall way students can use them. So that way, regardless of the tool, we have the use cases in mind. Now, in terms of the general large language models versus the school-specific tools.
[00:14:17.620] – Josh
The school-specific tools are generally wrappers on top of the large language models. So they’re basically using the LLMs, but then adding a user interface on top. Anything the school-specific models can the large language models can do also. But what is sometimes better about the school-specific tools is, one, it provides a friendlier user interface. It allows for, if it is a school-wide or classroom-wide license, it allows oversight from the teacher to see how students are using it. It also provides guardrails for students. Sometimes with the school-specific tools, it will really guide students back to the topic at hand, or if a student asks it to generate a five-paragraph essay, it will say, no, that’s actually not how we’re using it today. So it’ll give them some guardrails. So that is all helpful. And it will also be compliant with privacy laws, so FURPA regulations and all of that. So I would say it’s a good place to start. But, yeah, students can access most of these tools for free, which can be a good or a bad thing, right? As long as we’re providing students the guidance to use it effectively and safely, I would argue it It’s a good thing.
[00:15:30.540] – Josh
But that’s a really important and big step in the process.
[00:15:34.980] – Steve
Well, working with the teacher, playing with the thought of, first of all, teaching students how to get the feedback. But we were looking at writing and imagining that students got feedback on their first draft from AI, on their second draft from AI, and then their third one went to the teacher. But when they handed the third one in, they also handed in copies of the feedback that they got from AI so that the teacher could track the student’s learning process in the student learning, how to use the feedback.
[00:16:14.910] – Josh
It really is, both with the assessment process and just the entire learning process with AI, so much of it is about documenting that process so students can show their learning. Because now it’s really impossible to tell if the output is from the student or from the tool they’re using unless we see the steps they took to get there and they can articulate that. So it is definitely an important process. And I like that approach of starting with AI feedback, ending with teacher feedback to really give students opportunities for more feedback and more learning without overtaxing the teacher and making it impossible for them to work with all their students.
[00:16:53.480] – Steve
I was going to ask you for some cautions for teachers who are looking at taking the first step into using AI for assessment. And I think I’ve heard one that you labeled for it says, if you can do it within the school LMS, that’s the better place to start. Did I hear that right?
[00:17:16.140] – Josh
Yeah, I would start with embedding something in the LMS or even just having a school-specific tool that is built around a teacher license or even a school-wide license, because that really provides a level of oversight and guidance that might not be necessary forever. But I think as we’re all learning about how to use AI, starting with more of those guardrails in place will be helpful. And take some pressure off of teachers to be responsible for acceptable use because that’s a challenge in itself. Anything we can do to make it easier for teachers in that regard is helpful.
[00:17:57.120] – Steve
Other questions you’d suggest teachers to be considering?
[00:18:01.570] – Erin
I highly recommend to start small. Don’t expect to get 100% of your assessment right off the bat. Because you need to learn how the AI works, and the AI needs to learn how you work as well. So if you are starting with a small part of it and then working your way up from that. Don’t get overwhelmed. There are a lot of tools out there, but it doesn’t mean you have to use every single one. But also it’s okay if a certain tool is not for you. Maybe find something else that works, but don’t give up on AI just after one attempt. If you try and use Magic School and you’re like, no, this is not for me. I don’t like it. There are other tools that could help. But definitely start small and just build your capacity up with how you’re working with AI.
[00:18:54.540] – Josh
I would add to that as a teacher, you have the unique opportunity to really experiment with a lot of things that are out there. And as part of that process, I would circle back with an administrator in your building, in your district, to check and just ensure that your thinking is aligned on how AI can best be used in your context. So if you’re circling back, I think it allows you to continue experimenting without having to worry about going down the wrong path with it or going down a path that maybe your context isn’t quite ready for.
[00:19:29.270] – Steve
Thanks. So in closing out, I wonder what your hopes and dreams are. What are you hopeful for teachers and students to discover about AI’s ability to empower learning?
[00:19:47.850] – Erin
I am hopeful that teachers and students alike are learning how AI can really leverage their learning and enhance their learning I hope that they are understanding that they’re not alone. And students and teachers alike, as well as administrators, there are many people going through the same process. And it is not one size fits all. It is definitely a learning curve with certain aspects of it, but it definitely can just enhance your teaching. It can enhance your learning. It can enhance the way you interact with content in so many different ways. And I really hope that teachers continue to ask questions, and they continue to use it, and they continue to just enhance their learning.
[00:20:43.340] – Josh
I would add that it really is such a powerful tool that there isn’t a teacher, there isn’t an educator out there who can’t augment their own practices with it in some way. And that will be different for everyone. It’s not a one one size fits all in terms of how to use these tools, which tools to use. But there is some benefit out there to everyone. So I hope everyone starts to explore what that individual benefit is and then shares those with their communities and with their larger educational communities because it is something we’re all learning from each other. Like Erin said, we’re all in this together and there’s no one size fits all. And bigger picture, I really hope that teachers, really everyone in schools starts to consider how this can allow us to transform education systemically and really empower students, empower teachers to create really, really meaningful learning experiences that allow for more of that relationship building, more personalized and contextualized learning, and really use this as a tool for positive change.
[00:21:53.540] – Steve
Well said. Thank you both. I really appreciate it. What’s the best way for listeners to follow up, find out more about the work you’re doing, or come back to you with some questions?
[00:22:04.850] – Erin
Probably the best way to get in touch with me would be through email. It’s erinc@cciu.org. I can also be found on LinkedIn if you Google Erin Cummings.
[00:22:15.410] – Josh
And for me, same option. So my email is joshe@cciu.org. Linkedin, you can just Google my name and search LinkedIn and I’ll pop up. And then if you want to learn more about what we’re doing at Chester County Intermediate Unit, you can go to our website, cciu.org.
[00:22:36.230] – Steve
Thanks so much, guys. Have a great rest of your day.
[00:22:39.770] – Erin
Thank you. You, too.
[00:22:40.050] – Steve [Outro]
Thanks for listening, folks. I’d love to hear what you’re pondering. You can find me on Twitter or LinkedIn at Steve Barkley, or send me your questions and find my videos and blogs at barkleypd.com.
