PODCAST TRANSCRIPTSteve [Intro]: 00:00 Hello and welcome to the Steve Barkley Ponders Out Loud podcast. Instructional coaches and leaders create the environment that supports teachers to continually imagine, grow and achieve. They model an excitement for learning that teachers in turn model for students. This podcast is dedicated to promoting the important aspects of instructional leadership. Thanks for listening.
Steve: 00:28 I’m excited today to be joined by by Tom Schimmer. Tom has joined me earlier on a podcast looking at his expertise in the in the area of assessment. And today I’ve asked him to join me as one of the authors on “Instructional Agility,” subtitled, “Responding to Assessment With Real-time Decisions.” So Tom, thanks so much for agreeing to jump on this with me.
Tom: 00:59 Yeah. Glad to be here, Steve. Looking forward to the conversation for sure.
Steve: 01:03 I’m wondering for starters, if you’d give us a working definition of instructional agility and then kind of wondering what led you and your co-authors in that direction of writing the book.
Tom: 01:18 Well, to start with, instructional agility kind of came from my background in athletics, the idea of being agile and being able to make in the moment or on the spot maneuvers. So we thought about this idea of using assessment evidence to make real-time instructional maneuvers, being able to kind of read and react the same way that an athlete would. You read the situation, then you react to it. We wanted teachers to also be able to think about using assessment evidence as a kind of action as a way to make a maneuver to guide our instructional decisions. The reason that became so relevant for us is because we have seen over the past couple of decades as my colleagues and I have been deeply invested in assessment work, what we’ve seen is a massive influence by online grade books and electronic grade books.
Tom: 02:08 And that kind of led to a lot of places over-quantifying learning. And we noticed that everything was sort of being spreadsheet driven and number-driven and level-driven and what we were losing was some of the art of formative assessment, which is gathering evidence, which was really the foundation of what formative assessment was supposed to be. I gather evidence to make an instructional maneuver. I use the evidence formatively to either provide you with feedback or provide you next steps or engineer a self-assessment opportunity or something like that. So what we saw as we were calling the research and thinking about assessment was that so many folks looked at assessment as a kind of thing that they conducted, as opposed to thinking about assessment more organically as a way to help me guide our instruction. So an easy way to think about instructional agility is to think about formative assessment, but think of it as informative assessment. It informs my ability to make decisions instructionally and helps me make those maneuvers on behalf of the students and for students to do it on their own behalf themselves.
Steve: 03:15 As I read your writing, you talk about the difference between seeing assessment as a verb and assessment as a noun.
Tom: 03:28 That’s a big one. That’s definitely a big one because – and that’s part of the fallout of the electronic grade book, which is the over quantification. So whenever I quantify learning, it means I’m going to have to conduct an assessment. So what was happening was so many teachers were of the mindset that whenever you assess you had to stop teaching and conduct an assessment, as opposed to assessment being a natural part of the instructional process, the same way that you would think about a coach. You know, there isn’t a moment during practice where a coach isn’t assessing his or her athletes. It’s just not – we’re not running to the computer to quantify it. We’re giving feedback, it’s instantaneous and it’s happening. So the world of athletics really did have it. And you could say the arts too – dance is also something where it’s this more organic interaction and it’s less about the number or less about the quantity and it’s more about the what’s next, how do I make that maneuver?
Tom: 04:22 So the idea of assessment as a verb is the idea that assessment leads to action, but it’s actionable, right? And that’s the part that I think gets missed as well, is this idea that formative assessment is not formative because you labeled it formative, like labeling something formative doesn’t make it so. It’s only formative when it’s used formatively. And that part about instructional agility is a main sort of driver in the idea that assessment is a verb. It causes action, it causes instructional maneuvers, and it helps us take students to the next level on their learning continuum.
Steve: 05:01 It’s interesting for me with your connection to the performing arts and to athletics, because I ended up zeroing in on the term in my coaching work, the student learner production behaviors. In other words, it’s what the student does that produces the the learning outcome. So like like a coach observing an athlete or a performing artist coaching a musician, it’s seeing that the learner is not engaged in the right behavior to cause the learning to happen and moving in at that moment. So that’s the same piece that I was that I was clicking on as I was reading through your work. I was picturing, as I read through this, looking at assessment on this continuum where at one end is almost that instantaneous of the teacher reading student non-verbal cues as to how they’re responding to what’s happening, to a more teacher designed informal assessment to maybe something that is a district common formative assessment to then some something more structured. Does that play out?
Tom: 06:26 Absolutely. The whole continuum of assessment, and I think one of the things that we mistakenly think about assessment is we equate – my friend Leanne Young is really great in phrasing this. She often talks about formality is not akin to validity, meaning the more formal and assessment is, it doesn’t make it more valid. So we often underestimate the value of observation, you know, nonverbal cues, paralanguage, the paralinguistic patterns that students show can tell us a lot. A student can even utter the words and actually communicate the opposite. The example I often use with teachers is is the example of, you know, our lives with significant others. I say, how many of you have ever had a significant, other, other utter the words to you, “I’m fine,” and you knew they weren’t fine.
Tom: 07:16 Right? We have all been there. And that’s because you heard the tone, or it was the parallel. The word said I’m fine, but everything else said I’m not fine. And so not to make light of that but we’ve all been in that situation. But the point is that a student could say, “yeah, yeah, yeah, I got it” or “I understand it,” but everything else surrounding that student is telling you that they’re confused or they’re stressed or anything like that. So that helps us, once we start noticing those things, that helps us make instructional decisions. Not every maneuver is necessarily going to be epic, it’s just an opportunity to maximize the efficiency of our instructional minutes, right? So everything from observation to very informal, exit tickets or, you know, thumbs up thumbs down, or four quarters activities, or hinge questions, all the way to more formal. The format itself, every assessment can be used formatively and summatively. It’s really how you use the information that determines or distinguishes the formative from the summative. So assessment is just gathering evidence of learning and then how you use it will determine whether or not it was used formatively or whether it was used to verify in that summative paradigm.
Steve: 08:33 How about the role of feedback to students as part of this process? Where does it fit in?
Tom: 08:43 Feedback is monumental. Feedback gets almost unanimous support in the academic literature in terms of that’s how you raise student achievement. That’s how you increase learning. So learning increases through feedback, but the challenge we all face is that feedback is both simple and complex. It’s simple because we know feedback works. It’s complex because there is no absolute answer to the question of what works with feedback. Everything about feedback is context dependent and nuanced. So what works in one classroom may not work in another classroom or another school. So when we provide feedback to students, there are some fundamentals that we can follow but one of the most important things you can do is you can watch how your students respond to your feedback routine. So rather than chasing the perfect strategy, the goal would be to chase the perfect response, because feedback can only be called effective if it elicits a productive response from the recipient.
Tom: 09:45 So no matter what you do in feedback, if your students don’t take your feedback and don’t use it to advance their learning, you can’t call it effective. So I’ll often tell teachers, a teacher actually can’t say, “I give my students effective feedback, but they ignore it.” You just can’t. That’s not true. What you can say is, “I give my students feedback and they ignore it.” That you can say, but you can’t call it effective because to be called effective, it must trigger a learning response. That’s the point. So does it elicit more learning? Are you triggering more learning? Are you focused on feedback that is longetudinally applicable. Often, Steve, when I talk about feedback, I’ll say something very flippant but it’s not flippant, but it sounds flippant when you say to a group of teachers, “make sure your feedback focuses on learning.”
Tom: 10:40 And then everybody kind of looks at me like, “oh, great idea, Tom. Never thought of that in my 25 years of teaching,” but here’s my point. The research on feedback is nuanced and I’ll try to be brief with this, but the research on feedback is nuanced to the point where you could provide effective feedback to a student that helps them improve acutely on performing on a particular task, but it doesn’t impact their long-term learning because you’re focused more on the immediate task. So when you change the task, the feedback’s not transferable, right? So a quick example, if we were talking about making an argument, formulating an argument, argumentative writing, and the first task I asked you to do was write a letter to the editor of a local newspaper or some sort of like local issue that you wanted to argue in favor or against or something like that, and we did that and I provided you feedback on your letter.
Tom: 11:33 Now, the next time I ask you to formulate an argument, it might be an argumentative essay. The feedback on your letter, the structure of your letter, et cetera, is not transferable to an essay. But if my feedback on the first task was focused on your ability to formulate an argument, make a claim, support it with details, acknowledge the counterclaim, all of those things, that feedback is transferable. And that’s how you can actually, in some cases, not improve the short-term task, but actually have an impact on their long-term learning. And that’s what we should be focused on is how do we impact their long-term learning? So that’s what I mean, it sounds flippant to say focus on learning, but there is method to that madness, which means we have to have an eye on the – this is like a long game. We want to impact their long-term memory or their long-term learning so that how they formulate, because the third activity might be a debate. Now, what do I do? I’m not even writing anything. It’s a debate format. So how is that feedback transferrable? It’s one of the most important nuances that we have to remind ourselves about feedback. So simple, we know feedback works. Complex, because we have to pay close attention to how our students respond and that will tell us whether or not our feedback was effective.
Steve: 12:50 You’ve actually built in for me, in effect, I need to be assessing the impact of my feedback as a constant. I’m triggering – years ago, I was coaching an administrator who was having a conference with a teacher and he had a lot of notes and he would give the teacher a piece of approval of something that she did well. And right as he finished it, he would look down at his notes to see where he was going to go next. And what I realized is, he totally missed the non-verbals of the person that he had given the feedback to. So in effect, he has no idea how the person processed. It’s almost, I’m thinking of working with a beginning teacher and you’re observing in the classroom and the teacher’s giving directions. And it’s like totally obvious that kids are not following. They’re not getting the directions. And then she cuts the class loose and the new teacher is totally surprised that the kids aren’t ready to move in to execute because she wasn’t processing what was happening to the students as she was in that delivery mode.
Tom: 14:19 Yeah, no, it’s an incredibly sophisticated body of evidence. And I think my frustration with feedback comes from the way feedback is talked about, especially on social media. Everything is reduced to kind of these pithy comments about feedback. And, you know, for example, “give a student a grade and the learning stops.” And, you know, that’s just never been proven. That causal relationship has never been proven in the academic literature. It is true that grades and scores can interfere with a student’s willingness to keep because it sends the signal that you’re done, but that doesn’t always happen. That’s not, you know – the famous expression is correlation is not causation. We have a lot of students who receive formative grades. So my advice to teachers is if you are producing formative scores and providing feedback to students, pay attention to how they respond, because if they take your feedback and try to advance their learning, there’s no harm, no foul.
Tom: 15:17 Now, if they ignore your feedback, now you might rethink that whole formative score piece. But to predetermine that this is going to be this is going to be an automatic response, I think is definitely, at best, it’s an overreach in terms of what the research is saying, but we all know that that nuance and and balance doesn’t sell on social media. So if you want to be provocative and you want to get a lot of retweets and followers, you have to sell, and often that comes in the form of exaggeration.
Steve: 15:48 And there’s no always, which I think it makes this so intriguing. The strategy I just used with six kids in a row that worked great doesn’t work with the next.
Tom: 16:01 Yeah. I tell people assessment is often like relationships in the sense that if you start a sentence with the words always or never, you’re probably wrong. And that’s just the way it is.
Steve: 16:17 I’m working with quite a few schools that are looking at getting folks focused on acceleration rather than remediation when returning to opening schools up. And I’m seeing a strong play there of the need for teachers to have the strong instructional agility to pull off acceleration. Could you talk to that a little bit?
Tom: 16:51 Yeah. Without using assessment to guide those instructional decisions, it’s very likely that teachers are over teaching and under teaching certain things. So trying to accelerate and trying to move, because sometimes the evidence tells you that you need to remediate, or you need to go back, but sometimes the evidence is going to tell you that you can move ahead rather quickly. And sometimes the evidence tells you that what you had planned to do in the next 15 minutes is exactly what you should do. So it isn’t just about always making a maneuver because it’s the potential that’s most important. So you gather the evidence and you have the potential to make an adjustmen should it be necessary. If it’s not necessary, then keep going. It’s also, like I said, possible that you may be spending too much time and that your students are way ahead of the curve right now.
Tom: 17:48 And so, you would be wasting instructional minutes by just forging ahead with your lesson plan, even though you’re not really – what’s missing is the interaction. It’s that constant interaction between the teacher and the students that makes a classroom instructionally agile allows us to make those maneuvers. But one of the keys of course, is to make sure that you plan for that. The misunderstanding of instructional agility, it sounds really loose, and it sounds, you know, kind of off the cuff, but there is very much precision to the planning. So what Cassandra Nicole and I often say is the more you plan with precision, the greater you can respond with agility and that will allow you to do that. So it’s the very purposeful, strategic placement of those assessments. So to the students, you know, a four corners activity might seem random in the middle of a lesson or a exit ticket at the end of the class or a pinch question, but you’ve spent the time planning in advance to say, this is the moment I need to do this because I need to take inventory on where my learners are and whether or not I need to accelerate to make an adjustment, or whether I just keep going with the plan that I’ve developed.
Steve: 19:04 So Tom, for administrators observing in the classrooms and conferencing with teachers, are there some specific kind of look-for’s listen-for’s that administrators should should have in mind and to be sharing teachers on the whole concept of instructional agility?
Tom: 19:27 I think there’s a couple that come to mind. One is the intentionality. So when you walk into a classroom, just on a macro level, just to see how much constant back and forth there is between the teacher and the students. Is there a question, a response, another question, a prompt? Are they moving around the classroom? And it isn’t just about making the move around the classroom, but you’re constantly looking for this back and forth between, is it direct instruction? And that’s fine. Listen, there’s a place for direct instruction and there’s a place for all of that when you’re building a foundation of knowledge. But what you would look for is the constant interaction between the students and the teacher and what that looks like.
Tom: 20:13 And you would then see those maneuvers. I say to people, when you walk into a classroom and there’s instructional agility going on, you’ll know when the teacher is assessing. You’ll know when the teacher is giving feedback and, you’ll know when the teacher is teaching, but the lines between them are blurry. There’s a really smooth transition between them and the students may not even realize they’re being assessed. So I’ll give you an example. The teacher provides the students with a prompt for a group at their tables. Well, the students are talking about the question or the prompt. Now, the role of the teacher and all of that of course, is to circulate around the classroom and listen in on those conversations, because you might hear certain misunderstandings start to emerge in that conversation.
Tom: 21:03 And if you hear it enough, it would allow you to say, okay, hey, everyone, let’s stop and let’s make a – let me go back for a second. I need to remind you of these. So what seems like a pretty loose activity, the prompt was very precisely planned. This is exactly what I want them to talk about. And I’m very purposeful as what I’m listening for. So you would see this kind of constant interaction. You would also maybe be able to look for maybe, you know, in a conference with a teacher or a conversation with the teacher in their planning, right? So you could look for the ways that they plan. And typically, when I talk about instructional agility, I talk about guiding your decisions with three questions. So when somebody says, well, Tom, how do I plan with precision?
Tom: 21:49 I say, okay, start with these three questions. The first question is, what are the most common misunderstandings that typically emerge when I teach this lesson? Like, what am I looking for? And I knew, for example, when I taught eighth grade math, if I was teaching equivalent fractions, often in the early days, my students would, would conflate reciprocal fractions with equivalent fractions. So I knew to look for that. So what are the most common misunderstandings? The second question is, how will I know whether or not any of those misunderstandings are emerging? That’s the assessment question. When am I going to check as to whether or not that’s occurring? And then the third question is, what will I do instructionally should one or more of those misunderstandings emerge? And then to flip it, you might even ask the question, what will I do if everybody has it? What is my acceleration plan? Now that plan can be as informal or as formal as you’d like. I mean, obviously we’d want to be more formal, but even if you’re just doing it informally, just that level of awareness will allow you to sort of consume what’s happening in the classroom in a very different manner.
Steve: 22:59 One of my favorite post-conferencing questions with teachers is to ask them as they were planning the lesson and the movie was running in their head, when during the class did the action kind of match the movie? What were the signals that were off of the movie and decisions did you make as you as you saw those go off? And talk me through that, then I get the sense of the teacher reading the situation.
Steve: 23:41 I was wondering about the role of professional learning communities supporting teachers in their instructional agility.
Tom: 23:50 Well, the role of a PLC and the collaborative teams within a sort of PLC framework are essential because listen, making all of these
decisions on your own can be daunting. Now, clearly a mid-lesson adjustment is going to be happening while you’re teaching, but sometimes we use common formative assessments and we come together as a collaborative team and some standards are very complicated. They’re sophisticated and complicated to teach, they’re challenging for students to learn. And so therefore, as a result of those, maybe on that other end of the continuum, we might have a more formal formative assessment, a common formative, where we engage in a conversation about next steps and instruction. And we might take the evidence, you know, that the students have produced and begin to sort it by what’s next for the learners and begin to do some lesson planning.
Tom: 24:40 So these decisions are challenging at times when you’re doing them all on your own. So the role of the collaborative team is to put our collective heads together and begin to say, okay, what’s the best, most favorable course of action for this cohort of students who need to tighten up this aspect of their learning. And then there’s another group. Now, if we were teaching at the same time, we could even trade students. I could take all of the students from both classes who need to tighten up this one piece, and then you could take the others and we can sort of maneuver that way. So there’s just a – there’s a power in the collective, if you will, where we put our heads together and we’re just going to make, you know, I don’t mean to sound flip it, but we’re just going to make better decisions about what’s next for learners if we do it together, as part of that routine, around our collaborative teams.
Steve: 25:28 I’m also hearing peer coaching here, that anytime there’s another set of eyes in my classroom and ears, they’re going to see and hear things that I didn’t see in hear and all that extra information is going to help drive those decisions.
Tom: 25:49 Exactly. You have another set of eyes in the room, looking at other things, maybe looking at the class in its totality, looking at body language, listening to certain groups on one end of the classroom while you’re – you know, there’s all sorts of ways that that can be, can be deployed. So I love that. That to me, speaks loudly to our opportunity to put our collective heads together and make instructional decisions. And that is only going to make us more efficient and more effective in what we’re doing.
Steve: 26:17 Well, Tom, I really appreciate the time you’ve given me here. I’m wondering if there’s any last closing thoughts you might have as to how administrators can support and build – the more I think about it, it’s a career long piece – building that instructional agility.
Tom: 26:41 Yeah. Principals, leaders, have to give sponsorship – real, authentic sponsorship to these ideas. And one of the first things you can do is ease up on these requirements around entries into grade books per week. The idea that you have to have two entries – you’re emphasizing the summative purpose and you’re not allowing for the space. You know, again, principals often look at formality, as you know, it’s not really a valid assessment, unless it’s a stapled packet of paper and you know, and that’s not all principals, don’t get me wrong. There’s a ton of progressive principles out there in terms of assessment practice who deeply understand. But on the other end of the continuum, there’s this idea of thou shalt have three entries in the grade book per week. Well, there’s not a lot of space for assessing formatively when I’m constantly having to mark and score in grade everything.
Tom: 27:31 So the one piece is, you know, give sponsorship to the idea. And that probably will come as a leader, as a principal builds their own assessment literacy. They have to understand sound assessment practices. And so I often, when I work with principals, I’ll say, look, you have to know what you’re talking about. You can’t lead a transformation and assessment in grading practices and just get out of the way and empower people that doesn’t – that’s cliche, it’s old school. It doesn’t work. You have to be fully invested because you need to have a credible conversation. And the second piece would be, there’s only so many minutes. So the more you’re requiring entries in the grade book, the more teachers are going to be focused on that summative purpose because teachers don’t want to get in trouble. They don’t want to be in violation of expectations of their boss.
Tom: 28:19 And, you know, we don’t want to jeopardize our employment and all of that. So we’re going to do what the expectation is, but that doesn’t mean it’s good for kids. You know the old farmers you know, expression about you don’t fatten the pig by continually weighing it comes to mind when I think about it, right? So weighing things, measuring things, measurement is grading. Measurement is verification. Measurement is summative, and there’s a place for that, but we should be spending the majority of our time really in that formative realm. And principals have to not only acknowledge it, but they’re going to have to run interference with parents. And I don’t mean that from an adversarial way, but it’s just, when parents ask questions, we’ve got to help parents understand that this is teaching.
Tom: 29:08 This is what coaching is. This is what dance teachers do. This is what teachers do. When I see evidence of learning, I react to cause more learning just the same way if I see you serve the volleyball poorly, I’m going to make a correction. I’m going to help you make a maneuver right then and there. And I’m not going to run to my computer and quantify it. I’m going to help you get better at serving the games. There’ll be games, there’s practice, there’s games and we spend more time in practice than we do in games. So if we could ease up on some of those requirements, run a little interference with parents and help parents understand why this is an important part of teaching and deepen our own understanding of assessment, then you’ll start to be able to see that you can nurture that kind of culture in your schools.
Steve: 29:52 As I was listening to you Tom, the the word trust was coming to mind and in effect, conversation is critical to trust. So when I require you to get those marks into the grade book, somehow, we fool ourselves into saying that that has purpose and meaning that I understand, but it’s really my ability to have a conversation with the teacher where he or she can describe what it is they’re doing, what they’re learning and how they’re using what they’re learning to assist kids and that builds the trust. Now, if I have to answer parents’ questions, I’m much more ready to answer parents’ questions now, because I have that trust and I know what’s happening in that classroom. They don’t know that I want to turn to a grade book to kind of be my coverage.
Tom: 30:46 And look, it’s tough because we have promised parents, we have open grade books and we’ve promised parents updates. You can’t say to parents, hey, we have an open grade book, you’re going to be able to log in and see how your child’s doing, but everything’s formative so we never populate the grade book. That’s a bit of a bait and switch on parents. So we’ve got to find our balance there, but you and I both know that more often than not, I don’t know if it’s 51%, I don’t know what it is, but more often than not, it comes from a lack of trust that we expect those entries in the grade book, because we want to make sure you’re doing the hard learning. We want to make sure that it’s not just fluff and that lack of trust is fairly apparent in those types of policies. Again, not always, sometimes they’re driven by updating parents. I don’t want to cast everybody with the same sort of – or paint everybody with the same brush, but definitely, it’s enough to notice that those policies are often driven by a lack of trust in what teachers are doing in their schools and it happens more often in larger schools.
Steve: 31:45 And with parents, it’s communication too. If I can communicate the growth that the child is making and the work that we’re doing, then the grading’s not near as important. It’s when I don’t have anything else though, to hang my hat on. Well, Tom, thank you so much for all the work that you do. And I really appreciate you giving me this time.
Tom: 32:08 Yeah. You’re welcome, Steve. I appreciate it. And anytime you want to talk assessment, you know where I am.
Steve: 32:12 All right. Take care.
Tom: 32:14 All right. Thanks Steve.
Steve [Outro]: 32:16 Thank you for listening. You can subscribe to Steve Barkley Ponders Out Loud on iTunes and Podbean. And please remember to rate and review us on iTunes. I also want to hear what you’re pondering. You can find me on Twitter @stevebarkley, or send me your questions and find my videos and blogs at barkleypd.com.
