PODCAST TRANSCRIPTAnnouncer : 00:00 Steve Barkley Ponders Out Loud is sponsored by the AAIE Institute for international school leadership. Preparing educators for the unique challenge of international school leadership through online courses led by international school leaders. Learn more at aaieinstitute.org.

Steve [Intro]: 00:18 Hello and welcome to the Steve Barkley Ponders Out Loud podcast. For over three decades I’ve had the opportunity to learn with educators at all levels, both nationally and internationally. I invite you to listen as I explore my thoughts and learning on a variety of topics connected to teaching, learning and leading with some of the best and brightest educators from around the globe. Thanks for listening in.

Steve: 00:46 A conversation on assessment and grading – now and then with Tom shimmer. Tom is the co-author of “Standards Based Learning in Action: Moving From Theory to Practice,” and he’s also the author of “Grading From The Inside Out.” In an earlier podcast, Tom joined me and discussed the topic of modernizing assessment and I asked Tom if he would return to look at assessment and grading in our current virtual setting and what ramifications there are for the opening of classroom doors. So welcome Tom, thanks for joining us.

Tom: 01:29 It’s great to be here, Steve. Thanks for inviting me.

Steve: 01:32 So Tom, in looking at the term of modernizing grading and assessment, I’m wondering what you would identify as some of the components where you might say that our doing isn’t currently aligned with our knowing?

Tom: 01:52 Well, I think generally speaking, I think there are probably three areas that come to mind immediately. First being the use of formative assessment. I think it’s fairly ubiquitous that people understand the concept of formative assessment. But in far too many places, and I wouldn’t say it’s the majority, but I would say in far too many places, formative assessment is still about gathering information that doesn’t count toward a student’s grade, where the real power in formative assessment is actually using the assessments formatively. In other words, making instructional decisions, you know, being able to see assessment as more of a verb. So labeling something formative, does it make it so, it’s only formative when it’s used formatively. And that means we take action, we make adjustments, we make decisions about intervention. So I think we’re in a good place conceptually, but I still think that a more granular day to day ongoing approach to formative assessment that really makes assessment and instruction almost synonymous.

Tom: 02:54 There’s a blurry line between the two. That would be one. The second area that came to mind immediately was in the area of feedback. I think we all understand the concept of descriptive feedback and how that can improve learning and how grades and scores can often interfere with a student’s willingness to keep learning. But I do think we need to refine our approach to feedback because sometimes it’s not automatic that the existence of feedback will improve learning. In fact, there are many studies that show that feedback does not improve learning, and I think a lot of times that’s the result of feedback that is really focused on helping kids do better on a task, but not really focused on impacting longterm learning. And so really focusing on the learning goals and having our feedback be a little bit more refined to impact longterm learning versus just doing, you know, in layman’s terms we say doing better on a task.

Tom: 03:44 And of course the third area continues to be the area of grading, where there’s just an overwhelming amount of information right now that speaks to, and I think we talked about this last time a little bit around the way that the percentage scale and the idea that there are 101 levels of performance and the whole notion of gradations of quality and rubrics. We have so much information and yet grading has a grip. Traditional grading still has a grip on many schools and many individual teachers, so again, I can’t suggest that it’s the majority, but it’s enough to be noticeable, that the traditional grading still has that grip on many schools and also many parents and students and just looking at that traditional way that we’ve kind of sorted and ranked kids through that percentage system. I mean, there are probably others. I think that the fourth one that now comes to mind I would say is in the area of student self assessment and peer assessment.

Tom: 04:37 I think we all understand the power of metacognition, but the idea of of incorporating that into our classrooms, it’s ironic in a way that teachers might say, you know, I know how important self assessment is and metacognition is, but I just don’t have the time when in fact that is one of the most powerful strategies that we use. So it’s interesting to say I don’t have time to utilize one of the most impactful instructional strategies because I’m too busy teaching, right? So there are four now, I mean identify that fourth one as we were thinking as I was thinking aloud here, and I’m sure there are others, but those are the first three and then four that kind of came to mind.

Steve: 05:12 Tom, I’m wondering if you’d touch on assessment for feedback during this time of virtual learning and what should be happening in that process.

Tom: 05:24 You know, it’s interesting because I think, you know, I’ll pick up on what I said earlier, which is really focusing on the learning. And there is that difference between, because again, you know, learning is really about your longterm memory. It’s about retrieval. It’s about sort of that impact and if our feedback is too shortsighted and just helping you do better on a task, and what I mean by that, if I could, I’ll just take a moment to kind of explain what I mean cause some, some folks might not, they might just think this is semantics, but a task is really a means to an end. What I have you do as a student has meant to your learning and the learning is transferable. So this is really where we, from our feedback perspective, where we craft our criteria. We want our criteria to be focused on the learning and that’s transferrable.

Tom: 06:11 In other words, the question about transferability with criteria is really one where you would ask yourself, if I change the task, could I use the same criteria? And if the answer is no, then you have what I would refer to as a task specific rubric or or task specific criteria. If the answer is yes, then you would have what I tend to favor, which is task neutral criteria so that as we continue down the pathway of the same learning, I can use the same criteria which starts to help students embed the criteria so it’s almost as though we transitioned to a place where students don’t need the rubric or don’t need the criteria. So from a feedback perspective, we want feedback to be focused on that learning side. I think that’s a really important aspect of it.

Steve: 06:55 I’ve done some reading and just a little bit of writing in my own blogs of the difference between having kids practice for performance versus practice for learning. And is that similar to what you’re describing?

Tom: 07:10 Yeah. Dylan William and others would suggest that performance is more task-based and more doing better on how I perform versus impacting the learning. Yes, absolutely.

Steve: 07:21 The, the example that really sunk strong for me is the basketball player who stays after practice and shoots foul shots and gets better and better and better at shooting foul shots after practice. But their foul shot shooting during the game doesn’t improve because there’s not enough connection between the performance and how that comes off. And I guess you’d call it deeper learning, huh?

Tom: 07:47 Yeah. And you know, that transferability for me is really important and this is one of those areas where you can do both. There’s no absolute right or wrong way, but you know, most of us don’t love making rubrics and most of us don’t love, you know, creating criteria. And what I try to look at is, is leveraging the, the ability to transfer criteria. Otherwise you get into what I refer to as the death by rubric spiral. And you start having to create criteria for every single task and that’s a heavy lift for people. So it’s that connection to longterm learning makes it transferable. And now the other part of that of course, is that you can now branch out across subjects. So for example you could create a rubric that is about criteria that’s about analyzing and interpreting data, which of course is a science and engineering practice, but it’s also something students do in mathematics.

Tom: 08:38 It’s also something students would do in a history class. And it may also be something that folks do have students do in other classes, right? So there’s an example of not only a task neutral rubric, but it’s actually a subject neutral rubric that we can all teach the analyzing and interpreting data of the same way. Making argument is a ELA standard, but it’s also in other science and engineering practice. So to me, focusing on the learning and focusing on task neutral criteria makes it more connected to the longterm learning versus what you refer to Steve, as the performance. So now, you know, as far as the online environment, some of it would depend upon a number of factors. One, is the learning environment synchronous or asynchronous. Two, you know, what is the time afforded. So I’ve often asserted to teachers that the thing that a lot of teachers are actually guilty of, and I don’t mean it in a negative, but we’re often guilty of giving kids too much feedback.

Tom: 09:33 And what I mean by the term, too much, is in proportion to the amount of time you’re prepared to give students to act upon the feedback. So if you’re having a quick turnaround time, then we would be wise to focus on more frequent, more, you know, smaller, shorter cycles and making it manageable. Because the one thing we don’t want to do with students is overwhelm them with the amount of feedback disproportionate to the amount of time. If you’re prepared to give students a longer period of time to act upon their feedback, then of course we can provide them with more feedback. Now online environment or not, we always want to balance our feedback between strengths and areas that need strengthening. So we want to make sure that in an online environment, we don’t want to overemphasize efficiency by only pointing out what is deficient with the learning.

Tom: 10:22 We want to make sure we balance, here’s where you’re strong, here’s where you’re actually doing some things that align with the criteria and here are some aspects that need improving. And I’ve often suggested a a very informal kind of two to one, three to two kind of ratio of strengths to that which needs strengthening to sort of maintain the efficacy and the confidence of the learner. That that to me is really important because it’s really hard to separate the emotional side of feedback no matter how technical and clinical a teacher is at zeroing in on the learning and not making the feedback about the person, kids can still take that information personally. And so we have to be aware of the emotional impact that feedback can have. And so we can all imagine as adults imagine being told by our supervisor, here’s the one thing you do well, here’s the 12 things you need to do better, right?

Tom: 11:13 I mean we all know how that’s going to feel. I’ll give you a week in your job before you’re deciding whether or not this is the career for you. So, that emotional side, but in the online environment of course if it is an asynchronous self paced kind of environment, then sure provide, you know, a good amount of feedback to kids and have them then, you know, methodically go through and try to make the enhancements to the work. If it’s a synchronous environment where students are expected to do that work in real time, then we want to balance that feedback with the amount of time that they’re provided.

Steve: 11:42 I’m reflecting on the fact that we need to teach students how to use feedback. During this time I’ve been doing some podcasts for parents on supporting learners at home because I was seeing parents get confused thinking they were now the teacher and I was trying to switch the language, what you’re doing is supporting the learner. And one of the ways you support learners is understanding what feedback means and how I can use feedback. So if that doesn’t set in the student’s head, I think we do waste a lot of feedback.

Tom: 12:16 Yeah. I, you know, I think that that our work in over the last 30 years or so in feedback has been, has been noble and really well-intended. But the challenge is that I think too many of us have thought that just the existence of feedback would equate to improve learning. And that’s not the case. I mean, we learned a lot. Kluger and Denisi, did a very groundbreaking study in 1996 and it kind of showed that the odds are stacked against us because there really are only two positive responses that students can have to feedback. If the feedback illustrates to students that hate that they have fallen short of their goal, then the only positive response is to exert sort of more focused effort toward meeting that goal and therefore doing what it takes to get there. If the, I mean they could ignore the feedback, they could decide the goal was too hard and lower their but you see like there are just too many negative reactions.

Tom: 13:10 Now if the feedback shows that they’ve met the goal, there was only one positive response, which is to set a new goal to set a loftier goal. Otherwise the students, if they think they’re done, if they sort of say they ignore the feedback, it’s good, all of those types of things are negative. So we really do have to be very mindful about the fact that just providing students with feedback and then handing it to them is going to be insufficient and teachers will predictably be disappointed with the results of that. So your point about teaching kids how to use feedback and teaching them how to use that feedback to improve the quality of their work is really important. And then as students gain proficiency, we start to provide feedback in ways that causes thinking. So we’re not prescribing them what to do, we’re using cues and questions and prompts and highlighters and ways in which we can cause them to think and use the criteria to figure out what it is they need to do. But that comes as students gain proficiency, right? Our more novice learners are going to need more specificity and more hands on, whereas our more proficient learners can probably have a little bit more independence where that’s concerned.

Steve: 14:19 I’m making a interesting connect in in coaching where good reflective questions during the preconference with a teacher will actually end up with the teacher improving the lesson that they haven’t, haven’t taught yet with no suggestion, actually no feedback from the – the teacher gave herself feedback that got triggered by the reflection that the coach caused. That’s powerful learning.

Steve: 14:52 Absolutley. And you can imagine that someone who’s in their pre-service teaching or just starting in the profession may not know how to answer those questions. They don’t have that depth of experience and all of that. So you can see that as someone is a novice teacher versus a veteran teacher and an experienced teacher, you could see the same thing with learners, right? Because I think it’s just about –

Steve: 15:11 You gotta build it. You got to build it. Yeah. Yeah.

Tom: 15:13 Sure.

Steve: 15:13 Tom, I know that this has been a difficult time for both systems and teachers in trying to figure out how to address the grading issue. And I’m just wondering if there are any thoughts that you put out to people as to what they should be thinking about or considering as they’re looking at grading in this last for some, a month of school left.

Tom: 15:41 Yeah. I don’t envy schools and having to make these decisions because it has appeared to me as though they’re, you know, what we think is a right decision in one area could create differences in an issues in other areas. I think the universal advice right now is to make sure that nothing that happens during this COVID-19 pandemic does anything that harms students both in the grade book, but also, again, speaking to the emotions, speaking to the anxiety and the stress. I think that, you know, it’s that kind of the the idea of first, do no harm, right? Let’s make sure that what we’re doing in grading is not to anyone’s detriment. Schools right now are wrestling with and have been for a while, issues of equity, issues of access. And depending on where you are, I think schools on a local, I don’t think there is one universal decision that we can make.

Tom: 16:39 I think that schools have to look at the circumstances and see if they can continue what they’ve done. So I’ve worked with a few international schools who said, you know, our, our classes are year long. We all the way up until, you know, the pandemic and even the third report card of the year in April, we’ve been able to have sort of valid evidence and been able to kind of go in that direction and maintain ourselves. And we haven’t identified issues of access and equity within our student population. So we’re going to carry on the way we have with obviously a reduced workload with the online environment and all of those things. And yet other schools will look at their situation, whether it’s, you know, public schools in the United States and Canada or even some international schools where access and equity is a very overt issue that they have to to wrestle with.

Tom: 17:36 And so while there might be, you know, great internet access online environments, if you’re at the school, there may be other places
around the city or around the town where you live, where internet is not as readily available, et cetera. I think schools are really struggling right now to make decisions that they feel, you know, check all of the boxes. And what I’ve noticed recently is a lot of reports coming out where students are being given the choice. And so I do like the idea of student agency where students can say, listen, for me a pass/fail system is what works best for me in this situation. And I think, you know, going down this road a little bit, I think that one thing that people misunderstand about pass/fail is they just think that pass/fail or pass and incomplete is somehow setting the bar low.

Tom: 18:26 You know, the, the idea of pass is about what is an acceptable level of performance. And that of course can be rigorous, right? I mean that can still be a very rigorous sort of expectation. So a lot of schools are going down that pass/fail. A lot of prestigious universities in the United States, MIT, Harvard, Stanford are kind of doing the same thing. So it’s very doable. But at the same time, some schools are saying, listen, if students want to continue to try to pursue the sort of, to keep their A or to satisfy what an adequate sampling would be, then they can do that too. I have yet to hear how those, the ripple effect of those decisions. So if you have a district where they’ve said it’s up to the student to decide or up to the family to decide, I have yet to hear how that’s going to play out. I think the thing that many have to keep in mind is that a lot of universities and colleges are saying, we’re treating this spring, year 2020 spring grades as being quite neutral. So there may not be the advantage of remaining in an A through F system even though you feel like you could continue it.

Steve: 19:35 Gotcha. Gotcha. Well, kind of close us out here, Tom. One of the things that I’ve been spending a lot of my time on in the recent weeks is getting educators to think about what they’ve learned during this timeframe. And especially, what have they learned that they, that they think they want to carry over? Just give you one that I just recently wrote on is I’ve had teachers complimenting their principles on the empathy that the principal has shared by being understanding of a teacher’s current condition at home. You know, I got a, I have a husband and wife teaching team with four kids and you know, what can I expect that they’re gonna get done with wifi access? And then equally I’ve heard I’ve heard kids complimenting their teachers for teachers being understanding of what students are dealing with.

Steve: 20:36 And you know, it’s kind of struck me – when the when the doors of the school opened back up, the teachers still have the same issues of what their houses like and students still have the same issues of what they’re dealing with. And I’m hoping we can carry that empathy and understanding of people into this into the classroom when the doors open back up. And I’m wondering, across the concept of assessment and grading, if you think there’s some things that people are learning now that they might want to consciously take back with them?

Tom: 21:10 Well, I think the most obvious thing that comes to mind is the use of technology. You know, it’s a really interesting time right now because I’m hearing two things sort of on from educators online, especially like Twitter and other places, I’m hearing on the one side, this is a chance for us to reimagine education. And then on the other side we’re hearing, you know, this just emphasizes how important what we used to do is. That face to face contact. So I think that one of the things that I think teachers will take with them is the best of of what technology has to offer. Recently heard a teacher describe her use of video conferencing for feedback and where not only was she using not beyond just a Google doc, she actually was like recording herself, giving feedback to students where students can see facial expression, nonverbal cues, et cetera, and really get that kind of personalized feedback.

Tom: 22:07 And her assertion was that that was something she was going to bring with her as schools begin to reopen up as she’s going through
work at home. For her, she said, it takes less time for me to record myself than it does for me to type a comment in a Google doc or something like – so she’s thinking about using that. I think the other part, there’s several. I loved your reference to the empathy idea. I think we’re maybe rediscovering you know, the emotional side of learning and the empathy that we have to have for circumstances that are diverse and, and, and things that students are having to deal with at home. This may be giving teachers a little more insight as to what the big picture at home is like and how it’s just not as simple as saying, you know, here’s your homework, 500 questions, odds only, go. Understanding what the dynamics are like and what students are sort of dealing with in terms of their environments.

Tom: 23:07 Obviously this is a very acute situation that has added to the stress of students. You know, we won’t always be in this predicament, I’m assuming. And therefore, you know, things may settle, but at the same time, that part of it I think is really important. I think the identification of what’s essential. I think teachers and schools have had to make some very important decisions about what assignments or what demonstrations of learning actually matter. What criteria is important. What’s been revealed here in this environment is that you just simply can’t assign things that are easy to copy or just about recall and rope memory because you just wouldn’t be able to rely on the evidence. So I think teachers are, I mean teachers who had already identified and made those moves ahead of time are probably in a better place. But I think for a lot of teachers they’re discovering that there are some things that are more essential.

Tom: 24:00 What is meaningful evidence look like and do I have to get the kids to do so much or can I actually get an adequate sampling? So I think what we’re seeing is maybe a bit of a shift to effectiveness and thinking about what is effective versus volume and thinking that kids have to go full throttle for 180 plus school days a year. I think that’s a big part of it. And then the criteria itself, making sure that it’s meaningful, making sure that it’s substantive and making sure that it’s connected to that learning. So flexibility is another thing that comes to mind in terms of accomplishing goals and staying on pace that it doesn’t all have to be lock step with everybody on the same day, that maybe we’re learning that within the structure and parameters of a school year, we can build in some flexibility in how students meet learning goals. So I think there’s a lot of lessons that teachers are going to take with them as they transition back into school, whatever that looks like.

Steve: 25:00 A favorite that I walked away with from an interview with a a coach in in China who was six to eight weeks ahead of us in experience, and this was they were at the point now where learning had to trump curriculum. And I think that’s that – when you were describing that weighing of what really is important here versus I got the time and I’ve got to put that all in. A lot of people had to go through that. So I mean I had high school teachers tell me that they, after three or four weeks they realized what they had to do was cut the curriculum in half and now decide what was important. And I don’t know that that’s – in my mind that’s pretty good thing to take back into the back into the classroom with me.

Tom: 25:48 I think that’s a wonderful point because we’re, you know, a lot of times as teachers we’re not comfortable with space in terms of thinking space and time. We feel like every day has got to be another topic, another lesson, full throttle. Let’s get them through. And we’ve really, I think to the detriment of learning, we have emphasized coverage and that we’ve always had that tension, coverage versus depth and how can I get to that place where students are digging deep. Metacognition takes time and space. Self-Assessment takes time and space. Inquiry based learning takes time and space. Helping students sort of navigate sort of their curiosity takes time and space. Being creative takes time and space. So if we don’t, you know, we can’t look at kids and say we want you to be innovative creative thinkers in the 21st century, but do it by Monday.

Tom: 26:39 We have to really provide that space for kids and teachers often aren’t initially with that because they wonder, well what am I going to be doing during that 15 minutes of that 20 minutes? And as you gain comfort with the messiness of learning and the messiness of some of the more sophisticated learning goals that require that thinking space, I think you start to see you’ve got a very robust learning environment that looks at quality over quantity. And that’s an expression I use so much in assessment. So much in learning. It’s always about quality over quantity. It’s not how many questions you ask. It’s the quality of what you’re asking. It’s not how much you’re asking the students to do, it’s the depth of what you’re asking them to do. Those types of, you know, I think you can always be in a good place when you emphasize quality over quantity or you know, from a grading perspective, I would say quality over counting where you’re looking at the idea of what is the depth and breadth of what the student has done in terms of the demonstration of learning.

Steve: 27:33 The thought that’s connecting for me that I’m also seeing as a common pattern is this forced us to slow down. And all those things that you said need time, they need us, they need us to slow down. So that could be a strong piece for us to be a caring back into our into our classrooms with us.

Tom: 27:54 Yeah, and I know, I know this is going to sound a little cavalier because you know, I’m not currently working in a school, et cetera, but you know, in my now 29 years of working in education, I’ve never had the curriculum police show up and look at my checklist of standards that I’ve covered, right? I think when you look at learning through the lens of broad categories and strands and domains, the way the standards are organized, you can get an adequate sampling within that strand without having to use the checklist or the decimal points in your standards and identified they don’t show up. And we have to remember that schools were not created so teachers could cover curriculum. Schools were created so students could learn. And so if I’m emphasizing my end of it, which is coverage and sacrificing student learning, I’ve completely lost the plot of what schools are supposed to be about. So the learning has to be emphasized and if it takes us slowing down a bit to ensure that that which is essential is embedded into the student’s longterm learning and they have a retrieval and they’re able to sort of use that work on an ongoing basis, then that’s what learning is all about. It’s not about what the teacher covered. It’s really about making sure that this experience is highly impactful to the learner.

Steve: 29:10 Well, Tom, thank you so much. I’ll make sure that in the lead into the podcast, we put your website. Why don’t you just go ahead and and mention it for people who are doing this while they’re exercising.

Tom: 29:25 Sure. My website, a personal website is tomshimmer.com. I blog on our Solution Tree website called allthingsassessment.info. And you can also find me on Twitter @tomschimmer. Follow me there and you’ll be able to connect if you have questions, anything like that. I never hesitate for people to reach out and just, you know, if you’ve got things you want to talk about, let me know. I’m happy to support schools, individual teachers or principals or whatever the case might be. So thanks again Steve. Really appreciate the opportunity.

Steve: 30:02 You bet. Thank you. Thank you for joining us and don’t be surprised if I circle back to to catch you at another time as the new issues pop up for us. Take care. Have a great day.

Tom: 30:14 Thanks Steve. I appreciate it.

Steve [Outro] : 30:16 Thanks again for listening. You can subscribe to Steve Barkley ponders out loud on iTunes and Podbean and please remember to rate and review us on iTunes. I also want to hear what you’re pondering. You can find me on twitter @stevebarkley or send me your questions and find my videos and blogs at barkleypd.com.