Ep 79. Math Fact Crisis, Two Years Later with Brian Poncy
This transcript was created with speech-to-text software. It was reviewed before posting but may contain errors. Credit to Canadian Podcasting Productions.
In this episode, Anna is joined by returning guest Dr. Brian Poncy to discuss math fact fluency and whether improving students’ fluency with basic number operations leads to better overall math achievement.
Brian shares new results from the first year of a large implementation of the free Facts on Fire program in a Texas school district. They discuss what the data shows, how the program has evolved based on teacher feedback, and what educators are saying about it.
They also revisit five evidence-based reasons why fluency—being able to do math both accurately and effortlessly—matters. With a new school year underway, this episode offers practical and encouraging ideas for helping students build automaticity with math facts through clear instruction, practice, and feedback.
This episode is available in video at www.youtube.com/@chalktalk-stokke
TIMESTAMPS
[00:00:04] Introduction
[00:04:25] What’s changed with Facts on Fire?
[00:08:15] Measuring fluency in problems per minute
[00:14:47] How Facts on Fire builds fluency through practice and feedback
[00:17:29] How much practice do students need?
[00:20:11] Facts on Fire goes school-wide in Texas
[00:23:22] Teachers’ experiences implementing Facts on Fire
[00:24:56] Did Facts on Fire improve math achievement?
[00:30:10] What the MAP data reveals
[00:34:40] The impact of Facts on Fire and the results
[00:42:40] Five reasons why fluency matters
[00:43:01] Cognitive load and working memory
[00:46:15] Increasing opportunities to respond
[00:47:31] Effort, reinforcement, and building confidence
[00:49:23] Maintaining skills and promoting generalization
[00:52:14] Recapping the five reasons fluency matters
[00:54:26] The future of Facts on Fire
[00:56:19] A call to action for the new school year
[00:57:58] Final thoughts
[00:00:04] Anna Stokke: Welcome to Chalk & Talk, a podcast about education and math. I'm Anna Stokke, a math professor and your host.
Welcome back to another episode of Chalk & Talk. If you're a regular listener, you'll know that I believe fluency with basic math facts is extremely important. And it's not that math fact fluency is the goal on its own. It's because math facts are the foundation for more complex math.
And I hear from teachers all the time that their students are struggling in later math because they can't access the material easily, often because they don't know their basic facts. But does improving math fact fluency lead to better overall math achievement? That's what today's episode is all about.
My guest is a popular returning guest, Dr. Brian Poncy, professor of school psychology and co-developer of the free Facts on Fire program, which targets fact fluency and fluency with basic number operations. He first joined me over two years ago for the most downloaded episode of Chalk & Talk. Since then, educators around the world have implemented Facts on Fire, and he's tweaked the program based on teacher feedback.
So, I was delighted to have him on to get an update. In this episode, Brian shares exciting new results from the first year of a large implementation in a district in Texas. We discuss why he made some changes to the program, what teachers are saying, and we talk about the data from the first year of the project.
Did the program improve student outcomes? Stick around to find out and check out the resource page for promising data. Remember, fluency involves both accuracy and speed.
It's being able to do something both accurately and effortlessly. Brian and I revisit five evidence-based reasons why fluency matters. I think you'll find this conversation both practical and encouraging.
As many listeners begin a new school year, here is my call to action. Make sure children gain automaticity with Math Facts. Give them clear instruction, lots of practice and feedback, and better yet, use a program like Facts on Fire.
It takes four to six minutes a day, every day, and it's totally worth it because it sets kids up for success. Now before we get started, I'd like to encourage listeners to also check out the podcast Teachers Talk Radio. It's created by teachers for teachers and covers any education topic you can think of.
Check them out at ttradio.org or find them wherever you get your podcasts. Now, here's my conversation with Brian Poncy.
I'm joined today by a returning guest, Dr. Brian Poncy. He is a professor of school psychology in the College of Education at Oklahoma State University. He has a PhD in school psychology. His research focuses on academic interventions and behavioural principles of learning, specifically for mathematics.
And he is a co-author of the book, Effective Math Interventions, a Guide to Improving Whole Number Knowledge. And he co-developed with Dr. Gary Duhon, Measures and Interventions for Numeracy Development, MIND, the mind, which is a set of free empirically validated resources designed to supplement core math instruction and provide intensive remediation targeting early numeracy and computational skills. And part of that program, I believe, is Facts on Fire. So welcome, Brian. Welcome to the podcast.
[00:04:16] Brian Poncy: It's great to talk to you again. It's wonderful to be back. And shocking to me, that was almost two years ago or maybe a little bit over two years ago.
[00:04:25] Anna Stokke: Yeah. I think when this is released, it'll be just maybe a little over two years. So yeah, you were on the podcast and honestly, that remains one of my most downloaded episodes.
And you've told me that you heard from educators from around the world after that, wanting to learn more about your MIND program and Facts on Fire. And I certainly share Facts on Fire with a lot of people. Today I wanted to catch up and hear what's happened since we last talked on the podcast.
So, let's do that. What's gone on since then?
[00:04:59] Brian Poncy: Yeah, ever since the podcast, the reception has been phenomenal and fantastic. And I don't know that my wife enjoys you as much as I do because I'm busy all the time, but the teacher response has been great. And so obviously two years ago, where the program was, was totally different than it is now.
Teachers started reaching out to me and they would use my materials. And one teacher, I can just think of her name's Beth Madden, and she contacted me and she's like, we're using this. And she's like, I don't like this and I don't like this.
And you know, I think this could be better and this could be better. And so, it was so cool about the whole process was these teachers, they were superstars. They would like initiate this on their own or they would coach it in their schools.
And they started giving me feedback. This is what my teachers like. This is what they don't like.
This is what the kids like. This is what they don't like. And then that started me down the path of creating the MIND 2.0 because these materials really needed to be scaled to school-wide implementation. And again, the reason why that episode is so popular, I have to imagine is because teachers understand and I think they feel the frustration of if my kids just had these basic facts memorized and they could immediately retrieve those, I could be so much more effective as a teacher. And so, they really like just teachers, Laura Stamp and Samantha Leopard, and there's just the list goes on and on. But they really gave me tons of feedback to try and retool the resources so I could present something to teachers that was much more manageable, and easy for them.
And in the meantime, we went back to the drawing board on some of the instructional components so that we got also increase the effectiveness for students as well.
[00:06:52] Anna Stokke: Yeah, that's awesome. So, let's talk a little bit about some of the changes so that listeners know. So first of all, most of your changes were to Facts on Fire, right?
So, you now have the Facts on Fire website. And the way that the program used to work is essentially the teachers would figure out where students were at and they'd be given different fluency materials. But my understanding is now it's class-wide. Is that correct?
[00:07:19] Brian Poncy: Yeah, it was always class-wide. But what teachers had to do was assess kids, put them all where they needed to be. So that could be in six different skills.
But they would do their four to six minutes of daily practice, but then they'd have to score and then they would have to score to see if the kid needed moved up to the next skill or not, which then required them to do printing every single week. When I was talking with teachers, they were just like, it's effective. The kids are liking it, but it's like when I try to get my other second grade or fourth grade teachers to do it, they're just like, we don't have time.
It's just too much printing. It was being done in pockets of schools, but not school-wide. And I think all of us, fourth grade teachers especially, they're just think of the power of this if this was school-wide and I wouldn't have to keep remediating and pulling kids up so they could access my instruction.
[00:08:15] Anna Stokke: I want to also talk about your fluency benchmarks and just a reminder for listeners that fluency is accuracy plus speed. So, if we want to ensure students are fluent, we need to measure, first of all, can they do it and how quickly can they do it, right? And we'll talk a little bit more about fluency and just recap some of why that's important later.
Last time we talked on the podcast, your fluency benchmarks were in digits correct per minute. So, a student wrote 41 for seven times six, that's one digit correct. And I understand you've changed that now.
So, the fluency measures are in problems correct per minute. So why did you make that change?
[00:09:01] Brian Poncy: Yes. So, a couple of different reasons. First off, Dr. Solomon, who's been on your podcast, him and Dr. Vander Hayden released a study a couple of years back and it was something, it had Goldilocks in the title, but they looked at digits correct per minute and problems per minute. And they saw very little difference between the psychometrics between the two. So, you could use either one. Both of them are good.
So, when I would have interactions with teachers, one of the frustrating things was the scoring piece. One of my ideas was, well, what if we just had kids score it? And then I'm like, well, if I was trying to teach second graders how to calculate digits correct per minute, that would be a headache, but they can easily count the number of problems they completed.
And that also took away the scoring piece from the teachers. And so, with the new materials, so some of the things we changed was we shrunk set sizes and did take problems so that we ensured accurate responding for maybe those bottom kids that didn't have it. And then we would do a one minute timed kind of facts on fire.
And in that we had, we would focus on four items that they were taught, but then we also interleaved items before. With that was a goal setting component. And there was an image to the right that said hot or warm, hot, on fire, nuclear.
And so, what my thoughts were is we can absolve teachers from needing to score things. Kids will set their own goals. They'll count their performance and they'll get feedback on where they're at.
And the old version, they would just kind of climb that hill from like 10 correct digits to 40 correct digits. And that got a little old. Now, if they go from 10 problems to 14 problems, they'll jump from warm to on fire.
They can raise their hand, get their teacher's attention, get feedback and pumped up on that stuff. Right. And so, it put more onus on the goal setting component with the student.
And also, it allowed them to access teacher praise as they moved up in these smaller increments. So, they got higher levels of reinforcement. They felt much more successful.
And what I really like about it is I'm trying to facilitate really positive interactions between the child and the teacher about, my gosh, you know, I'm working hard on my math facts and I'm getting better at math. I like math. Math is fun.
So, to me, one of the ways that we unlocked that for kids was to go to that problems per minute. Now, number two, Beth Madden would be like, Poncy, why is it that you want 40 digits correct per minute in sums to 81 and 40 digits correct per minute in division? She's like, you realize that's like 25 problems for multiplication, but it's 40 problems for division.
And what people were finding, especially in subtraction, is they could never get kids off subtraction. And in reality, we were asking kids to complete many more problems. If you do 25 problems per minute in sums to 18, that's about 40 digits correct per minute.
If you do 40 problems, that's almost 55 digits correct per minute. But we were expecting 40 problems with that subtraction. And so, teachers sniffed that out right away.
And so, then what could we do? Then I could say, well, it's 40 for addition and it's 30 digits for subtraction. And it was just all confusing for teachers.
So, I said, if we use a problems per minute, we can just use a common sense approach. One problem every two seconds. That's 30 problems per minute.
It doesn't matter whether it's addition, multiplication, subtraction, or division. And plus, it's got face validity to teachers because they hate giving kids partial credit when they get one out of, because they don't have the right answer. They just have one digit.
And so, these are things traditionally that teachers have raised an eyebrow with. And so, I was like, okay, let's go to problems per minute. It's better for the kids.
It's easier for the teachers and it's more uniform. And it makes sense within that construct of what's automaticity. If I can get a problem within two seconds or one second, that's something that's very comprehensible for teachers and parents.
People get that. I'm really glad I made that jump. And I'm glad for Dr. Solomon for leading up that research because I think it's really improved the program.
[00:13:32] Anna Stokke: Yeah. And by the way, I'm for getting the problem correct. Because they could just keep getting the same digit incorrect.
And that's not being picked up by that measure, right? If it's just digits correct per minute. So, I'm in favour of this change.
[00:13:47] Brian Poncy: Every positive thing about the program, there's a little bit of pushback. One of the pieces now, though, is teachers don't have to score it. Well, kids just count the number right.
And so, then teachers are like, but what if they get them wrong? Because we want to be brief with things. We don't have kids check whether they're right or whether they're wrong.
But hopefully with our pre-assessment, kids are already accurate. And then also we do have that tape problems piece, which has immediate feedback with small sets. So, it shouldn't be an issue.
We shouldn't have students with high rates of inaccurate responding. And the good thing is, is the new system, a slide deck kind of reads all the directions and does all the timing. So, teachers should be circulating the room.
And they can go ahead and, you know, as teachers know the five or six kids, they probably need to make sure that they're on task, that they're putting forth effort and they might be worried about accuracy. They can certainly, I'm not telling teachers not to score. I'm just saying you don't have to.
[00:14:47] Anna Stokke: Yeah. Okay. And I just want to back up a little bit in case there are some people that aren't familiar with the program to explain a couple of things.
So first of all, taped practice. What that means is in this case, there's a computer, it's actually a PowerPoint that you can download and they'll say the fact, so six times five and pause for two seconds. And the idea is that the child needs to write down 30 or they should try to write down 30 before the computer says the answer, 30.
So, six times five, pause, pause, pause, 30, right? So, they get immediate feedback on that problem as well. And I think if I remember correctly, the way that your program is working now is you have maybe four or five iterations of the facts that you do a day, correct?
[00:15:43] Brian Poncy: I have four facts in the tape problems per day.
[00:15:46] Anna Stokke: Four facts and six repetitions. So, they do that one day. And then the next day, a new set of facts comes in and one problem from the last day is in there, right?
[00:15:58] Brian Poncy: One problem gets folded out and one new one gets folded in. So, the lesson structure is 10 lessons with Facts on Fire and we introduce 12 items or eight if it sums to 10. But basically we systematically introduce every single item and we try to balance it so everybody gets the same amount of OTRs.
And then as set gets bigger, we'll always on the tape problems, we go ahead and we focus on the problems they just got like for the first eight problems, but then we interleave the practice with the others. And that wasn't in the old system either. We just did 12, then we went to the next 12 and the next 12, which was a problem because you would have kind of decay.
And so, with this one, we went ahead and ensured that things were interleaved. So, when you start on lesson one, like on sums to 10, for example, you have four items and you practice four items. And what's cool is you baseline them on 16 items.
And so, they normally like that first day they'll jump up and they'll do way better. So, they're really excited about it. But then we move from those four and we'll add another four and then they'll focus on the new four, but then they also practice on the past four.
And then as we get through the notebook, when we're on items 13 through 16, those will be brand new and they'll kind of get blocked practice in that. Our mass practice, but then we also interleave and provide the distributed practice. It's all built in and baked into the program.
[00:17:29] Anna Stokke: Yeah. So, it's a ready-made program. It's free, right?
And hopefully it helps kids with their math facts and multi-digit multiplication, that sort of thing. We're going to talk about that a little more in just a minute. And just a reminder, how frequently should teachers be doing this every day, every other day, every Friday? What do you think?
[00:17:53] Brian Poncy: Every day.
[00:17:54] Anna Stokke: Absolutely.
[00:17:55] Brian Poncy: The research is really clear. It needs to be done every day.
[00:17:54] Anna Stokke: Absolutely. And how long does it take each day?
[00:18:02] Brian Poncy:
We've seen results with as little as four. We do six now.
[00:18:06] Anna Stokke: Six minutes.
[00:18:07] Brian Poncy: Six minutes. But I also have that tape problems to ensure that accurate responding as well.
And then we also have that obviously one minute daily timed practice as well.
[00:18:19] Anna Stokke: Okay. And what's your fluency benchmark for multiplication facts? What are we aiming for?
[00:18:27] Brian Poncy: Students, if they follow my prescribed, if schools follow my prescribed sequence, they'll start practicing multiplication at the end of their third grade. And so, 30 to me is mastery at that point.
And fourth and fifth grade and obviously above, it's 40 problems per minute. And so that would be a problem every 1.5 seconds or so.
[00:18:49] Anna Stokke: Yep. And if you keep practicing it, you can get even better. 60 problems correct per minute, right?
[00:18:56] Brian Poncy: Yeah. I use it at home with my kids and my little second grader, she got up with sums of 18. She was doing 42 problems per minute and that's a second grader.
So, there's no reason why we can't get majority of fourth graders in that 40 to 60 problems per minute range.
[00:19:15] Anna Stokke: Yeah. And I'm just going to say one more thing because there's another question I often get asked about this kind of thing. Even if the child knows their times tables and they're already pretty fast, they can always get faster.
This doesn't take much time. This practice at the beginning of the class, it would be like lifting weights, right? You might be able to lift a lot of weight, but you should keep doing it and you can always keep better.
[00:19:41] Brian Poncy: Or being a distance runner, like you still need to do it. And furthermore, all the science of learning people, they just ooh and ah over the learning and forgetting curve. Well, it's probably very similar with fluency as well.
So, every time you come back in your practice skills, that's positive. So, in my program, they'll do multiplication at the end of third grade. They do it at the beginning of fourth grade and they do it at the beginning of fifth grade.
And I think that kind of repeated exposure is important for kids.
[00:20:11] Anna Stokke: Absolutely. I think we should talk about some of the outcomes, like what's been going on, because there's some big news that you've actually got some work with a school district. They're using facts on fire in an entire district in Texas.
And the district has just finished their first year, from what I understand. First of all, how did this come about? Like how did the opportunity come about?
[00:20:37] Brian Poncy: Yeah. So first of all, it's being used all over the world and there's a lot of schools using it, but I don't necessarily get to train them, and I don't get my eyes on it and see what's going on and get the direct feedback from coaches. Because of the podcast, yeah, Fort Worth Educational Partnership, they kind of reached out to me and they were like, we'd love to pilot this in one of our districts and see how it works.
We had schools in the area apply to be a part of the project. So, we found a school that wanted to do it. And so, we did start it this year and it's been a wonderful year.
But just to describe the demographics of the school a little bit, so they're in obviously the Dallas-Fort Worth area. They're, I think, 86% or in the 80 to 90% Hispanic population, economically disadvantaged around 90%. The school leadership is fantastic.
And so, I'm going into a school that on some of those demographics sounds like, oh my gosh, 90%, but they do more with less. They have amazing superintendent and leadership and principals. And so, I was really fortunate for that.
So anyway, we went in and I made these workbooks and we delivered them down and I trained the administration and coaches. That took three hours. And then me and Dr. Duhan went and I trained K through two and he trained three through five and that took three hours. And that was all the training that we provided them. I went back to Stillwater. They had the books, they had the slide decks.
I trained them. I couldn't sleep because school was starting and they were going to put it in place. And I wasn't going to be able to see it.
Right. So, it literally was, I mean, the school was just doing it. And so, three days went by and I never heard from them.
And so, I sheepishly gave them, I emailed them and I was like, how'd things go? Because they were like, oh, everything's great. The kids got it.
We got it. Teachers got it. Perfect.
And so, what they did was their instructional coaches, after we trained them, they incorporated some of this into their PLCs and they modeled what it would look like. The teachers participated. And I mean, what's so exciting about this is the school-wide implementation.
It wasn't like there was a bunch of bugs and things they had to work out. They went ahead and started it within the first week of school and it was off to the races.
[00:23:04] Anna Stokke: Wow. How many kids are using it?
[00:23:07] Brian Poncy: In that school, it's a K through five in that district that we're using it with. And there's about 240 kids per class. So, I think we're looking about, I think there's 60 classrooms, 1,500 kids, that kind of thing.
[00:23:22] Anna Stokke: Wow, that's a lot. Okay. So not a lot of training was required.
It was fairly easy for them to implement. So, what kind of reaction are you getting from the teachers and school leaders? Are they happy with it?
[00:23:35] Brian Poncy: Yes. They report being happy with it. I know that everything's always a heavy lift because teachers are so busy.
But the administration, they set it. So, there was a set time every day that they did it. The principals would go ahead and announce it every day.
And so, we had some really animated principals that would sing songs. But I mean, the whole point is, is the coaches, the teachers, the principals, everybody knew this was a part of the school culture. And the principal would be like, were you on fire today?
Were you this? Were you that? And so, it really was a school-wide effort.
And again, I was blown away because I've done plenty of research, but it's always like my grad students go out and they deliver it and they look at integrity and everything's really tightly controlled. This was really scary for me because it was more program evaluation. And we don't want to see how a school does if I have my thumb on them all the time, right?
We want to just see if these materials are good, right? They need to be accessible to teachers and effective enough to where teachers don't have an uprising and a revolt to be like, we want our time back. And so again, just a testament to the school because I wasn't there, right?
I created the program and gave it to them, and they ran with it and just did a fantastic job.
[00:24:56] Anna Stokke: Okay. So, let's talk about the results. Have you got any data that you can share with us?
Did their back fluency improve? Did they improve in other areas even in math? So, I'm really excited to hear about it.
[00:25:11] Brian Poncy: Yeah, great questions. With the project, part of it was we wanted to measure, did they get better in their facts? Which that's kind of a no brainer.
Of course they're going to. The more important question really is, does this bleed over to broad math outcomes, right? Because there's some people out there that kind of intimate that if kids become fluent and memorize their math facts, that this is somehow going to really disrupt this ability to understand and apply what they've learned in multiple strategies in classrooms.
And so, we also looked at their NWEA map scores, which is a broad math achievement score. And obviously we looked at state test scores. The other great thing about this project is it's a three-year project.
So, we're in year one. And one of the reasons I'm so glad I get to talk with you about it is I'm not waiting for everything to be done and then cherry picking later and bragging. Everybody that listens to this is going to know like, what about the state test scores next year or year three, right?
So, I can't be selective. These are the three things we looked at. And we want to see increases in fluency.
That should bleed over to math. And that should bleed over to state test scores eventually. Obviously, if we look at those, the facts are the most proximal and most close to what you do.
And that's been the biggest criticism of the types of research I do, right? Is that like, well, yeah, you intervened on this kind of finite set of items. I know it got better, but that doesn't make a kid a good mathematician.
And it's a very fair criticism. But again, I've never done a three-year, 150-day-a-year intervention. In fact, very few people have.
So, what the outcomes are going to be of this should be of interest to everybody. But anyway, what we found with the fact data is most kids started around 8 to 12 problems per minute. And after they would go through one of our units, which is usually 30 to 45 lessons, the group average would get up between 20 and 25 problems per minute.
And then when they got into like fourth and fifth grade, they would start maybe a little higher at 15, but they would get between 25 and 30 problems per minute, which again, my mastery criteria is obviously 30 and 40, but we're looking, this is a tier one supplement, right? And so, what we want, but doubling and taking kids at the beginning, one problem or 10 problems per minute is doing one problem every six seconds. And so even taking kids from doing a problem every six seconds to every three seconds, which is what the project did.
I mean, that doubles the rate of their responding, which is we were excited about that. Where we saw the most amazing or really taken aback by the results is with the MAP results. So NWEA, MAP, math, RIT scores, and that is a computer adapted, a kind of broad achievement test, highly reliable.
And the growth percentiles we found the previous year, the school was a 50th percentile school in between 40 and 50. And that was their level. And then their growth was around 50th percentile as well.
And so, they were doing good work. They were progressing according to MAP like most other schools were. And so, after a year of facts on fire though, grades two through five, and this was very consistent across the grades, their growth scores for the district was from the 76 to I think the 83rd or 84th percentile.
And so impressive gains there. Impressive gains.
[00:28:55] Anna Stokke: And just to back up a bit there. So, what you're saying, so first of all, there were clear gains in fact fluency, which is exactly what Facts on Fire targets. And you said you'd expect that.
Well, and I bet there are lots of facts fluency programs out there that probably don't really improve fact fluency. So first of all, it did what it was supposed to do, right? They improved in their fact fluency and relatively rapidly because that matters too.
We don't want to be wasting too much time on this, right? But in addition to that, they improved in other areas of math quite a lot, like a lot more than what the school was seeing in previous years.
[00:29:44] Brian Poncy: Yeah. And the MAP scores, they break it down in the computation and three other areas and all of them all just kind of went up the same.
[00:29:53] Anna Stokke: And just to make sure, you know, did the teachers use any other different mathematics programs than what they'd been using previously?
[00:30:03] Brian Poncy: Yeah. Not that I'm aware of. And so, everything was pretty much status quo other than facts on fire change.
[00:30:10] Anna Stokke: Okay. Is there anything else you want to share with us about the data, or do you want to show it to us?
[00:30:15] Brian Poncy: Yes. I would love to show you these data. I really would like to share these data.
I know a lot of schools use MAP data. And so hopefully these will kind of elucidate and give a visual to what we're seeing. And so, here's the second grade level growth and achievement data.
And so basically, I have grade two stack. So, this was their 25 or 24-25 year. And then this was this year.
And so, in second grade, for example, their school growth percentile went from the 48th to the 80th and second grade from the 44th to the 76th and third grade. And in fourth, they went from the 40th to the 81st and fifth, the 47th to the 84th. And so that's just a massive amounts of growth.
The good thing about these data is it not only looks at growth data, but it looks at level data, right? So, you could be two grade levels below, grow a lot and still be behind. And so, the other thing I want people to think about and look at is, you know, we have these colors here.
So red, orange, yellow, green, and blue. And those represent their national norm. And basically that divides these scores into percentile ranks.
And so obviously they're split into five equal percentile pieces of 20 percent piece. And so, we're just to illustrate. And the other thing is you got to be careful when you're looking across cohorts, right?
Because as teachers know, sometimes grades differ. And so, you could see second grade this year did better than second grade last year. But there could have been an issue.
We saw that back in COVID, right? So, we'd see COVID grades. But also in these data, you can look at second graders last year to third graders this year.
So, you follow the cohort. So, let's look at fourth grade. Last year, the fourth graders were at a 40th percentile growth on average, and their achievement level ended the year at the 50th percentile.
And so, lo and behold, when they started in fifth grade, their percentile was the 50th, right? So, you see that right here, 50th here, 50th here. MAP is a great measure.
It's highly stable. But with these same kids that grew at the 40th percentile, now this year, those same kids grew at the 84th percentile, okay? And if we begin to look at like these scores right here, at the end of last year, you had around 53 percent of kids that were in the bottom 40 percentile.
Okay, you compare that to the end of this year, and in fifth grade, it was down to 24 percent. What's also is encouraging is in their growth. So, it was 53 percent grew in the bottom 40 percent.
And that went down to the 16 percent, which is fantastic. And then when we look at the other end, my goodness, last year, only 27 percent of their kids grew in the top 40. This year, 72 percent of the kids grew in the top 40.
And 52 percent of those kids are actually in the top 20 percent of growers across when compared to the MAP sample. And so, we were excited about that. And then even with the level at the beginning of this year, we had 35 percent of the kids in that top 40 percent.
So, we're super, super excited about that. So, when this podcast gets published, we definitely need to post these with data, you know, so that your listeners can see for themselves the data patterns.
[00:34:09] Anna Stokke: And we will do that for sure. And so just in terms of the MAP data, so you mentioned that it breaks down the math by categories, right? So, computation was one of them.
What are the other categories like geometry?
[00:34:25] Brian Poncy: Yeah, I don't know offhand. I saw them one time, all of them got better. And so, because I thought, oh, well, maybe we'll see computation spikes, but the other ones don't.
And they all just kind of all raise together.
[00:34:40] Anna Stokke: Yeah, that's really, really promising. And so, at this point, it's correlational, right? But this is really promising data because as you said, you can see how the same cohort did in the previous year.
And they're doing a lot better this year. So, it sounds like Facts on Fire has had a really positive impact both on the fact fluency and on other areas of math, which I would expect. You know, that's, I've been banging that drum for a long time, that if you want to do well in math, you really need to have that foundation strong.
And without it, you're probably going to struggle. So that's likely what's going on here. And the kids must be feeling really good about themselves.
[00:35:27] Brian Poncy: Obviously, there are kids that this doesn't work for because they're low. And some kids need additional dosages. We didn't look at that this year.
We didn't look at tiered supports along with this. Yeah, most kids really seem to like it. When I would walk in and observe, I'd have the kids thank me, like, oh, you're the Facts on Fire guy.
And that was fun. But the other thing I want to point out too is I showed that example of fourth to fifth grade. When your listeners look at the data, the same thing happened for the cohort from second to third, third to fourth, and fourth to fifth, which is what we had.
And I want everybody to be very cognisant that the teachers didn't change either. And so, the very teachers that got 40th to 50th percentile growth the year before got 80th this year. I'm not going to just sit there and say, oh, well, they learned their facts.
So, you know, that makes them great mathematicians. The teachers were doing a really nice job, obviously, of delivering instruction, interacting with their students, and being great teachers. But they also had to teach the scope and sequence at the pace they were told to.
And obviously, the huge amount of standards they're forced to teach. When we increase those students' kind of core tool skills, as your Precision Teaching guests have talked about, now, all of a sudden, these same teachers, all the great things they were doing now was being recognized. Sometimes people will blame teachers for the low scores.
And what was so interesting to see is that, well, they were teaching the same way last year as this year, but we just kind of changed the kids' skill level. And then they were able to access the wonderful teaching that the teachers were doing. And so, you know, not only is this program good because kids like it, and they feel more confident and get better, but teachers also get some benefit out of it because kids can access their lessons and their teaching better in their daily instruction.
[00:37:24] Anna Stokke: Yeah, I mean, it's better for everyone. And of course, a good resource can make all the difference. But if you're working with poor resources, that can really hold both teachers and students back. So, I think that's what you're saying there, right?
[00:37:39] Brian Poncy: It takes a village, so to speak. And when I say you got to have good curriculum, what you teach, how it's sequenced, to what degree you listen to, teach to. You need good instruction.
That's how you teach. That's where the instructional hierarchy comes in. And we know what teachers are doing in their core math.
They're teaching accuracy of procedures and concepts, and then they're teaching generalization of those. And what's missing? Fluency.
What the facts on fire do? Plugs in the fluency, and then it unlocks that instructional hierarchy.
[00:38:10] Anna Stokke: Absolutely. Is there anything else you want to say about the data before we follow up on fluency?
[00:38:16] Brian Poncy: Let's talk about state test data. State test data, we saw some positive. If you squinted hard enough, right?
You're like, oh, that's getting better. That's getting better. And so, we're seeing increases for the amount of students and meets or seeing anywhere from a three to six point jump there.
But what's really interesting is when you begin to look at things longitudinally, right? So, for the third graders on the STAR test, that's all we could see. So, we have nothing to judge that.
We can judge them by last year's third graders, but who knows, right? That's playing with fire. But for fourth grade, we got to see their third grade scores versus their fourth grade scores.
And for the fifth graders, we got to see their third grade and fourth grade STAR test scores versus the fifth. And when we did that, the longitudinal, we saw that it was having an effect. And what we're just so excited about, and that's the whole point of this project being a multi-year project, is we will get to see the compounding effects of fluency practice.
On your podcast, you talk relentlessly about evidence, and that's what the science of math is supposed to be about, right? It's not an ideology. Let's define things in a way that is translatable to a teacher, whether you're in Australia or America or Canada.
You can take it, you can replicate how it's supposed to be implemented, and then we see similar results across setting. We're really excited by what we're seeing. We're excited to see where it goes, but does the data maintain?
For example, when we come in here in a few weeks and start school back up, do we see the same type of aggressive growth? It could be that facts works really good to a certain point, and then it stalls out. These will all be fascinating, fascinating things that we can use data to inform and hopefully stoke good conversations about what we need to be doing for kids and teachers to help them achieve these instructional goals.
[00:40:22] Anna Stokke: So really promising first year, and they're keeping you around, I take it, because why wouldn't they? And so, I'll find out from you what's going on at this time next year, and I bet we're going to see the same thing.
They're just going to keep getting better. That's my guess.
[00:40:37] Brian Poncy: And again, when I redesigned the program, I literally was like, there's something called Herrnstein's Matching Law and behavioural theory. And it basically says at any one point in time, a human being is going to behave to what's most reinforcing.
And so, you look at effort and reinforcement. And so, if I give people a product and it's easy to do and it's highly reinforcing, then they're more likely to do it in the future. And so, with the instructional adaptations we've made for students, they should really make really good gains and feel a lot of success.
Teachers will see that that's why they get into it. When they can see kids enjoying, having fun, increasing their confidence and can have those positive interactions. And all they have to do is press play, walk around the room and encourage kids and have conversations about effort and growth and congratulate them.
That's a good cocktail. That's going to win. And so, there's lots of things that we're going to do and we are doing.
We're adding skills. We're making directions more efficient. I'm giving all the grant teachers and kids surveys next year to get their input on what they like, what they don't like.
I'm so proud of this project, but I know it would have never happened without the Beth Maddens of the world or Laura Stams or all these teachers that have came in and just, hey, can we Zoom and let's talk about this. And they're literally like we're, we're building this together. So, it's not me or literally it's just, it takes a village again, like I said earlier.
And so, I'm excited to see where it's going to go. And we'll continually kind of tinker with this until hopefully in a couple of years, we got a really tightly designed and replicable and scalable product that we can provide teachers and we'll continue to provide it free of charge. And my goal to help kids and the best way to help kids is by empowering teachers.
And so, you know, that's what we'll continue to try and do.
[00:42:40] Anna Stokke: Awesome. So, let's just recap a little bit before we close off today. Why Facts on Fire likely is producing such large gains in other areas of math, not just fact fluency. I thought we'd recap the five reasons fluency is important. Does that sound good to you?
[00:43:01] Brian Poncy: Oh, it sounds great. And again, biting my tongue because I believe this to be true, but it's not about what I believe, it's about what the evidence says. But just to review these, my saying is fluency rocks.
And I'm going to start with cognitive load theory, just because I'm going to do these out of order. Cognitive load theory, because that's the thing that teachers just clicks for, right? It makes sense.
And we know kids have a finite amount of working memory. And so, when we build that automaticity and skills that opens up and think about it, a teacher's job generally is to teach accuracy using procedures and multiple representations and then how to use those. And so, as they try to build that concept or build those procedural repertoires, they have to be cognizant of competing things and how they link, right?
But if you have to link those procedures and then offload all that working memory to, oh, I got to count on my fingers or draw a race or grab my calculator or use this scratch paper, it takes away from the very goal of what progressives want, right? Which is a good goal. It's just, again, how do we get there and how do we support that?
And so back fluency, number one, should reduce that cognitive load and teachers get that. Thank God. And that's awesome.
And thank you to the cognitive psychs and Dr. Sweller and all the work done to that. But there's also other reasons. From the behavioural standpoint, the most overlooked things is the faster you are with your foundational component and tool skills, the more opportunities per instructional minute you get to practice.
I went and observed some schools in New York and they were doing a lesson and they literally would give kids 10 minutes, 5 minutes, 10 minutes to do a singular problem. And in Facts on Fire, when you're in the warming up stage activity, if a kid completes every problem, they get 172 opportunities to respond. Over the course of this year at that school, I want to say that there's probably between 10 and 11, maybe 12 million responses.
I mean, that's how many problems kids answered. And so, when you're like, well, how did six minutes, how did this change kids' learning trajectory so much? It multiplied the amount of math they practice exponentially.
And then when they would get out of Facts on Fire and they would work and do independent C work, a problem that would have taken them 5 minutes now takes 1 minute or 2 minutes now it takes 20 seconds. And think about people get reading so well. But if I read 240 words per minute versus a kid that has a learning disability that may be reading 60, I'm reading four times as fast.
But not only that, in a more temporal space, I am juxtaposing meaning and structure and all of these things so I can just put it together better. The working memory is kind of the instantaneous, but the multiple opportunities to respond is really what allows people to put things together. So cognitive load, opportunities to respond.
[00:46:15] Anna Stokke: Let me just recap something there. So, the first two you talked about, cognitive load, and the second was increasing rates of response, like more opportunities to respond. So essentially more practice.
And just to remind people again, that that's why the speed piece is important. And that's what fluency really encaptures, right? It's being both accurate and being able to do something effortlessly.
[00:46:41] Brian Poncy: So, an example of this would be, you could be like, yep, opportunities to respond is so important. I'm going to make every kid do 48 problems. And what would happen?
Well, you would be sitting there for two hours for like three of your students. And so, the point being is, and they're going to get fatigued. They're not going to be able to stay on task.
Yeah. Rates and the speed is so important in supporting kids learning.
[00:47:07] Anna Stokke: Yeah, definitely. Because there is sometimes an idea out there that just being able to do something or having many strategies for doing something, no matter how long it takes, is just fine, right? But it really holds students back in other areas.
So, we just want to keep that in mind. Okay. So, you've gone through two.
Okay. What's the next one?
[00:47:31] Brian Poncy: So, the third one this leads into is effort and reinforcement. When I present, I always ask people, who here does a to-do list? And everybody raises their hand.
And I'm like, who here writes things on their to-do list that they've already done? And so, they can cross it off. And it's like 70% of us raise our hands because we do that.
We have been conditioned through life that completing things is reinforcing. And so, the faster, the higher rates you can complete problems, the more reinforcement you get. When you associate math tasks with high rates of reinforcement, now you choose to engage in those tasks and you will persevere with those tasks longer, right?
And so, behaviour has been showing this since the beginning of time. And people will argue, well, it's intrinsic motivation. And what a behaviourist would say is, well, no, it's just a past learning history of these contexts producing higher rates of reinforcement.
And so, you continue to do that. When we have kids that struggle with math, right? And they don't know their facts.
It is a teeth-pulling, just not a very fun experience. And then people wonder why they don't have a math mindset. And they can get a math mindset.
But like set them up in structurally small sets, high intensity, you know, lots of success. And then because what will happen is they'll be like, I love doing Facts on Fire. But then when it comes to the regular math class, then you're like, I don't know that I like that.
But again, and our job is we want to shape kids, like just getting kids successful in something mathematical and really being able to praise and show them you're a man. It's not about being a math person or not. It's like if you practice, you will get better.
And so, we want people not to think of internal attributions of ability, but internal attributions of effort.
[00:49:23] Anna Stokke: So, what next?
[00:49:25] Brian Poncy: And the last two is maintenance. So, the faster we are, the more fluent we are in behaviours and the more we practice them, then we're more likely to maintain those. So, Matt Burns and Vander Hayden have some great research on this.
And then the final one is really what the whole MAP and state test scores are going to get to. Unfortunately, we don't have a lot of experiments that show this because doing a consistent three-year intervention is very hard. But hopefully what we will be able to demonstrate through this project is that basically fact fluency, it helps with skill generalization, which is the whole point.
This is what everybody's fighting over, whether it's progressives or we need to be explicit or we need to do this or expertise reversal effects. Let's just be simple. If you got your foundational skills and you can do those to fluency, it increases the probability that you're going to learn to generalize and adapt that skill better.
And what we do know is that even in an instructional hierarchy, adaptation literally is inquiry-based learning. You know, they're at this point, the kids have the skills, they have the procedures, let's give them something and let's allow them the space and the freedom to put these things together because we can't teach every single combination of these facts and skills and the situations we're going to use those. And so, it's these silly arguments, it's define, measure, know where the kid is and what they need, provide that to them.
And again, that last one is what is going to unlock teacher success, right? And that's kind of what we're seeing is, oh man, because in that fourth and fifth grade, again, 50% in both of those grades grew in the 80th to 100th percentile. We targeted three skills all year, basic multiplication fluency, division fluency, and multi-digit multiplication, that's all.
And for that to happen, that means either apparently just teaching those facts kind of outpaces the rest of the country, or it means we armed kids with subskills that they got the benefit and basically generalized, learned those in that generalization stage. And I think that's what we did. And we'll have two more years so we can repeatedly demonstrate and strengthen.
Hopefully, I mean, by the end of this, it'll be quasi-experimental and it won't be perfect, but it's also in a school context without researchers running around. So, it's real, it's in the field and more data's coming. And as we get better data, we can have more precise conversations and move our science forward.
[00:52:14] Anna Stokke: Well, that's awesome.
I'll just do a quick recap because I like to do that. I've been teaching for a long time. So, number one, you told us about cognitive load.
Fluency reduces cognitive load. Okay. Number two, fluency increases rates of opportunities to respond.
So, students get more practice. Number three, fluency increases rates of reinforcement. And then the tasks become easier.
So, students persevere too, right? So, they feel better about themselves. Number four, it improves knowledge maintenance.
So, fluency makes it more likely that students are going to maintain those skills over time. And number five, fluency promotes skill generalization. In other words, it's going to be easier to learn related skills.
That doesn't mean you don't teach the related skills. You still have to teach those other skills. We're not just saying only fact fluency and that sort of thing.
You still have to teach other things. But when you teach that, it's going to be easier for students to pick it up when they have those basic facts and multiplication, addition, subtraction, division, when they're good at it.
[00:53:30] Brian Poncy: And one thing I want to reiterate there too is when you go to that generalization stage, you need to explicitly teach the boundaries. So, this is what I use it. This is when I don't use it.
Here's the context. As a behaviourist, it's all about discrimination, right? Like when I use this versus when I use that.
And obviously, instructional cues are what I need to discriminate which skills to use. Once I have that, now you can begin to release some of that teacher control. And now you want to see the kid actually do that with instructional cues that are different than the boundary conditions you control that you trained on.
And that would be that adaptation phase, which again, the Morningside model, I'm in awe of the generative model of instruction. If you're like, you know, this Poncy guy's making a lot of sense. It's because I read Morningside and precision teaching people are fantastic.
[00:54:26] Anna Stokke: Okay. Is there anything else you want to add today?
[00:54:29] Brian Poncy: Nah, just thanks to you and the listeners.
But if you could see my inbox and the amount of teachers and even sometimes parents that are reaching out and we're slowly gaining on this. And again, the whole goal of the science of learning and science of math movement is, you know, let's continue to add to our knowledge base and our evidence base so that we can provide teachers with the multitude of tools that they're going to need to serve the students that they're teaching. And so, you see all the straw man and the ridiculous stuff out where they're just trying to make it like we're trying to fight with others.
And we were just trying to say, there are certain goals. There's a way to get to those goals. Let's look at the evidence and what it says.
And so, I'm excited for the next two years. And hopefully we'll add something that can be published in a peer-reviewed journal and kind of add to this discussion moving forward.
[00:55:28] Anna Stokke: Absolutely. I'm excited for the next two years too. And I want to note this episode will be coming out around the time school starts or shortly before school starts in Canada anyways, because we generally start after Labor Day.
But I think in the U.S. a lot of schools start maybe end of August, right? So, this is a good reminder that basic math facts and fluency with the four operations is really important and it sets kids up for later math, right? It's the foundation.
[00:56:00] Brian Poncy: And it enables teachers to be more successful. It's about the kids, but teacher burnout is a problem. And anything we can do to make teachers feel more efficacious in their everyday practice, if we target these skills, it will help with that.
And it's important too.
[00:56:19] Anna Stokke: So, this is maybe a bit of a call to action. Okay.
So let this be the year that we make sure this happens. We make sure that kids actually know their math facts, that they're fluent with the four operations and it's the beginning of the year. It's a new start.
So, let's do it. Is there anything else you want to say, Brian?
[00:56:37] Brian Poncy: No, just thank you so much. And thanks for all the teachers that reached out and gave me feedback and suggestions and have been so kind with their time and all their help. So, it's just been such a fun two years as we've hopefully created and refined this tool to share with educators.
[00:56:57] Anna Stokke: Yeah. Well, thanks for creating it because, you know, you've done great work and it's free. And remember, it's only six minutes a day we're talking about here.
That's not much, but it could actually be the most important six minutes of a child's educational day for some children.
[00:57:15] Brian Poncy: I think in the future, if we get three years of very consistent data on this and I go to administrators and say, I can increase your state test scores, let's say over the course of three years, let's say we increase the percentage of kids that meets standards by 15 or 20 percent. I mean, for six minutes a day, that's 15 hours of instruction per year. I mean, and cost wise, it's pretty low, but we need to put our money where our mouths are, and we need to show that it works.
And we're doing that. And Rome wasn't built in a day, but data is coming.
[00:57:52] Anna Stokke: You bet. And we'll watch for it. OK, thank you, Brian. It was great to see you again.
[00:57:57] Brian Poncy: Great to see you.
[00:57:58] Anna Stokke: Thank you.
Thank you so much for listening. If you enjoy this podcast, please consider showing your support by leaving a five-star rating on Spotify or Apple Podcasts. Don't forget to subscribe on your favourite podcast app or on YouTube so you never miss an episode.
You can stay connected with me on Instagram, Facebook, TikTok, X, Blue Sky or LinkedIn. All links are in the show notes. And check out my website, annastokke.com for more information. This podcast is funded by a grant from La Trobe University and from the Trottier Family Foundation through a grant to the University of Winnipeg to fund the Chalk & Talk podcast.