Ep 66. Desirable difficulties for learning with Elizabeth Bjork and Robert Bjork
This transcript was created with speech-to-text software. It was reviewed before posting but may contain errors. Credit to Canadian Podcasting Productions.
In this episode, Anna is joined by two world-renowned experts in human memory and learning, Dr. Elizabeth Bjork and Dr. Robert Bjork, to explore what makes a difficulty desirable in learning.
They explain why some learning strategies that feel effortful often leads to stronger, longer-lasting retention while strategies that feel easy, like rereading or rewatching lectures without self-testing, or cramming, can create an illusion of mastery. The conversation unpacks retrieval practice, spacing, interleaving, variation, and addresses common misconceptions about what makes a difficulty desirable, and why what looks like slow progress in the short term may actually lead to more durable learning in the long-term.
This episode will be of interest to educators at all levels and is especially important for high school and university students and their teachers who want to develop effective study habits grounded in science.
This episode is also available in video at www.youtube.com/@chalktalk-stokke
SHORT COURSE
La Trobe Short Course: Evidence-informed Mathematics Teaching – An Introduction https://shortcourses.latrobe.edu.au/evidence-informed-mathematics-teaching
TIMESTAMPS
[00:00:22] Introduction [00:04:48] Guests’ background: moving from math to psychology[00:09:04] Researching effective learning techniques [00:10:35] Who does this research apply to? [00:14:47] The introduction of ‘Desirable difficulties’ [00:18:40] Understanding desirable difficulties [00:20:13] Importance of retrieval practice [00:30:46] The spacing effect [00:33:43] Variation and the benefits for students [00:36:55] Retrieval practice is the anchor in desirable difficulties [00:43:19] Blocking and interleaving [00:51:58] Final Thoughts
[00:00:00] Anna Stokke: Welcome to Chalk & Talk, a podcast about education and math. I'm Anna Stokke, a math professor and your host. Welcome back to another great episode of Chalk & Talk.
Today I'm joined by two influential researchers in the science of learning, Elizabeth Bjork and Robert Bjork. Just a quick note that today's episode is audio only. And honestly, this is one you'll really want to listen closely to.
If you've heard the term desirable difficulties, retrieval practice, spacing, interleaving, and variation, you have them to thank. This episode is especially important for students, particularly high school and university students, and their teachers who want to develop effective study habits grounded in science.
In this conversation, we talk about what makes a difficulty desirable in learning and why simply making learning harder isn't the point. We explore why cramming can work in the short term but fail in the long term. We discuss why rereading textbooks or rewatching lectures without forcing yourself to retrieve the information or in math without forcing yourself to work through the problems independently feels productive but often isn't. And we go through each of the desirable difficulties discussing how and why they work and the research that supports them.
Importantly, we clarify something that often gets lost. A desirable difficulty is only desirable if the learner has the background knowledge to succeed. Otherwise, it becomes an undesirable difficulty. If you're a student who wants to learn about effective study skills or a parent, teacher, or professor who wants to help students build durable knowledge, this episode will give you practical research-backed guidance you can use right away. I hope you like it.
Before we get started, I want to let you know that I'll be co-delivering a four-session short course on evidence-based math teaching through La Trobe University's School of Education starting in April 2026. The course is open to teachers anywhere in the world. I'll include a link in the show notes for registration. It would be great to see you there.
Today I am joined by two leading experts on human memory and learning.
It's a real honour to have Dr. Elizabeth Bjork and Dr. Robert Bjork with me here today.
Dr. Elizabeth Bjork has a PhD in psychology from the University of Michigan and a BA in mathematics. She was a professor of psychology and senior vice chair at the University of California, Los Angeles. That's UCLA.
She chaired UCLA's academic Senate and was a distinguished research professor. Her research in human memory focused on inhibitory processes and their role in creating an adaptive memory system. Supported by the James S. McDonald Foundation, her recent work explored applying laboratory learning principles to enhance instructional practices.
And Dr. Robert Bjork has a PhD in psychology from Stanford and a BA in mathematics. He served as a distinguished research professor in psychology at UCLA. His research delved into human learning and memory, exploring the applications of learning science in instruction and training.
A fellow of the National Academy of Science and the American Academy of Arts and Sciences, he has held editorial roles, chaired committees, and led academic departments. He has won numerous awards, including UCLA's Distinguished Teaching Award. Robert and Elizabeth were jointly awarded the James McKean Cattell Fellow Award for outstanding contributions to applied psychological research.
So, congratulations on that. And it's a pleasure to have you here today. Welcome to the podcast.
[00:04:27] Dr. Robert Bjork: Oh, we're happy to be here.
[00:04:29] Anna Stokke: I have to ask because I'm a mathematician and I'm so pleased that you both have BAs in math. I think you're the first psychologist I've had on the show that have had a math background. Do you know that?
[00:04:42] Dr. Robert Bjork: Is that really?
[00:04:43] Anna Stokke: Yeah. But I have to ask why you moved from math to psychology?
[00:04:48] Dr. Robert Bjork: Well, maybe I'll answer that first and then Elizabeth can get in. So, in my case, I was a physics major through most of my undergraduate career at Minnesota.
And it was kind of a crisis in my senior year. I got to see what physicists were actually doing. And it was, you know, big, huge projects.
And it was not the era of the 1600s or something. But anyway, so I talked to an advisor about what and to make a long story short, this advisor is very helpful. She suggested I go talk to a prominent faculty member of Minnesota named David LaBerge, because he was in psychology, but mathematical psychology, she said, whatever that was.
So, I went and talked to David LaBerge. He had won the university's teaching award. He was all excited about things.
I didn't understand it, but I decided, oh, this is perfect for me. So, I switched to math. And then in the middle of a graduate year at Minnesota, Professor LaBerge at that time came in and said that Stanford University had just gotten this very large program and grant.
And that was the place to go and work on mathematical psychology. I was kind of shocked in a way. I mean, it was exciting, but I thought, does he want to get rid of me? But anyway, I did go to Stanford.
And that was very a exciting place at the time where all sorts of, the mathematical psychology program was very active with good students and faculty members who went on to become winners of the National Medal of Science and so on. So, I should switch and give Elizabeth as a somewhat similar story, if at different places.
[00:06:27] Dr. Elizabeth Bjork: My story is a little more embarrassing than Bob's in that he was worried that this professor was trying to get rid of him. As a woman in math, and when I was studying math, I was a pretty rare bird. I didn't have women in any of my classes.
I didn't have a woman professor and so forth. So, it was a little bit discouraging in that respect. Then my senior year, I needed one more course, one more GE course to complete my degree requirements.
So, I asked around my friends, you know, I'm going to be taking two hard math courses this term. So, I need something, this is the embarrassing part. I need a course that I can cram for.
Now, as Bob and I know, and we can say cramming is one of the worst ways to actually study to learn anything for the long term. But they said, well, have you had intro psych yet? Because it qualified, it would qualify this GE requirement. And I said, no.
And they said, well, they teach it to 1000 people at once, because it's such a popular course. They only give multiple choice and you can do well if you just cram for the exam. So, I said, sounds like what I want.
Then I went in. And I realized this is a very interesting field to study. And also, mathematical psychology was just beginning to take foothold in psychology.
And so, I decided, kind of what the heck, I'll apply and see what happens. And I applied to the math psych programs in Michigan. A number of ones accepted me, but Michigan gave me a scholarship to come there.
And they also said, by the way, if you want, you can take math graduate courses while you're here. And they'll count for your degree. And you can also make that part of your decision.
Do I want to stay in math, or do I want to come get in psych? So, I thought, wow, these are very understanding people, very attractive offer. So, I and then the other thing that really got me interested is the kind of research you do in psychology is largely collaborative.
You work with other people, you work in a lab, you work with a lot of graduate students. Whereas math is much more solitary kind of research. And I decided that I really enjoyed that interaction with colleagues, being able to discuss problems and theories and what would be the best way to try to test this hypothesis with other people.
[00:08:50] Anna Stokke: So, you're known for your work on desirable difficulties. And, so you moved into studying effective learning techniques, really, as a research area. So why did you become interested in that field?
[00:09:04] Dr. Robert Bjork: At the time, I had the great good fortune as a grad student at Stanford to be in a program with terrific faculty members, other grad students.
And so, we became very excited with just trying to fit our results mathematically, rather than just saying that something would make something better or worse, to actually say the format. And I was one of the grad students to the Institute for Mathematical Studies in the social sciences there, and a lot of excitement about that. And faculty members who were exceptional, two of them went on to get the National Medal of Science in the White House.
But anyway, what happened gradually is I became very interested in just the basic phenomena and their applications. I mean, it's one thing to say that you found evidence that you can enhance subsequent learning by doing some things to introduce forgetting. I mean, it sounds crazy, but like, what then does that mean for how schools should be optimized or skills, so a great deal of excitement.
And that has really continued. I think I can speak for Elizabeth in this case, that these kinds of questions just kept motivating us across the following decades.
[00:10:20] Anna Stokke: Okay. And then I wanted to ask about the research that you did and or that you're still doing. So, does most of your research apply to students who study independently, like university students or high school students?
[00:10:35] Dr. Elizabeth Bjork: Well, I'm not sure. It certainly applies to individual study, wherever you're doing it.
Doesn't mean you don't have to be a student. You can be trying to learn a new way of doing something at business, like a new program that you're going to introduce to make things work more efficiently. The other thing is one of our students just before we left UCLA had gotten very interested in this idea when students work together, is that an optimal way to be doing things? And I think we've mentioned here and there that it's one thing to tell people, here's what you should do to learn effectively.
But because it requires a little more effort in the beginning, people are reluctant to go that way. And particularly students want to just keep doing what they've been doing. And it's gotten them into a good university.
It's gotten them, you know, they're being successful so far. Why don't I just keep doing it? And one of the most favorite things students do is read chapters over and over and over again. And they like to do it back-to-back.
So, how are we going to try to get people to stop doing those sort of things and doing what we know is much more effective, if you want to learn something for both the long term and for transfer reasons, so that it'll be accessible to you when you can apply it in another context. And so, one of the students we were working to, well, what if we had students working together, and they have the task that brought in desirable difficulties, even without them sort of realizing that? So testing, for example, students don't like to test themselves. I mean, they like to test themselves as long as it's something easy, and they can get the right answers, but they have to struggle and so forth.
So, we formed, you know, like study groups. And now a lot of people really like the idea of study groups, and they put students in study groups all the time. But the problem is that they don't sort of orchestrate what those students should be doing in those study groups.
And one problem is the brightest or the best student in that study group will sort of take over and do most everything, and the other students will just say, okay, oh, so that's the right answer to this question. Okay. So, we looked at that in some experiments of one kind or another, where we had students work together, and then see versus students who just work, did what they usually did alone.
And then later on the final exam, say several weeks later, who did better? And it was the students who had worked with somebody else who all did much better on their final exam. So, it was those sort of things that really intrigued us both. How can we take what we learned in the lab about how we learn and how we remember best and apply it to education in a way that more and more people will be willing to do these sorts of things? And therefore, sometimes you hear about the 10-minute education? Because after 10 minutes, this is as much as you're going to remember 10 years from now.
I know some comedian made that up and was very successful in pursuing that joke. We don't want to work against that. We want people to be able to remember all the various things that they learned when they were going through their graduate education, so they can apply that later on when it's necessary.
I know, Bob's heard me say this many times when we're driving through the countryside, I think, gee, I studied all these formations when I was taking geology. I'm not being able to remember right now what it is exactly that caused those rocks to look that way or so forth. And we think if people learned and studied during their undergraduate years, using desirable difficulties, that wouldn't be a problem.
10 years from now, they'd be able to see a geological formation and remember what were the forces that led and produced such an object.
[00:14:41] Anna Stokke: Okay. And did you actually introduce the phrase desirable difficulties? Is that your phrase?
[00:14:47] Dr. Robert Bjork: Yes. I introduced it in two papers in 1994. One was in a kind of summary. I had chaired a committee of the National Academy evaluating training in the military and use it there in another chapter.
But it had a nice alliteration and has been used a lot. But sometimes we wonder if it was the right term, because sometimes people thought, well, you just make things difficult. And we have to remind them it's desirable difficulties.
It's that desirable is a key word. And being really skilled as a teacher, as some of the audience that you're preparing this to show this work to, are very skilled at adjusting the level of difficulty across students in their class or whatever. So what's an optimal difficulty that requires a substantial effort to recall, and then hence later recall, that's going to vary depending on your level of initial learning.
So, we've had to do quite a bit of trying to clarify what all is meant by desirable difficulty. But those principles, those spacing instead of massing, varying the condition of learning rather than keeping them constant, all of those difficulties can make a huge difference. And the difference shows up later, long-term recall, transfer of skills.
And so that's a challenge that students confront, because when they cram for an exam, right now, cramming will make things very accessible.
What also happens is that people have difficulty realizing this, the positive effects have to do with the exam tomorrow morning, or you cram all night for something. The negative effects are the next academic term or a year later, where this material is important for your understanding and comprehending new material. Then it's a very bad idea because now it's gone.
I mean, it's almost you never got any exposure. And students get very susceptible to getting fooled because they're in an environment where, by themselves, with other students, where the book is right there, their notes are right there. And it's sort of not difficult in a sense to come up with the right answer, but now the exam is going to be sometime later.
It's maybe going to be in a different place. And if you really want to create the kind of learning that will transfer, massing is an awful thing to do.
[00:17:31] Anna Stokke: And just, I'll sort of start by saying that the desirable difficulties that we're going to talk about today, they are retrieval practice, spaced practice, interleaving and variation.
And so, we've talked about those on the podcast before, maybe not variation as much, but you mentioned that this word desirable is really important, right? Like just if learning's difficult, that doesn't mean that that's an effective way to learn. It has to be a desirable difficulty. And the thing with these desirable difficulties is that you've studied them, they've been studied and they've been shown to be effective.
That's what makes them desirable difficulties. And so, you mentioned in your chapter, you have this great chapter in a book. In fact, it was Paul Kirshner that gave it to me, ‘making things hard on yourself, but in a good way.’
And it maybe has a little more to the title, but one of the things you mentioned there, and this really struck a chord with me, and I really want to stress this, is that if the learner does not have the background knowledge or skills to respond successfully, that desirable difficulties become undesirable difficulties.
[00:18:40] Dr. Robert Bjork: So, yeah, that's a very good point. Students bring, depending on their home environment, depending on prior experience in educational environments, whatever, they bring to a particular learning set, a great variety in terms of what their background is.
Some of them may have had parents, a home situation that included a lot of educational opportunities where others have not. So, it's kind of like a key is almost, to put it one way, is what's the level of these difficulties, whether it's spacing, retrieval, practice variation, that's appropriate for a student who's at a certain level. And sometimes if you, we've had some interactions with teachers who are terrific, and it's pretty clear that they develop, without even thinking of it this way, they will ask a question in a different way to different students in terms of how much that, there are clues and whatever in the question itself to try to get the level of this.
So, to get some students to produce certain answers or certain concepts, whatever, is going to require a lot more support in how the question is phrased than it will for other students. And so, really, we greatly admire the teachers who are there trying to optimize things across a classroom of kids who bring varied backgrounds and various levels of prior learning to their current learning.
[00:20:13] Anna Stokke: To follow up on that, something like retrieval practice, if the students haven't actually learned the concepts in the first place, retrieval practice just is going to have no impact, right?
[00:20:25] Dr. Robert Bjork: Yeah, we sometimes feel like there've been good things about coining the term desirable difficulties, but then a lot gets lost in what all is meant by that word desirable. And it's translated sometimes just make things difficult.
So, really, and we've had, over the years now, particularly the last several decades, lots of interactions with teachers at every level in real world environments, from teachers working with almost kindergarten level all the way up to college teachers, and a lot actually with teaching in medical schools and teaching in law schools. And that really has illustrated the importance of having this kind of flexibility that students can take advantage of depending on what they bring to the task. And some of that's been very exciting in the sense that it looks like optimal teaching can overcome some disadvantage.
So, one of the studies, without going into details, we worked with a dean in a Florida law school who did a terrific job at trying to change the law school curriculum to incorporate desirable difficulties. And after doing that, three years in a row, the students there scored higher on the bar exam than the students of any other program in the state of Florida. And what's exciting about that, as far as they're entering credentials, those students should have been about fourth.
But by optimizing the learning process, they went to first. So, they actually exceeded on the bar exam the performance of students who came in with more knowledge, more background.
[00:22:09] Dr. Elizabeth Bjork: I don't know if this will help, but sometimes I've said this to teachers, and it seems to help.
But you want to, for retrieval practice to be really effective, you want that retrieval practice to occur at a point where if you waited any longer, the students wouldn't be able to retrieve that information or manipulate it or work with it. But if you wait until it's almost ready to be forgotten, but hasn't been yet, that is a very powerful learning experience. It's like, I don't want to get too much in the model thing, but the idea of this retrieval strength versus storage strength.
When storage strength is low, and you present a lesson of some sort, or you represent that material that you're trying to learn, when retrieval strength is low, so if you waited any longer, you would just forget this information, but you represent it again, that increases the storage strength of that information by a huge amount, much more than if you did it immediately when retrieval strength was still high. When retrieval strength is high, you don't get much out of that learning lesson. If retrieval strength is low, but not gone, you can build up a lot of new storage strength going through that material.
So I don't know if that helps, but I think it sort of the idea is that to just keep repeating things, which is what a lot of students like to do, a lot of teachers like to do, because students are sitting there nodding their heads, okay, yes, yes, I've got this, I understand it, doesn't really do much. You've got to wait for a while until the retrieval strength of that information has gotten lower, and then represent it again.
[00:23:49] Dr. Robert Bjork: And every time you retrieve it at one delay, you then are able to retrieve it at about twice that delay.
That's what we called expanding retrieval practice years ago, but it's a very interesting feature of human learning memory that when it's produced at some delay, when it's produced in a different environment, then it can be successfully produced at a longer delay or in an environment that's still more different. So it's kind of like these difficulties kind of feed on each other in a sense, and of course, there's a whole metacognitive side of this, students at every level, they're forming some of their own assessment of how, you know, do I know this or do I not, can I stop studying, and it's very easy to be fooled, you know, and any teacher, certainly us, but any teacher in the college environment and probably high schools too, will have students come in afterwards and they're just, they're so distressed by their performance, you know, I knew it all. They get the sense that they've got it all kind of mastered and now they're in the exam and answers don't come to mind and they're very distressed often.
[00:24:59] Anna Stokke: Well, particularly with things like, so what students will often do is if you post solutions for them for a problem set and they don't know how to do the problem, they'll read the solution and then they read it and it makes sense to them, but then that doesn't mean they can do it, it just means they could read the solution, right?
And so that's the difference between retrieval practice, if you don't know how to do it, okay, get help, read the solution, whatever, but you need to be able to, you have to put it away and be able to do it without looking at the solution. So I get your point and then the other thing that students do a lot, especially now, is they watch YouTube videos of people doing problems and then again, they think they know how to do the problem, but they don't because they just watch someone do it, they didn't actually do it themselves.
[00:25:53] Dr. Robert Bjork: That applies to so many different situations. There was one famous study done on training in artificial respiration and people watched the steps being done on a dummy or whatever and yep, yep, got it, got it, got it, and now a little while later, now you got to do it and you realize it's one thing to follow it, but to kind of produce it independently is another level and that's very true in sports as well and a great range of situations.
One thing, we may not understand the engineering or physical details of a recording device, but we can kind of understand its functionality.
It gets recorded and it can be played back, but humans in all key ways work almost the opposite of recording devices, our learning machinery, so to speak, doesn't work. We wrote one paper one time about just all the ways we differ from any such recording device, so to really learn, you have to be producing, not just reading, and so all of these kind of principles that go beyond the surface of information are what make it very recallable in the long term.
[00:27:16] Dr. Elizabeth Bjork: It's probably one of the most frustrating things about how humans learn and remember effectively is that during the learning process, there's nothing, there's not a little gauge up there telling you, oh yeah, you're doing the right thing, you're doing the right thing. You get fooled into thinking you're doing the right thing, so we did give talks to a lot of students who were honour students and who were now working in classes where they were helping the other students, classes that they had already had and had done well in, and these students that was the most frequent question I got from those students was, was there any way that I can know when I'm studying that I'm really learning or is this something that, you know, tomorrow I just won't be able to reproduce or I won't understand again, and unfortunately there isn't any, unless Bob you can add to this, there isn't any way that you can know except to test yourself after a delay and that is a great informer of whether you really learned something in a way that you're going to be able to retrieve it on your own unaided later.
[00:28:25] Dr. Robert Bjork: We have the privilege at UCLA of having these terrific students, so as you may know, UCLA gets more applications for the freshman class than any other university, even though there's about half as many spots as some other schools, and so we have the privilege across our years of working with those terrific students and often when we're trying to say there's some better way of studying, they think, wait a minute, I got in here, I'm doing well, and one of the, one of the few ways we could really make pretty convincing case was to say, if you do everything we're talking about, you can achieve the level of performance in less time. Time is going to be, given the goals and aspirations you have, time is this critical resource, and if instead of just doing something that's low efficiency and going over this, if you incorporate these more effective techniques, you have more time to go into other things and so on, and that, not for every student, but for these highly motivated students for the great record, that was kind of a convincing thing, because they, a lot of those students want to take on more and more things, they want to get a apprenticeship of kind in the UCLA Medical School, they want to work outside with their, working with teachers in their school that they had years ago, you know, they're so ambitious and they want to do things, but there is only so much time, and so the argument that desirable difficulties can reduce the amount of time on task is very influential for some of those students.
[00:30:05] Anna Stokke: Yeah, definitely, that's a good point. Of course, you can study for a long time and not learn much at all if it's ineffective, so we should get into the difficulties, the desirable, desirable difficulties, not the undesirable ones. We could talk about the undesirable ones all day, that's kind of what this whole podcast is about, but let's talk about desirable difficulties.
So, we'll talk about spacing first, and you mentioned this thing about how it's good to retrieve at the point where you've almost forgotten it, okay, so I kind of think that leads into spacing, is that right?
[00:30:46] Dr. Robert Bjork: It does. In fact, really, if you just think about the spacing effect, you know, at first it actually seems pretty crazy, namely that that I studied something and now I let more forgetting happen by increasing the delay till I studied again, and now I studied again and I more than make up. I let all this forgetting happen and my performance goes down, now I studied again and now I'm maybe twice as good as I was if I didn't let that forgetting happen.
So, one paper we wrote had a title, Forgetting as a Friend of Learning, a pretty odd kind of thing, but in a real sense that's true for a lot of these things. You produce forgetting by the delay, then when it's presented you remember more, or you can produce forgetting by a context change. Now, instead of studying, go back to that same place, you go to some very different environment, you will perform worse, but if you restudy it then, then you'll perform better, especially if you're now in a third environment like the exam room.
And so, all of these manipulations that produce forgetting, there's four different ones, will enhance learning when the material is studied, practiced again. And so, that was the basic idea of the one chapter called, you know, we're just trying to get attention partly by calling it Forgetting as a Friend of Learning, but there's a real sense in which it is. If you want really long-term learning, you have to do something that produces some forgetting, delay, a context change, maybe the social environment and there's a whole lot of, well, there's too many different aspects of this, but performance can be very cue dependent, and that's another advantage spacing does, is it lets you be more effective recalling that information in more different environments.
And when you combine those two, that I will space and change environment, I can add benefits of both to, and now I'll be able to perform better in a variety of situations. But it's a hard thing to convince people of, and in some ways it's a hard thing for a student who is convinced to think of ways to vary, because, you know, for example, students on a college campus, students like to find some one place to study, you know, something, maybe it's in the library somewhere, whatever, and we had to tell students, if that helps you just sit down and get to study, that's a good thing, but just be aware that you're making your learning be context-bound, you'd be better if those separate learning sessions were in different places, as far as transferring to the classroom and other settings.
[00:33:29] Anna Stokke: So, there you're talking about variation, so you're saying that when students are studying, it helps if they vary their study locations, right? I'm quite curious about that. What kind of research has been done on that?
[00:33:43] Dr. Robert Bjork: Well, some of the just simply more straightforward and gave people a learning task in a particular setting, where it's been done on college campuses, we would then vary, they would come back to the same place and restudy again, or they'd go to another place on campus and restudy. And the change to another place, when you test them there, will produce more forgetting, but then after the same amount of restudying, they will perform better, especially in a new novel environment, like the classroom during the actual exam. And so, that seems very strange to students often, because they feel it's a good thing on their part to find some place where they don't fool around, they sit down, they're in a library, they sit down and start to study, and that is true.
But if you want to actually optimize everything, that will make your learning more sort of context bound and not as flexible. You know, you probably remember yourself, people have this thing, you're in an actual exam in a course, you remember, oh gosh, I remember I was sitting there, and to some degree, you mentally reinstate that study environment.
And one interpretation of the advantages of varying the study locations is there's more opportunities to reconstruct and use that contextual reinstatement as a clue to coming up with an answer.
[00:35:06] Anna Stokke: Yeah, it's kind of like when you hear a song, and it reminds you of something that you were doing when you were listening to that song at another time, right? Like, I can hear a song from the 80s and it takes me back to that time, right? And I'll remember like an old friend or something like that. So, it's something like tying whatever that was that you were learning at that moment, I guess, to that study space.
[00:35:30] Dr. Robert Bjork: Yeah. And even just anecdotally, it's interesting how often we find that you can do a mental reinstatement. So, over our decades of research at UCLA, we had sabbatical leaves.
Once we were at Dartmouth College, we were in England, and sometimes some question will come up, one of us will say, oh, what was that whatever? And now we start talking about, well, remember there was that little town, what was the name of that little town we used to go to there when we were at Dartmouth? Remember they had the little windmill thing there? And gradually it comes to mind, you know. So, the social aspect of reinstating, and there's something positive about replaying events in our lives. If we were a different kind of researchers, I think we might have explored that because really, particularly as you get older, as we are now, there's more and more time when you kind of want to replay, there's something about replaying events of our lives that's sort of reinforcing, but it also leads us to recall things we wouldn't be able to recall otherwise.
[00:36:35] Anna Stokke: Yeah. And so, all of this, retrieval practice is kind of tied into all of it, right? Like it's kind of like the, I guess, the mother of all the desirable difficulties, wouldn't you say?
[00:36:46] Dr. Robert Bjork: They're all important, but I think you might be right in that respect.
[00:36:49] Anna Stokke: And do you think that it has like the most research backing it up?
[00:36:55] Dr. Robert Bjork: I think probably, because you can go back, I mean, some of these do have a long history.
And for example, Benefits of Spacing, there were some studies in the 1950s, very well done, very convincing. And then it wasn't until decades later that people kind of started to emphasize theoretical reasons or practical applications. But I do think retrieval practice, it goes back to even early things in the 1900s, optimizing recitations of poems and things of that kind. So, it's a good question.
I mean, historically, both spacing and retrieval practice have a very long history. But often it was early with, the research was like with animals in, you know, rats or other animals in cages and stuff. I mean, the spacing effect is so general. It applies well, it goes across all the species.
[00:37:54] Anna Stokke: Okay. And so, spacing again is just spacing out your practice.
So, leaving a delay, forcing yourself to kind of recall it again, which is the opposite of cramming. So, cramming, of course, everybody knows what cramming is.
[00:38:10] Dr. Elizabeth Bjork: Spacing usually implies that you're getting represented with the material you're trying to learn again. So, you're taking a second look at that chapter or second look at that kind of problem or whatever. Whereas retrieval practice, you're not getting the whole thing back again to look at.
You're asking to, maybe you're giving a few parts and then you have to retrieve the rest. So that's kind of one slight difference there. Retrieval practice is probably what's underlying the benefits of spacing.
But just the procedure itself is you're not really requiring the student to outwardly retrieve anything. They're getting it here and now they're getting it again and you're spacing the time before they see it.
[00:38:55] Anna Stokke: So, it's important for students to know these things, particularly that retrieval practice is really important. Forcing yourself to recall the information instead of say, reading, watching, highlighting, that sort of thing.
So, you know, like flashcards and stuff like that are really helpful, I think. And then for teachers, I think the spacing piece is really important to know that. Like a lot of people don't know that, about that space practice is really beneficial.
[00:39:24] Dr. Robert Bjork: The idea that, you know, the very notion that I would cover something less completely than usual. And now in the second half of the course, I come back and go over it again. That's almost a formal, that's a very strange idea.
But the work where actual classroom research, some of it done by colleagues in England, show it makes a huge difference at the end of the year. So, at those places where there's a final test of a measurement of what's been learned across the year, this spacing during the year has a huge effect on those final tests. And we're prone in this country, I think, to think we've been over this.
Why would we go over it again? You know, my teacher can think I've covered that. But basically, once is not enough. And if it's important, it has to come up again.
[00:40:17] Anna Stokke: And even if they get it, like, even if you give the students a test, you've given them practice problems. And the evidence at that time is showing that they learn that information. You really need to force them to recall it again, because they forget it.
Right. And then eventually, the hope is, like Elizabeth said, that we'll remember everything we learned, right?
[00:40:39] Dr. Robert Bjork: I remember one event at UCLA where the students were saying that when there were data analyses in later courses, they were telling them, they were saying that this was never covered in their statistics course. And so, I was one of several faculty who went to the people teaching stat and was saying, well, this is pretty crucial.
Why isn't this being covered? And they all took out the course syllabus, and it was their feature. So, they just could not recognize that this earlier learning, so it can become kind of context-bound. And now where it's applicable in some new settings, they just can't see it.
They can't bring it to mind. And it doesn't transfer them.
[00:41:27] Dr. Elizabeth Bjork: I think in education, we've gotten too focused on how much of a subject we can cover, as opposed, and the more you can cover, the better teacher you are. Whereas that's sort of completely independent of, well, have the students really learned this information? So, if we could give instructors the freedom to give more tests, to bring back things again, to insert more of these desirable difficulties, I think, into their classroom teaching, students would go away with the benefit much more than just covering everything about American history. Cover, you know, less, but cover it such that I'll be able to remember when I'm an adult that this, that, and the other thing happened during the formation of the country and so forth.
[00:42:19] Dr. Robert Bjork: And not be so afraid of students giving wrong answers.
I mean, Elizabeth and a student working with her was one of the first to do research on pre-testing. This is something where in a class before your lecture, you ask questions on the material about to be covered. They don't know the answers.
They must get most everything wrong, but then that turns out to enhance what is learned from the subsequent lecture. Somehow trying to answer these questions where you haven't studied the material yet activates some relevant domains and now you become more efficient. I mean, this is, this is a newer result compared to the other things we've been talking about. Very active right now in different schools, different labs.
[00:43:09] Anna Stokke: So, we should talk about the one we haven't talked about yet, and that's interleaving.
Can one of you explain the difference between spacing and interleaving?
[00:43:19] Dr. Robert Bjork: Well, interleaving is often contrasted with sort of blocking, which is also kind of massing. And the experiment that had the biggest impact here was one we did with students who were learning the styles of different painters from examples of their painting. So basically, they saw a series of paintings, saw who the painter was, prominent kind of painters.
And then their later tests, they'd be shown new paintings and have to say who painted it. So, they hadn't seen those before. They had to learn something about the style of the painter to do that.
And so, it shows how we're not as smart as we often think we are. We did an initial experiment to show that there is one such type of situation where massing is better than spacing. And that's where, to use this example again, if I show you five or six paintings in a row by a given artist, you start to see their style, how they vary, what colours they use, and so on.
And that's going to help you on later tests pick up what their style is, which would help you identify new paintings. And that was very convincing, but see that would incorporate a kind of massing. I would show you six paintings by a given artist.
And so, we decided, we did a little experiment to compare that with one where the painting, same six paintings, how many artists do we have? Do you remember Elizabeth? Something like, it might've been four paintings by six artists. I think I've got that wrong. But so, if they saw 24 paintings, and then it would contrast that with the case where those were all interleaved, or you would see four paintings by one, so on.
Anyway, without getting into more details, that was a case where we predicted that here's one case where massing or blocking will help you to see this, that person's style. But lo and behold, when we actually did the final test, it was interleaving, not spacing that led to better performance. And one thing that was made that results have a lot of impact and created a lot follow-up studies, is at the very end, after they had done this final test, on which they had performed substantially better on the interleaved artists, we said, what helped you learn better, blocking or interleaving? 80% of the participants said blocking helped them learn better.
The same people had just performed better on the interleaved artists. Sometimes when you would inform them that they were better, they would think, oh, those artists must be easier.
They get such a strong feeling that blocking is helping them see what kind of colours that artist is using, whatever, and so on. And then people went on to look in other domains, then artists on learning the names of birds and all sorts of other things, turns out to be very general effect, but very, very unintuitive.
[00:46:25] Anna Stokke: Okay. So, interleaving essentially means taking different types of problems and sort of mixing them up as opposed to doing the same type of problem over and over and over again. And, and so in math, I think the reason that this is important is because it really forces students to discriminate between, particularly if the problems look similar, so that they look at it and they actually have to think about what strategy do I have to use to solve that problem? Whereas when they're blocked, they just kind of, oh yeah, I was supposed to use the quadratic formula the last time, so I'm going to do it again this time, right?
[00:47:04] Dr. Robert Bjork: And to this day, if we go somewhere and pick up paperbacks of practice problems, you'll see they'll have 10 in a row that are Pythagorean theorem.
You know that that's what solves this problem. Whereas that in many real world and academic things, knowing what to apply to solve the problem is a key important thing. I'm not telling you in advance what to use.
[00:47:27] Anna Stokke: The only thing I would say to that is, I think you do need the 10 Pythagorean theorem problems first, and then whatever else you're going to learn, say similar triangles or something like this, and then you block that, and then you start interleaving afterwards. Otherwise, I think it gets too confusing, like if you're just interleaving right off the hop with math problems.
[00:47:49] Dr. Robert Bjork: That touches on another thing that we did pursue, which is shifting from what's the optimal way to start the learning process with some massing or blocking, and then to shift to interleaving.
And we're pretty convinced that there is an optimal combination, but never pursued that topic very far.
[00:48:09] Dr. Elizabeth Bjork: But students are a little more receptive to interleaving if you give them some block trials first, and then you go into the interleaving. Even though on the test, they did not do better that way than if they just had nothing but interleaving.
[00:48:27] Anna Stokke: It's interesting. This one is the one that I think is the trickiest to me of all of them.
Just because I think for math problems, it probably depends on how difficult the concept is too, about how quickly you could move from blocked to interleaved. I wonder if blocking's really the issue half the time, or if it's just that it's always blocked, or that people tend to stay too long on blocking without varying the problems.
[00:48:58] Dr. Robert Bjork: There have been several experiments run where you put it under the learner's control what they see next. There's one on learning the names of birds, where, you know, the birds could look different, but be examples from the same. And they ask questions like, you just saw a thrush.
What do you want to see next? And people want to see all the thrushes before they move to any other. So, it's under their control. They will see all the birds of one type before they look at any other birds, even though, if you run the experiment, that will produce worse performance. Yeah. Something compelling about it as a learner.
[00:49:40] Anna Stokke: That's interesting. I guess it goes back to what you kind of said at the beginning. It's like, if it feels easier, it's what you want to do. I mean, someone else that was on the podcast used this phrase, which I thought was really good, that we're all kind of cognitive misers, that we want to do what's easiest cognitively, right?
[00:50:01] Dr. Elizabeth Bjork: And it applies to things like, Bob and I went, a few years ago, went to the Cleveland Indians baseball training camp to talk to the various players there. And they have this amazing equipment for those players to use. And one is these ball pitching machines that they have.
And they can be set to do nothing but, you know, fastballs, or they can curve balls, or whatever they want. They can mix them all together. And again and again, now, these aren't people just learning to play baseball.
These are people trying to get into the major leagues. And they want to practice with the machine that they can set, so it would be nothing but fast strikes down the middle. And, well, why do you want to do that? Well, that's the kind of ball I can really get out of the ballpark.
Well, what about at a game? Does the pitcher only pitch you fast strikes down the middle? Or does he sometimes do this, that, and the other thing? Oh, he does this, that, and the other thing. So, wouldn't it be better to spend your time practicing what's going to happen in a game than what the ideal thing is for you to learn?
Particularly on a, you know, like a scholastic test of some sort that's important for getting into, going higher, they're not going to be blocked like that for you. They're going to be all mixed up together.
The big thing there will be to decide what kind of problem is this? Well, of all the formulas I've learned, and all the procedures I've learned, what is appropriate for this problem? And that's been, I haven't been asked to do that.
[00:51:43] Anna Stokke: So, I think we'll start closing off now. We've had such an interesting conversation today, and it's been awesome to hear from you both. If you could eliminate one persistent myth about learning, what would it be?
[00:51:58] Dr. Robert Bjork: Well, there's so many answers to that. I mean, one, I think the practice makes perfect. It would be more accurate if you said practice makes permanent.
You know, that is how you practice is not going to produce a perfect outcome. It's going to make something good or bad more permanent. And so, that's one, but there'd be a lot of others.
I don't know if Elizabeth has one.
[00:52:24] Dr. Elizabeth Bjork: I might want to go after this whole idea of learning should be fun. And if you're not having fun while you're learning, then that's not a field for you. You should look elsewhere. Whereas, if learning is a lot of just fun, you're not really learning anything. You should have to struggle a little bit.
You should have to conquer a little bit. And then, you'll come to appreciate the satisfaction of that.
[00:52:51] Dr. Robert Bjork: And that's one other thing. The whole, when Elizabeth says your thing, all this research shows that in very, very early mathematics, one bad score or one thing makes people decide math is not their thing. And then, they make that a self-fulfilling prophecy by the decisions they make after that. And sometimes, they pick that up from the family context.
Their parents, instead of encouraging them to stay involved in topics like that, we may tell them something, well, our family's never been good at math. You don't know if you're good at math.
[00:54:26] Dr. Elizabeth Bjork: I guess the other thing would be to try to change the attitude that failing is a terrible thing.
Failing sets the stage for greater learning and have them understand that, you know, nobody, all the great inventors in the world that we've heard of didn't make a light bulb the first time they tried something. They went through hundreds of possibilities before they finally got something that worked. So that just sort of that same thing with students, help them understand that failing is not a bad thing. It's actually an opportunity for greater learning.
[00:54:04] Anna Stokke: Well, thank you so much for joining me today.
It was an honour and a pleasure to meet you both and to hear about your work. So, thank you so much.
Thank you so much for listening. If you enjoy this podcast, please consider showing your support by leaving a five-star rating on Spotify or Apple Podcasts. Don't forget to subscribe on your favorite podcast app or on YouTube so you never miss an episode. You can stay connected with me on Instagram, Facebook, TikTok, X, Blue Sky or LinkedIn.
All links are in the show notes. And check out my website, annastokke.com for more information. This podcast is funded by a grant from La Trobe University and from the Trottier Family Foundation through a grant to the University of Winnipeg to fund the Chalk & Talk podcast.