Revisiting Retrieval Practice | New Journal Club | TAPP 68
The revered martial artist actor and philosopher Bruce Lee once wrote I fear not the man who has practiced ten passing kicks once but I fear the man who has practiced one kick ten thousand times. Welcome to the amp professor a few minutes to focus on teaching human anatomy and physiology with a veteran educator teaching mentor your host. Kevin Pat in this episode. Iv reveal our new journal club. And I talk about putting retrieval practice front and center in the course. Hey I have some exciting news. We have a new kind of segment plan for the next episode. Or maybe the one after that these days. I've learned that one must not make plans based on the world staying on. Anyone assumed trajectory but soon I hope what is it I can tell you. It's not a book club we already have one of those and won't be getting a new book club recommendation soon. So now it's not a book club. It's a journal. Club the ADP Professor Journal Club. And we have a new voice joining in for that one. It's a voice that we've heard before on this podcast and a voice that you may have heard before if you've been active in town hall meetings or in Triple A. Or any of a number of other venues including her own journal articles on teaching and learning. It's my friend. Krista Room Polski who is an associate professor at Moravian College in Bethlehem Pennsylvania and part of the ANC authoring team over at McGraw Hill? Every other month or so she'll be bringing us a journal Article She's found. That helps enlighten us about evidence based approaches to teaching amp. I have a link to the first journal. Article in the show notes in the episode. Page if you WANNA read ahead and perhaps send in your recorded reaction but you don't have to read ahead. Don't worry about that. Crystal will be summarizing. The article in the Journal Club segment. And then she and I will be discussing it a bit as always if you have any ideas to share about that or any other feature of this podcast or perhaps a new feature. You'd like to suggest. Please let us know so that we can better meet your needs. Hey let's have a peer review. Wait we already have that. It's called the listener survey. I don't have nearly as many of those in is. I'd like to have so why not go to the? Ap Professor Dot Org Slash Survey Right now and tell me what you think anonymously. Of course this is professional peer review or after all right searchable transcript and a captioned audio Graham. This episode are funded by AAA the American Association for Anatomy at Anatomy Dot Org. Did you know that if you're looking microscopic images to use remote teaching triple a? Has You covered? They have something called the virtual microscopy database or Vm. De for short that has more than thirty five hundred virtual histology and pathology slides that you can use in your course. Just go to anatomy dot org and Click on the resources tab then. Select Virtual microscopy database by the way this is a free resource for both members and non members a longtime listeners. Know that I'm a big believer in the power of retrieval practice as a learning strategy. When I say believer I mean zealot but I also mean that. There's a huge body of experimental evidence to support my believe that it's one of the most powerful ways to learn for the long term so when I throughout the idea of using retrieval practice particularly in the form of open book on timed online tests as I did in the Pandemic Teaching Book. I told you about episode sixty seven and also in a segment of that same was based on my presentation in the hats townhall meeting. I'm advocating a really really really powerful strategy but I also know that there is great resistance to it I know because I also resisted the idea for a long time. I don't know maybe for ten years or maybe even longer than that but I do know that when I started doing these oddly designed online tests in my web enhanced face-to-face AP courses way back in two thousand and two. I felt like I might be jumping off the proverbial cliff. I thought I might go. Splat the bottom of that cliff and a lot of my colleagues. Were very very very unsure of what I was trying to do now. They didn't try to discourage me or anything like that but I could tell from the look in their eyes and things big sad to me that they were worried that I truly lost my mind that is gone from quirky and eccentric to full blown mad but it turns out that I should have trusted experimental evidence. Even though it kind of went against my gut feeling and really flew in the face of the way. It's always been done on the other hand. Part of my gut also knew it was the right thing to do. You know that part of my gut that actually trusts the science and when I thought about how we learn nearly everything we learn outside of the classroom things like writing a by doing any sport or hobby or even on the job training like when I was an apprentice Lion Tamer. I realized that we do it. By retrieval practice. That is we try something it fails we fix it maybe with someone guiding us and we try again and fail again and get more advice in coaching and eventually we get it and learning things that way. We often never forget it. We don't read about Lion Taming or soccer or I dunno trout fishing. Take a quiz on what we've read and then we're able to just go out and do it. That just doesn't make sense right yet. That's what we do. In our academic courses and what we're often afraid to let go of the free distribution of this podcast is sponsored by the master of science in Human Anatomy and physiology instruction the happy decree. I'm continuing to get all kinds of feedback from our current and former happy graduate students who are out in the trenches during this time a pandemic teaching and they're expressing their gratitude for giving them the tools techniques and the flexible mindset to cope with the sudden move from on campus teaching to remote learning. And you can get this to. We have many students who already have doctorates and Masters degrees in our program. And they love it and yes. Most of them already have other fulltime responsibilities. It's Kinda hard. Yeah but it's doable. Checkout this online graduate program at NYC DOT EDU slash happy. They'LL TEACH API. Click a link in the show notes or episode page A. I've talked about this retrieval practice thing in past episodes so you may want go back and listen to the back catalogue you know as you walk your dog around your neighborhood at a safe distance from others but right now. I do want to summarize the essential way my implementation of retrieval practice works in case you WanNa try something like it or in case it sparks an idea for some little tweak that you can make to what you're already doing or are already planning for the next semester. I it's based on online open book tests. That open book thing seems heretical. I know all but really this is real life so I just needed to get over that and I'm glad I did because it turns out that this is the way that students achieve mastery by getting help when they need it. Just like in soccer or trout fishing and you know what isn't looking things up and consulting with peers. Something that the truly competent health professionals do all the time. Hey if they're taking care of me I'd rather they double check the proper dosage of that script. They're about to write for me or ask a colleague. Check my medical imaging before making a diagnosis. Why wouldn't I want my students to use those options in amp? The next thing is that I set my learning management system to serve the questions one at a time just like they do in the online board exams that most health professionals have to take to get licensed or to proceed from one stage of professional training to the next. I heard so many of my former students told me that this one time format raised their stress levels even higher than they already were walking into that board exam meaning maximum stress. Right after I'd been doing that for a while in my class I started hearing from former students that think. We're so glad that they've gotten used to doing that in my class. And then unlike some of their classmates who didn't have me for amp. My students didn't freak out when they had to do test items one at a time on their board tests. But you know what that's just gravy. The main reason I do the one in a time thing is that I'd found out that during in class testing some students had difficulty focusing when they were faced with many items on one page. Now I know you and I are used to. That are more likely. Our brains are just wired a certain way to allow us to focus on just one item on a page of many items but for some students probably each with their own unique set of neural pathways and connections the ability to see just one test item at a time helps them focus which means that it helps them to succeed next. I don't strictly time mine tests. There's usually a start date and an end date but not a limit of a certain number of minutes way back in two thousand and two when I first started doing you miss. I didn't plan on giving untying tests them up in my first one with a one hour time limit and it had to be on a certain date and within a certain limited window of time like ninety minutes guess what because I had at one of the highest number of students per semester at our college and because I was apparently the first professor in history of our college to give students one test item at a time there were too many hits on our server per hour and the learning management system started weasing and coffee in will. It simply stopped working for a while. Not what you'd want to happen. When two hundred and fifty students have a limited time to take a test right especially in amp test because those are always scary and even more so when it's an online test because remember this was almost twenty years ago when not very many students had ever taken an online test or at least. We're very comfortable with them then. Matt at our college certainly so I went to our staff and they advised me to stretch out the day to my test taking window and make my tests on timed of course that sticky frozen stay in my comfort zone. Part of my brain rebelled. What an untying tasks that they could take over the course of several hours or days but the alternative was the shutdown our server for my students and all students that are so. Yeah okay I relented and I think I kinda did pout. A little bit at least in my head I was pouting. You know what though it turned out to be a breakthrough on time? Testing is so freeing to the learning process and in so helpful to students with almost any kind of learning challenge. Which is everyone in my opinion. Sure eventually medical professionals will need to be able to answer things quickly on the spot but we should not expect them to start their should we? So okay maybe closed. The books notebooks and have some kind of time limit on a midterm or final exam. Maybe but really. I'm thinking of the closed. Books and timing should come at or near the end of their degree program. Not In my course at all and AIM P. They're still beginners even at the end of my course marketing support for this podcast is provided by. Half's the human anatomy and physiology society promoting excellence in the teaching of human anatomy and physiology for over thirty years. As I've mentioned in the last episode if you haven't been stopping by for the virtual town meetings you're missing a great opportunity to visit with other amp teachers and learn about what they're doing in their courses. You can just listen in or you can ask questions or you can share. What torque for you or what's not worked for you. It's really fun and it's uplift thing it's a way to stay connected to others in the same boat as you are. You're a member. Watch your emails and if not cope is at Habsburg at the AP Professor Dot Org Slash helps that's H. APS and join today In the next step of my recipe calls for a lot of retrieval practice more than you are used to way way more than you and I ever saw in our own education retrieval. Practice the topic of the very first episode of this podcast. Is this decades favored jargon for plain old fashioned testing or quizzing or whatever you WANNA call it that is being challenged with test items case studies problems specimens to identify any and all manner of challenges to go into our memory banks and find the pertinent facts and sort them out and put them together and well retrieve the solution to the answer. The test item. That's the retrieval part then to that retrieval with a slightly different problem again and then again so that we get better and better at it while also improving our long term retention of relevant memories. That's the practice. Part of retrieval practice looking at retrieval practice from the testing angle. We see that this isn't the kind of testing that most of us I think of when we think of a test we usually think about summit of testing that measures. How well a student has learned something protesting as retrieval practice is in stowed a formative kind of testing measuring how much a student has learned while they are still learning it. It's sort of like a practice. Soccer game you find out what's working and what's not working before you play a regular game that counts for your team's season record and possibly for a championship so the practice sessions and practice games are formative and the regular game is summit back. In the olden days when I was an undergraduate that was done through self graded homework it had been impossible for professors even those with a large team of. Ta's to immediately grade a big stack of homework coming in every single day including weekends but thanks to modern learning management systems. I can do that. Through automatic grading a student can take an online formative test and get immediate results. Then take another attempt and another one gaining all the benefits of retrieval practice and being able to truly master concept and remember it for the long term again back in the olden days. A lot of what I learned was held onto for a few days or weeks just long enough to pass that next in class test. And then I'd forget it forget most of it would. I did hold onto the long-term term or concepts that came up again in a later section or later course the came soon after the previous one. I've found that giving students. Multiple ATTEMPTS GIVES THEM ENOUGH. Retrieval practice for to really do some good in some courses. I gave him up to three or four types with the highest score counting for the course grade in other courses I've given them unlimited attempts and you know what all approaches work just fine. The trick of course is to make sure that each test is different. That is it's still tests. The same objectives are learning outcomes but with a different set of test. Items Support for this episode comes from our friends said. Ad instruments who've come out to help support amp teachers and our time need. They've come up with a plan to let any of US use their lt online learning platform for free as I mentioned in the last couple of episodes lt is an online environment that you can set up very quickly and it contains ready to use content for anatomy physiology and other biology courses even medical courses. It has over three hundred and sixty interactive and fully customizable lessons and labs. That are just the thing for remote learning. You ready no that. Ad instruments provides only very high quality products and have access to a whole set of the remote learning resources. Right now for free is an incredible thing especially during our adventures with pandemic teaching. Just go to. Ad instruments dot com slash lt slash Kobe. Nineteen to find out more once again. That's a D instruments dot com slash L. T. SLASH COVA. Nineteen A if we want to have a different version of anyone attempt of anyone test and we want to allow at least three attampts. Whoa that's a lot of different tests to right. Isn't it allow unlimited tests? Which is the direction which I've been having then. Wow we need a Jillian versions of each test. Any of that is pretty much impossible right or is it impossible. Not only is it possible. It's fairly easy not super easy but fairly easy. All the learning management systems have the ability to use. What I'M GONNA call a question set but each has its own name for this feature. Here's how it works for test item. One I ride three versions. They could be three different ways to ask the same question or they could be three different aspects of the concept that I want to assess in test item one for example it could be three different examples of cardiac muscle tissue to identify or it could be one. Each of three major types of muscle tissue when a student takes attempt one one of those three items in the questions that will be presented as test item one in their next attempt. It could be the same item presented or more likely one of the other two versions because it's randomly selected by the learning management system. Even if it's the same item one that they saw in their first test attempt each item in the task. Let's say fifty items total we'll be similarly randomized so yeah okay. Test Item? One is the same as they've seen before. But maybe probably all or most of the other forty-nine items on attempt to will be different than they've seen before as with a slot machine having three items in each question site for fifty test items will generate now on your seats here. It'll generate seven point two times ten to the twenty third different possible test attempt versions. Yeah that's all right. It's it's more than a mole of different versions. The thing is I usually have more than three items per questions. I divide up. I want my students to know in the fifty categories some of which overlap because I want to attack some things at different levels of understanding or application or style of test item. Then once I get going and I'm constructing three items from my first category or my first question set you know what I always think of a fourth item which boxing idea for fifth item Mandalay that but for multiple choice and matching items I can set the LMS to randomize the choices. So I've got another billion Brazilian versions of that test being generated because not only is the LMS picking random items out of a question set of three or four or five. It's also scrambling up the choices within a multiple choice or matching item. When I first started doing this back in two thousand and two I got a math. Professor help calculate the possible number of versions on any one of my tests and her calculator could not go that high. It just gave an error being a math professor. She had a pretty powerful calculator. And you know what that's all I needed to know. I didn't need to know the exact number I just needed to know. It blew out the math professor's calculator so that was good enough for me. That is good enough to ensure the my students weren't getting the same test and every attempt of course I waited until the semester started to begin making my first time hoops ship. It started that way and advanced so well so heavy days that first semester. When I came home from school I immediately set my timer for twenty minutes and wrote as many test items as I could before the timer went off doing that every day. Yeah it was a chore. But really twenty minutes was not that big of a chore and it became a habit so it really wasn't that hard to sit down and do it for twenty minutes every day. Because that's just kind of what I got used to. After my first test was done I was really getting good at writing test items and getting faster and faster about coming up with good test. Items and his students were taking attempts of their first test. While I was working on test to they were taking the first test. And they were giving me feedback on. Which items were poorly constructed or just odd or Fulla? Typo is or something was wrong with it. I could claim that I did that on purpose. But I didn't know I just got started late but it did turn out to have that silver lining then the next time I taught that course it was easy to add a few more test items to some of the question sets of course I was by now a seasoned pro at test item writing something. I'd always dreaded it because I didn't feel like I was any good at it. But all that retrieval practice gave me a certain level of mastery mentally mastery of test item writing but I also gained insights about the AP concepts that. I was considering for test items. I lived through it and I'm a better person for it or at least a slightly better person for and so yeah like any new project. It seems intimidating but it's like a very long walking trip. You take one step at a time and I don't know if you walked for twenty minutes a day four days a week for two sixteen week. Semesters you'll end up having walked more than one hundred twenty five miles probably with no injuries. No collapsing on the roadside. No dehydration just a smile on your face. So do it one step at a time and you can do it to me and I also want to mention some icing that I put on that retrieval practice cake and decorations after all. Isn't that cake that much better when it has icing and grains one thing? I did that helped a lot. The icing was give students an online pre-test before much of the course and online attempt taken from that huge test bank that built over that coming modules content because I told them to expect to fail it but they had to take it to unlock the videos and other learning resources for the coming module honestly it took a while to convince them not to read ahead and not to study not to prepare for it. It was merely to give them a peek at what they be expected to know later. But we're not in any way expected to know yet and you know what those pretexts helped a lot now. The cake decorations consisted of making my online tests. Cumulative every test had questions from all the previous tests the really important questions the core concepts the big ideas. Yeah of course. They balked at that one. I I told them who wouldn't. We've all learned. The cumulative tests are scary. But it didn't take them long to realize that those questions from prior modules were the easy questions to answer on a new test and by the time they got through a few tests. Those questions got even easier because they kept seeing those kinds of questions again and again questions on those topics cap coming up so they grew to love cumulative testing. I know that sounds weird. I know don't believe me but I swear it's true and I know what you're thinking that if we're doing all formative testing and it's open book and students can even consult with each other than well. Can we trust that? They've actually learned anything that they have any knowledge in their heads that they can use without a book without asking anyone and also there on the right track. Well the answer to that is an emphatic. And wholehearted yes. How do I know that? And how can I be so confident about it? It's because when I first started this and for many years thereafter I also did to summit of exams to see whether all that retrieval practice worked. And you know what it did. Not only students walk into their exams with more confidence. They did way better on them. Then my pass classes had done on their exams before I started doing. Any of his formative testing possibly part of that improvement came from their confidence. But I'm sure that a lot of it came from all those open book formative tests and my students told me that they not only felt confident on exam day. They also told me that they tried to cram for the exam but found that well. There just wasn't any need to as they prepared to cram the night before an age old ritual. They felt compelled to perform they realized as they reviewed the course content that kind of knew it all and knew it pretty. Well so yeah. I don't have any problem thinking that. The formative testing alone gave them what they needed. There's more to my story of retrieval practice in the AP course and a lot of that story is told in previous episodes and in seminar that I have available in the links and the show notes episode page. But before I wrap it up for now I want to mention that I really think it works. Best if we write our own test items most of the time sure the test items at the end of textbook chapters in study guides in online study or quizzing programs and the like are all useful tools and have their place and learning. But I've grown to like making my own and I think if I use my own test items that integrate easily with the way I tell the story of amp it works better for learning. But that's a discussion for another day. A in the previous episode that is episode sixty seven. I mentioned a free e book that I recently published. It's called pandemic teaching a survival guide for College. Faculty. It's a little over one hundred pages and it's a quick easy read. Oh about two hours. And it's chock full of ideas tips and mindset. That help us move to remote teaching and be successful at it. I'd really love it if you could do me a favor please. Could you take a moment and share it with three colleagues at your school? One of them being whoever organizes the professional development for your faculty really after going to all the trouble of putting it together. I'm like folks to get some use out of this book. It's a free download from all the major bookstores to share this link. I'm going to give you to see a list of different places to find it like Amazon. Kindle Barnes and noble nook apple books and all the list goes on so that link is books to read that books the numeral to read dot com slash pandemic teaching once again books to read dot com slash pandemic teaching. Oh Yeah and could you also take a moment to mention it in social media and share that link when you do it? I sure do appreciate your help with this. A I always put links and the show notes and at the episode page at the AP Professor Dot Org Slash Sixty eight in case you want to further explore any ideas mentioned in this podcast or if you want to visit our sponsors and you're always encouraged to call in with your questions. Comments and ideas at the podcast hotline. That's one eight three three Lion Ben or one eight three three five four six six. Three three six or send. A recording are written message to podcasts at the Professor Dot. Org I'll see you down the road the A. N. P. Professor is hosted by. Dr Kevin Pat An award-winning Professor And Textbook Author. In human anatomy and physiology any resemblance to actual persons living or dead is unintentional and purely coincidental.