Retrieval Practice is arguably one of the most popular learning strategies in education. Its popularity is based on a wealth of research that has shown it can accelerate student learning.
Chat GPT, an Artificial Intelligence processing system which can interact with its users, has recently gained a lot of attention in the education world and beyond for its potential wide-ranging implications. But can it (and more importantly, should it) be used by students when they are doing Retrieval Practice?
Plato once famously remarked that “If men learn this, it will implant forgetfulness in their souls”. The revolutionary new technology he was talking about? Pen and paper. Ever since then, each wave of new technology that emerges seems to offer great promises of advancement but comes attached with a warning that if it is used poorly, it can hinder more than help.
What Is ChatGPT?
To answer this question, we’ll let Chat GPT explain it to you…
So, to summarise: ChatGPT describes itself as an AI assistant that can help out with various things such as providing information, answering questions, and more. It is not human, but it is designed to communicate like it is.
ChatGPT is capable of a range of Retrieval Practice questions
ChatGPT has many potential applications. One such function is that it can very quickly generate a list of questions for students to retrieve. For example…
ChatGPT can generate general Retrieval Practice questions
If you ask ChatGPT for 10 questions on Romeo and Juliet for example, it immediately lists 10 relevant questions that vary in breadth and difficulty, from the more simple “Who wrote Romeo and Juliet?” to the more challenging “What is the significance of the balcony scene in Romeo and Juliet?”.
ChatGPT can generate Retrieval Practice Multiple-Choice Questions
If you were to ask ChatGPT for three multiple-choice questions based on Romeo’s fight with Tybalt, it quickly provides you with four response options for each of them. For each question, the answer options provided are all plausible and within the same scope so that they aren’t too easy to answer.
ChatGPT can generate Retrieval Practice essay questions
Finally, if you ask ChatGPT for an essay question, it instantly generates an open-ended question that is refined and in-depth. In this example, the generated question for Romeo and Juliet elaborates on possible ambiguities and specifies the number of examples needed in the answer.
This range of Retrieval Practice activities is encouraging, as evidence suggests that the testing benefit can be found if doing multiple-choice questions, short answer questions, or essay questions. ChatGPT being able to generate all these different types of questions is handy.
ChatGPT is so fast at generating Retrieval Practice questions
The most obvious initial benefit to be gained by using Chat GPT for Retrieval Practice questions is the speed that it can generate questions at. All too often, students spend a lot of time creating their questions on their flashcards, which leaves them less time to retrieve the answers – which was the original intention of using flashcards. If used correctly, this benefit of ChatGPT can ensure Retrieval Practice activities are more time efficient.
So, should students use ChatGPT to write their own Retrieval Practice questions?
So, even though using ChatGPT can have so many benefits for Retrieval Practice, we still need to answer the most important question: does it mean students should be generating their own questions for Retrieval Practice? Or should someone else do it?
At the time of writing, we don’t have a definitive answer from the research on this, so we have to look at some related studies to draw inferences.
In this recent study, students in a lecture were divided into one of three groups:
- Group 1 had to generate their own questions based on the lesson and answer them
- Group 2 answered questions that they had been given based on the lesson
- Group 3 had to go through the lecture slides and memorise the content
One week later, the researchers found that students who generated questions and answers performed equally well as those who had answered questions that had been given to them (with both groups performing better than the one that only re-studied the material).
Unfortunately, getting students to write their own questions, which may have some generative learning benefits, has a big significant drawback: many might not know what counts as a good question.
This is also true for Chat GPT. When we asked it to come up with multiple-choice questions based on Romeo and Juliette, some of the questions that it generated either didn’t make sense or didn’t have a correct answer (see questions 2 and 3 below, for example).
A teacher is likely to spot these errors, but a student using ChatGPT independently may not be. The phrase in coding to describe this is known as the acronym GIGO (Garbage In = Garbage Out). If the input (in this case, the Retrieval Practice questions Chat GPT asks students) is substandard, then the output (the quality of the retrieval) will be as well.
Can we use ChatGPT to ask challenging retrieval questions?
One of the reasons why Retrieval Practice works is that it prompts students to think hard. This means that students do not benefit from retrieval tasks being too easy. This is shown in this recent study, which found that although students prefer easy questions (and rate them as more effective), these are actually not as beneficial for learning as challenging questions are.
ChatGPT works better the more specific and targeted the commands you give it. This means you can ask it, for example, to generate a list of “challenging questions on Romeo and Juliette for a 16-year-old studying for their GCSEs”. Here is the list of questions it generated using this example:
However, these questions are probably too challenging. When we discussed these with English teacher, Mark Roberts, he said “these are such huge questions. Typically, exam questions focus on one theme, character, or extract. Or as teachers, we look at one aspect of the context. These questions are asking them to cover everything at once, which requires massive amounts of knowledge and writing skills to summarise. There are so many different features of language to cover in Question 3 for example.”
Finding the sweet spot of "difficult, but successful" for Retrieval Practice takes knowledge about what your students do and don’t know. As mentioned above, even students may struggle with that self-awareness. Outsourcing that decision to ChatGPT is unlikely to lead to personalised, specific, and targeted retrieval.
ChatGPT for Retrieval Practice: The importance of feedback
There is a wealth of evidence that suggests that Retrieval Practice is beneficial for learning. However, numerous studies also found that it is even more effective with feedback. In some instances, such as with multiple-choice questions, ChatGPT does provide answers to its Retrieval Practice questions (this can be done by simply typing “show me the answers”).
However, that’s not always the case, typically for free recall/open ended retrieval questions. This is frustrating and a real drawback, as it feels like a potential missed opportunity.
Final Thoughts
The power and possibility of ChatGPT shouldn’t be underestimated. One of its many uses could be to provide Retrieval Practice questions.
However, as it currently stands, we think the positives (i.e., its speed as well as the range of questions it can provide) are seriously mitigated by some of the limitations (i.e., errors in questions or limited feedback).
However, the biggest problem is the same as why students can’t just google an answer. To use it effectively, you need a good understanding of a topic to allow you to detect what is a good or bad question.
It therefore stands to reason that it may be of more use for teachers in generating retrieval questions, compared to students doing it for themselves.
