I first wrote this in 22nd December 2022. Looking it feels like a lot has happened since then but what I said about GenAI assessment didn't change at all. I've published 2 papers and gave several talks about this in the last 2 years. If you are interested, here is one of my recent article titled "Generative AI in Higher Education Assessments: Examining Risk and Tech-Savviness on Student’s Adoption.
What did I do 2 years ago? I simply go over different popular assessments we used and talk about the impact of ChatGPT on them. Let's look at them briefly.
Coursework
I simply copy paste one of the coursework briefs that I used in my Marketing Communications course onto ChatGPT, and the result was okay. It is undoubtedly not a distinction (70+) or merit mark (60+) in the UK system. There are several problems, such as:
It writes the same paper for everyone if you don’t change the assessment brief. So, plagiarism cases will skyrocket, I presume.
No theoretical support. It doesn’t know the module content, obviously.
A low number of references.
All reference resources are internet resources. We ask students to use academic articles and textbooks in their reading lists.
You can fix those problems by asking ChatGPT to rewrite the paper by incorporating concepts from a particular textbook, but it is still insufficient. We want students to use relevant theories and frameworks, and to be able to do that; students need to know what those theories are. ChatGPT can certainly help them summarise articles, cases, or textbooks, which is excellent because many students don’t read them at all.
On the other hand, many students can use this tool if they don’t aim for a good mark. The response of ChatGPT is definitely a pass mark for sure. It is well-written and shows an okay degree of internet research, but it is not more than that.
What can we do about it?
First of all, definitely use a plagiarism checker for essay-type assessments.
More importantly, as we always tell students, “Be creative” and “Think outside of the box”. This time you do it. We have advocated authentic assessments in higher education for a long time. So, instead of essay-type assessments, it is better to use more authentic assessments like
asking students to do a podcast,
or develop a creative idea and prepare a pitch to present it,
or use simulations,
or ask them to create portfolios.
If you are short of ideas, ask ChatGPT. It will give you so many authentic assessment ideas (no pun intended, it really gives you some ideas :))
And, of course, the safest assessment will be in-class ones if you like to keep essay writing as the core of your assessment. Yes, in-class will increase its popularity again after getting a severe hit during Covid.
Multiple Choice Quizzes
Oh, Multiple Choice Quizzes (aka MCQs) are doomed if there is no proctoring tool in place. Copy-pasting the question is enough for ChatGPT to give you the correct answer and the explanation for it It is incredible.
What can you do about it?
Use pictures and graphs in the questions and ask students something based on them, like the one below.
Use multiple correct answer options with negative markings. Currently, I am conducting research with my colleagues on the increasing difficulty of MCQs by providing multiple answers and negative marking features. Our study found that it helps students pay more attention to the module content and prepare them better for the assessment. It turns out ChatGPT is not great at answering multiple answer types of questions. I used the below question, and ChatGPT failed to answer it correctly. Well, I’ll definitely use this while presenting our research for sure 🙂
So, what?
For academics
It is not the end of the world for assessments in higher education. We know students already use so many tools for their studies. I actually encourage them to use different tools because it is what they will do in their professional lives. However, we need to keep pushing ourselves to create meaningful and relevant assessments. And, no need to say that we still need proctoring and plagiarism tools.
For students
We need to emphasize the importance of having moral and ethical values. If you cheat in your assessments, you will never have the confidence to do something on your own and need some sort of help in your life. Use allowed technologies to develop your ideas or check your writing but always do the job on your own. It feels good to earn something that you work for.
Well, I can keep digging into this topic, of course, but I would like to hear your opinions and suggestions about it as well.
Comentarios