Student report: Open University trains markers to spot generative AI work

The Open University has carried out research into the strengths and weaknesses of assessment types as generative artificial intelligence (GAI) tools develop.

AI is becoming more present in both education and workspaces, with more than half (56%) of college students using a form of GAI on their coursework or essays, according to the Open University. This, the university, suggested could cause an increase in plagarised work.

The Open University conducted 55 different assessments to discover which are the most robust, and easiest assessment types for learners to answer if they use GAI. As part of the research, markers were given 16 answers, ten of which were authentic answers and six of which were GAI generated. The scripts were split up to prevent the markers from knowing who wrote which answer. A total of 950 answers were included in the research, making it the largest GAI investigation at the time.

Liz Hardie, Director of SCiLAB and Senior Lecturer in Open Justice, said that the markers had a positive reaction to the training, with many feeling more confident in pointing out where GAI has been used. Hardie said that training markers to recognise GAI can be extremely beneficial, however it could also lead to some students being falsely accused of using the tools. Studies by The Open University have shown how work produced by GAI is similar to some lower ability students work.

An anonymous marker said: "I could see how the ways in which GAI cannot really come to a conclusion is different from the ways in which students just fail to put forward an argument. GAI tends to provide both sides of an argument and then it’s not coming to a conclusion, while students just fail to put forward a convincing argument throughout.”

The investigation concluded that the higher the level of qualification, the less useful GAI technology is. The study found that GAI allowed Level 3 learners to achieve higher grades than those at Level 5, which suggests that the tools are more useful for college students rather than university students.

It’s hoped that the project will help markers to be able to pinpoint when a learner has used AI and ultimately reduce the amount of plagiarism that is taking place within schools and colleges.

Charlie Wheeler is studying UAL creative media production: Extended Diploma in journalism at Exeter College. He was reporting from the Association of Colleges Annual Conference and Exhibition.