Can AI understand feminist discourse?
There is no shortage of suggestions about how AI can help us with our research, and the authors have found it useful in supporting some aspects of their work. As we all know, however, the results generated by AI can also be fundamentally flawed and misleading. In this blog, we explore the implications of a recent experiment in which we asked AI to summarise sections of interviews conducted with activists in the Rape Crisis movement. The issue that arose was whether AI is capable of understanding feminist discourse.
For the past three years we have worked on creating the Herstory collection: a National Lottery Heritage-funded oral history of the Rape Crisis movement in England and Wales. Once catalogued by National Life Stories (NLS), it will form part of an open-access audio collection. In addition to depositing the sound recordings, we were asked to produce 250-word summaries for every 30 minutes of interview. These summaries help listeners navigate the recordings and allow search engines to make connections across catalogues.
We soon discovered that high-quality summaries are time-consuming to compile, taking an average of two days per interview. We began to discuss whether we could speed up the process without compromising quality. After consulting with NLS, we asked interviewees whether they would consent to our using AI to generate initial drafts of the summaries, on the understanding that we would carefully check their accuracy against the interview transcripts. As a result, 17 summaries were produced using ChatGPT Edu within the University of Oxford’s closed, enterprise-grade digital and AI environment, which uses secure cloud storage systems.
The summaries produced by ChatGPT Edu were useful in a range of ways. It was helpful to have blocks of 250 words to work with in order to ensure that we kept to word limits. ChatGPT Edu was also very good at picking up on factual details such as interviewees’ place of birth, the number of siblings they had, their parents’ occupation, and where they had attended school or worked. But as our work progressed, we both became independently concerned about the extent to which it accurately captured the essence of more complex discussions about the ongoing objectification of women, the sexualisation of girls, narratives about trauma, sisterhood, radical listening, the dynamics of collectives, and intersectionality, all of which featured prominently in the interviews.
The following example, used with the permission of those involved, aims to illustrate the mismatch between the AI-generated summaries and our understanding of the feminist discourse being engaged with in interviews. Word limits prevent us from reproducing the relevant 500-word section of the transcript, in which an interviewee reflected on the need for specialist services for Women of Colour within Rape Crisis centres. However, a comparison between the ChatGPT Edu précis and the one we prepared is insightful:
‘[Interviewee] describes context as everything, [a] way of thinking that shifts attention from identity alone to the social crossroads where harm is produced.’
Our preferred summary read as follows:
Detailed description of the systemic context in which rape occurs including racism, misogyny, patriarchy, capitalism, family and other oppressive social constructions. Describes intersectionality as a crossroads of social inequality.
A comparison of the two summaries raises three interesting issues. Firstly, ChatGPT Edu did not pick up on the structural causes of sexual violence with which feminist debate, philosophy and activism have long been concerned; instead, violence was merely seen as having a ‘context’. Secondly, and somewhat bizarrely, the systemic causes of sexual violence were seen as a location rather than as pervasive, as the interviewee was at pains to stress. Finally, the notion of intersectionality was not referred to at all and instead understood merely as a crossroads unconnected with identity. It is interesting to note in this context that we asked ChatGPT to refer only to the interview transcript and not online sources to avoid hallucinations. Significantly, the terms ‘intersectionality’ and ‘intersectional’ appear a total of 47 times in the interview concerned, and the concept is a major point of discussion. Yet ChatGPT Edu appeared not to have learned from this.
There is nothing original in our concerns that algorithmic bias produces systematically prejudiced results that reflect and amplify existing global inequalities, though it is less common to come across socio-legal examples of how such bias manifests itself. This bias is likely to be particularly prevalent where sexual violence is concerned, given the sheer volume of pornographic and misogynistic material on the internet and social media, as well as the silences concerning rape and sexual abuse. Viewed in this way, ChatGPT Edu can be excused for not understanding feminist discourse. It has been taught to extract and abstract from the material to which it has access, reproducing hegemonic bias rather than challenging it. However, our research suggests that feminist scholars should remain vigilant in exposing and challenging instances of narrative erasure.
About the Authors
Professor Linda Mulcahy
Director of the Centre for Socio-Legal Studies, University of Oxford
Linda is the Director of the Centre for Socio-Legal Studies, University of Oxford. Her research focuses on how people experience the justice system. She is currently working on an oral history of radical lawyering and on the impact of video-technology on legal ritual and space.
Dr. Ellie Whittingdale
Fellow in Law, The London School of Economics and Political Science (LSE)
Ellie is a Fellow in Law at LSE Law School. Previously, Ellie worked at CSLS with Professor Linda Mulcahy to create the first permanent oral history collection of the English and Welsh Rape Crisis movement, lodged in perpetuity with National Life Stories at the British Library.