New blog post: Beyond the Redesign Rhetoric
Calls to redesign assessment for AI often ignore the labour, priorities and power structures that shape what is possible in practice.
This post asks how we can support assessment that is not only technically robust but also educationally purposeful and institutionally sustainable.
Read the post here:
https://www.e-learning-rules.com/blog/0027_beyond_the_redesign_rhetoric.html
#AIinEducation #AssessmentDesign #AcademicLabour #DigitalPedagogy #eLearning
What Makes Writing “Academic” in the Age of Generative AI?
If we see a significant change in the nature of academic writing, would that actually be a problem? As Molinari (2023: 18) notes the “genres, jargons, grammar, syntax and overall forms have been pejoratively described by writing scholars as straightjackets, chains, pigeonholes, frauds and hoaxes” not to mention the many critics for whom it is “discriminatory, elitist, exclusionary, colonial, dull, zombified and confusing”. I’ve often felt sceptical about these critiques because of their tendency to suggest that specialised writing is by its nature problematic, as opposed to reflecting the specific needs of an expert audience. To write in a vernacular which is adapted to the interests of this audience is only a problem if it’s the only way in which academics can write. Billig (2009: 9) suggests that public intellectuals can be thought of as bilingual in that “they use one language for addressing the public and another for addressing fellow specialists”. As long as there are at least some academics within each field who are willing and able to switch between these languages, I’m not convinced the problem with specialised language is that it excludes non-specialists.
The problem instead is that technical language often substitutes for clarity about what is being expressed in this technical jargon. When these terms are used in specific and precise ways they serve a purpose by simplifying communication. It’s easier for me to use the adjective ‘computational methods’ than it is to list a series of techniques for working with research data that have been made possible by technological advances in recent years. There might be uncertainty about how widely I’m drawing the category, but it can be safely assumed that everyone who recognises the phrase will understand the kinds of method I am talking about with this phrase. There will be people who are specialists who don’t understand the phrase. In which case I might have to list examples of the methods I’m describing, such as text mining, explaining how this relates to established forms of qualitative and quantitative work with texts that they will be familiar with. If they’re not a specialist then it’s likely another approach would be necessary, in which I pitch my explanation based on an assessment of their prior knowledge. The phrase would be a problem with this latter audience but the issue at stake is how to address non-specialist audiences in clear and effective ways.
This illustrates how, as Molinari (2023: 13) puts it, “what makes writing academic – its academicness – emerges from a complex stratified ontology of structures and agencies”. There’s not a defining feature of the text, a specific grammar and style, which gives it this academic quality. Nor is it simply a reflection of the fact it’s being written by people who occupy the social role of the academic. Academics write non-academic things continually. It seems implausible to say that our e-mails are a form of academic writing yet we write thousands of words of this form each week. I wrote a short story a few months ago. Is this academic writing? There are certainly academic themes to the writing, in that it was set in a university and explored themes of political crisis and civilisational crisis. But it didn’t feel like academic writing to me, nor do I suspect that anyone who reads my academic work would regard it as such. It’s not simply the fact we get published in a particular location either, because the unpublished manuscripts sitting in the (by now figurative) filing cabinet are academic writing even if they’ve not been made public. It could be argued that these are at least intended to be published, which would imbue journals with the capacity to render writing academic if the author seeks to publish it there. But as someone who has been blogging for over twenty years, I find this an obviously limited way to account for the meaning of the adjective. The six thousand posts on my blog are mostly academic but there are vast quantities of material which have nothing to do with my academic life either. I would struggle to offer a general account of the difference between these categories, even if it would be easy to make the distinction on a case-by-case basis. It’s not straightforward what makes writing academic even if we might have been taught in a way that suggests otherwise.
It follows from this that academic writing will always be subject to change and development. This doesn’t mean that there will be continual change but rather that its nature is not fixed, even if it sometimes feels like it is. The role of power in enforcing these judgements, whether the supervisor presenting their stylistic preferences as received wisdom or the journal reviewer demanding adherence to their instructions, compounds this sense of fixity. They are at least in those interactions fixed elements of the landscape which the academic writer is pressured to adapt to. While you don’t have to accept their judgements as the gospel truth, rejecting them incurs costs which you must negotiate in the development of your academic career. This gives them a force which can often be an object of resentment, leaving academic writing positioned precariously as both an expression of intellectual creativity and something we are forced to do in ways that count as a condition of employment. It can be hard to be reflective about academic writing when it so frequently feels so ambivalent to us. But unless we are reflective about it, we are likely to continue forward in the same way we have until something forces us to change.
This isn’t uniquely a feature of machine writing. The encounter of academics with digital media, first through blogging and then through social platforms, provoked similar anxieties. The change it required led some to worry that digital engagement would unavoidably lead to the abandonment of scholarly virtues. Now we’re experiencing similar concerns about AI’s impact on academic writing. But perhaps we’re focusing too much on the technology itself and not enough on the cultural and economic context in which we’re using it.
Instead of treating LLMs as an actor in their own right, we need to examine “the cultural and economic values that currently shape AI’s affordances”. As Vallor (2024: loc 2077) points out, “we ourselves are measured in terms of our ability to resemble our own mechanical mirrors”:
“Whether it’s the pressure to produce your next album, or publish enough to get tenure, or film enough videos to see your subscribers – our dominant values favour those who don’t get writer’s block, who don’t struggle to find the right words, or images, or notes, or movements, who never get caught up in the swirling drag of inexpressible meanings. Our economic order has long rewarded creators who work like machines. Should we really be surprised that we finally just cut out the middleman and built creative machines?”
If academics are inclined to draw on machine writing in order to maximise their productivity, we need to ask why this is. What is it about the training they undergo and the incentives they confront which leads them to seek to produce as much as possible, as quickly as possible? The affordances of LLMs offer an opportunity to radically expand the quantity of what is being produced but what is it which lead some academics to want to take advantage of that opportunity? In this sense we need to treat AI as a mirror, to use Vallor’s (2024) metaphor, which is held up to our existing social arrangements. If machine writing is bringing out the worst in academic writers we need to examine what it is about academic writers which leads them to engage with machine writing in this way. If we simply treat machine writing as if it is an enemy at the gate, a foreign intruder which needs to be repelled lest an otherwise well-functioning system be damaged, we lose the opportunity to grapple with the real changes taking place in academia.
Perhaps the most valuable question isn’t “How will Generative AI change academic writing?” but rather “What does our response to Generative AI reveal about the state of academic writing today?” The anxieties, temptations, and opportunities presented by these technologies hold up a mirror to the pressures and contradictions already embedded in academic practice. These are pressures that existed long before ChatGPT or Claude appeared on the scene.
#academicLabour #academicWriting #digitalScholarship #JuliaMolinari #michaelBillig #writing
The allure of LLMs as professional support at a time of crisis within higher education
Machine writing has arrived at a time of intensifying pressure within many higher education systems. Financial constraints lead to changes in the organisation of academic work, particularly with regard to the role played by teaching. Political polarisation drives a greatest contestation of academic authority, sometimes even harassment of academics. The shifting plate tectonics of knowledge, stemming from social and technological transformation, create the risk that recognised expertise will be rendered redundant. Universities are increasingly torn asunder between leaders who see themselves as equipping their institution to survive in a hostile climate and academics who see the ensuing disruption as an expression of that very hostility (Rosenberg 2023).
Within this challenging landscape, large language models have emerged not just as technical tools, but as psychological presences in academic life. It can be immensely difficult to work in these conditions. This is exactly why we need to give serious thought to how LLMs might feel to academics under these circumstances. These friendly assistants are constantly available, willing to consider any request and always encouraging. They are never irritable, distracted, passive aggressive or tired. They never prioritise someone else over us. They don’t impose expectations on us. They can make mistakes, confuse us or act in ways contrary to our intentions. But as we become more skilled at talking with them, these occasions come to feel like the exception rather than the rule. In the seething cauldrons of ambient stress and interpersonal antagonism which many universities have become, at least some of the time, these are evocative characteristics. If we see our working life as assailed on all sides by hostile forces, if we see our jobs as under impending or future risk, the omnipresent ally able and willing to support us through the working day is going to be extremely attractive.
The psychological comfort offered by these systems creates a complex relationship that goes beyond their technical capabilities. When human relationships in academia become strained by institutional pressures, the consistency and apparent care of AI systems can feel like a welcome respite.
AI literacy is an important feature of how academics engage with the opportunities and challenges presented by LLMs; it’s essential that users of these models have a broad understanding of how they operate, how they’re trained and the limitations entailed by this (Carrigan 2024: ch 3). However it’s possible to have a cognitive understanding of these issues while still relating to the models in complex and potentially problematic ways. For example I’ve determinedly insisted on using ‘it’ if I have to refer to LLMs using a pronoun in conversation. Yet I recently slipped a ‘he’ into the conversation when referring to Anthropic’s Claude despite the fact I was half way through my second academic monograph on the subject. I immediately corrected myself but it stuck with me because it illustrates how these associations and assumptions can linger on in the psyche, complicating the reflective views we hold on a particular subject.
I know Claude isn’t a ‘he’ and I often remind my students of the same thing when I see them falling into this habit. Is there nonetheless part of me which feels that Claude is a ‘he’? Which imagines Claude as a ‘he’? Which wants Claude to be a ‘he’? The point I’m making is not one about my own psychology but rather illustrating how there’s more to our reaction to LLMs than can be adequately captured in the intellectual views and opinions we offer about them. You can’t ensure academics have an accurate and effective sense of what models are how to engage with them simply through providing routes to knowledge about LLMs, important though such knowledge undoubtedly is. I would suggest that we must go deeper and that writing is a fascinating frame through which to explore these issues.
#academicLabour #academicWork #claude #higherEducation #LLMs #support
We urgently need to talk about the temptations of LLMs for academics
If we want to understand how academics use large language models (LLMs) we need to begin with the reality of the conditions most of us are working within.
This is a temptation I’ve experienced in my own work. I felt it strongly for the first time when struggling to complete a co-authored piece for an impending deadline. There was an element in the article I believe it was important to include but my co-author felt much less strongly about.
I wasn’t happy with the contents of the article, as it was missing discussion of a topic which I felt was hugely important, yet I was tired and distracted in exactly the way that makes writing difficult. I knew what I wanted to include but not how to include it. The words were not flowing, the deadline was approaching and I didn’t want to let my co-author down. I knew there was material on my blog which I couldn’t directly reproduce but which could easily provide inspiration for Claude to write passages which matched my writing style. It was the first time I had seriously contemplated relying on machine writing to complete a formal publication. I could not see a satisfactory way of resolving my dilemma: I didn’t want to exclude this topic from the article, I didn’t want to let my co-author down but I was also too sleep-deprived to write the required text that afternoon.
It strikes me in retrospect that I wouldn’t have contemplated including machine writing if I hadn’t been confident that Claude could match my style. In previous months I had experimented with giving it samples of my writing, asking it to characterise the style in bullet point lists, then using these descriptions in order to refine a prompt to match how I write. I wasn’t certain but it felt like Claude could match my writing in a way which others would likely find utterly plausible. This was initially an exploration of how subterranean machine writing could become but in that moment of temptation I saw the consequences of this capacity for the first time. I wasn’t comfortable including machine writing that was declared to the reader, either explicitly in the text or tacitly by simply including passages written in a jarringly different style. Even if the publisher had been ok with this, which I hadn’t gone as far as to investigate, it would have felt like an abdication of my authoriality. I’m sure it would have made my co-author deeply uncomfortable as well. But if the machine writing wouldn’t be identifiable to anyone other than me? That was a different prospect which offered a way out of my dilemma. I could fill in the text with a short section, satisfying my intellectual requirement to cover the topic while also meeting the deadline for the article.
What made it even more tempting was this machine writing would have been expressing my own ideas. There was no sense of asking Claude to provide the ideas. I simply had the ideas in one form (notes on my blog) which I needed to translate into another form (a section of an article) but which I was not in the moment capable of acting on. In the end I couldn’t do it. I have rarely had such a vivid sense of the devil and angel on respective shoulders in a professional setting. I could see a practical case for acceding to the temptation, in that it would produce a better piece under the circumstances which I found myself in. But if I did then I felt I would inevitably do it in similar situations in the future. Even with the best planning, a standard which none of us can consistently meet, there will always be circumstances where we have writing responsibilities which outstrip our present capacities. If we develop a comfort with leaning on machine writing in those situations, I suspect the category will expand and we will gradually find ourselves relying on it in situations which would once have felt simply challenging rather than impossible. It’s a retreat from the trouble of writing, one which is particularly tempting when that trouble feels insurmountable, but which has the capacity to subtly unpick the moral psychology through which writing comes to be meaningful and satisfying to us.
What’s at stake here isn’t just a question of research ethics or academic integrity in the formal sense. There’s something more fundamental about our relationship to the creative process itself. The constraints we face as writers (whether time, energy or our own cognitive limitations) create the conditions in which genuine intellectual work happens. Without that productive friction, something essential to scholarly identity may be lost.
The use of machine writing in knowledge production is still in its infancy and, even with detailed empirical investigation, there is a limit to how far we could answer these questions in relation to an issue which is developing so rapidly. In raising them I’m trying to highlight the questions, rather than take a stance as to the answers. The assumption that human authoriality underpins what we write in monographs, edited books and journals is so axiomatic that it is difficult at this stage to think through what knowledge production looks like when it can no longer be assumed. Explorations of the potential implications often oscillate between feeling mundane, preoccupied by minutiae around the edges of practice which will otherwise feel unchanged, and feeling grandiose, making sweeping generalisations which tend to overstate the issues involved.
This is exactly why empirical investigation will be so crucial to stabilising our understanding of how academics are using machine writing, as well as what this use means for knowledge production. But what I’m trying to do is, rather than even offer a comprehensive review of the fragmented and pre-print heavy literature in its current stage, open up the conceptual issues involved with a view to supporting academics in reflecting on their writing practice in relation to the rapidly developing possibilities which machine writing offers in their mundane working life.
My suggestion is that difficulty is at the heart of how academics will tend to relate to the possibilities of machine writing. Conversational agents provide us with new ways of negotiating difficulties in the writing process. They can offer new perspectives on what we have written, help us elaborate upon what we are trying to say and provide detailed feedback of a form which would have previously required a human editor. The attempt to eliminate difficulty from the writing process will have downstream consequences for our own writing practice, as well as the broader systems through which (we hope) our writing makes an intellectual contribution.
The reason I’m focusing on the experience of joy in academic writing is not simply that this makes it less likely we will hear the siren song of machine writing in the first place. I will argue that if we rely on machine writing when confronted with difficulties, those experiences of joy are likely to become more elusive and perhaps even disappear altogether from our writing lives. It is only through staying with these difficulties, even when it’s uncomfortable and dispiriting, that we can make it through to the other side.
This isn’t to suggest we must reject these tools entirely. Rather, we might consider approaching them with the same deliberate intentionality that characterizes thoughtful writing itself. Perhaps the question isn’t whether to use AI writing assistance, but when, how, and with what awareness of what we might be surrendering in the process. The most dangerous temptation may not be using these tools, but using them unconsciously, without reflecting on how they reshape not just what we produce, but who we become as scholars through the process of producing it.
My concern is that the critical discourse, while accurate in many respects, fails to create the space for these conversations about practical reasoning by academics.
#academicLabour #academicWork #acceleratedAcademy #claude #higherEducation #writing
#Workshop: Academic Labour from the Peripheries of the Knowledge Economy
3-4 April 2025, #Groningen
This workshop will explore the #work done by #marginalized groups in #academia, with a focus on global North-South, or center-#periphery relations. We want to understand the challenges and additional workloads faced by scholars who are disadvantaged in this regard, and to explore ways to conceptualize and address these issues.
From Peter Mandelson’s intervention today:
In return, Mandelson said universities would need to make “more tough choices” to improve efficiency, noting that Italian state universities had one teaching staff for every 21 students while UK universities had one for every 13.
I’m increasingly convinced GenAI will be proposed as a way to reduce staffing. The automation agenda I wrote about in the final chapter of Generative AI for Academics will imminently hit UK higher education and we are not prepared for it.
#academicLabour #automation #finances #higherEducation #PeterMandelson
Academic associations: when you are looking for graduate students to do work for a conference, perhaps don't offer an honorarium equivalent to less than minimum wage in 12 out of 13 provinces and territories.
That's not even counting the unknown (uncompensated?) time for the mandatory orientation session.
#PhDchat #GraduateStudent #Universities #Academia #AcademicLabour #AcademicChatter #MinimumWage #LivingWage #GraduateSchool #AcademicConferences #CSSE2024 #CHSS2024 #CHSS #CSSE
Steve
Mark Carrigan
UG Faculty of Philosophy
Bridget Stirling
Alison Hearn