Nora Hoffmann, Head of Goethe University Frankfurt’s Writing Center, on the nationwide student survey “Shaping the Future of Academic Writing with AI”
UniReport: Dr. Hoffmann, the results of the student survey are now available. They show that ChatGPT & Co. have definitely found their way into the everyday lives of students, with 66 percent of respondents saying they have used generative AI writing tools. Does this result surprise you?
Nora Hoffmann: We were indeed uncertain about the extent to which students already use generative AI. After all, most of them do not necessarily communicate this usage openly, especially towards lecturers, for fear of negative consequences. In the protected setting of writing consultations or workshops at our Writing Center, on the other hand, it was already becoming clear prior to the survey that AI had found its way into students’ writing. In that sense, the result was not entirely unexpected, although I have to admit that I was astonished at the very large number of student users and the depth of their research, especially when considering that AI had previously not seemed to factor on the same scale across all student groups. The survey results show that students in all subjects and semesters are already using AI for numerous elements of academic writing, 25% of them at least once a week.
Survey participants were asked to rate their writing skills. Were you able to identify any correlations between the respective answers and the use of AI tools?
We were certainly able to observe minor correlations across many of the questions: Students who rated their writing skills higher used AI slightly less frequently and, more than that, they used it less to generate whole texts. We also found that they used it for a narrower range of tasks. In the free text answers, students often stated that their own individual style is important to them in their writing, or that they learn and develop ideas while writing and therefore do not want to use AI to generate texts.
Do the survey results offer room for statements on how the students themselves rate the quality of AI-generated texts? Where do they see potential deficits? Are there situations when they would dispense with using AI?
Our data show that students consider the output to be of a high linguistic quality, which is also why many use AI for textual and linguistic improvements. As far as the quality of the content is concerned, around half the students are aware that AI produces false outputs and invents sources. For them, the biggest disadvantages are a lack of transparency and inadequate precision. However, this awareness stands in stark contrast to the fact that AI is most commonly used to summarize literature. The poll’s free text answers offer a possible explanation for this contradiction: Some students said they only use these summaries as an initial starting point and for orientation and then read the texts themselves. Put differently: They do not rely exclusively on AI.
What can you say about students who have not used AI yet?
Given that but a few of the poll’s respondents said they were unfamiliar with or unable to handle the tools, most students who have not yet used AI seem to have made that decision consciously. The reasons vary: Many see no benefit in using AI, for instance, because they believe in the added value of the critical thinking it takes to independently formulate texts. Some also have reservations about using the tools, especially since most universities (including Goethe University Frankfurt) did not yet have clear rules on AI usage in place at the time the survey was conducted. Others were unsure about how to incorporate the tools into the writing process. If universities provide clear rules and information that remove the obstacles encountered so far, students’ willingness to use AI is likely to increase further in the future.
Where, do you think, is further research needed and what, in particular, should writing researchers analyze (or have analyzed) in more detail?
Given that the use of AI will presumably be part of both professional and academic writing in the future (or already is now), for the writing pedagogy community, teaching a reflective and expedient use of AI tools in the writing process is a (new) part of their task of imparting writing skills. However, it is not yet possible to predict exactly how AI will impact text production processes and products, writing skills and reading habits, and, accordingly, which kind of support will be needed and what impact it will have. That is something I believe requires more research. As a first step, our survey documented students’ attitudes towards and use of AI before clear rules, information, or wide-scale pedagogical support were available. The results make the status quo and support requirements more tangible, although many questions also remain unanswered, including: Does the use of AI as a tool to develop ideas inhibit our own creativity or, on the contrary, stimulate it? Does the use of AI and the incorporation of the turns of phrase it generates into our own repertoire upgrade our language development, or do we lose our own abilities? Is it possible to critically evaluate and revise texts generated with AI without any prior personal writing experience? On the one hand, research should explore such questions. On the other, the next step in writing pedagogy is to (further) develop support measures that build on existing findings and analyze their impact. To identify further (possibly subject-specific) methods for using AI that we could teach to students, corresponding studies by researchers could also be enlightening.
As the Writing Center, what suggestions and conclusions have you derived from the survey results?
The Writing Center has developed a wide range of information and support services related to academic writing with AI tools for students and teaching staff (see https://tinygu.de/KI-Schreiben – in German). On the one hand, the results relieve some of the pressure on us to expand our AI activities, as they show that students mostly use it with at least a rudimentary critical awareness of its strengths, weaknesses and dangers. On the other hand, there is certainly room for more: An entirely critical handling of AI has not yet developed – and certainly not across the board. The survey also reveals minorities who either use AI to generate whole texts or categorically refuse to use it at all. At the same time, most respondents voiced a significant need for information and clear rules. In this respect, we at the Writing Center have concluded that our mission is to continue monitoring technical developments, to engage in the lively debate in our field, to offer students and teaching staff help with incorporating AI into academic writing, and to further develop and explore suitable pedagogical measures.
Questions: Dirk Frank
“Shaping the Future of Academic Writing with AI”
Objective:
The aim of the Germany-wide student survey was to document how and why students use generative AI tools for academic writing, their attitudes towards it, and to what extent their own writing skills affect this use. The results are intended to give universities points of reference for aligning teaching, studying, examination formats and AI support services with students’ actual use.
Data collection:
Writing pedagogy researchers distributed the online survey, which ran from July 13 to August 20, 2023, to universities across Germany. At that time, ChatGPT had been generally known for about six months, and many universities had not yet adopted a clear position on it, nor for that matter procedures and rules on how to handle it. As a result, students were able to try out AI in a legal grey area and develop individual ways of using it for their written assignments.
Sample:
After data cleansing, 3,997 valid questionnaires remained. 43.5 percent of respondents were students at Goethe University Frankfurt.
Results:
• Most students already use generative AI writing tools on a regular basis; some have not used them yet. Many students use generative AI writing tools in a reflected and responsible manner and revise AI texts; some, however, use AI to generate entire texts.
• Most students are aware that the content of AI texts is unreliable. However, they are less familiar with data protection, copyright and ethical aspects.
• Most students feel confident about using generative AI writing tools. However, some do not know how to incorporate them into the writing process in an expedient way.
• Students voice a significant need for clear rules and information on how to handle generative AI writing tools.
• Students who rate their own writing skills higher use generative AI writing tools less frequently and agree less with the reasons for using them.
Initial analyses and details of the survey (in German) are available here.
