this post was submitted on 03 Jul 2025
48 points (100.0% liked)

Technology

72338 readers
2929 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
L3s@hackingne.ws
L4s@hackingne.ws
48
An analysis of 15M+ biomedical abstracts from 2010 to 2024 finds researchers using AI to write abstracts use certain words far more often than those who don't (www.science.org)
submitted 1 day ago* (last edited 1 day ago) by Pro@programming.dev to c/technology@lemmy.world
4 comments fedilink hide all child comments
 

Large language models (LLMs) like ChatGPT can generate and revise text with human-level performance. These models come with clear limitations, can produce inaccurate information, and reinforce existing biases. Yet, many scientists use them for their scholarly writing. But how widespread is such LLM usage in the academic literature? To answer this question for the field of biomedical research, we present an unbiased, large-scale approach: We study vocabulary changes in more than 15 million biomedical abstracts from 2010 to 2024 indexed by PubMed and show how the appearance of LLMs led to an abrupt increase in the frequency of certain style words. This excess word analysis suggests that at least 13.5% of 2024 abstracts were processed with LLMs. This lower bound differed across disciplines, countries, and journals, reaching 40% for some subcorpora. We show that LLMs have had an unprecedented impact on scientific writing in biomedical research, surpassing the effect of major world events such as the COVID pandemic.

top 4 comments
sorted by: hot top controversial new old
[–] renzhexiangjiao@piefed.blahaj.zone 11 points 23 hours ago

tbh I don't see anything wrong with using AI just to write the abstract, assuming the author redacts it afterwards. It becomes much more problematic if AI is used in the middle section of the paper, where it is crucial to present information as accurately as possible.

[–] trailee@sh.itjust.works 6 points 19 hours ago

Very interesting paper, and grade A irony to begin the title with “delving” while finding that “delve” is one of the top excess words/markers of LLM writing.

Moreover, the authors highlight a few excerpts that “illustrate the LLM-style flowery language” including

By meticulously delving into the intricate web connecting […] and […], this comprehensive chapter takes a deep dive into their involvement as significant risk factors for […].

…and then they clearly intentionally conclude the discussion section thus

We hope that future work will meticulously delve into tracking LLM usage more accurately and assess which policy changes are crucial to tackle the intricate challenges posed by the rise of LLMs in scientific publishing.

Great work.

[–] Plebcouncilman@sh.itjust.works 3 points 21 hours ago* (last edited 21 hours ago)

Analysis of over 15M+ bodies of water finds that water is wet.

[–] AmidFuror@fedia.io 0 points 21 hours ago

Did AI write the headline? Article abstract is about detecting how much AI is in use by looking for an uptick in AI-favored words. Headline is about how scientists found out that AI prefers certain words.

If it were actually about the headline topic, the paper would be fatally flawed.