A recent scroll through social media offers a stark reminder of the modern crisis of trust. For many users, about half of their daily feed is now saturated with what has come to be known as "AI slop"—machine-generated text, images, and content that astute users can easily spot, often to the embarrassment of those who unknowingly spread it. While some share these artifacts unwittingly, others do so deliberately, amused by the chaos or finding them useful tools for sowing general confusion across digital platforms.
This modern landscape of synthetic fabrication is a far cry from the early days of GPT-3, when users could ask an early chatbot a question and watch it instantly and with absolute confidence spit back a plausible yet entirely false answer. That specific phenomenon is when the word "hallucination," as applied to artificial intelligence outputs, first entered the modern lexicon. Six years and several iterations across a slate of different chatbots later, that underlying skepticism remains stubbornly relevant. Even when working with advanced models like Claude or Gemini to accomplish useful daily tasks, the core reality of current generative AI remains unchanged: Trust, but verify. And verify you must.
Apply that maxim at scale to maintain the integrity of the scientific record, and one begins to grasp the monumental challenge currently facing publishers of peer-reviewed scholarly research. In an insightful feature for IEEE Spectrum titled "I Used AI to Fake a Scientific Paper," freelance writer and doctor of applied mathematics Benjamin Skuse set out to explore this vulnerability. He investigated the human gatekeepers trying to beat back the rising tide of machine-generated deception by actively attempting to push his own entirely AI-generated papers past the stringent defenses of IEEE.
As features and AI beat editor Eliza Strickland noted in discussions about the project, Skuse was genuinely surprised at just how effortless it was to generate a research paper that looked, sounded, and felt like legitimate science. For one of his experiments, he essentially hit "run experiment" and "write paper," bypassing any traditional period of deep reflection, laboratory trial, or rigorous human oversight. As Strickland described it, Skuse "didn’t do a damn thing to it" before launching it directly into the maw of the professional publishing engine.
What Skuse ultimately discovered during his journalistic experiment is that the combination of human-review processes and automated research-integrity checker programs are currently holding their ground in what amounts to a relentless cat-and-mouse game. This conflict pits publishing platforms against individuals exploiting free, widely available tools to flood the system. However, as large language models continue to evolve, the newest generations of advanced LLMs are no longer making the same obvious, elementary mistakes that characterized earlier free versions. This leaves traditional academic publishers locked in an ongoing technological arms race against increasingly sophisticated, highly plausible slop, striving to outpace the detection systems designed to catch them.
Scholarly publishers are far from alone in their desperate efforts to hold the line against the tide of unverified digital noise. Another primary bastion of human-generated knowledge, Wikipedia, faces an identical existential crisis. In an interview for this month’s Five Questions series, Strickland spoke with Ellia Lebleu, a founding member of WikiProject AI Cleanup. This grassroots volunteer effort is dedicated to detecting, correcting, and deleting AI-generated material from the platform before it contaminates the historical encyclopedia. Lebleu’s team finds itself confronting the exact same systemic problem that commercial and academic publishers face, but they must do so with significantly fewer people, narrower institutional resources, and far fewer automated tools.
Wikipedia has long been universally recognized as a premier repository of human-curated information, and its community is fiercely committed to holding onto that identity, though nobody can say with certainty how long they will be able to sustain that defense against automated generation.
If society can no longer collectively assume that the scientific record is fundamentally trustworthy, or that basic facts about the world reported across media are accurate, our shared understanding of what constitutes reality unspools into deep uncertainty. Perhaps that environment delights quantum scientists who study probabilities, or the digital grifters who thrive precisely in an information ecosystem where a shared consensus of reality no longer exists. Yet, this widening circle of uncertainty ultimately undermines public trust in the core institutions that were built to curate the corpus of human knowledge and to promote the safe advancement of science and technology for the ultimate benefit of humanity.
Alternatively, looking further ahead into a potential future where artificial intelligence is both conducting the underlying science and writing the resulting research papers, it may ultimately matter very little whether human beings place their trust in the outputs. As Strickland observed while reflecting on the trajectory of these technologies, it feels surreal to contemplate a future where AI systems are writing content exclusively for other AI systems to read and analyze, leaving human beings entirely out of the loop.
