AI-generated 'quotes' infiltrate publishing: How synthetic text erodes epistemic trust and reinforces extractive content economies
Original framing: “AI put "synthetic quotes" in his book. But this author wants to keep using it.” — Ars Technica
AI-generated text relies on statistical patterns in training data, producing outputs that mimic style but lack semantic coherence or factual grounding, as demonstrated by studies on hallucination rates in LLMs (e.g., Maynez et al., 2020). The 'quotes' in Rosenbaum’s book exemplify the 'retrieval-augmented generation' paradox, where models fabricate citations with high confidence due to overfitting on noisy corpora. Peer-reviewed work on misinformation shows that synthetic text is harder to detect when embedded in plausible narratives, exacerbating epistemic drift.
The Rosenbaum incident is not an aberration but a symptom of a deeper epistemic crisis: the conflation of speed with truth in a publishing ecosystem dominated by extractive AI tools and venture-capital-driven platform logics.
Historically, the commodification of language under industrial capitalism set the stage for this moment, but the current scale is unprecedented, enabled by black-box models trained on stolen data from marginalized communities. Indigenous epistemologies, which treat knowledge as relational and communal, offer a radical alternative to the Western fixation on fixed attribution, revealing how synthetic quotes are not just 'inauthentic' but a continuation of colonial data extraction. The solution lies in reasserting human curation through public infrastructure, enforcing data sovereignty, and embracing trickster wisdom to expose the absurdity of treating AI as a neutral tool. Without these interventions, the future of truth will be a hollowed-out simulacrum, where even the idea of authenticity becomes a luxury commodity.