How much AI-generated text is found in philosophy journals today? (guest article)

“It appears that philosophers have begun to make significant use of AI in their articles, and it is not uncommon for AI-generated texts to be incorporated into philosophy published in decent journals.”

This is not uncommon. How much are we talking about here?

This is the question recently asked by Maximillian Noichl (Utrecht).

His response? It is estimated that “12% of philosophy published in 2026 (involves) some AI assistance,” although he notes that this figure will vary depending on the AI ​​detection software used and how the false positive threshold is set.

Writing about AI in philosophy articles is therefore not yet normal. But, he notes, this figure is increasing.

In the following post, Noichl explains how he investigated the problem.

(A version of this article first appeared on Noichl’s site.)


How much AI-generated text is found in philosophy journals today?
by Max Noichl

Reading philosophy in recent months, I’ve often had the nagging feeling that what I’m reading has a distinct whiff of AI. So I wanted to study the prevalence of AI use in philosophy in a data-driven way. This article presents the results of this investigation. Long story short, it appears that philosophers have begun to make substantial use of AI in their articles, and it is not uncommon for AI-generated texts to be incorporated into philosophy published in decent journals. It also appears that philosophy preprint servers are becoming increasingly clogged with AI-generated content.

Here’s what the results look like, at a glance:

Use of AI in a sample of published philosophy articles (n = 11,957). Click on the figure to see it in full size.

Methods and data

AI detection is funny in that about three years ago it was extremely bad and unreliable. Many people thought that it might even be impossible to detect AI-generated content. This has changed over the last year, with more reliable AI detectors like Pangram entering the market (Emi & Spero, 2024). Now, the best AI detection systems seem quite reliable (Jabarian & Imas, 2025), at least under decent conditions (enough text, English language, etc.). This means that for some time now, many people have seriously underestimated how detectable AI writing is in practice. This does not mean that the problem is entirely resolved. On the one hand, it remains difficult to distinguish between AI-based editing, translation and full generation, especially for very short texts, and there is a risk that new large language models could, at least temporarily, preempt detectors. But at present, AI detection with reasonable confidence is possible.

For this project, I will use an open source AI detector called MELD, published by Li et al. (2026). MELD is based on a transformer model, Ettin-1B, from the ModernBERT family.

Transformer models work (very) crudely in such a way that text is divided into small subword units called tokens. These tokens get individual digital representations called embeddings, and as text is pushed through the model, in a process called attentionindividual token embeddings change depending on the tokens around them, encoding contextual semantics, grammar, etc. In LLMs, these embeddings are typically initialized randomly at the beginning of the training process and then iteratively updated, so that embeddings that drop out at the end of the model become more useful for specific tasks, e.g., predicting the next token for text generation in GPTs or predicting a token from its environment (in BERT-style models). For an interactive walkthrough of these steps, I recommend Transformer Explainer.

In the case of MELD, a model pre-trained on general text tasks is trained further. To make it work for the AI ​​detection task, an additional small neural network, which projects the large token embeddings that come out of it into a much lower 48-dimensional space, is added on top, along with two small sets of anchor vectors, one that contains vectors intended for different families of AI models and one for humans. The model is then trained further, now on a mix of texts that we know whether they are AI or human generated. As the model is trained, each token of each incoming text is compared to all anchors, and its similarity to the most AI- and most human-like prototype is noted. These scores are used for general prediction about the text, and based on how correct its predictions are, the prototypes used in each training step, as well as the entire network leading to it, are updated (Li et al., 2026). MELD can, in principle, try to distinguish between different types of AI use (raw generation, polishing, etc.), but for our current investigation we are looking for any content that appears to have come from or passed through an AI model in any shape.

In my experience so far, MELD works pretty well. I suspect it’s not as good as Pangram, especially when it comes to the newer models. It does, however, have the big advantage of being open source, which allows us to check how it works under the hood, what it has and hasn’t been trained on, and how it performs on our specific datasets. This also means that we can run it ourselves if we acquire a powerful enough GPU, which costs a little money, but is much cheaper than running a large number of philosophy articles through Pangram, which would cost around $5 per article. This also means that potentially copyrighted material may remain on machines we control. I haven’t been able to access MELD’s canonical weights, so I’m using this mirror.

MELD uses a threshold approach. When using the frozen weights after training is complete, each token gets its own AI score, and we can decide for ourselves at what threshold we want to assume that the text is actually generated by the AI. The MELD authors recommend setting a threshold for material that we know is not AI-generated, which we do for each individual dataset in this project based on the 2021 and 2022 papers, as ChatGPT in its first version was introduced in November 2022. For the results presented, we set the threshold to accept a 1% false positive rate in the pre-AI years, which is the strictest thresholds suggested by the MELD authors for the results reported in their paper. On their own evaluation dataset, with a false positive rate of 1%, they successfully detect 99.7% of AI texts.

I should note that this is significantly less stringent than the overall false positive rate reported by Pangram, which is one in 10,000 (Emi, 2025). As I’m not trying to judge the work of individual authors for potentially game-changing decisions, but to get a general idea of ​​how things are in philosophy, I’m just as worried about the model missing more subtle cues of AI writing (false negatives), as I am about false positive judgments.

For the data, we consider four publicly available sources of philosophical articles, namely journals Imprint of philosophers And SOwhich are, in my opinion, very respected places which have the big advantage of being completely and freely available on the Internet. I have carefully scraped these two publications, as well as all preprints from the PhilSci-Archive. I also downloaded all preprints available on PhilArchive from PhilPapers via the CORE collection. After limiting ourselves to articles from 2021 containing at least 1,500 words of text (and, for PhilPapers, English-language articles of no more than 30,000 words), this leaves the full texts of 197 articles from Imprint of philosophers374 of SO4723 from PhilSci-Archive and 29954 from PhilPapers. For the latest year 2026, CORE had about a third of the records available, compared to about 78% available for previous years, so we have a still somewhat incomplete dataset to work with at the moment. I noticed that many preprints were backdated (published in 2025/26 but claimed to be from 21/22) and therefore could not reliably be considered AI-free. So I removed them from the dataset and used only the articles actually downloaded in those early years for threshold calibration.

For PhilSci-Archive and PhilArchive, I linked the records to OpenAlex, which allows us to check the publication status of each article. For this, I consider all articles that appear in a journal whose ISSN appears on the PhilPapers journal list as published in philosophy. Even if these articles ended up being published in one form or another, we must keep in mind that their filed version is not always entirely identical to the final published version. Since different genre compositions and text sources will be treated differently by MELD, I calculated separate thresholds for (1) the two open access journals, (2,3) published and preprint-only works in PhilSci-Archive, and (4,5) published and preprint-only works in PhilPapers.

All of these articles were converted from their various HTML/PDF formats to a uniform text format using BeautifulSoup (for journal HTML pages) and PyMuPDF (for PhilSci-Archive PDFs). For PhilPapers, I use the text that CORE had already extracted. The articles were then analyzed with MELD using GPUs rented from Google Colab. MELD read the texts in windows of 2,046 tokens (approximately 1,500 words) at a time and scored each token; these symbolic scores were then summarized into passages of 256 tokens (approximately 190 words). Following the procedure outlined in the MELD article, MELD scores were then calculated by taking similarity to human prototypes as opposed to AI prototypes, and taking the average of the highest 25% scores as the AI ​​score for the entire passage.

I generally publish all code and data for the analyzes I do, and I think it is critically important to do so for published work. In this case, however, I will not share the data and code (not even privately), at least for the immediate future. The use of AI is a very sensitive subject and accusing the perpetrators of it could have career implications. THE…

Gn Health

Logo