In my recent post, I listed four broad topics which I think our profession should urgently try to address. Here I want to say something about journals.
Whenever I have tried to write one of these posts, I always get stuck on the issue of incentives. There are many things I would like to see, but there is not much point saying what I want to happen without providing any mechanism for us to get there. There are two broad categories of incentives: professional, namely what criteria we use to hire people, give people tenure, give people prizes, etc.; and personal, the inherent drivers that motivate us. The former we have some control over, the latter is a little more slippery.
A lot has been written about the beauty of understanding in the last few months. However, even a cursory examination of the arXiv shows where the money is. There has been a mad and honestly undignified rush to be “the first” to prove one result or another. There is a very wide spectrum of people using AI, and I am definitely not trying to imply that everyone is using it badly. I have seen some excellent uses of AI to prove interesting things, and I’m excited by the math that is to come!
But who could blame people for rushing? I have the luxury of taking this time to use AI to explore interesting new math without needing every experiment to turn into a publication. I also have a good tenured job. For younger people, more papers in better journals could mean a better job and a better life. We cannot continue to reward the production of papers in exactly the same way and then be surprised when people organize their work around producing more papers.
This brings me to the Leiden declaration, which calls for transparent disclosure of the use of automated tools. It is already creaking at the seams. Its spirit is not being observed: we are seeing everything from vague admissions of “assistance” to complete denial. The point of a disclosure should be to tell the reader exactly what happened, not to leave them guessing how much is concealed inside weasel words like “assistance.” I’m not saying that every use of AI should come with the full prompts, but it would be nice to see them now and then!
I am really uncomfortable with a default middle ground where AI is being used in an ambiguous way. That is one of the reasons I found the disclosure discussed in this post so refreshing. It told us plainly where the proofs came from. We should be encouraging that candour, not making people feel that they have to minimize the machine’s contribution to make the paper professionally acceptable. Of course, papers that are genuine collaborations between man and machine exist, and naturally these cases are more subtle, which only makes the point we should have more practice being clear.
One of the few levers we have to affect how mathematicians act is through hiring and through journals. And what are journals doing? As far as I can see, far too little. This cannot be business as usual. Editorial boards have to decide now what kinds of papers they want to publish, and what those publications are supposed to contribute.
I do not think this is simply a question of allowing or forbidding AI. We should be willing to reconsider what constitutes an interesting mathematical contribution. We should also be willing to consider new forms of papers, whether using AI or not. Being more demanding about what is worth publishing is perfectly consistent with being less constrained by traditional ideas about who must have produced it.
For example, let’s consider this new paper. For context, as part of my thesis, I proved that there did not exist any non-zero semistable abelian variety \(A/\mathbf{Z}[1/N]\) when \(N=6\) or \(N=10\) assuming GRH. This paper removes the dependence on GRH.
To make a point, I have listed the authors as Astra and Fable, and instead the paper has a “Human acknowledgment” section. Suppose (for arguments sake) I thought this was interesting enough that it should be part of the mathematics literature in some way. Would putting my name at the top, and relegating the machines to an acknowledgment, make it a better paper? Would writing it under my name and saying that AI “contributed to some of the arguments” make it more or less transparent who did what?
There was a comment on my earlier post asking why Constantin (or someone else in this position) should list themself as an author at all if the mathematics was not due to them. Here is one institutional obstacle: the arXiv does not allow AI tools to be listed as authors. Not only would the paper above not be considered by any journal in its current form, but I can’t even post it on the arXiV.
Naturally, the reason to insist on papers having human Authors is to put responsibility onto humans for what they post. That is a real issue, but a brief glance at the arXiV should convince you that a human listed as an author is hardly a guarantee of anything. Why must responsibility and credit for discovering the mathematics be bundled together? Someone could be responsible for presenting a paper, explaining what has been checked, answering questions, and dealing with corrections without claiming to have discovered its arguments. The best AI disclosure is one which honestly says what the person presenting it actually did.
(The arXiv does, however, now insist on a full English version of every submission. At least we have got rid of the French; remember Agincourt!)
You may well ask what I am doing about this, given my editorial duties.A&NT (Algebra and Number Theory) is a journal with plenty of prestige, especially for a specialist journal, while also being relatively new and so perhaps less constrained by “tradition.” I strongly believe that it is a perfect journal to experiment with new forms of papers, so what better place than A&NT?
I emailed the board asking, as a thought experiment, whether each editor would accept a paper of the kind recently discussed here if it were submitted and passed the referee process. There were many who would. But the discussion made it clear that we had substantially different views about how the journal should respond to AI. I don’t think there is any point running a journal right now as if AI doesn’t exist, and if it was just going to be business as usual, there isn’t much sense for me to remain on the board.
So I resigned.
The point is not that journals should publish more AI papers, or fewer AI papers. It is that they should make some decisions about what is worth publishing, and be willing to experiment with how it is presented. Papers that have a nice idea which didn’t work. Papers with speculative ideas that the author has no idea how to prove. Papers with good expositions, papers which only have conjectures, the list goes on. Unfortunately, right now, the claim that there is “so much more to mathematics than theorems” is not exactly what one might conclude by looking at any recent journals. What we absolutely should not be doing is encouraging everyone to rush out conventional-looking papers while refusing to consider experiments that make it harder to pretend that nothing has changed. If mathematical understanding is what we want to defend, then it ought also to be what we reward.
And yet, I am not pessimistic. The pace of change is so fast that what is obvious to me now (and has been obvious to others for far longer) may simply be obvious to everyone in a years time.
Added: Link to Hexagon Math.
Younger people have to be careful when they do this. When you start producing “top-level” results written up as pure AI slop, what do you think I’m going to say when I’m asked to write a letter for your promotion a couple of years from now? Why ruin your career with results you did not bother to digest properly? When I read the paper, I can immediately tell that the proof is 100% AI-generated and has not been understood by the “authors.” The proof ideas do not address the known obstacles. The proof sketch makes no sense. The author doesn’t even claim responsibility for the proof. There are many other tells. Even if the proof is correct, do people really think everyone is going to give them credit for pushing a button, especially when the AI disclosure is obviously dishonest?
I’m less concerned with AI slop than with hybrid papers of unclear provenance; where real mathematicians do write the proofs themselves, but where the AI did the real intellectual heavy lifting and the authors either obscure that or don’t even realize it.
Yes, I’ve seen those cases too. There was an arXiv paper without an AI acknowledgement, and I absolutely thought the authors had solved it themselves. Then the paper was submitted to a journal and came to me for refereeing, and it said AI had proved this one lemma. That lemma was the main new idea.
Have you seen hexagonmath.org? It was created for papers like the one you discuss here. Please consider uploading it there!
There are four categories of paper (ignoring correctness for now)
1. AI slop with a human author
2. AI slop with a computer author
3. clearly written papers with a human author
4. clearly written papers with an AI author
My impression of hexagonmath was that it was for 1 (or maybe 1 and 2), and I was aiming for 4. Am I wrong?
It’s for all of the above. When you submit you indicate which of these categories it falls under. To be precise, you select whether the paper is primarily AI-written, primarily human-written, or a mix; you select whether the paper has been fully understood by a person before submission, not at all understood, or a mix; and you tag the submission with the AI tools you’ve used. You also list the people who are contributing the paper to Hexagon (in this case, that would be you), and this does not include the AI tools.
It is possible to search/filter on all of the above criteria, so if someone only ever wants to see the category 3-4 papers, you can set it that way. Certainly we would love to have submissions in category 4 and to encourage people writing papers in that category.
We definitely need to think about what authorship means, but I’m strongly opposed to listing current AI models as authors. Maybe it could make sense in the future for fully autonomous AI systems with persistent mental states over indefinite time periods, but not for AI as tools. Basically, I think it’s appropriate only if the AI is a person, and current tools are pretty clearly not people.
As long as the paper clearly states where the results came from, there’s no real problem with having only human authors. (I agree that many AI-assisted papers aren’t clear, which is dishonest.) As far as I can tell, the primary objection is that it may mislead people who judge others by counting papers without looking at them, but this was already silly before AI, and by now it’s so ridiculous that I can’t believe anyone’s going to do it.
“I have listed the authors as Astra and Fable” This is really bad on so many levels, it’s hard to describe.
“It is that they should make some decisions about what is worth publishing, and be willing to experiment with how it is presented. Papers that have a nice idea which didn’t work. Papers with speculative ideas that the author has no idea how to prove. Papers with good expositions, papers which only have conjectures, the list goes on.”
But here I agree completely!
And it’s not only about papers or formalizations btw, I think we should also encourage to produce other kinds of interactive artifacts and websites of various forms (e. g. https://www.erdosproblems.com/ or https://elliptic-rank.icarm.cloud/ or someone just shared this example https://p.migdal.pl/equations-explained-colorfully/#navier-stokes among others).
> “I have listed the authors as Astra and Fable” This is really bad on so many levels, it’s hard to describe.
Apparently this is a joke which went over your head. To spell it out, the crap being thrown out on the arXiV with “human” authors and dishonest disclosures makes a mockery of what authorship means.
Yeah, no, I understood that it’s a joke, but decided to comment on it, because in some sense it has been already implemented by OpenAI in all of their papers released, and it’s very painful.
What would you prefer? That openAI attribute each result to a randomly chosen mathematician?
Well, before today it was around 13 papers; now it’s much worse, true
Now that Algebra & Number Theory is not an option, are any of you editors at serious journals that would consider the Odlyzko-Poonen paper for publication?
I am open to experiementing with the paper I posted. And I would find it fun if it were among the first papers with ideas due to an AI that are accepted by a serious journal. I think it is unreasonable to send it to a very top journal as this is unfair to people who submitted a paper say last year when the top journals were so relevant for careers. But a serious subject journal like Algebra & Number Theory or a serious general journal could make sense. I’m not sure I will submit it though — maybe a traditional journal isn’t the right home for a paper that arose out of a new way of working.
The journal system as a whole has been bursting at the seams for a long time. Formerly it served a direct purpose, actually disseminating new ideas to people in the field; now that function has long-since been supplanted by arxiv and preprint circulation, so the principal function has been to assess the value of mathematical contributions and thereby, indirectly, manage career advancement. Obviously there is also refereeing: but if refereeing were only about catching mistakes and offering suggestions, and not making a recommendation on whether the paper should be accepted, it would be straightforward enough to implement through other institutional structures, even before AI.
The discipline is currently having a come-to-Jesus moment in general about what its deep priorities are or should be, and the extent to which measurable criteria diverge from these. “Proofs and Progress” is being cited a lot, rightly. But I wonder whether the journal-system-as-prestige-metric can or should survive a genuine pivot toward actually valuing “the advancement of human understanding” and not just theorem-credits. Should we focus the effort on trying to reshape journals so that a “prestigious journal publication” is still a reasonable unit to measure someone’s career in (even if the publication is expository, or about a failed attempt, or one of the other types listed in this post)? Or should the effort be put into shifting hiring committee criteria so that journal publications receive much less weight, full stop?
I’m genuinely not sure! Some of this is much more sociological/office-political, and might come down to tactical decisions about what the best routes are toward change (and not just the endpoints), what bodies have more institutional inertia, etc. I think there have been many blog posts over the past month or two with vague pronouncements about how “incentives should change”, but maybe less concrete action being taken to do so. Of course, in the background, the specter hanging over all of this is NSF and other funding cuts and the fiscal future of academia as a whole.
“Some of this is much more sociological/office-political” — I agree, but I also think that these type of questions are more important than mere philosophical hand-wringing.