Xena on Counterexamples

There is an interesting post here on Xena’s blog.

I don’t centre “proof” in mathematics to anywhere near the extent that Kevin does, but I think emphasizing that distinction obscures the fact that our opinions about mathematics, mathematicians, and, I suspect, AI are quite similar.

It is fascinating to observe the progress of AI in mathematics (other people might use different adjectives). The result that a finite group scheme of order \(n\) need not be annihilated by \(n\) is the first AI-assisted result that I actually “care about.” I most closely associate this problem with a remark Hendrik Lenstra (the human equivalent of an LLM in the 90s) once made to me in the elevators of Evans Hall: namely, that there is a weaker result asserting that every finite flat group scheme \(G\) of order \(n\) is annihilated by \(n^{c(n)}\), for some integer \(c(n)\) independent of \(G\). The point is that there is, in an appropriate sense, a universal family of group schemes of order \(n\) over a finite-type base \(S\), where \(S\) is more or less the parameter space required to write down all the necessary Hopf-algebra data.

I absolutely agree with Kevin’s reaction to reading LLM-generated mathematics. Reading mathematics is already extremely hard; we are used to the fact that we might have to spend weeks understanding a few lines in a difficult paper. But what gives us the spirit to persevere is that we have been trained to give the author the benefit of the doubt: it is we who are missing the idea, and if only we think a little more about it, we will work it out. For better or worse, this is a cultural truth of mathematics. Once that is taken away (and it absolutely should be, for now, when reading LLM-generated mathematics) it becomes almost impossible to read mathematics. If there is one thing that LLMs have mastered, it is writing with the confidence and airs of someone possessing great authority.

Low-Hanging Fruit:

Some people have dismissed the recent AI-assisted breakthroughs because they were “obvious in retrospect” or because “not enough people tried to do them.” The first criticism seems clearly ridiculous, since “obvious in retrospect” is very far from “obvious.”

The second criticism reflects, in part, the way humans solve conjectures. In my experience, the process goes as follows. First, you learn about the conjecture and why it might be interesting. Second, you learn a little about why it is hard. You might think about it for a while and fail, and then file it away in the back of your mind. Later, when you encounter new ideas, you perform some pattern recognition and ask whether those ideas have any relation to previous problems you care about. If you are lucky, you “see a connection,” and then you can begin. Sometimes the first inspirational connection is most of the work; sometimes it is only the beginning, and there is much more work to do. In either case, that initial step is crucial.

An LLM, on the other hand (to anthropomorphize), can conduct an extremely thorough literature search and then throw a thousand different ideas at the problem to see whether anything sticks. If one of those ideas does seem relevant, and if the journey from that realization to the end of the proof is not a long one, then the LLM can solve the entire problem. As it becomes possible to chain together longer and longer stretches of reasoning, the only problems that can resist attack are those requiring a genuinely new idea (whatever that means).

Finally, there is the observation that these results are all counterexamples to conjectures that, as far as I know, were generally regarded as true. This phenomenon has a clear diagnosis, given by Deligne: “All problems in mathematics are psychological.” Well, to be precise, that quote was ascribed to Deligne^* by Kisin in a lecture at Luminy, but it captures the essence of something clearly true. And the good news is that AI is an insane psychofreak with no hangups.

Addressing the more interesting—and controversial—question of what we, as a profession, are to do about all of this will have to wait until later. This week, I will be attending my first ICM in person. It is a much-maligned conference whose most exciting moment has already been blown by incompetent website design and which is being held in a city with slightly less appeal to me personally than St. Petersburg; but we shall see!

^* Since I always check the original source, here is a slightly more nuanced version of that quote:

Dear Calegari,

I don’t remember the exact words, but I expect it is roughly accurate. Of course, it is not always true. The meaning was that we often have blocks which prevent us from seeing things which later will seem obvious to us.

Best,
Pierre Deligne