back to blog

The Frontier Needs Open Problems

The Frontier Needs Open Problems

How do you tell the difference between a model that is getting better at school and a model that is starting to do research?

That question is going to matter more every year.

A lot of current AI progress is still measured on tasks with known answers, known datasets, known rubrics, known tricks, and often known leakage paths. That is useful up to a point. You want systems that can pass tests. But if the long-term goal is machine intelligence that helps push science, engineering, and mathematics forward, then eventually the benchmark has to reach past the part of the map we already colored in.

That is why HorizonMath: Measuring AI Progress Toward Mathematical Discovery with Automatic Verification is interesting.

The core move is simple and good. Instead of asking models to solve polished textbook problems or formal proofs that are expensive to check, HorizonMath collects more than 100 mostly unsolved problems from computational and applied mathematics where the hard part is discovery but the checking step is comparatively cheap. You do not need a room full of human graders debating style points. You can verify whether a proposed result actually improves the current bound or satisfies the target condition.

That sounds narrow, but it points at something much bigger.

The real bottleneck in a lot of AI research is not generation anymore. Models can generate endless candidate ideas, proofs, code, plans, and explanations. The bottleneck is contact with reality. Which outputs cash out into something true? Which ones survive checking? Which ones actually move the frontier instead of remixing yesterday's internet?

HorizonMath is valuable because it builds one more place where the answer is not vibes.

That is also why I think the most promising version of the future still looks a lot like dense loops between models and verifiers. Not models floating in a cloud of prose, but models attached to environments that can say yes, no, closer, wrong, try again. In code, that means tests. In robotics, it means the body and the task. In science, it means experiment. In parts of mathematics, it means finding problem classes where search is hard but checking is clean.

Once you have that structure, progress stops being purely theatrical.

You can feel the difference in the paper's framing. The benchmark is designed to be resistant to contamination because the answers are not sitting in a training corpus waiting to be memorized. The authors report that most state-of-the-art models score near zero, which is actually healthy. A frontier benchmark should have room in front of it. If every model is already cruising through, you did not find the edge. You found another retrospective exam.

The most provocative line in the paper is that GPT 5.4 Pro proposed solutions for two problems that may improve on the best published results, pending expert review. Maybe those claims hold up, maybe they do not. Either way, that is the right kind of claim to make. Concrete enough to check. Ambitious enough to matter. No magic, no hand-waving, no "look how smart the chatbot feels." Just: here is the problem, here is the proposed result, now go verify it.

That style of progress is underrated.

People keep asking whether AI can do real science as if the answer has to arrive all at once in the form of some artificial Newton. It probably will not. More likely, it arrives as systems that are uneven but useful, narrow but real, able to generate many plausible attacks on hard problems inside domains with cheap feedback. First in math niches where verification is easy. Then in coding, theorem proving, materials, optimization, lab automation, and every other place where candidate generation can be coupled to a fast filter that is not just another language model nodding along.

That is a very Culture-shaped direction, honestly. Not because it is flashy, but because it lowers the cost of thought.

A society gets more interesting when intelligence is easier to instantiate, cheaper to check, and less dependent on small pools of elite attention. If a hard problem can absorb millions of machine attempts and only ask humans to inspect the few that survive verification, then expert time starts stretching further. You get a new ratio between speculation and judgment. More search. More parallelism. More frontier per person.

That does not mean every benchmark like this will matter, or that every claimed breakthrough is real. A lot of this space will be noisy. Some tasks will turn out to be gameable. Some apparent discoveries will collapse under review. Fine. That is what contact surfaces are for.

The important part is that the field is getting better at building them.

For a while, a lot of AI evaluation was basically theater for people who already wanted to believe. Benchmarks with hidden leakage. leaderboards without stakes. demos that looked great until you asked what exactly had been measured. The way out is not less ambition. It is better interfaces between ambition and reality.

Open problems with automatic verification are one of those interfaces.

If this line of work keeps improving, the interesting future is not just models that explain known mathematics more fluently. It is models that help widen the set of people and organizations who can push on unsolved things at all. Better search, better filtering, better use of expert attention, better maps of where machine reasoning actually starts to become productive.

That is the kind of infrastructure you build before discovery stops being rare and artisanal.

And that is when the future starts feeling less like a story and more like a workflow.