Lecture 18 · Act V · Applications

Science and mathematics.

A model can propose a proof, a structure, a program, or an equation. Science begins when something independent checks it.

9 sections 5 systems 1 interactive lab Self-paced
00 · Learning objectivesLecture 18 · Science and mathematics

By the end of this lecture

Four things you can do.

01

Separate proposing from verifying

Name the proposal, the artifact, the checker, and the warrant — and keep them apart.

02

Read a discovery system

Use the tuple: neural part, symbolic part, coupling, checked artifact, guarantee.

03

Explain symbolic regression

Say what the search enumerates, how constants are fitted, and what the complexity penalty does.

04

Audit what a check guarantees

Distinguish a proof from an experiment, and name the five gaps between a check and a claim.

S/V · Lecture 18
00 · The hookLecture 18 · Science and mathematics

A dataset, two candidates

The fit is not the warrant.

x y
16 noise-free points from an expression the lab hides.

A search finds a short expression at complexity 6 that fits the points exactly.

It also finds longer expressions that fit the same points. Training error does not choose between them.

A warrant is what chooses. A derivation, a held-out test, a units check, or a proof. The candidate is not knowledge until something independent supports it.
S/V · Lecture 18
00 · The hookLecture 18 · Science and mathematics

The shape of the lecture

Propose, then check.

01 · Propose

A search, a language model, or a scientist returns a candidate.

02 · Artifact

A proof, a program, an equation, or a structure — something a checker can inspect.

03 · Check

A kernel, an evaluator, a held-out set, or an experiment.

04 · Warrant

Accept, reject, score, or revise. The result is a reason, not a new candidate.

The checker must be independent of the proposer. Otherwise the loop is a mirror, not a check.

The proposer can be neural or symbolic. The checker can be a proof engine, an evaluator, or the world. The design question is where the independence comes from.

S/V · Lecture 18
01 · Discovery vs verificationLecture 18 · Science and mathematics

Not every check is the same kind of check

Five warrants, five scopes.

WarrantWhat it checksWhat it cannot support
Formal proofEach step follows by a rule from axioms and prior stepsThat the formal statement is the intended claim
Exhaustive checkEvery case in a finite, correctly enumerated spaceAnything outside the enumeration
Held-out predictionError on data not used to fitExtrapolation, causation, or the truth of the hypothesis
Controlled experimentAn intervention against a comparisonThat the design controls every confound
Benchmark scorePerformance on a fixed task setThat the benchmark measures the intended capability
A check is relative to a specification. “Verified” without the artifact, the checker, and the specification is a slogan.
S/V · Lecture 18
02 · AlphaGeometryLecture 18 · Science and mathematics
02 / 08

A model proposes; a deduction engine checks.

Auxiliary constructions, DDAR, and a proof you can check.

S/V · Lecture 18
02 · AlphaGeometryLecture 18 · Science and mathematics

The hard part is not the derivation

The missing line is the hard part.

the construction is not in the statement
The model proposes; the engine derives.

Formal language. Points, lines, circles, and predicates such as collinearity and equal angles.

DDAR. A deduction engine with sound rules; it derives consequences or stalls.

Language model. Trained on synthetic proofs; when the engine stalls, it proposes an auxiliary construction.

The engine cannot invent the construction; the model cannot certify the proof.
S/V · Lecture 18
02 · AlphaGeometryLecture 18 · Science and mathematics

The loop

Four steps, one certificate.

01 · Formalize

K is the diagram, g the goal.

02 · Propose

The language model samples auxiliary constructions A.

03 · Derive

DDAR applies sound rules to K ∪ A.

04 · Check

A proof π, or a stall and another proposal.

A ∼ pθ(A | K, g)     π = DDAR(K ∪ A, g)The model is a proposal distribution; the engine is the checker.

The proof is a sequence of rule instances from the premises plus the proposed construction. The model can be wrong; the engine rejects wrong steps.

S/V · Lecture 18
02 · AlphaGeometryLecture 18 · Science and mathematics

What the proof check buys

The proof is about the formal statement.

It guarantees

Each accepted step is an instance of a sound rule. The goal follows from K ∪ A. The proof is a checkable artifact, not a plausible sentence.

It does not guarantee

That the formalization matches the problem you meant; that the engine’s implementation is correct; that the proof is short, new, or general.

A rejected construction is not a disproof. A proved theorem about the wrong formalization is still the wrong theorem.
S/V · Lecture 18
03 · AlphaFoldLecture 18 · Science and mathematics

From sequence to structure

A structure is a prediction.

01 · Input

Amino-acid sequence plus a multiple sequence alignment and pair features.

02 · Evoformer

MSA and pair representations, updated with pair-biased attention and triangular operations.

03 · Structure module

Backbone frames and invariant point attention produce Cartesian coordinates.

04 · Output

3D coordinates plus pLDDT and PAE confidence estimates.

The symbolic content is geometry and chemistry — frames, equivariance, bond geometry — encoded as inductive biases, not as a logic program.

This is not a NeSy reasoner in the strong sense. It belongs here because it makes the propose-and-check pattern visible at scientific scale.
S/V · Lecture 18
03 · AlphaFoldLecture 18 · Science and mathematics

The check is empirical

The experiment is the check.

Experimental structure

X-ray crystallography, cryo-EM, or NMR. Blind assessment against structures released later, such as CASP.

Supports agreement for that construct and condition.

Confidence

pLDDT is a per-residue confidence; PAE estimates relative position error.

Learned estimates, not guarantees. They say where to look, not what is true.

A snapshot is not a mechanism. A structure does not tell you how the protein folds, what it binds, or how it moves.
S/V · Lecture 18
04 · FunSearchLecture 18 · Science and mathematics
04 / 08

Programs that are executed, not believed.

An LLM proposes code; an evaluator checks it.

S/V · Lecture 18
04 · FunSearchLecture 18 · Science and mathematics

Evolutionary program search

The evaluator is the check.

01 · Specify

Signature, docstring, contract, and an executable evaluator.

02 · Propose

The pretrained LLM writes candidate programs against the specification.

03 · Execute

The evaluator runs each candidate and returns a score.

04 · Select

A program database keeps diversity and feeds the best back into the prompt.

No fine-tuning. The LLM is a proposal operator inside an evolutionary search. The evaluator, not the model, is the source of trust.

The model is prompted with the specification and the best programs; the evaluator runs the instances.
S/V · Lecture 18
04 · FunSearchLecture 18 · Science and mathematics

What is checked

A program, not a paragraph.

A construction

The program builds an object; an independent check verifies the defining property. The paper’s cap-set result is this kind.

A heuristic

The program is scored on instances. Online bin packing is this kind: the score is evidence, not a proof of optimality.

The evaluator is the specification in executable form. If it is incomplete, the search will find the gap and exploit it.
S/V · Lecture 18
05 · Symbolic regressionLecture 18 · Science and mathematics

Search over expressions

Search the grammar, not the weights.

+ 0.5 × 2 sin x complexity 6 · exact on the 16 points
The selected expression at λ = 0.20.

Enumerate expression trees by complexity — nodes, operators, constants.

Fit each expression’s constants by deterministic coordinate descent.

Keep the best error at each complexity; the front is the non-dominated set.

Expressions7,626
Max complexity9
Front points4
S/V · Lecture 18
05 · Symbolic regressionLecture 18 · Science and mathematics

The objective

Fit alone gives no reason to stop.

score(f) = RMSED(f) + λ · C(f)λ is a prior about simplicity; C is the number of nodes.

λ small

Fit wins. A complex coincidence can beat a simple law, especially with noise.

λ large

Simplicity wins. A short approximation can beat the expression that generated the data.

Pareto front

Keep every expression not dominated in both error and complexity; choose a point, not a rank.

The penalty is a prior, not a fact. The data do not choose λ; the modeler does, and then defends the choice.
S/V · Lecture 18
05 · Symbolic regressionLecture 18 · Science and mathematics

Interactive · move the penalty

Move the penalty. The data do not move.

COMPLEXITY ERROR better x sin(x) 2·sin(x) 2·sin(x)+0.5
Complexity6
RMSE0.000
Score0.000
0.5 + (2 · sin(x))

λ = 0.00 selects complexity 6: the exact expression on the synthetic data.

C4 is on the front, but no linear penalty selects it: it lies above the line between C2 and C6. A front is a set of trade-offs, not a ranking.

S/V · Lecture 18
06 · Theorem provingLecture 18 · Science and mathematics
06 / 08

The kernel checks the proof, not the problem.

Lean, Coq, Isabelle, and the formalization gap.

S/V · Lecture 18
06 · Theorem provingLecture 18 · Science and mathematics

A small trusted base

A small checker, a large search.

Γ ⊢ π : φIn context Γ, the proof term π has type φ.

Kernel

Type-checks the proof term against the stated theorem. The trusted base.

Proposer

Tactics, lemmas, and language-model suggestions. Untrusted; wrong steps are rejected.

de Bruijn criterion

The proof can be checked by a small independent program, so the search can be huge.

The kernel makes the result trustworthy, not the search fast. Finding the proof remains the hard part.
S/V · Lecture 18
06 · Theorem provingLecture 18 · Science and mathematics

The scope of the check

What “verified” does not mean.

The axioms are consistent

Gödel’s incompleteness theorem rules out a sufficiently strong system proving its own consistency.

The statement is the one you meant

The formalization can be weaker, stronger, or about a different object. This is the formalization gap.

The search will find a proof

Proof search is not guaranteed to terminate or to be cheap. Type checking is not proof finding.

The kernel has no bugs

The trusted base is small, not empty: type theory, parser, compiler, runtime.

“Verified” is relative to the kernel and the statement. Both are part of the claim.
S/V · Lecture 18
07 · What failsLecture 18 · Science and mathematics

Two gaps that survive a passing check

The artifact can be wrong.

Formalization gap

The formal statement, grammar, or specification is not the intended claim.

Symptom: a proof of the wrong theorem; an equation outside the grammar. Mitigation: independent formalization and explicit assumptions.

Evaluator gap

The check measures a proxy, not the goal. The evaluator is a program and can be gamed.

Symptom: a high score on a gameable proxy; benchmark overfitting. Mitigation: independent checks and held-out evaluators.

S/V · Lecture 18
07 · What failsLecture 18 · Science and mathematics

The rest of the taxonomy

Three more gaps.

Search gap

The proposer misses the candidate, or the budget is too small. The result can depend on the seed.

Restarts; better proposals; completeness bounds where available.

Underdetermination

The data do not identify the hypothesis. Many expressions fit; extrapolation can fail.

More data; units and symmetry; stability tests.

Trust gap

The checker, axioms, or measurement is wrong. A kernel bug or a biased experiment.

Small trusted base; independent replication; calibration.

Five gaps, one habit: ask what the artifact is, what the checker is, and what the check is relative to.
S/V · Lecture 18
08 · Tuple & recapLecture 18 · Science and mathematics

Read the system, not the headline

Five systems, one tuple.

SystemProposerCheckerInferenceArtifactGuarantee
AlphaGeometryLM constructionsDDARDeductionProofFormal, relative to formalization
AlphaFoldEvoformer + structure moduleExperimentsPredictionCoordinatesEmpirical
FunSearchLLM programsEvaluatorInductionProgramEmpirical, relative to evaluator
Symbolic regressionEnumerationHeld-out dataInductionEquationEmpirical
Lean + LLMTacticsKernelDeductionProof termFormal, relative to kernel
The strongest guarantee comes from a checker that is independent of the proposer. AlphaGeometry’s engine and Lean’s kernel do not care how the candidate was produced.
S/V · Lecture 18
08 · Tuple & recapLecture 18 · Science and mathematics

What to retain

Three patterns.

Independence buys trust

A checker that does not care who proposed the candidate can reject it. A model grading itself cannot.

A proxy is not a proof

An evaluator gives evidence relative to itself. A benchmark is a proxy, and a proxy can be gamed.

Empirical is a scope

An experiment is the right check for a structure, just as a proof is for a theorem. The mistake is the vocabulary.

The verifier is independent; the benchmark is a proxy; the experiment is a scope.

S/V · Lecture 18
08 · Tuple & recapLecture 18 · Science and mathematics

What you should be able to do

Four moves.

Separate

The proposal from the warrant; the artifact from the world.

Name

The artifact, the checker, and the specification the check is relative to.

Explain

Symbolic regression’s enumeration, constant fitting, and complexity penalty.

Diagnose

Formalization, evaluator, search, underdetermination, and trust gaps.

S/V · Lecture 18
08 · Tuple & recapLecture 18 · Science and mathematics

Next · hands-on

Now run the front.

Open the lab. Enumerate 7,626 expressions. Move λ and watch the selected expression change while the data stay still.

Next in the arc: robotics and embodied AI. The same loop, with the world in the checker role.

S/V · Lecture 18
Speaker notes