Course / Lecture 18 / Lab Lab 18

Symbolic regression: the front, not the answer.

Sixteen points from a hidden expression. Enumerate 7,626 expressions, fit their constants, keep the Pareto front, and move the complexity penalty. Watch the selected expression change while the data stay the same.

16 data points 7,626 expressions ≤ 9 complexity No install · deterministic

The symbolic-regression lab

Data · 16 pointsnoise-free synthetic · x ∈ [−3, 3]
Pareto front · error against complexityselected point highlighted
Ready. Press Run search to enumerate expressions up to complexity 9.

Legend. Teal dots are the data. The orange curve is the selected expression evaluated on the same x-range. On the front, lower-left is better: lower error and lower complexity. The highlighted point is the one the current penalty selects.

The selected expression

—
Complexity—
RMSE—
Score—
0.20

Score = RMSE + λ × complexity. The penalty is a prior about simplicity, not a fact. At λ = 0 the search selects the lowest error; at a larger λ it will pay for a shorter expression.

The front

Best error at each non-dominated complexity. The selected row is highlighted; the expression is the fitted form, not a claim of truth.
CRMSEExpression
Reveal the generating expression

The data were generated by y = 2 sin(x) + 0.5 on 16 equally spaced points in [−3, 3], with no noise. The search can recover it because the grammar contains sine, addition, and constants, and because the constants grid contains 2 and 0.5. That is a luxury real data rarely provide.

Trace

No search yet. The trace will list the best error at each complexity, the Pareto filter, and the selected score.
WHAT TO NOTICE

Three choices, one dataset.

CHOICE 01 · THE SEARCH

Enumeration by complexity

Every expression up to complexity 9 is generated, its constants are fitted by two coordinate-descent passes over a fixed grid, and its RMSE is computed. The best expression at each complexity is kept; the trace lists all nine. There is no sampling and no random seed.

CHOICE 02 · THE PENALTY

λ selects, the data do not

At λ = 0.20 the score selects the exact generating expression at complexity 6. At λ = 0.35 it selects sin(x) at complexity 2; at λ = 0.60 it selects x at complexity 1. The data are identical in all three cases. The penalty is the modeling choice.

CHOICE 03 · THE FRONT

Pareto is not a ranking

Four points survive: C1, C2, C4, and C6. C4 is on the front, but no linear penalty selects it: it lies above the line between C2 and C6, so a linear score never prefers it. A front is a set of trade-offs, not an order. The selected point depends on the scalarization.

What the lab is not

It is not a benchmark, a real scientific discovery, or a proof that the selected expression is true. The data are noise-free, the grammar contains the generating expression, the constants are fitted on a fixed grid, and the complexity measure is the number of nodes in the tree. It isolates one mechanism: enumeration, constant fitting, a complexity penalty, and a Pareto front. Real symbolic regression has noise, a grammar that may not contain the law, and a check that must be defended outside the training data.

Read the symbolic-regression section again · Read the five gaps · Slides