Lecture 07 · The five interfaces

Logic as a loss.

A constraint can be a penalty. Compile the logic into a differentiable function of the network's output, and the gradient does the reasoning — softly, approximately, with no guarantee.

9 sections 4 systems 1 soft-constraint lab Self-paced
00 · Learning objectivesLecture 07 · Logic as loss

By the end of this lecture

Four things you can do.

01

Soften a constraint

Turn a logical formula into a differentiable penalty, and say which t-norm you used.

02

Compute a semantic loss

Write −log P(α) for a small constraint and evaluate it on a distribution.

03

Read the family

Tell semantic loss, LTN, and DL2 apart by their semantics and their object of belief.

04

Name the limit

State why a loss is a prior, not a guarantee, and when to use an architecture or a verifier.

S/V · Lecture 07
00 · The hookLecture 07 · Logic as loss

Two digits, one sum

The answer is a distribution.

A classifier reads two digits. It does not output “3”; it outputs a distribution over 0–9 for each position.

The task says the sum is 8. That is free information: 90 of the 100 pairs are impossible.

The loss interface turns that information into a penalty on the distribution.

p(A)p(B) A + B = 8 P(valid) = Σ p(a) p(8 − a)
The constraint sums probability mass over the diagonal. A soft penalty lowers it when the mass is off the diagonal.
S/V · Lecture 07
00 · The hookLecture 07 · Logic as loss

Two terms, one weight

Who wins when they disagree?

L(θ) = Ldata(θ) + λ · Llogic(θ)
(1)

Data

Fits the observations. With noisy labels, the data term is confident about the wrong thing.

Logic

Penalizes distributions that violate the constraint. It does not know which valid answer is right.

The weight λ is the whole negotiation. There is no universal value; there is a validation set and a metric.

S/V · Lecture 07
01 · Hard to softLecture 07 · Logic as loss
01 / 08

A truth value between 0 and 1.

The hard indicator is not differentiable. The soft semantics is a design choice.

S/V · Lecture 07
01 · Hard to softLecture 07 · Logic as loss

The semantics is a hyperparameter

Three ways to say AND.

FamilyANDORGradient
Producta · ba + b − abSmooth; shrinks near zero.
Łukasiewiczmax(0, a + b − 1)min(1, a + b)Piecewise linear; saturates.
Gödelmin(a, b)max(a, b)Gradient to one operand only.

A constraint that is easy under one t-norm can be a plateau under another. Papers rarely report the choice as a hyperparameter; you should.

S/V · Lecture 07
01 · Hard to softLecture 07 · Logic as loss

Two readings of 0.8

Truth is not probability.

Probability

p(3) = 0.8 means 80% of the mass is on 3. A conjunction is a sum over assignments, not a product of marginals in general.

Fuzzy truth

t(3) = 0.8 means the degree of truth is 0.8. A conjunction is a t-norm of the degrees; the result need not be a probability.

a = 0.8, b = 0.8ProbabilityFuzzy (product)Fuzzy (Łukasiewicz)
a ∧ bdepends on dependence: 0.64 if independent0.640.6
What the number meansa measure over outcomesa degree of satisfactiona degree of satisfaction
The two coincide for a single atom and diverge for compounds. A paper that calls a t-norm “probability” is using a metaphor, not a theorem.
S/V · Lecture 07
02 · Semantic lossLecture 07 · Logic as loss
02 / 08

The probability the constraint holds.

Sum the mass on satisfying assignments, then take the negative log.

S/V · Lecture 07
02 · Semantic lossLecture 07 · Logic as loss

The definition

Put the mass on valid assignments.

Ls(α, p) = −log Σx ⊨ α Πi pixi (1 − pi)1 − xi
(2)

Each satisfying assignment contributes the model's probability for that assignment. The loss is small when the mass is on valid assignments, large when it is on violations.

If α constrains one variable, this is exactly cross-entropy. That is the sanity check.

ConstraintWhat the loss becomes
One variable: x = 1−log p — cross-entropy
Exactly one of n−log Σ pi Πj≠i(1 − pj)
Deterministic αZero if the model is on a satisfying assignment; infinite if it is not
S/V · Lecture 07
02 · Semantic lossLecture 07 · Logic as loss

A worked case

Exactly one of three.

Ls = −log Σi pi Πj≠i (1 − pj)
(3)

Only three assignments satisfy “exactly one”: (1,0,0), (0,1,0), (0,0,1). The sum has three terms.

p = (0.9, 0.05, 0.05) P(valid) = 0.9·0.95·0.95 + 0.05·0.1·0.95 + 0.05·0.1·0.95 ≈ 0.821 L = −log(0.821) ≈ 0.197 p = (0.5, 0.5, 0.5) P(valid) = 3 · 0.125 = 0.375 L ≈ 0.981

The sharp distribution is closer to satisfying the constraint; the flat one is penalized more. The gradient pushes toward a one-hot assignment.

S/V · Lecture 07
02 · Semantic lossLecture 07 · Logic as loss

What the gradient does

Reweight the assignments.

The derivative of −log P(α) increases the probability of satisfying assignments and decreases the probability of violations, weighted by their current mass.

Near a violation

The gradient is large: the model is putting mass where it must not.

Near satisfaction

The gradient is small: the constraint is nearly satisfied, and the data term dominates.

This is why the loss behaves like a prior: it matters most when the model is wrong, and fades when the model is right.

S/V · Lecture 07
02 · Semantic lossLecture 07 · Logic as loss

Interactive · exactly one of three

Move the probabilities.

p10.90
p20.05
p30.05

P(exactly one) / semantic loss

0.821 loss ≈ 0.197 The sharp distribution puts most of its mass on one variable.
(1,0,0)p1 (1−p2)(1−p3)
(0,1,0)(1−p1) p2 (1−p3)
(0,0,1)(1−p1)(1−p2) p3

P(exactly one) is the sum of these three terms; the loss is −log of the sum.

S/V · Lecture 07
02 · Semantic lossLecture 07 · Logic as loss

The catch

Counting is hard.

The sum over satisfying assignments is a weighted model count. In general it is #P-hard, even when satisfiability is easy.

A Boolean constraint with many variables can be cheap to check and expensive to count.

Compilation: turn the constraint into a tractable circuit (d-DNNF, SDD) and evaluate the count on the circuit.

Structure: exploit independence, small treewidth, or a factor graph. The loss is only as cheap as the constraint's structure.

If the constraint has no structure, semantic loss is as expensive as the reasoning it was meant to replace.
S/V · Lecture 07
03 · LTNLecture 07 · Logic as loss
03 / 08

First-order logic in real values.

Predicates are networks; quantifiers are aggregators; the knowledge base is a loss.

S/V · Lecture 07
03 · LTNLecture 07 · Logic as loss

Grounding and aggregation

From atoms to formulas.

Ground atom: a predicate network maps an entity tuple to [0,1].

Connectives: a t-norm, chosen once for the whole knowledge base.

Quantifiers: aggregate over the domain; min/max are the hard versions, pMean is soft.

; a knowledge base ∀x (Digit(x) → (Zero(x) ∨ ... ∨ Nine(x))) ∀x∀y (Sum(x, y, 8) → ...) ; a loss L = L_data + λ · mean(sat(KB))

The formulas can quantify over entities and relations, which is exactly what a fixed Boolean constraint cannot do.

S/V · Lecture 07
03 · LTNLecture 07 · Logic as loss

The quantifier is a choice

Min or pMean?

AggregatorBehaviourGradient
minOne bad instance falsifies the formula.Flows to the worst instance only.
pMeanA soft minimum; every instance contributes.Spreads across instances.
sum / meanEvery instance counts equally.Easy to train; can dilute a violation.

A hard quantifier is faithful to the logic and hostile to the optimizer. A soft one is trainable and no longer exactly the logic.

S/V · Lecture 07
04 · DL2 & fuzzy logicLecture 07 · Logic as loss
04 / 08

A semantics, not a formula.

DL2 makes the t-norm explicit and uses it for training and for queries.

S/V · Lecture 07
04 · DL2 & fuzzy logicLecture 07 · Logic as loss

Four pieces

Ground, evaluate, train, query.

01

Ground

Map symbols and network outputs to [0,1].

02

Evaluate

Choose t-norms and aggregators; compute satisfaction.

03

Train

Minimize data loss plus a weighted knowledge loss.

04

Query

Optimize the truth value over free variables.

The same semantics is used for learning and for inference; that is the elegant part.

S/V · Lecture 07
04 · DL2 & fuzzy logicLecture 07 · Logic as loss

Before you trust a number

Two warnings.

Truth ≠ probability

A satisfaction of 0.9 is not a 90% chance. It is a degree under a chosen t-norm.

Semantics ≠ neutral

Changing the t-norm can change the solution. A result that depends on product may not survive under Łukasiewicz.

t(a) = t(b) = 0.9ProductŁukasiewiczProbability
a ∧ b0.810.8not determined by the marginals alone

Report the semantics. Compare systems under the same one, or do not compare them.

S/V · Lecture 07
05 · The ledgerLecture 07 · Logic as loss
05 / 08

What it buys, what it costs.

The loss interface is the easiest to deploy and the easiest to overclaim.

S/V · Lecture 07
05 · The ledgerLecture 07 · Logic as loss

The trade

A prior, not a proof.

BuysCosts
End-to-end training, one optimizerNo hard guarantee; violations can remain
Soft guidance with noisy or sparse dataSensitivity to λ; task-specific tuning
Works with any differentiable model#P-hard counting without compilation
No external solver at training timeMis-specified constraints teach the wrong thing
Background knowledge as a priorThe letter can be satisfied; the intent violated

If a violation is unacceptable, the loss is the wrong interface. Use an architecture or a verifier.

S/V · Lecture 07
05 · The ledgerLecture 07 · Logic as loss

How it goes wrong

Three failure modes.

Gaming

Collapse to a satisfying assignment regardless of the input. The constraint is met; the task is not.

Weight collapse

A large λ makes the data irrelevant. The model is constraint-valid and task-wrong.

Semantic mismatch

The constraint is a proxy for the real requirement, and the proxy is wrong.

None of these is fixed by a better optimizer. They are fixed by a validation metric that includes constraint satisfaction and task accuracy, and by a human who checks the semantics.

S/V · Lecture 07
06 · Four casesLecture 07 · Logic as loss

Read them along the tuple

Same interface, different semantics.

They share the anatomy, not the semantics. The guarantee is none in all four.

S/V · Lecture 07
07 · DesignLecture 07 · Logic as loss

Before you add the term

Six questions.

01

Is the constraint soft or hard? Hard → architecture or verifier.

02

Can you count satisfying assignments? If not, is there a tractable circuit?

03

Which t-norm and aggregator? Report them.

04

How will you tune λ, and on what metric?

05

Which failure mode is most likely: gaming, collapse, mismatch?

06

What does the interface guarantee? A loss guarantees nothing at inference.

A loss guarantees nothing at inference.

S/V · Lecture 07
08 · RecapLecture 07 · Logic as loss

The four moves

What to carry forward.

01 · Soften

A constraint becomes a differentiable penalty; the semantics is a choice.

02 · Count

Semantic loss is −log P(α); counting is #P-hard without structure.

03 · Read

Semantic loss, LTN, DL2: same anatomy, different semantics.

04 · Limit

A loss is a prior, not a proof. Hard constraints need architecture or verification.

A prior, not a proof.

S/V · Lecture 07
08 · RecapLecture 07 · Logic as loss

Next · hands-on

Sweep the weight.

Open the soft-constraint lab. Train with λ = 0, then raise it. Watch the constraint probability, the MAP pair, and the data loss trade places.

Lab 07 · semantic loss → open the lab

S/V · Lecture 07
Speaker notes