The most useful question about a neurosymbolic system in an institution is not “what model?” It is “what would count as an audit trail?” A decision that cannot name the rule that produced it, the facts it used, and the person who signed the exception is not yet a decision an organization can defend.
00 / CyberThe attack graph is a plan written by the defender.
A log is an observation; an alert is a claim that a pattern matched. Neither explains how an attacker moved. An attack graph does: nodes are security conditions, edges are exploits with preconditions and effects, and a path to a goal composes steps.
Denning’s 1987 model made intrusion detection statistical. Phillips and Swiler (1998) made vulnerability analysis a graph search; Sheyner et al. (2002) generated attack graphs with model checking; Ammann, Wijesekera, and Kaushik (2002) scaled the analysis; MulVAL (Ou, Govindavajhala, and Appel, 2005) expressed generation as a logic program. The vulnerability model is a knowledge base; a path is a plan that composes exploit rules.
Alert correlation adds abduction. Ning, Cui, and Reeves (2002) modelled alerts as hyper-alerts with prerequisites and consequences and correlated them into scenarios; Cuppens and Miège (2002) built a cooperative framework. The engine asks which scenario would make the alerts expected; several can, so ranking and ambiguity are part of the output.
Neural anomaly and alert classifiers. Symbolic an attack graph from a vulnerability model, plus correlation rules. Interface detector scores become facts; an analyst maps them onto edges. Guarantee if the model is correct, a path is a valid composition of exploits; correlation is a ranked hypothesis, not proof. Limit model drift, graph explosion, and an attacker who need not follow the graph.
| Artifact | What it asserts | What it does not |
|---|---|---|
| Log line | An event was recorded | That it is malicious or complete |
| Alert | A detector matched a pattern | That the pattern is an attack |
| Attack path | A sequence of exploits is possible under the model | That it occurred, or that no other path exists |
Attack graphs inherit stale feeds, incomplete topology, and exploits absent from the rule base. Correlation inherits detector base rates and false-positive bursts. The guarantee is relative to the model and the alert semantics.
01 / Law & complianceThe statute is a program with an open texture.
Law looks like a production-rule system: conditions, exceptions, definitions, cross-references. It is also interpreted: terms are open-textured, precedents compete, and a principle can defeat a rule that was never written as an exception. The formalization is one reading, made by someone with authority.
Catala is the clearest modern attempt to make that reading explicit: a domain-specific language in which legal text and rule are written together, so the source is a literate document a lawyer can audit. Its semantics includes default logic, giving exceptions a principled place rather than an if-chain. The prototype encodes fragments of French tax and family-benefit law. The guarantee is modest: the result follows the encoded rule and the encoding is readable against the source. It does not certify fidelity, constitutionality, or what a court would do.
Case-based reasoning takes the other route. Ashley’s HYPO (1990) represents cases with dimensions, compares a new case to precedents, and builds three-ply arguments: claim, response, reply. It can show which case is closer and which dimension does the work; it cannot weigh a fact no precedent anticipated. Statutes supply the rule, precedents supply the interpretation, and hard cases live where they meet.
COMPAS is the cautionary case: an actuarial risk instrument used in U.S. criminal-justice settings. ProPublica’s 2016 analysis argued its error rates differed by race; Dressel and Farid (2018) disputed that conclusion and argued the answer depends on which error rate one chooses. This lecture does not adjudicate the dispute. The narrower point: a score is not a trace. It can rank risk and be explained post hoc, but it does not name the rule or facts that produced the ranking. Explainability is not derivation.
Neural document classification, clause extraction, and entity resolution. Symbolic a statute or policy encoded as rules with defaults, plus a precedent base with dimensions. Interface extracted facts become predicates; the engine reports the derivation. Guarantee the decision follows from the encoded rule and facts; fidelity, truth, and justice are separate. Limit open texture, conflicting precedent, and the knowledge-acquisition bottleneck.
Rules as code is not neutral translation. Choosing what to encode, which exceptions to make explicit, and which terms to leave to judgment is a policy decision with distributional consequences. A deduction is only as legitimate as the process that produced the rule base. The trace should name the policy version, not only the rule.
02 / HealthThe guideline is a protocol; the patient is not.
Health has the oldest case study, and it predates the neural part. MYCIN (Shortliffe, 1976) was a rule-based consultation system for bacterial infections. Its rules carried certainty factors, a hand-designed calculus for combining evidence that was meant to be inspectable and to avoid demanding priors clinicians lacked. It asked for findings, chained backward, and recommended therapy, but remained a research system. Its lesson is the trace: a recommendation came with the rules that supported it.
INTERNIST-1 (Miller, Pople, and Myers, 1982) and its successor QMR took another route: a large internal-medicine knowledge base and a scoring scheme over competing diagnoses. Curation was expensive, and the score was not a probability. Both teach the same lesson: inspectability costs maintenance.
Modern support usually formalizes the guideline rather than the disease. GLIF, Asbru, PROforma, and EON represent a care pathway as a graph of decisions, eligibility conditions, actions, and temporal constraints; Peleg et al. (2003) compare them on a shared case study. The guideline is a protocol: it says what should happen for a patient who fits its criteria. A constraint or rule engine can check the pathway; it cannot check whether the patient fits.
The neural part enters at the boundary. A classifier can flag a study; a language model can extract findings; a triage model can rank a queue. Esteva et al. (2017) showed a convolutional network matching dermatologists on a skin-lesion benchmark; that is a benchmark result, not a deployment licence. The interface is the pattern: the classifier produces a probability, the guideline decides whether it changes the next step, and a clinician owns the exception.
Neural image, signal, and note models that produce findings with probabilities. Symbolic a guideline with eligibility criteria, actions, and temporal constraints, plus a drug-interaction knowledge base. Interface findings become facts; the guideline derives the next step and can request a missing test. Guarantee the recommendation follows the guideline for the documented facts; it is not a diagnosis or an outcome. Limit outdated guidelines, exceptional patients, and a calibrated model that is still wrong.
Clinical reasoning is also abduction: which condition would explain this constellation of findings? A guideline engine can enumerate hypotheses and check consistency, but it cannot supply the judgment that decides when a guideline does not fit. That judgment is the exception the audit trail must make visible.
03 / Finance & auditEvery posting is a proof obligation.
Finance is where the symbolic side is least optional. Double-entry bookkeeping, described by Pacioli in 1494, is an invariant system: every journal entry has equal debits and credits, and the trial balance follows. A machine can verify these identities exactly; it cannot verify that the amounts are honest, the counterparty exists, or the treatment is appropriate. The invariant is necessary, not sufficient.
Around the ledger sit controls already shaped like rules: segregation of duties, the three-way match between purchase order, receipt, and invoice, duplicate-payment detection, vendor-master review, and materiality thresholds. Decision tables are a good notation; the OMG’s DMN standardizes tables with hit policies such as first, unique, and collect. The hit policy is precedence in disguise, and it needs the same documentation.
The neural part is at the edges: OCR, merchant normalization, entity resolution, anomaly scores, extraction from filings. XBRL gives reporting a symbolic schema; a validator can check an extracted fact against the taxonomy and arithmetic. Benford’s law (1938) is the classic digit-distribution check; it detects, it does not prove fabrication.
The lab isolates the mechanism: a neural component produces facts and flags; a rule engine applies policy in fixed precedence; the decision names the rule that fired; exceptions are policy objects or logged overrides. The audit log lets a reviewer reconstruct the decision without rerunning the model.
| Control | Symbolic check | What it catches | What it misses |
|---|---|---|---|
| Double entry | Debits equal credits; trial balance | An unbalanced posting | A balanced posting that is fraudulent |
| Segregation of duties | Requester ≠ approver ≠ payer | Self-approval | Collusion across roles |
| Three-way match | Order, receipt, and invoice agree | Overbilling without delivery | A collusive vendor supplying all three |
| Duplicate detection | Same vendor, amount, and date | Resubmission and double payment | Split invoices below the threshold |
Neural OCR, merchant normalization, and anomaly ranking. Symbolic ledger invariants, policy rules, decision-table hit policies, and the exception register. Interface extracted facts and flags enter the engine; it writes a decision and a trace; a human reviews the rejected and the ambiguous. Guarantee the decision follows the policy for the recorded facts, and the accounting identities hold; it is not a finding of fraud. Limit false positives, collusion, and a policy that is itself the weakness. Lab rules are fixtures, not accounting advice.
A rule engine that treats an absent fact as false will reject a claim for a missing receipt that exists elsewhere. That is often the right audit default, but it is an assumption and should be visible: “not found in the case file” differs from “does not exist.” That difference is what review is for.
04 / ArtThe grammar generates; the artist selects.
Art is where the symbolic component is easiest to dismiss and most instructive to keep. AARON, Harold Cohen’s drawing program, is the canonical case. In its classic form its rules were hand-written by Cohen: composition, figures, colour, and the relations that make a drawing hold together. It did not learn a style from a corpus; it encoded a practice. That makes authorship precise: the program explored a space the artist defined, and the artist chose the space.
Generative grammars generalize the idea. Lindenmayer systems (1968), popularized in The Algorithmic Beauty of Plants (Prusinkiewicz and Lindenmayer, 1990), are rewriting rules that produce branching structures; shape grammars produce arrangements of shapes by rule. The symbolic object is a language of possible forms. Membership is checkable; quality is not.
The neural side enters as a critic, stylist, or proposer. Gatys, Ecker, and Bethge (2015) showed that a pretrained convolutional network can separate content and style and recombine them by optimizing a content loss and a style loss. The interesting interface is generate and criticize: the grammar guarantees a candidate is well-formed, a neural critic ranks candidates, and a human selects. The symbolic part guarantees the space; the neural part navigates it.
The guarantee is narrow. The system can guarantee membership in the grammar, satisfaction of a constraint, or reproducibility from a fixed checkpoint. It cannot guarantee that the output is good, original, or authored by anyone in particular. Those are judgments; in the copyright case, legal questions.
Neural a style model, a generative model, or an aesthetic critic. Symbolic a grammar, a shape rule system, or composition constraints. Interface generate-and-test: the grammar proposes a well-formed candidate; the critic ranks; the human selects. Guarantee membership in the grammar and reproducibility from the recorded rules and checkpoint; no guarantee of quality, originality, or authorship. Limit training-data provenance, critic bias, and “creativity” as a non-benchmark.
A system that optimizes a learned aesthetic score produces what the score rewards, including shortcuts. A grammar that enumerates a space produces everything in it, including the boring parts. Selection is where intent enters; if the interface hides it, it hides the authorship.
05 / EducationThe hint must be licensed by the next step.
Education has a symbolic core that is easy to miss because it is called a curriculum. A model-tracing tutor represents the domain as a graph of solution paths, including correct and incorrect steps, and compares the student’s step to the graph. Anderson, Corbett, Koedinger, and Pelletier (1995) described the Cognitive Tutors built on this idea: the tutor can say which step is legal next, hint at that step, and update a model of the student’s knowledge. Knowledge tracing (Corbett and Anderson, 1994) adds a latent model of skill acquisition.
The neural part is now the fluent part. A language model can grade a free-text answer, generate a hint, or explain an example. The failure mode is equally fluent: a hint can skip a step, contradict the graph, or solve the problem. The interface that makes the system trustworthy is a verifier: the model proposes, and the tutor checks that the hint is licensed by a valid next step from the student’s state. Without a solution graph, the check becomes a rubric or a constraint, and the guarantee weakens.
Effectiveness is separate. VanLehn’s (2011) meta-analysis compares human tutoring, intelligent tutoring systems, and other tutoring; the literature is more textured than the “two sigma” slogan. This lecture takes no position on the average effect. It claims only that step-level feedback can be checked against a domain model, and that check separates a pedagogical agent from a chatbot with a lesson plan.
Neural free-text grading, hint generation, and engagement signals. Symbolic a domain model with solution paths, a student model, and a pedagogical policy. Interface the model proposes; the tutor checks the proposal against the next licensed step and the prerequisite rules. Guarantee the feedback is consistent with the domain model for the student’s state; it is not a guarantee of learning. Limit a wrong graph teaches the wrong procedure, and a fluent hint can still harm.
Model tracing is only as good as its solution graph. If the graph encodes one procedure and the student uses another valid one, a correct step is marked wrong. The symbolic component does not remove the need for interpretability; it relocates it. The graph is now the object to inspect, and teachers should be among the inspectors.
06 / The common patternSix domains, one audit loop.
Put the six cases side by side: the differences are vocabulary, not form. Every one has a normative text, a case, a decision, a trace, and a human who owns the exception. The neural component sits at the boundary; the symbolic component is the part that can say why.
| Domain | Normative source | Symbolic artifact | Neural role | Guarantee |
|---|---|---|---|---|
| Cyber | Vulnerability and topology model | Attack graph and correlation rules | Alert classification | A path is valid under the model |
| Law | Statute, regulation, precedent | Rules with defaults; cases with dimensions | Clause extraction | The decision follows the encoded rule |
| Health | Guideline and formulary | Care pathway and interaction rules | Image and note models | The recommendation follows the guideline |
| Finance | Standards and policy | Ledger invariants and decision tables | OCR, matching, anomaly ranking | The decision follows the policy |
| Art | Grammar or composition rules | Rewrite or shape grammar | Style and aesthetic critic | The output belongs to the language |
| Education | Curriculum and solution graph | Model-tracing graph and hint policy | Grading and hint generation | The hint is licensed by the next step |
The pattern explains why a system can impress in a demo and fail in an institution. An institution cares about rejected, reviewed, and overridden cases: who decided, on which rule, with which evidence, and who signed. A system that cannot produce that record has not solved the institutional problem.
Accept if no rule blocks, or a named exception waives the only blocker. Reject if a blocking rule fires. Review if a required fact is missing or two rules conflict. Override is not a fourth verdict; it is a human event that accepts a rejected or reviewed case, logged with a reason. Collapsing review into accept or reject hides exactly the cases a human should see.
A rule engine can be sound — it derives only what follows from its rules — and still be wrong about the world, because the rules are wrong. It can be complete relative to its rule set and still miss an exception the rule set never encoded. That is the difference between a guarantee and a certification: the guarantee is conditional on the specification, a human artifact with an owner and a version.
07 / The tupleRead a case as a tuple.
The sections used the anatomy informally; the tuple makes it explicit. Write a case as:
N is the normative source and version; F is the case file with provenance; R is the rule set; ≺ is precedence; E is exception policy; d is the decision; τ is the trace; o is the override, a human event with a reason and an owner. The decision is a function of the first five; the trace is the evidence. An override changes the decision and is recorded alongside it, not inside it.
The tuple makes failure attribution possible: a wrong decision can come from a bad fact, rule, precedence, exception, or override. These are different bugs with different owners. A system that reports only “rejected” cannot distinguish them.
| Order | Rule | Result | Why |
|---|---|---|---|
| 1 | R3 · prohibited category | pass | meal is reimbursable |
| 2 | R5 · duplicate claim | pass | no earlier match |
| 3 | R6 · currency conversion | pass | USD; no conversion |
| 4 | R1 · receipt threshold | pass | receipt attached |
| 5 | R4 · travel pre-approval | not applicable | not a travel claim |
| 6 | R2 · meal cap | fail | $62.00 per attendee > $60.00 |
| — | E1 · meal exception | applied | MEAL-EXC waives R2 |
| — | R0 · default accept | skipped | decision came from R2 + E1 |
The trace is a small proof, but only relative to the rule set and facts. Five properties are worth naming separately:
| Property | What it means | How to test it |
|---|---|---|
| Deterministic | Same facts, rules, and policy → same decision | Replay the case; compare the trace |
| Sound | The decision follows from rules and facts | Re-derive the decisive rule |
| Complete | Every rule was evaluated and reported | Compare trace length with the rule set |
| Auditable | The record names facts, rules, exceptions, and the human | Give it to a reviewer who did not build the system |
| Contestable | A rejection or review has a path to a human decision | Override it; confirm the original remains visible |
Fairness is deliberately absent. A rule set can have all five properties and still treat people unjustly, because the rules can encode a disparity or the facts can be measured differently across groups. Fairness is a property of the policy and its effects, not of the engine. The audit trail is a prerequisite for asking, not the answer.
08 / RecapWhat you should be able to do now.
Explain why an attack path is a symbolic composition, and why correlation is abductive.
Distinguish encoded statute, case-based precedent, and an actuarial score.
Read MYCIN, INTERNIST/QMR, and a guideline engine as three symbolic cores.
Name the finance invariants a rule engine checks exactly — and what they miss.
Describe generate-and-criticize in art, and why a grammar cannot guarantee quality.
Explain model tracing and the interface that licenses an LLM-generated hint.
Write a case as (N, F, R, ≺, E, d, τ, o) and attribute a failure to a slot.
State the conditional guarantee: sound relative to rules and facts, not correct about the world.
Exercises
- Write five rules. Pick a policy you know. Write five rules, a precedence order, and two exceptions. For each exception, say whether it is a policy rule or a human override.
- Break the precedence. Check all claims in the lab’s default policy, then in strict mode. Which verdicts changed because of a rule, and which because of a data gap?
- Make an ambiguous case. The lab’s ambiguity is a missing FX rate. Construct one that is ambiguous because two rules conflict. What should the engine do?
- Audit an override. Override EXP-1046, then re-check it. What do the trace, the decision, and the audit log each say? Are they consistent?
Check your understanding
Ten questions, with explanations. The score is not the point; the questions you cannot answer are the reading list.
Reading list
The first six are the spine of the lecture. Classics without an arXiv identifier are cited by author, title, venue, and year.
Computer-Based Medical Consultations: MYCIN
Shortliffe · Elsevier, 1976 · rules, certainty factors, and the trace
Internist-1: An Experimental Computer-Based Diagnostic Consultant for General Internal Medicine
Miller, Pople & Myers · NEJM, 1982 · the large knowledge base
An Intrusion-Detection Model
Denning · IEEE Transactions on Software Engineering, 1987 · the statistical baseline
Automated Generation and Analysis of Attack Graphs
Sheyner et al. · IEEE Symposium on Security and Privacy, 2002 · paths as counterexamples
Constructing Attack Scenarios through Correlation of Intrusion Alerts
Ning, Cui & Reeves · ACM CCS, 2002 · hyper-alerts and abductive scenarios
Catala: A Programming Language for the Law
Merigoux, Chataing & Protzenko · ICFP, 2021 · literate law, default logic, executable rules
Modeling Legal Argument: Reasoning with Cases and Hypotheticals
Ashley · MIT Press, 1990 · HYPO, dimensions, and three-ply arguments
Comparing computer-interpretable guideline models: a case-study approach
Peleg et al. · JAMIA, 2003 · GLIF, Asbru, PROforma, EON
Cognitive Tutors: Lessons Learned
Anderson, Corbett, Koedinger & Pelletier · Journal of the Learning Sciences, 1995 · model tracing
The further exploits of AARON, painter
Cohen · Stanford Humanities Review, 1995 · rules, composition, authorship
A Neural Algorithm of Artistic Style
Gatys, Ecker & Bethge · arXiv:1508.06576, 2015 · content, style, and the critic
The figures are schematic redrawings, not reproductions. The lab’s rules, thresholds, and claims are illustrative fixtures; they are not legal, tax, or accounting advice. Systems are cited by author, title, venue, and year. The lecture makes no claim about deployments or benchmark scores that it does not state.