Lecture 15 · Act IV · Composition

Agents and world models.

A robot sees a window, remembers a map, imagines a path, and moves. The agent is a loop closed through a model — and the model is where the interfaces meet.

9 sections 3 world-model families 1 interactive lab Self-paced
00 · Learning objectivesLecture 15 · Agents and world models

By the end of this lecture

Four things you can do.

01

Trace the agent loop

Perceive, model, plan, act, monitor — and name the interface each stage uses.

02

Compare world models

Latent, symbolic, hybrid: what each one buys, and what each one costs.

03

Separate the layers

Task planning, motion planning, control — and the feasibility condition that links them.

04

Diagnose agent failures

Attribute a failure to grounding, model, planning, execution, or specification.

S/V · Lecture 15
00 · The hookLecture 15 · Agents and world models

A robot in a kitchen

One sentence. Six questions.

counter fridge 5 × 5 view “put the apples in the fridge”
The fridge is not in the window. The sentence still has to become torques.
Which apples? Grounding: words to objects, places, relations.
What happens if I open it? A world model: consequences of actions.
Did it work? Monitoring: prediction versus observation.

The instruction is not a plan. It is a specification that has to be grounded, modelled, planned, executed, and checked.

S/V · Lecture 15
01 · The agent loopLecture 15 · Agents and world models

The loop

Perceive, model, plan, act, monitor.

PERCEIVE sense · ground update belief MODEL predict · rollout latent / symbolic / hybrid PLAN search · optimise task · motion ACT execute · control close the servo loop MONITOR compare · detect replan or revise BELIEF b(s) what the agent thinks is true now THE MODEL IS THE INTERFACE · THE MONITOR KEEPS IT HONEST

The loop is the composition. Each box is a stage; each arrow is an interface from an earlier lecture.

S/V · Lecture 15
01 · The agent loopLecture 15 · Agents and world models

Composition

Every stage is an interface.

StageInterfaceQuestionFailure if omitted
PerceiveGrounding; logic as dataWhat is the world, given what I see?The plan is for the wrong state
ModelRepresentation; logic as architectureWhat would happen if I did a?The plan is for the wrong model
PlanLogic as searchWhich sequence reaches the goal?No plan, or a plan that cannot reach it
ActControl; architectureWhat command do I send now?The plan is not executable
MonitorLogic as verifierDid the world do what I predicted?The agent never notices
If you cannot name the interface, the stage is a black box. The composition is not a pipeline of models; it is a set of contracts.
S/V · Lecture 15
01 · The agent loopLecture 15 · Agents and world models

Interactive · model or no model

Same world, two agents.

A hand-built 16 × 10 grid. Both agents act in the same world; only one plans over it.
mode —
path —
status —

Choose a mode, then press Run. The reactive agent follows the distance gradient; the model-based agent computes a path.

S/V · Lecture 15
02 · World modelsLecture 15 · Agents and world models
02 / 08

What does the agent imagine?

Latent, symbolic, and hybrid world models — and what each one costs.

S/V · Lecture 15
02 · World modelsLecture 15 · Agents and world models

Definition

A model of consequences.

Latent

Learn the state

Encoder, transition, reward/value heads. Plan by rolling out in a continuous latent space. Dreamer, MuZero.

Symbolic

Write the state

Facts and operators with preconditions and effects. Plan by search over operators. STRIPS, PDDL.

Hybrid

Both, with a seam

Objects and relations, learned skills, symbolic constraints. Plan across the seam. SayCan, LLM+P.

A model that cannot answer “what would happen if?” is a representation, not a world model. The difference is counterfactual use at decision time.
S/V · Lecture 15
02 · World modelsLecture 15 · Agents and world models

Latent

Learn the state; imagine the rollout.

observation o_t encoder q(z | o) latent z_t state transition p(z′ | z, a) heads reward · value · o′ imagination: repeat with the next action
World Models (VAE + MDN-RNN), Dreamer (actor-critic in imagination), MuZero (value-equivalent model + MCTS).
p(zt+1, ot, rt | zt, at) = p(zt+1 | zt, at) · p(ot | zt) · p(rt | zt)

What it buys: scale and no hand-written state. What it costs: opacity, distribution dependence, and no guarantee that a good reconstruction is a good plan.

S/V · Lecture 15
02 · World modelsLecture 15 · Agents and world models

The cost of imagination

One-step error, multiplied.

‖fH(s) − f̂H(s)‖ ≤ ε · (1 + L + ⋯ + LH−1) = ε · (LH − 1) / (L − 1)
If the true and learned transitions are L-Lipschitz and the one-step error is at most ε. For L = 1 the bound is εH.

Longer horizon

H grows; the error compounds. Planning far ahead is a bet on the model.

Optimistic model

The optimiser finds the trajectory that looks best under the model, including its mistakes.

The loop

Re-observe and replan. Closed-loop correction is what keeps the bound from being the whole story.

The bound is worst-case and open-loop. A monitor can make the real error much smaller — but it cannot make the model true.
S/V · Lecture 15
02 · World modelsLecture 15 · Agents and world models

Symbolic

Facts, operators, effects.

state s on(cup, table) holding(hand, cup) … operator a pre(a) ⊆ s add(a) del(a) applicable? state s′ (s \ del) ∪ add … successor plan search
STRIPS add/delete lists; PDDL adds types, quantifiers, conditional effects, and durative actions.

The plan is checkable relative to the model. That is the guarantee — and its limit. A sound and complete planner proves the goal in the model, not in the world.

Explicitness is not completeness. The domain can be readable, inspectable, and wrong.
S/V · Lecture 15
02 · World modelsLecture 15 · Agents and world models

The boundary

Frame, qualification, ramification.

Frame

What does not change?

A model must say what an action leaves alone. The “alone” is enormous. McCarthy & Hayes, 1969.

Qualification

What else must hold?

Preconditions are never exhaustive: the door can be locked, blocked, too heavy, or opening inward.

Ramification

What else changes?

Actions have indirect effects: moving a cup changes reachability, visibility, and support.

All three are reasons a hand-written model is expensive. They are not bugs in a planner; they are the cost of formal representation.

The same three problems reappear in a learned model as distribution shift, spurious correlation, and unmodelled state.

S/V · Lecture 15
02 · World modelsLecture 15 · Agents and world models

Hybrid

Put a seam inside the model.

Objects + relations

A detector builds a scene graph; a planner or program operates over it. NS-CL.

Language + planner

An LLM writes a PDDL problem; a classical planner solves it; errors feed back. LLM+P.

Skills + task layer

A task planner sequences learned skills; each skill has a precondition and an effect. SayCan.

Dynamics + constraints

A neural proposal is checked by collision, kinematic, or temporal-logic monitors.

Hybrids relocate the interface problem. A detector labels a closed door as open; the fact is used; the plan is valid; the robot collides. Each component can be correct and the system still fail.
S/V · Lecture 15
03 · Planning & controlLecture 15 · Agents and world models
03 / 08

Three layers, three clocks.

Task planning, motion planning, control — and the feasibility condition that links them.

S/V · Lecture 15
03 · Planning & controlLecture 15 · Agents and world models

The stack

What, where, how.

LayerStateDecisionRateSuccess
Task planningFacts and objectsWhich operator nextSeconds to minutesGoal formula satisfied
Motion planningContinuous configurationCollision-free path0.1–10 sPath exists and is free
ControlContinuous stateTorque or velocity100–1000 HzStable tracking
A task plan is a sequence of names for motions that have not been checked. A motion plan is a geometric object that has not been checked against the task.

The layers have different notions of “possible.” The interfaces must reconcile them.

S/V · Lecture 15
03 · Planning & controlLecture 15 · Agents and world models

Task planning

Search over operators.

start goal
A* explores by f(n) = g(n) + h(n); the heuristic is the distance estimate.
Domain: operators with preconditions and effects. Problem: initial state + goal. Plan: a sequence or a partial order.
Search: A* with an admissible heuristic; delete-relaxation heuristics make it practical.
Validation: check the plan against the model without executing it. A real guarantee — about the model.

f(n) = g(n) + h(n) · A* is optimal with an admissible heuristic, efficient with a consistent one.

S/V · Lecture 15
03 · Planning & controlLecture 15 · Agents and world models

Motion + control

Continuous, collision-free, closed-loop.

start goal configuration space
A path can be collision-free and dynamically infeasible: too fast, too tight, or unreachable.
Motion planning: RRT, PRM, trajectory optimisation. The collision model is a world model.
Control: PID or MPC at the servo rate. It tracks a reference and rejects disturbances.
a*0:H−1 = argmin E[ Σ c(sk, ak) + cf(sH) ]

Receding horizon: optimise, execute one step, observe, optimise again.

S/V · Lecture 15
03 · Planning & controlLecture 15 · Agents and world models

The interface

A plan must be realisable.

Task

grasp · move · place

A sequence of operators. It knows the door is a precondition — if the domain says so.

Motion

pose · path · reach

A collision-free trajectory. It knows the cup is behind a chair; the task layer may not.

Control

track · force · slip

A feedback law. It knows the cup is heavy; the motion plan may not.

The cupboard door is closed: a missing precondition, not a search failure. The plan is valid symbolically and impossible physically. TAMP solves the layers together because each constrains the other.

The feasibility condition: there exist trajectories that connect, stay collision-free, and end in each action’s geometric goal. If not, the task plan is a wish.

S/V · Lecture 15
04 · Language agentsLecture 15 · Agents and world models
04 / 08

Language proposes. The world disposes.

SayCan, ReAct, LLM-Modulo, and the end-to-end contrast.

S/V · Lecture 15
04 · Language agentsLecture 15 · Agents and world models

Language as a scorer

Multiply semantics by affordance.

score(a | i, s) ∝ pLM(a | i) · V(a | s)
Language model probability of the skill text × learned value of success from the current state. Ahn et al., arXiv:2204.01691.
Candidate skillLanguage fitAffordanceSelected?
find spongeHighLow (none visible)Not yet
wipe tableHighLow (no sponge)No
look aroundMediumHighYes

Greedy selection, no backtracking. The value function is the grounding; the language model is the prior. Neither is a guarantee.

S/V · Lecture 15
04 · Language agentsLecture 15 · Agents and world models

Language as a trace

Thought, action, observation.

THOUGHT “open the fridge first” ACTION tool call · skill OBSERVATION result · error · state REVISE next thought repeat until done or budget exhausted
Yao et al., arXiv:2210.03629. The trace is a scratchpad; the observation is the grounding.
What it buys: feedback-driven revision. The plan can change when the world answers.
What it does not: guarantee a faithful trace, a safe action, or termination. It can loop or drift.
S/V · Lecture 15
04 · Language agentsLecture 15 · Agents and world models

Two opposite designs

Who checks the plan?

LLM-Modulo

Propose, check, repair

An LLM generates candidate plans. External critics — a planner, a simulator, a formal verifier — check them. The LLM repairs. The critic is the guarantee.

Kambhampati et al., arXiv:2402.01817

VLA · RT-2

Instruction to action

A vision-language model emits action tokens directly. No plan, no map, no symbolic model. The prior is the policy.

Brohan et al., arXiv:2307.15818

The guarantee lives where the check lives. If nothing checks the plan, the agent has a prior and a hope.
S/V · Lecture 15
04 · Language agentsLecture 15 · Agents and world models

Failure modes

Five ways a sentence fails.

FailureExampleDetection
Referent“the red cup” when there are twoCross-view consistency; ask
AffordancePlans to wipe with a sponge that is not thereValue function; perception
Precondition“put in fridge” without opening the doorPlan validator; monitor
Ambiguity“clean the spill” with no standardHuman feedback; clarification
DriftA small error changes the state; the plan does not noticeObservation; re-grounding
Language is a specification, not a goal. The literal instruction can be satisfied while the user’s goal is not.
S/V · Lecture 15
05 · Memory & toolsLecture 15 · Agents and world models

Memory and tools

What the agent carries and calls.

Working

The current belief, plan, and context. Finite, and easy to overflow.

Episodic

Past trajectories and outcomes. A retrieved episode is evidence, not a rule.

Semantic

Facts, maps, attributes. A map is a belief, not the world.

Procedural

Skills and policies. A skill has preconditions; a library does not check them.

A tool is a contract: preconditions, inputs, outputs, cost, failure. The guarantee moves only if the tool has one. A tool call is an observation, not a fact.

Taxonomy: CoALA, Sumers et al., arXiv:2309.02427.

S/V · Lecture 15
06 · EvaluationLecture 15 · Agents and world models

Evaluation

Measure the loop, not the answer.

Success — did the goal hold?
Cost — steps, path, energy, interventions.
Generalisation — new instructions, objects, layouts.
Recovery — detection and time to recover.
Calibration — does predicted success match observed?
BenchmarkTests
ALFWorldText plans → embodied execution
ALFREDLong-horizon grounded instructions
BabyAIGrid-world compositional language
EAIModule-level diagnosis
Each benchmark is a model of a task; each has a blind spot.

Prediction accuracy is not planning performance. A model can reconstruct and still be useless for control.

S/V · Lecture 15
06 · EvaluationLecture 15 · Agents and world models

Diagnosis

Four modules, not one score.

Goal interpretation

Did the agent read the instruction correctly?

Subgoal decomposition

Did it break the goal into the right intermediate steps?

Action sequencing

Did it order the actions correctly in the current state?

Transition modelling

Did it predict what each action would change?

Final success conflates the modules. Module-level metrics attribute the failure to a stage — which is the difference between a score and a diagnosis.

Li et al., arXiv:2410.07166.

S/V · Lecture 15
07 · Failure modesLecture 15 · Agents and world models

The sharp version

A* is correct. The map is wrong.

Soundness and completeness are properties of the planner relative to its model. They are not properties of the agent relative to the world.

Monitoring is the gap between the model and the world.

S/V · Lecture 15
08 · RecapLecture 15 · Agents and world models

What you can do now

Four moves, one lab.

Trace

Perceive, model, plan, act, monitor; name the interface at each stage.

Compare

Latent, symbolic, hybrid: state, transition, cost, guarantee.

Separate

Task, motion, control; a plan is realisable only if the layers agree.

Diagnose

Grounding, model, planning, execution, specification.

S/V · Lecture 15
Speaker notes