Lexicon

A

Adversarial — The concept of setting up some work to be done in opposition to the “positive” result of the outcome that we actually want, in order to determine whether the work was in fact finished. “Poke holes in this work according to these criteria”. It is the stance we ask a validator to take toward a doer’s output.

Agent — A model placed inside a harness, so it can take action in the world — read files, run tests, query databases, call APIs — rather than only produce text. The model supplies judgment; the harness manages its lifecycle (loop, tool dispatch, permissions, stopping); a skill can define its process. Contrast a chatbot, whose world ends at the edge of its context window.

Agentic loop — Repetition of the AI sandwich: prepare context → model reasons or calls a tool → process output → feed the result back → repeat, until a final answer or a stopping condition. The loop is controlled entirely by the harness, not the model.

AI sandwich — The structural pattern of every agentic system: deterministic code bookends a model call. The top slice prepares context (prompt, history, tool definitions); the filling is the model inference (reasoning, synthesis, judgment — where you have least control); the bottom slice processes output (parsing, validation, action, error handling — where correctness lives). Your code is the bread; the model is the filling.

Andragogy — Malcolm Knowles’s model of adult learning (vs. pedagogy, child learning). Adults learn best under six conditions: self-direction, use of prior experience, readiness to learn (“why should I learn this?”), a problem-centered orientation, internal motivation, and immediate relevance to their work.

Assembly line — An ordered sequence of machines, arranged as a directed graph: each machine’s output becomes the input of the next, and the graph may branch or run paths in parallel. The order is fixed by whoever designed the line, not chosen by the machines while it runs. An assembly line is always part of a factory, which is what runs it — see factory. A pipeline is the simplest case — a straight line with no branching.

C

Calibration (trust calibration) — The judgment skill of knowing when to trust AI output and when to verify it. Because the tools are probabilistic, we teach calibration rather than blanket trust.

Cognitive load — From working-memory research (Sweller): the limit on how much a person can process at once. Justifies keeping exercises short and raising difficulty gradually.

Compaction — A dedicated summarization step — a model call whose only job is to compress prior history — that extends the life of a long-running conversation. A practical trigger is to compact at a fraction of the window (e.g. 70%), not at the ceiling.

Context editing (context trimming) — Pruning stale tool results and old reasoning from the transcript as a loop runs, to keep per-call input lean without summarizing.

Context rot — The degradation of model performance as context grows. Bigger is not better: a longer window is not a free win.

Context window — The total amount of text a model can “see” in a single call — system prompt, conversation history, documents, tool definitions, tool results, and the model’s own response. It is the model’s working memory: if something is not in the window, the model does not know it exists.

D

Defensive prompting — Building reliability into prompts: be explicit about output format, include negative constraints (“return only the code block”), and use structured-output features that constrain the model at sampling time.

Deliberate practice — Anders Ericsson’s concept of practice that drives improvement, distinct from ordinary repetition. Four traits: specific well-defined goals, immediate accurate feedback, a slight stretch beyond the comfort zone, and repetition with variation.

Doer — The agent that does the job. It produces the work product — the code, the plan, the document, the edits — and nothing else; checking the result is the validator’s job. We keep the two separate because a model that has just produced an answer is bad at finding its own errors in it. This is our canonical name for the role; older material and the wider industry also call it the worker, the actor, or agent A.

Doer-validator loop — The pairing run repeatedly: the doer produces output, the validator checks it against the criteria for done, the findings go back to the doer, and the cycle repeats until the validator has nothing significant left to report. In practice this settles within a few passes, but set a maximum iteration count anyway — a doer and validator can oscillate, each pass introducing a new problem.

Dreyfus model of skill acquisition — Five stages a learner moves through in any domain, each with different needs: Novice (follows context-free rules), Advanced beginner (recognizes situational patterns, needs maxims), Competent (deliberate planning, needs decision frameworks), Proficient (intuitive perception, deliberate response), Expert (fluid intuitive action; needs edge cases and chances to teach).

E

Eval — A test for AI behavior: does the agent produce correct outputs, behave safely, complete the task, and stay within bounds? The unit-testing instinct applied to a probabilistic system. Main kinds: rule-based (exact match / regex / structure checks — fast, brittle), model-graded (a second LLM judges quality), human-in-the-loop (a person labels outputs — the ground-truth source), and behavioral / end-to-end (did the agent accomplish the goal, not just emit good-looking text).

Eval-driven development — TDD applied to agents: write the eval first to define correct behavior, run it against current behavior, improve the agent, re-run, iterate. A passing eval means behavior is probably reliable at some confidence level, not guaranteed.

External selection channel — A model-independent check that can actually break a shared blind spot: executable tests, type checkers, proof checkers, numeric invariants, a ground-truth lookup, or even the same model run in fresh context. “External” refers to context and mechanism, not just vendor.

F

Factory — A piece of software containing one or more assembly lines and the orchestrator that manages them. The factory is the unit you build, deploy, and operate; the machines inside it are components of it rather than things you ship on their own.

Four Cs (Training from the Back of the Room) — Sharon Bowman’s workshop design sequence: Connections → Concepts → Concrete Practice → Conclusions. Use it as a facilitation check: connect learners to relevance, teach only what they need for the next move, let them practice, then have them make meaning and choose a next step.

Freshness (stale index) — The RAG concern that vector indexes go stale and produce confidently wrong answers from outdated content. For rapidly changing data, tool-based retrieval (always current) beats indexed retrieval.

G

Grounding — Anchoring an agent’s answers in actual, specific context — your code, docs, systems, history — via retrieval. The difference between an agent that sounds smart and one that is actually useful.

H

Hallucination — When a model generates plausible-sounding but factually wrong content (invented citations, wrong API signatures, fabricated dates). Not lying: the model is producing statistically likely continuations as trained.

Harness — The code and infrastructure wrapping a model call — everything except the model itself. The moment you add input preparation, output handling, error handling, retries, routing, permissions, logging, or stopping conditions around a raw API call, you have a harness. It is ordinary code around an extraordinary component, and it is what turns a proof-of-concept call into a workflow you can trust. Common shapes: simple wrapper, loop harness, pipeline harness, and multi-agent orchestrator. To understand an agent, understand its harness.

Human escalation — The harness-defined conditions under which the system stops and asks a human: low confidence, unexpected output shape, a cost threshold, or a domain-specific heuristic.

Human-in-the-loop — A review checkpoint inserted before high-stakes, side-effecting actions (sending email, writing to a database). The model saying it will do something is not the same as it being done, so those steps are kept separated. Also the gold-standard source of eval ground truth.

I

Identity threat — The reaction AI tools can trigger in experienced engineers (“Is my expertise being devalued? Will I become obsolete?”). Not irrational, so dismissing it breeds defensiveness; best addressed by framing AI as an added capability layer, not a replacement.

Idempotency — The property that an operation can be safely repeated: a harness can be re-run after a partial failure, and calling the same tool twice with the same arguments is safe. Design for restartability from the start.

Iteration — A bounded batch of work handed to an agent between check-ins, sized to a task rather than a clock. You (or the agent) scope what’s in for this pass, let it run, then review before deciding the next one. Distinct from the agentic loop (the model-tool cycle inside a single call) and from sub-agent (a separate context window): iteration is the human-facing unit of how much to hand off before looking again. The word is overloaded in the wild, so confirm what scope is meant when it’s ambiguous.

Input/output cost asymmetry — Output tokens cost several times more than input tokens (typically 3–6×), because generation is autoregressive (one pass per token) while input is processed in parallel. A long prompt is usually cheaper than a long response.

J

Judge (LLM judge / judge agent) — A second model that scores or selects output: the grader in a model-graded eval, or the arbiter in a tournament. Distinct from a validator, which checks one piece of work against its criteria; a judge compares candidates or assigns a score. A judge is only as good as its prompt, which is itself a skill that needs testing.

K

Kolb cycle (experiential learning) — David Kolb’s four-stage cycle in which learning sticks: Concrete Experience → Reflective Observation → Abstract Conceptualization → Active Experimentation → (back to experience). Reflection is the stage most workshops skip and where pattern recognition begins.

L

Lens — A focused angle of attention brought to a wall of text: security, performance, architecture, reliability, UX, maintainability, or product risk. Choosing a lens narrows what you look for, so the signal you care about surfaces instead of drowning in everything else.

M

Machine — An agent running in a non-interactive harness: it is handed its inputs, runs to completion, and returns a result, with no human in the conversation while it works. Non-interactivity is the whole of the distinction — see agent for everything else. The internals can be a model call or ordinary deterministic code; from the assembly line’s point of view it makes no difference. A machine can be a doer or a validator.

MCP (Model Context Protocol) — An open standard (created by Anthropic) for connecting models to external systems, data, and tools, replacing the M×N custom-integration problem with one protocol. An MCP host (the AI app) runs one MCP client per connected MCP server; servers expose three primitives — tools (executable functions), resources (read-oriented data), and prompts (reusable templates) — over stdio (local) or streamable HTTP (remote), with runtime capability discovery.

Memory systems — Injecting summaries or records of past interactions so an agent has continuity across sessions instead of starting from scratch each time. A form of RAG.

Model diversity — There is no single “the AI.” Frontier models are not interchangeable; the same prompt often yields materially different answers. Which model you pick is itself a design decision. We use the three big brains — Claude (Anthropic), Gemini (Google), ChatGPT/GPT (OpenAI) — as shorthand to break the single-oracle mental model.

Model drift — Behavior changing when the model version changes underneath you. The reason prompts must be re-tested on version changes and periodically.

Model routing (cascades) — Sending each task to the cheapest model that can handle it, often as a cascade: try a cheap model first, escalate to a stronger one only when confidence is low or a check fails. Requires a reliable signal for “not good enough.”

Model tiers — The flagship / mid / small ladder (e.g. Opus / Sonnet / Haiku). The small-fast tier is often 5× cheaper or more; picking the right-sized model is the cheapest cost “feature” available.

N

Need-to-know concepts — The minimum vocabulary, model, example, or demo learners need to do the next practice or make the next decision. Everything else is nice-to-know and belongs in references, details, or a later debrief.

O

Orchestrator — The part of a factory that manages its assembly lines: starting a line, handing each machine its inputs, choosing what runs next where the graph branches, handling failures and retries, and deciding when the line is finished. It does not do the work itself; it decides which machine does.

Overfitting evals — When an agent aces the eval set but fails in production because it was tuned to the eval cases rather than the underlying behavior. Mitigated by held-out cases, evals drawn from real usage, and rotating in new cases.

P

Permission model — The tiers a harness uses to gate tool calls: always allow, ask once per session, ask every time, never allow. Design it before deployment; tightening permissions after users build workflows on permissiveness is much harder.

Pipeline — As a multi-agent pattern: a doer produces output, a validator reviews it, findings go back to the doer or a human — sequential, correctness over speed. As a harness: a fixed sequence of agents where each output is the next input, with deterministic routing so every input runs every stage — an assembly line with no branching.

Projection — Re-rendering the same information into a shape a human can actually use, such as a table, diagram, map, lifecycle, flow, or marked-up page, instead of reading it in its raw form. The content does not change; only its shape does, to match how a person reads.

Prompt framing (wording sensitivity) — Because the model is a context-completion engine, how a task is framed — persona, wording, order of information — is itself a major input that strongly affects output. Prompt changes are system-behavior changes: track, test, and version them.

Psychological safety — Amy Edmondson’s concept: a prerequisite for group learning. Participants must feel that trying and failing won’t damage their standing among peers — especially important with probabilistic tools that produce embarrassing results.

R

RAG (Retrieval-Augmented Generation) — Any process that dynamically adds context-relevant information to a prompt at runtime. The mental model is “find the right stuff → put it in the prompt → ask the question.” The mechanism is an implementation detail and includes: classic RAG (vector embeddings + similarity search), keyword / full-text search (BM25), tool-based retrieval (the agent calls a search/API/DB tool — dominant in agentic systems), structured retrieval (query a database, inject the rows), file injection (pass a file straight into the prompt), and memory systems. We use this broad definition deliberately: if it dynamically shapes the prompt with relevant information, it’s RAG.

Relevance vs. context bloat — The RAG tradeoff: more retrieved content raises the chance of including what you need, but also raises cost, noise, and latency and can bury the signal. The goal is precision, not volume.

S

Selective inclusion — Including only what’s relevant to the current step, rather than adding content out of habit or caution. The core discipline of context management.

Self-efficacy — Albert Bandura’s theory that belief in one’s own ability predicts persistence. Early small wins predict willingness to push through harder challenges, so opening exercises must leave participants feeling capable.

Semantic interface — The LLM programming model: natural-language (semantic) input in, natural-language output out, with the contract defined by meaning rather than structure. Powerful (expresses hard-to-formalize tasks) but fuzzy by definition, so it requires empirical testing and degrades unpredictably rather than failing cleanly. A key consequence: the model generates what is statistically plausible, not necessarily what is true.

70-20-10 principle — Effective professional development draws roughly 70% from challenging on-the-job experience, 20% from social learning (feedback, coaching, peer discussion), and 10% from formal instruction.

Skill — A markdown file that encodes a process — a structured, step-by-step way to approach a class of problem — which the agent reads and follows (often invoked via a slash command like /review-pr). A skill is not a task; it separates the task (“review this PR,” given at invocation) from the process (“here’s how we review PRs,” encoded once). A well-formed skill states its goal, steps, criteria for done, and when to escalate to a human. Skills are crystallized prompt engineering, kept in the repo and improved via pull requests. Tools are the hands; skills are the mind that directs them.

Structured outputs — A schema-constrained response, tighter than free prose. Serves double duty: an output-token cost control and an output-validation technique.

Structured prompts — Writing system prompts like dense internal documentation — clear, no fluff, no redundancy — so every sentence earns its place. About token efficiency, not terseness for its own sake.

Sub-agent — A separate model call with its own fresh context window. Decomposing a task across sub-agents keeps each one’s context small and relevant, at the cost of coordination overhead and more total calls.

T

Token — The atomic unit of model input and output: a subword piece, roughly 4 characters of English or about ¾ of a word. Tokens are the unit of both capacity (how much fits in the context window) and cost (you pay per token, on both input and output).

Tool — A named, typed capability the model can invoke to act in the world or retrieve information it doesn’t already have. Tools are what turn a language model from a text autocompleter into an agent.

Tool schema (tool definition) — A tool’s name, description, and JSON parameter schema, provided to the model as context. The schema is the contract for valid inputs — like defining an RPC interface where the caller is non-deterministic.

Transfer — Applying learning on the job. Its strongest predictor is how closely training content resembles the actual work — the case for exercises built from participants’ real scenarios.

V

Validator — The agent that verifies the job was done satisfactorily. It is given the doer’s output plus the original criteria, and reports what is wrong, missing, or unsupported. Because it did not write the work, it has nothing to defend, so it looks for problems instead of justifying choices. Different mandates find different things — check the output against the spec, try to falsify a specific claim, attack it as a hostile user would, or argue the whole approach was the wrong one. A validator can be wrong too, so it should report issues with evidence rather than asserting corrections. This is our canonical name for the role; older material and the wider industry also call it the critic, the reviewer, the verifier, or agent B.