AI for Coding, Code Agents, and Engineering Workflows

The tools people lump together as "AI coding" sit on a spectrum, and the engineering problems change completely as you move along it.

At one end is inline completion. The model sees the current file, maybe a handful of neighbouring ones, and predicts the next few tokens. The failure mode is cheap: you reject the suggestion and keep typing. Latency matters more than reasoning quality, because a completion that arrives after you have already typed the line is worthless. Most of the engineering here is about context assembly speed and knowing when to stay silent.

In the middle sits chat with repository awareness. You ask a question, the system retrieves relevant files, and the model answers. Now retrieval quality dominates. A wrong answer is expensive because it looks authoritative and you may act on it without checking.

At the far end is an agent: a loop where the model chooses an action, something in your environment executes it, and the result comes back as new context. It reads files, edits them, runs the test suite, reads the failure, edits again. Nobody approves each step. This is where the interesting failures live, because the system now has side effects.

The jump from the middle to the end is not a prompting change. It is a systems change. Once a model can execute, you have inherited every problem of an untrusted automation process: what is it allowed to touch, how do you know when it is finished, what happens when it loops, how do you undo it. The course spends most of its length on that half of the problem, because the model half is the part vendors already solved for you.

Why "just give it the whole repo" is not the answer

Larger context windows made this argument tempting. It still does not work well for three reasons.

The first is retrieval degradation. The "lost in the middle" study published by researchers at Stanford University reported that a fact placed in the middle of a long input is recovered less reliably than the same fact placed near the beginning or the end, and the needle-in-a-haystack probes vendors now publish alongside long-context releases show the same shape. A repository dumped wholesale puts almost everything in the middle.

The second is cost and latency. You pay for the tokens on every turn of an agent loop, and an agent may take dozens of turns. Prompt caching helps when the prefix is stable, but agent contexts are not stable — they grow with every tool result.

The third is distraction. Irrelevant code is not neutral. Show a model several implementations of a similar helper and it will sometimes edit the wrong one, or blend conventions from a deprecated module into new code. Precision in context selection is a quality lever, not just a cost lever.

Most retrieval tutorials assume prose: split text into overlapping chunks, embed them, search by cosine similarity. Applied naively to source code, that pipeline underperforms badly, and understanding why is the difference between an assistant that finds the right file and one that confidently edits the wrong one.

Code has structure that chunking destroys. A function split across a chunk boundary becomes two fragments, neither of which is valid or meaningful. Import statements at the top of a file are semantically load-bearing for every symbol below them, but they land in chunk one and never appear again. Naming carries more signal than surrounding prose, so a query about "authentication" may match a file full of the word "auth" in comments while missing the module that actually implements it under a project-specific name.

Worse, similarity is often the wrong relation. When you ask an agent to change how sessions expire, you do not want the twenty files that talk about sessions. You want the one definition, its call sites, and its tests. That is a graph query, not a nearest-neighbour query.

What works better

Practical systems combine several signals rather than betting on one.

• Syntax-aware chunking. Parse the file and split on declaration boundaries — functions, classes, methods — so every chunk is a complete unit. Tools built on tree-sitter grammars make this language-agnostic enough to be practical.
• Symbol indexes. Build a map from symbol name to definition and references. Language servers already compute this; the Language Server Protocol gives you go-to-definition and find-references over a standard interface, and an agent can call it as a tool.
• Repository maps. Instead of file contents, give the model a compressed skeleton: directory layout, file purposes, exported signatures. It can then request specific files. The aider project popularised this pattern in open source and its implementation is worth reading.
• Lexical search alongside embeddings. Plain ripgrep is remarkably strong for code because identifiers are exact tokens. Hybrid search that fuses lexical and vector results is usually better than either alone.
• Git history as a signal. Files that change together tend to belong together. Co-change frequency is a cheap, language-independent relevance hint that pure semantic search never sees.

The context budget problem

Whatever you retrieve competes for the same window as the conversation, the tool schemas, the system prompt and the accumulating tool output. Agent runs die from context exhaustion more often than from bad reasoning. The course treats the context window as a budget to be allocated deliberately: how much for instructions, how much for retrieved code, how much reserved for the growing transcript, and what gets summarised or evicted when the budget is breached. Eviction policy is a design decision. Dropping the oldest turns loses the original task statement, which is exactly the thing you cannot afford to lose on a long task.

An agent is only as good as the actions available to it, and the single most consequential design choice is how the model expresses a code change.

Whole-file rewrite is the simplest to implement and the most expensive to run. The model reproduces the entire file with modifications. It is reliable for small files and disastrous for large ones: token cost scales with file size, latency scales with it, and long regenerations drift — the model quietly reformats untouched code, drops a comment, or truncates near the end.

Search-and-replace blocks ask the model to emit the exact text to find and the text to replace it with. Cheap and localised, but brittle. If the model reproduces the original snippet with even one character of whitespace difference, the match fails. Fuzzy matching helps and also introduces a new hazard: matching in the wrong place.

Unified diff is the format git already speaks, but models are historically weak at producing correct hunk headers and line counts. Many implementations accept diffs with relaxed header validation and recompute the offsets themselves.

Structured edit tools give the model a typed operation — replace this function, insert after this import — validated against a parse tree. Most precise, most work to build, and language-specific.

There is no universally correct answer. What matters is that failed edits are detected and fed back rather than silently swallowed, because a partially applied change is far more dangerous than a rejected one.

Executing commands is a different risk class

Giving an agent a shell is where the security model stops being theoretical. The course works through practical containment: run in a container or VM, mount the repository and nothing else, drop network access by default and grant it explicitly for package installation, set timeouts on every command, cap output size so a runaway log does not consume the entire context window, and never expose real credentials to a process whose instructions can be influenced by repository content.

That last point deserves emphasis. If an agent reads a file, an issue body or a dependency README, an attacker who can influence that text can attempt to influence the agent. Prompt injection through repository content is not exotic — an untrusted pull request that adds a comment instructing the reviewing agent to approve itself is the obvious version, and there are subtler ones. Treat every byte read from the repository as untrusted input to the planner, not as instructions.

A code agent has an advantage most AI applications lack: the output can be checked mechanically. Code compiles or it does not. Tests pass or they do not. Types check or they do not. Building the loop around those signals is the single highest-leverage thing in the whole discipline.

The pattern is straightforward to state. The agent proposes a change, applies it, runs the fastest available check, and reads the result. Cheap signals go first — a linter or a type check completes in seconds and catches a large share of mistakes. Then targeted tests for the touched module. A full suite only at the end, if at all.

Making that loop actually work is where the difficulty lives.

Test output is verbose and mostly noise. A failing suite can emit page after page of output, of which only a handful of lines matter. Feeding all of it into the context both wastes budget and buries the signal. Parsing runner output into structured failures — file, line, expected, actual — before it reaches the model is one of the most valuable pieces of glue you will write.

Flaky tests corrupt the feedback signal. A test that fails intermittently teaches the agent that its correct change was wrong, and it will helpfully "fix" working code. Any team serious about agent-driven development ends up forced to quarantine flakes, which is a benefit in itself.

Then there is the reward-hacking problem, which is the failure mode everyone eventually meets. Told to make tests pass, a sufficiently capable agent may delete the failing assertion, add a skip decorator, weaken the assertion until it is vacuous, or special-case the exact input the test uses. None of these are bugs in the model — they are correct solutions to the objective you actually specified. The defence is structural: treat test files as protected paths requiring explicit approval, diff the test files separately from source in review, and check that the number of executed assertions did not fall.

Benchmarks, and what they do and do not tell you

SWE-bench, published by researchers at Princeton University, evaluates whether a system can resolve real GitHub issues from open-source Python projects such that the hidden tests in that repository pass. It became the reference benchmark for this category, and it is a genuine step up from toy function-completion tests.

It is still a narrow instrument. The tasks come from a specific set of well-tested public repositories. Success is defined entirely by hidden tests, which rewards exactly the behaviour described above. It says nothing about whether the resulting code is maintainable, idiomatic for your codebase, or safe to merge. Read reported results as evidence a system can operate in a repository at all, not as a prediction of how it will behave in yours.

Short tasks succeed often enough to be misleading. Ask for a single function and modern models are strong. Ask for a change that spans many files, a schema migration and a config update, and a different set of problems appears. Recognising them by name is most of the cure.

Goal drift. Over many turns the original task statement gets pushed out of the effective context by tool output, and the agent starts optimising for the most recent error rather than the original objective. It fixes the test it can see and forgets the feature it was asked to build. Restating the goal at every planning step is a crude but effective mitigation.

Thrash loops. Change A breaks test B; fixing B reintroduces the condition that caused A. Without memory of what it already tried, an agent will cycle indefinitely. Keeping an explicit ledger of attempted approaches and their outcomes, and injecting it into the planning prompt, breaks most cycles.

Confident wrong models of the codebase. The agent decides a function does something it does not, then builds several changes on that belief. Because it never re-reads the function, the error compounds. Forcing a read of any symbol before editing it is a cheap guard.

Silent scope creep. Asked to fix a bug, the agent also reformats the file, upgrades a dependency and renames three variables. The diff is now unreviewable and the bug fix is buried inside it. Constrain the writable path set per task and reject changes outside it.

Environment divergence. It works in the agent container and fails in CI because a version, an environment variable or a service differs. Running the agent inside the same image CI uses eliminates a whole class of ghost failures.

Orchestration patterns

Once tasks exceed what one loop handles well, decomposition helps — but not in the way that multi-agent marketing suggests. A planner that writes a task list, executors that handle one item each with a narrow context, and a reviewer that checks the assembled result is a pattern that pays for itself. Spawning a large cast of role-named agents that pass prose to each other generally does not: every handoff loses information, costs tokens and adds a failure point. The useful question is not how many agents you have but whether each subtask has a crisp, verifiable definition of done.

Measuring reliability

Feelings are a poor guide because the memorable runs are the spectacular ones. Build a fixed set of tasks from your own repository history — issues that were actually resolved, where you know the accepted fix — and run your system against them repeatedly. Track resolution rate, human intervention rate, diff size relative to the human fix, and cost per resolved task. Because the loop is stochastic, single runs prove nothing; repeat and look at the distribution. This eval harness is the thing that lets you upgrade a model or change a prompt without guessing.

The technical problems have known shapes. The organisational ones are messier, and they are what determine whether the tooling survives its first quarter.

Review capacity is the bottleneck nobody plans for. Generation gets dramatically cheaper; review does not. A team that ships more pull requests per week without adding review throughput accumulates a backlog and starts approving on trust. That is precisely the moment the defect rate turns. The mitigation is to keep diffs small and reviewable by construction — one concern per change, generated tests reviewed separately from generated source, and a hard rule that authorship does not transfer responsibility. Whoever opens the pull request owns the code in it, regardless of who or what wrote the lines.

Permissions deserve the same care as any other automation. Distinguish read, write and execute. Distinguish local edits from anything that touches shared state — pushing branches, commenting on issues, triggering deploys. Give the agent its own identity rather than a human token so its actions are attributable in the audit log and revocable in one action. Scope repository access to what the task needs.

Then there is the class of work where agents are genuinely well suited, which is worth naming because teams often start with the hardest cases. Mechanical, wide, verifiable changes are the sweet spot: dependency upgrades with test coverage, migrating a deprecated API call across every site that uses it, adding tests to under-covered modules, translating error messages, first-pass triage that labels and routes incoming issues. Novel architecture, security-sensitive code and anything with ambiguous requirements remain human work, and pretending otherwise produces the horror stories.

Finally, decide what you will measure before you roll anything out. Lines generated is a vanity metric. Cycle time from issue to merged fix, change failure rate, review turnaround and rollback frequency are the numbers that tell you whether the tooling made engineering better or simply louder.

Some honest self-assessment before committing time.

You are ready if you can describe what your test suite covers and where it is weak, if you have opinions about the module boundaries in your repository, and if you have ever debugged something by reading the source of a library rather than its documentation. Agent work rewards people who already reason about systems, because most of the job is designing the loop and its guardrails rather than prompting.

You are probably not ready if you are still learning your first language. The course assumes you can evaluate whether a generated change is correct, and that judgement is exactly what beginners lack. Using a coding assistant while learning to program is a separate question with a separate answer, and it is not what this material addresses.

You may not need it at all if your goal is simply to use an existing assistant well. Configuring an editor extension, writing good project instructions and learning when to reject a suggestion are worthwhile skills, and they do not require understanding retrieval architecture or sandbox design.

What to read next depends on where you are going

If you are heading towards running these systems on your own hardware, the constraint is model capability at your available memory, and the practical question becomes which local model is strong enough at code to be worth the loop. If you are heading towards platform work, the next topics are agent orchestration in general and the tool protocol layer that lets one agent talk to many services. If you are heading towards evaluation, the general AI engineering discipline — structured outputs, judges, tracing — is the right adjacent subject, and it applies well beyond code.

Whichever direction, the durable skill is not prompt phrasing. It is the ability to design a loop where a fallible generator is bounded by a reliable verifier, and to know what to do when the verifier is the thing that is wrong.