Build Your Software Factory: Win an iPhone Duo, Mac Mini, or Switch 2

Join the Hackathon →

Building a verification stack for the software factory

A code change connects to four parallel evidence panels representing static checks, execution tests, formal checks and contextual review. Their paths join at an acceptance check.

In agentic engineering, we’re asking agents to take on more of the work of building software. That brings a familiar engineering question into sharper focus: how do we check their work well enough to trust it?

Lauren Tan, who works on Grok Bot and Cursor, has made verification part of her pstack workflows. They include instructions for creating formal checks and maintaining them as code changes.

That makes the meaning of “verified” worth examining. Formal verification uses mathematical reasoning to check stated properties under explicit assumptions. For a team considering it, the useful question is how those guarantees relate to the behavior they need from the application.

I wanted to explore that question in code. I worked with a coding agent on a standalone TypeScript component that limits how many requests can run at once. We compared tests, proofs, system models and code review, then examined what happened as inputs grew, requests overlapped and implementations changed.

Itamar Friedman and I discussed the broader question of trust with Deepak Singh, VP of Agentic AI at AWS, for The Agentic Review. As he put it, “trust that you can’t verify is just hope.”

Where does each check fit?

I’d build the workflow around a shared requirement and the exact code revision being accepted. Run type checks and lint rules during editing for fast feedback on type errors and known patterns. Execution tests check behavior on examples, generated inputs and real integration paths.

Use formal verification for selected properties whose importance justifies a precise contract and supported model. Qodo adds contextual review during development and on the pull request.

Together, these checks supply evidence for an acceptance decision. When a finding changes the implementation, rerun the affected tests, proofs and review on the revised code.

Intent and acceptance criteria lead to implementation, then static checks, execution tests and Qodo review. Findings return for revision. The latest PR receives CI and integration checks, applicable formal checks and Qodo PR review. Evidence is reviewed against team policy. Rejected changes return for revision and reruns; accepted changes proceed.
Recommended workflow: run checks while developing, then evaluate the latest PR revision. Qodo reviews local changes and the PR. Formal checks cover selected properties and can also run during authoring. The team sets required evidence and acceptance authority.

What can a formal proof establish?

My standalone admission component, inspired by Controlled Release Lab, allowed 16 active requests globally and four per client. Its contract required admission when a request without a permit had capacity, exact count updates, and release only once. Runtime guards restricted inputs to bounded integers, matching the proof’s mathematical-number assumptions.

LemmaScript translated annotated TypeScript into a model checked by Dafny. Exhaustive execution of 266 input/action pairs and formal checking each rejected eight deliberate teaching defects; seven examples rejected seven.

One TypeScript transition feeds two paths: execution across 133 states and two actions, and translation to Dafny to check the stated contract. Runtime counts are bounded integers.
The same transition feeds execution tests and a translated model checked against its contract.

The missed defect changed 13 active requests to 15 instead of 14. A capacity-and-ownership contract accepted it; the full exact-accounting contract rejected it.

Deepak described formal verification around proving that “the thing that you wanted to do is actually what’s happening.” The property we specify determines which behavior that proof establishes. Reviewing the requirement remains part of the work.

What does Qodo add to the workflow?

Qodo’s review agents examine changes using repository context, engineering standards and linked requirements when available. That supports review of logic errors, interactions across files and gaps between a change and its intended behavior. Findings connect concerns to relevant code, explain the impact and suggest a path to resolution.

Qodo’s findings give the developer or coding agent specific concerns to investigate. Recurring issues that fit a precise lint rule or regression test can become repeatable checks there.

This gives Qodo a useful role across the wider change. For the admission component, review questions include whether the contract requires correct accounting, whether callers use the checked implementation, and whether ownership assumptions match the surrounding code. Formal verification complements Qodo by establishing specified properties under explicit assumptions. Qodo’s contextual review helps examine the requirements, callers and design decisions around those properties.

I later submitted one clean input and one selected seeded defect to Qodo CLI, each covering the same five core, adapter and test files in isolated clones. Qodo identified the 13-to-15 accounting error. It also raised a reasonable question about multiple permits per client; those permits were intended and already tested.

To review during implementation, use an accessible repository base and provide the diff, intended behavior and deliberate design decisions. Have the coding agent assess findings, fix supported issues and rerun affected checks. Configure automatic PR review to examine the pushed revision before acceptance.

How do we check the surrounding system?

The function’s proof left request scheduling to its caller. In a separate HTTP workflow, a seeded race admitted two requests against a limit of one.

A bounded TLA+/TLC model helped examine request interleavings; failure paths replayed against the controller. Reviewing the mapping revealed that simultaneous requests with the same identity were excluded. A dedicated HTTP test caught that interaction.

Integration testing and model checking gave us evidence about how components interact, alongside the arithmetic contract.

Keep verification connected to the code you ship

In a pstack-guided maintenance exercise, the old function still passed its proof after its caller switched implementations. Checking the actual target rejected the seeded bug.

That gives a reviewer a continuing job: connect intent, checks, findings and running code. The team defines required checks and who can accept a change, with human judgment for ambiguous or high-impact decisions. Qodo contributes contextual review throughout that loop, while tests and selected formal checks establish complementary evidence.

We can give coding agents more responsibility when the team can explain why a change is ready to move forward. Keeping the requirements, checks and Qodo review connected to the code we’re accepting makes that confidence something we can maintain as the software changes.

Get started with Qodo for AI Code Review

Start trial
Share this post
Related reads:

More from our blog

Check out our musings on generative AI, code integrity, and other geeky stuff: