Build a Quality-First AI Coding Workflow

I recently gave a one-hour workshop using SignalPay, a small FastAPI payments API, to demonstrate a quality-first workflow for AI-assisted code changes.

The task I walked through was a specific code change in the repo: add a refund endpoint.

What I wanted to prove was that making this change AI with should be feasible without introducing AI slop. What would the best conditions be for the code then?

Ideally, for adding a new payment process endpoint like a refund, you’d preserve the quality of these relevant technical aspects of the code:

1. Authorization
2. Idempotency
3. Explicit state transitions
4. API and event contracts

In this post, I’ll take you through a workflow that made AI code changes testable, reviewable, and in better condition to merge. In other words, I’m actively avoiding AI slop through more structured agentic engineering.

The workflow is a loop I encourage engineers to adopt, especially if they don’t already have something similar in place:

task
  ↓
plan → gather repository rules → use agent skills
  ↓
behavior tests → implementation
  ↓
local deterministic gates
  ↓
local pre-PR review
  ↓ 
pull request → independent review
  ↓ 
remediation → verification again

Define the task requirements before you write code

A coding agent uses the context it receives to decide what “done” means. If the only instruction is “add a refund endpoint,” it can produce a route that works while breaking behavior the system already depends on. Before implementation, define the constraints that must remain true:

• authorization must happen before mutation;
• retries must not repeat side effects;
• state transitions must be explicit;
• public response and event shapes must remain stable;
• rejected requests must not leave partial state behind
  

Put these requirements where people and agentic tools can read them: an issue, a design note, or a plan in the repository. The format matters less than making the constraints visible before implementation begins.

In SignalPay, those constraints are recorded in different AI-friendly artifacts:

AGENTS.md sets the required workflow: plan the change, write tests before production code, run verification, and review the pull request.

.plan/ records the change, its risks, the relevant rules, the required tests, and the handoff into implementation. In a production workflow, this information can live in an issue tracker or design document. The key requirement is that the agent receives a clear task, constraints, and verification plan before it edits code.

rules/ contains repository-specific requirements for agents and reviewers.

skills/ turns those requirements into repeatable procedures for specific engineering tasks.

When you’re defining the task properly before you code, you can prompt your agent to identify the important rules and agent skills that are most relevant for the task at hand. Those are multiple sources of context that your agent can leverage:

• The task, which could be documented and codified somewhere and pulled in through integration from your task management tool
• The coding rules from your repo
• The agent skills that you use, that you’ve created locally or that are widely available to you from your repo or another centralized place

As you plan with your agent, these are important sources of context that will be used for the code generation implementation phase. 

Turn your plan into observable code behavior

One agent skill that I find to be very helpful is about test-driven development and behavior-driven development. It helps me identify the failure modes I expect the coding agent to implement and write code that handles failures and edge cases very clearly. This is also a process I use during planning mode as part of the required implementation process.

So I build that into the plan alongside all of the other contexts that will refine the code that is ultimately generated.

The refund endpoint used this API contract:

POST /payments/{payment_id}/refund

And a successful request returns this existing payload:

{
  "paymentId": "pay_1001",
  "customerId": "cus_9001",
  "amount": 12500,
  "currency": "USD",
  "status": "refunded"
}

And here is the error handling contract for potential failure modes:

• missing or invalid authentication returns 401;
• missing refund scope returns 403;
• missing Idempotency-Key returns 400;
• unknown payment IDs return 404;
• payments that are not captured return 409;
• attempts to capture a refunded payment also return 409.

Use requirements to write contract tests and failure-path tests before implementation. They define both the successful response and the rejection cases, so the agent has behavior to satisfy very clearly.

Run deterministic checks

After your coding agent has hit the ground running with implementation and prior to opening a pull request, There are several different deterministic tools available depending on your programming language and framework to run a local verification stack.

For example, here are all of the deterministic tools and their purpose that are baked into my project:

To reproduce the starter workflow:

git clone https://github.com/nnennandukwe/qodo-quality-workshop-signalpay.git
cd qodo-quality-workshop-signalpay
make doctor
make setup
make verify

A passing local run means the checks you configured passed. It doesn’t, however, prove that you wrote checks for every important interaction.

Use deterministic gates to answer the same bounded questions on every change. Do not treat them as proof that the entire workflow is correct.

Run a local pre-PR review

There are many ways to do this. There are workflows built into Claude Code and Codex, as well as independent code review workflows and skills available for developers to run.

In any case, running this locally is a strong best practice for identifying obvious issues that the coding agent might have generated. I highly recommend running this prior to submitting a pull request and ensuring your agent resolves the highest-value issues.

Use independent code review to find deeper code failures

The initial refund implementation pull request was generated after my agent followed clear steps satisfied the obvious feature behavior. It added the new endpoint, required the refund scope and idempotency key, returned the correct response, emitted the expected event, and passed local verification.

Qodo then reviewed the pull request as an independent code review step.

Multiple issues were surfaced, like these: 

• the new handler was missing a required docstring;
• new test helpers and tests were missing required docstrings;
• refund idempotency results were stored only in memory;
• the unknown-payment 404 path had not been tested;
• the header guard used an imprecise falsy check instead of explicitly distinguishing None and an empty string.

Fix findings, then run verification again

A code review finding is not resolved when a tool reports it. A developer must triage the finding, decide on the fix, implement it, and verify the result.

Treat remediation as part of implementation. Apply the fix, add or update the regression test, and rerun the targeted tests and full verification stack.

The remediation commit added:

• a guard preventing capture after refund;
• a regression test proving that the rejected capture leaves state and events unchanged;
• file-backed idempotency results;
• explicit missing and empty header handling;
• unknown-payment refund coverage;
• the required docstrings.

After remediation, run targeted tests and the full verification suite again. The pull request’s Qodo Fix Summary records all six fixes and the verification commands used to validate them before merge.

Apply the loop to your own repos

1. Define the requirements. Identify what must be true when new feature code is added to  the codebase.
2. Include rules. Put important constraints somewhere humans and agents can inspect and verify for code generation. 
3. Create an execution plan. Connect your relevant rules and agent skills to behaviorial expectations and tests.
4. Write failure-first tests. Prove that rejected operations fail closed and leave no partial side effects.
5. Run deterministic gates locally. Let linters, type checkers, security tools, tests, and repository-specific analysis answer the questions they handle well.
6. Use an independent code review boundary. A separate code review harness can uncover issues that your coding agent might not.
7. Remediate and verify again. An issue in your code can be resolved when a change is reviewed, applied or explicitly deferred, and reverified.

If you want to reproduce the full exercise, fork the SignalPay workshop repository, start from main, and implement the code change workflow. When you’re done, compare your result with pull request #4.