An AI-augmented QA workflow
A custom agent skill, live locator verification through MCP, and an enforcement layer that blocks a commit until the self-audit is clean.
- lines in the custom agent skill
- 467
- accessibility snapshots over seven months
- 796
- single-fact memory entries
- 89
The automation repository had strict, fast-changing conventions: page-object rules, banned waits, shared fixtures that must never be mutated, and a migration from REST to GraphQL waits. Those rules lived mostly in reviewers' heads and in a long checklist.
AI assistants without that context produced code that compiled and failed review: guessed locators, sleeps, inline test data. The same review comments came back merge request after merge request. The product also spanned three repositories that behave as one system, so answering "what does this component actually render" meant switching repositories by hand or clicking through the test environment.
What we built
A custom agent skill, 467 lines across three files: a role and guardrail file, nine task playbooks covering authoring, model tests, refactoring, pre-push self-review, review replies, failure triage, local runs and bug filing, and a self-update protocol. When review feedback or a failure root cause reveals a missing rule, the skill proposes an edit to the rule file, so the lesson is captured once instead of re-explained.
A workspace configuration treating three repositories as one system, with the test repository as the write target and the others as read-only references.
A pre-commit self-review layer, a 475-line rule file with four enforcement triggers. Every violation is reported in a table and the commit is blocked until that table is empty.
Live verification through MCP. Before a locator or assertion is committed, the agent opens the running application, takes an accessibility snapshot and confirms the actual role, accessible name or alert text, rather than guessing from the markup. 796 accessibility snapshots and 33 screenshots were captured this way over seven months.
Persistent memory: 89 single-fact entries holding non-obvious knowledge about component traps, error semantics and environment quirks.
The decision that mattered
The rules live in the repository and the skill only points at them. A rule embedded in a prompt is invisible to the team and goes stale silently. A rule in a file is reviewable, versioned, and improves for everyone when one person learns something.
What it produced
A workflow where AI-assisted work arrives at review already conforming to the team's conventions, with locators verified against the running application rather than inferred. This is the foundation of the quality tooling Kaycore is building as product.
