Case Study · QA & AI Delivery

Market Sentinel: building quality governance into AI-assisted delivery.

An evidence-led case study showing how I applied established QA and Quality Management disciplines to a working software product delivered with multiple AI engineering agents — governing requirements, live market-data integration, layered testing, performance, browser UAT and release decisions without handing quality authority to AI.

RoleQA & AI Delivery Consultant · Product Owner · QA/UAT Lead

Project contextIndependent product · self-directed delivery environment

Delivery modelHuman-led quality governance + parallel AI engineering agents

WorkflowGitHub-managed Agile/Kanban · BDD · TDD/RED-GREEN · controlled integration

2,275 passedcurrent deterministic regression checkpoint; 4 environment-dependent live-provider/PostgreSQL system validations run separately

1,500+ instrumentsFTSE 100/250/350, AIM and All-LSE universe scopes

184.5s → ~40smeasured FTSE-100 Analyze improvement

Human release gategreen automation never replaces final UAT judgement

Why this matters to a client

More delivery capacity without surrendering quality control.

The transferable value is not the financial-market product itself. It is the delivery model: convert product intent into measurable acceptance, let AI accelerate engineering work, keep evidence traceable across the SDLC, validate real integrations and runtime behaviour, and retain human authority over risk and release.

Measurable acceptanceRequirements become observable contracts rather than subjective “looks fixed” decisions.

Layered release evidenceComponent, integration, system, regression, live-environment validation and browser UAT answer different quality questions.

Governed AI deliveryAI agents can increase implementation capacity while ownership, review, traceability and release authority remain explicit.

Operational qualityData provenance, persistence, performance, usability and failure behaviour are treated as release concerns, not afterthoughts.

The product

A working platform before it became a QA case study.

Market Sentinel began as an independent financial-market intelligence platform. Its Command Center brings together instrument research, Capital Flow Radar, Global Health Radar, Loaded Springs, Market Scanner, historical validation, data status and scheduled market scans. As the product and AI-assisted development model expanded, it became a practical environment for testing how AI delivery can be accelerated without removing human accountability for quality.

Market Sentinel Command Center overview — **Market Sentinel — Command Center**The system-level view gives context before the case study moves into individual QA, UAT, performance and governance evidence.

Project scale & live-market integration

Real market data, not a demonstration dataset.

Market Sentinel had to ingest, normalise, reconcile and validate real external market data across different provider and exchange boundaries. The architecture keeps downstream research and scoring provider-neutral while preserving source provenance, freshness and failure state.

Market-data work included EODHD as the first broad engineering feed, plus Twelve Data and Finnhub adapter/fallback paths. The LSE universe is built independently from public London Stock Exchange Price Explorer and FTSE constituent data, then normalised and persisted through PostgreSQL snapshots.

The product exposes explicit FTSE 100, FTSE 250, FTSE 350, AIM and All-LSE scopes rather than inferring membership from ticker patterns or market capitalisation. Different feeds can also represent live, delayed, end-of-day or unknown freshness, so unavailable evidence is not silently presented as current or fabricated.

At a September live UI checkpoint, the All-LSE scanner reconciled 1,547 eligible instruments → 1,547 rows produced → 0 dropped/failed. An earlier source audit persisted and reloaded 1,558 LSE/AIM instruments, illustrating that the universe is maintained from external market sources rather than hard-coded to one fixed count.

The 100-stock scanner test was deliberately bounded — not small.

FTSE 100 provided an exact, auditable acceptance slice for proving the reconciliation contract. The same accounting principle was then exercised against an All-LSE universe of more than 1,500 instruments.

The challenge

AI increased delivery capacity. The quality question became more important.

The challenge moved beyond whether AI could write working code. Multiple development streams needed measurable requirements, testable acceptance criteria, traceability, separate review, safe integration, performance controls, trusted test evidence and explicit human release acceptance.

The central question became: how can AI development agents accelerate software delivery without weakening QA governance?

My role

QA & AI Delivery Consultant with explicit product and release authority.

I operated as QA & AI Delivery Consultant, Product Owner and QA/UAT lead: prioritising the backlog, defining product intent, shaping measurable acceptance criteria, directing the quality strategy, performing exploratory and browser UAT, challenging release evidence and giving final merge/release authorization.

AI agents handled substantial implementation, debugging and automated-test work. Separate review and test evidence informed the decision, but the model deliberately separated development speed from acceptance and release authority.

This was not QA added at the end of an AI coding exercise. Quality Management was part of the operating model: requirements, risk, test design, evidence, integration controls and release decisions were governed from backlog to main.

Quality strategy

Shift-left quality from requirement to release.

Different test layers answer different quality questions. No single green layer is treated as proof that the product is ready.

1. Requirement / BDDDefine observable user behaviour and measurable acceptance.

2. RED / TDDFail first: pin the contract or reproduce the defect.

3. GREEN / TDDImplement the minimum change that satisfies the contract.

4. Smoke / SanityConfirm the build and affected route are stable enough to continue.

5. RegressionRerun accepted behaviour across component, integration and system-level automated coverage.

6. System / UATRun environment-dependent system validation separately, then assess real browser usability and operational value.

7. Release gateReview evidence and risk before explicit human merge/release approval.

Test architecture

Different layers, different evidence — with “E2E” used carefully.

The automated estate is deliberately layered. The normal deterministic regression includes unit/component, integration and broader system/regression checks. Environment-dependent validations are opt-in so a missing external service, credential or PostgreSQL state does not make the everyday regression nondeterministic.

Component / unitFast checks of isolated business rules, mappings, scoring and behaviour at code-level boundaries.

IntegrationChecks across adapters, repositories, application services and other collaborating components.

System / regressionBroader application behaviour is protected by the deterministic suite before release evidence is accepted.

Opt-in live-provider validationReal configured providers exercise source-to-released-snapshot paths and live application behaviour outside the normal deterministic run.

Opt-in PostgreSQL validationStateful checks exercise persistence, history, momentum and historical replay through isolated PostgreSQL schemas.

HTTP / rendered-page validationSelected opt-in checks cross the application/server boundary through loopback HTTP and assert rendered HTML.

Human browser UATA separate activity: I use the real browser to assess usability, decision value and release readiness.

Release decisionNo single green layer is treated as sufficient; evidence is reviewed against risk before human approval.

Terminology matters.

The four environment-dependent checks are not described here as “browser E2E.” Some cross a broad source-to-rendered-HTML chain, but they use HTTP rather than browser automation. Browser UAT is separate and explicitly human-led.

BDD in practice

Turning a vague user problem into an auditable acceptance condition.

A Market Scanner defect originally appeared as a user-visible problem: the selected FTSE-100 universe did not produce a complete result set. Rather than accept “fixed” as a subjective conclusion, the requirement was converted into a measurable BDD-style contract.

Given FTSE 100 is selected

When Analyze completes

Then all 100 eligible instruments are accounted for

And 100 scanner rows are produced

And 0 instruments are silently dropped

And unavailable evidence remains explicitly unavailable rather than fabricated.

The FTSE-100 test deliberately used a bounded 100-instrument slice so the contract could be exact and auditable. It was not the size of the platform. The same reconciliation discipline was later demonstrated against the full All-LSE universe.

FTSE 100 Market Scanner acceptance evidence — **BDD/UAT evidence — FTSE 100 reconciliation**100 eligible → 100 produced → 0 dropped/failed, with unavailable evidence kept explicit.

Human UAT after green automation

“All tests passed” was not the same as “ready to release.”

Backtesting Lab historical detail and QA export

Following UAT feedback, decision-useful cohort and validation summaries were prioritised while the deeper historical evidence remained available through progressive disclosure and QA export.

One of the strongest examples came from the Backtesting Lab. Development and separate review both passed the automated suite, including more than 2,000 tests. Human browser UAT still rejected the release because decision-useful evidence was buried beneath a large wall of historical detail.

The UAT finding became a stronger corrective RED contract. The GREEN change reorganised the information hierarchy and introduced progressive disclosure. Review then caught an additional accessibility issue before final regression.

Historical checkpoint: development and separate review reached 2,050 passed, 0 failed in the deterministic suite; 4 environment-dependent live-provider/PostgreSQL validations remained opt-in. Human browser UAT then provided a separate acceptance gate.

QA lesson

Automation proved the coded contract worked. Human UAT challenged whether the contract represented the right behaviour for a real user.

TDD / regression evidence

RED → focused GREEN → complete regression.

A later defect provides a direct audit trail of the test-first workflow. The acceptance test first failed, the correction was verified with a focused test, and the complete suite was then rerun.

The four opt-in cases are deliberately separated from the deterministic regression because they depend on real external providers and/or PostgreSQL state. They cover live source-to-snapshot behaviour, live-provider-to-rendered-HTTP behaviour, stateful PostgreSQL history/persistence, and PostgreSQL historical replay. They are not silently ignored or counted as passing.

This prevents a local fix from being treated as sufficient release evidence.

Delivery workflow

GitHub Kanban, traceability and controlled integration.

GitHub is the source of truth for backlog, issue ownership, branches, commits, pull requests, test evidence, review history and acceptance decisions. Workflow status remains separate from developer ownership so the project can show what changed, who/what implemented it, who reviewed it and who accepted it.

Backlog / requirement→Developer owner→RED / defect→GREEN→Targeted + regression→Separate review→System/live validation as required→Human browser UAT→Explicit merge→Main / release

GitHub Kanban workflow — **Kanban & delivery traceability**Backlog, active development, testing and completed work are managed as a visible delivery flow rather than an informal AI coding session.

Performance

Non-functional quality was measured, not assumed.

A real FTSE-100 Analyze path degraded to approximately 184.5 seconds. Profiling, bounded concurrency and shared benchmark work reduced the same desktop workflow to approximately 40 seconds while retaining the functional regression gate.

The screenshots below show the platform’s later Performance / Diagnostics instrumentation. They demonstrate that performance became a first-class product quality concern; they are not presented as the direct before/after proof of the historical 184.5s → ~40s measurement.

Performance diagnostics bottleneck breakdown

Performance QA pattern

Measure → isolate → change → regress → validate in the real runtime.

Security & AI-agent governance

Process isolation looked correct — until QA challenged the boundary.

Parallel AI developers originally worked in separate Git worktrees. Investigation showed that worktrees were a process boundary, not a strong filesystem/security boundary when agents ran under the same Linux user.

Development stopped before merge. The architecture was strengthened around independent clones, controlled MAIN/REVIEW areas, enforced workspace restrictions, separation from production secrets and explicit human integration authority.

The first isolation acceptance test then exposed another quality problem: the test harness itself was defective and produced an apparent PASS that could not be trusted. The harness was corrected and rerun before the control was accepted.

A PASS is only valuable if the test mechanism is trustworthy.

Security controls were treated as part of delivery quality and release governance, not as a claim of specialist cybersecurity expertise.

AI-augmented efficiency

AI as an engineering accelerator — not an uncontrolled code generator.

The project demonstrates a high-leverage model in which one experienced QA/Product Owner coordinates multiple AI engineering streams, automated evidence generation and rapid defect correction.

The defensible conclusion is not that AI “replaces a team.” It is that AI increases delivery capacity, and that higher delivery speed makes quality governance, traceability, security and human release judgement more important — not less.

What changed because of QA

Quality activity changed the product and the delivery model.

Complete scanner accountingIncomplete output became an explicit eligible → produced → dropped contract, later exercised at full All-LSE scale.

Performance became measurableA degraded ~184.5-second FTSE-100 path was investigated and reduced to roughly 40 seconds.

Green automation was challengedBrowser UAT rejected technically passing behaviour when the evidence hierarchy did not serve the user.

Untrusted PASS evidence was rejectedA defective isolation test harness was corrected before its result could be accepted as release evidence.

Agent boundaries were strengthenedQA investigation changed the AI workspace/isolation model before unsafe assumptions became normal delivery practice.

Missing evidence stayed missingUnavailable analytical domains remain explicit instead of being replaced with neutral or fabricated values.

Source quality stayed visibleProvider provenance and freshness are retained so live, delayed, EOD and unavailable data are not treated as equivalent.

Release authority remained humanAI could accelerate implementation and review, but acceptance, risk judgement and final merge remained explicit human decisions.

Conclusion

What the project has demonstrated.

Market Sentinel started as a software product, but it also became a practical test of how AI-assisted engineering can be governed in a real delivery environment — with external data providers, changing market universes, persistent data, performance constraints and user-visible release decisions.

The work to date demonstrates that AI can materially increase engineering capacity, but greater delivery speed does not remove the need for established QA and Quality Management disciplines. In several cases, automated tests were green while browser UAT, live-data behaviour or deeper investigation still exposed problems.

BDD made expectations measurable. TDD helped control implementation changes. Component, integration, system and regression coverage protected accepted behaviour at different boundaries. Opt-in live-provider/PostgreSQL validation tested environment-dependent paths without weakening deterministic regression. Data-quality controls kept provenance and absence visible. Performance testing exposed operational bottlenecks. Human browser UAT challenged whether technically correct behaviour was genuinely useful. Traceability and release gates retained accountability across an increasingly AI-assisted workflow.

For a prospective client, the practical takeaway is a repeatable control model: accelerate engineering with AI where it adds capacity, but keep requirements, evidence, integration risk, user acceptance and release authority visible and owned by people.

AI does not replace quality governance — it increases the importance of it.

The strongest results came from using AI to extend delivery capacity while keeping product intent, requirements, evidence, risk decisions and release authority under explicit human control.

What this case study demonstrates

A broad QA discipline applied to a modern AI-assisted SDLC.

Quality leadershipStrategy, risk, acceptance and release governance.

Shift-left QAQuality involvement from requirements and design onward.

BDD & TDDObservable acceptance criteria and RED-GREEN development.

Layered testingComponent/unit, integration, system, regression, opt-in live/PostgreSQL validation and human browser UAT.

Data-integration QAProvider boundaries, provenance, universe reconciliation, freshness and persistence.

Non-functional qualityPerformance, usability, accessibility and operational behaviour.

Security governanceAgent isolation, credential separation and trusted test evidence.

Kanban & traceabilityBacklog-to-release audit trail through GitHub.

AI delivery governanceHuman accountability over accelerated AI engineering.

A2Z Consultancy · QA & AI Delivery

Need more delivery capacity without weakening quality governance?

I help organisations apply established QA and Quality Management discipline to AI-assisted software delivery — from measurable requirements and test strategy through data-integration QA, UAT, traceability, risk controls and human release assurance.

If you are introducing AI into software engineering and need a senior QA/QM perspective on how to govern it in practice, the conversation can start with your current delivery workflow, quality risks and release controls.

Discuss a QA & AI delivery engagement View consultancy services