What is still worth building when the feature is easy to copy?
The Moat Test takes one AI product category at a time, builds the part everyone assumes is the product, measures where it fails, and publishes the limits of the measurement next to the result.
Synthetic: this transcript was authored for the benchmark and depicts no real meetingDraft gold: annotations written by a coding agent and not reviewed by a human
Investigation 01 · Technical and commercial diligence
The Moat Test: What remains valuable when AI features are easy to reproduce?
If a model can summarise a transcript, what makes an AI meeting assistant worth paying for? This investigation builds a small challenger, measures where it fails, and separates what was actually measured from what is still a proposition.
- Held-out action precision
- 1.00
- Held-out action recall
- 0.24
- Cases accepted
- 0 of 12
- Critical defects
- 0
One deterministic method, one archived run, twelve held-out synthetic transcripts. There is no overall score because the measures do not share a denominator.
What this is not
- Not a product review. No commercial meeting assistant has been tested and no vendor is named anywhere in the results.
- Not a model evaluation. No model provider is configured in this deployment and no model has been called.
- Not user research. No interviews have been conducted, so the ledger holds 3 hypotheses and zero interview records.
- Not peer-reviewed. The gold annotations were drafted by a coding agent and are waiting for a human to check them.
The evidence ledger holds 14 records, 5 of them measured. Every claim in the investigation links to one, and a claim citing a record that does not exist fails content validation before it can be published.
How to read the labels
- Measured: Measured — archived experiment run
- Output from an actual execution of a named method, preserved with its run metadata and scored against the corpus.
- Illustrative: Illustrative — not a measured result
- Hand-authored by the author to show the shape of the output contract. It is not a result and carries no performance claim.
- Live trial: Live trial — unavailable in this deployment
- Would be output from a configured model provider. None is configured, so this mode never appears.