Noto
A reviewable execution layer between conversation and work.
A transcript records what people said. It doesn’t record what was decided.
A meeting transcript captures the conversation. It does not automatically establish what was actually decided, who owns an action, what the deadline is, whether a statement was a preference rather than a decision, or whether a later comment superseded an earlier one.
“The system shouldn’t just tell the user what it thinks happened. It should show why.”
Every extracted item carries a source quote and a transcript reference the user can inspect.
AI proposes. The system validates. The human approves.
The model proposes candidate decisions, actions, owners, deadlines, open questions and risks. Deterministic system logic checks evidence, ambiguity, conflicts, deferral language and (where supported) supersession. Nothing becomes authoritative until a human approves it, edits it, rejects it, or resolves the ambiguity.
“A preference looked like a decision.”
Sara preferred a 15-minute slot. Devraj preferred 5 minutes. Marcus said, “Let’s test both options before deciding.” Noto initially read that exchange as a finalized decision. It wasn’t one — it was a deferral. The system had silently converted uncertainty into certainty.
The deeper problem wasn’t semantic ambiguity in the abstract — it was that the product needed to understand the decision state of what was said (preference, proposal, discussion, disagreement, deferral, pending approval, finalized decision), not collapse all of it into “decision.”
A preference-vs-decision guardrail
Noto gained explicit handling for deferral language — “decide later,” “test both before deciding,” “wait for approval,” “revisit,” “decide after X.” Flagged items stay open for human review instead of resolving to a finalized decision.
The targeted scenario was retested and no longer resolved to a false decision, and the 41 regression tests pass (41/41) — but those are pass/fail checks on known cases. The full benchmark was not rerun after this guardrail shipped, so no accuracy, precision, recall or F1 improvement is claimed for it.
Frozen Gold Set v1.1 — 18 transcripts, 54 hand-verified ground-truth items. Stage 1 tested three prompt-only versions of the raw model:
| Version | Precision | Recall | F1 |
|---|---|---|---|
| C1 — Baseline prompt | 61.2% | 75.9% | 67.8 |
| C2 — + Evidence requirement | 65.1% | 75.9% | 70.1 |
| C3 — + Evidence + guardrail instructions | 71.9% | 85.2% | 78.0 |
C1–C3 are three prompt-only conditions, all scored on the same 18 transcripts and 54 items. They ran against the raw model before any validation code existed, so C3 is still a prompt asking the model to behave. That result is why validation became code that checks the model instead of instructions it can ignore.
Stage 2 scored the deployed pipeline (extraction, deterministic validation, evidence verification) against that same gold set: 77.8% precision, 90.7% recall, 83.8% F1, and 100% evidence grounding. It produced 49 correct items out of 63 predictions, with 14 false positives and 5 false negatives. The gold set is the same; the system being tested is different.
The Stage 2 (production-style) run predates the later preference guardrail, so it is not evidence the guardrail caused these numbers. The 41/41 regression suite is a separate check on known cases. It is not part of the frozen benchmark and is not a third benchmark configuration.
Tested with 8 real users — college club coordinators, students, a software developer, startup team members. (Some testers, including club coordinators, worked from the same starting transcript, so they aren’t independent scenarios.) Directional, not statistically significant.
“The benchmark didn’t catch this. Real users did.”
- —An owner was wrong in one test — why owner grounding is checked in code and “unclear owner” is a review state, not a guess.
- —One unresolved question wasn’t surfaced clearly enough. A known UX gap, not fixed.
- —Deadline and dependency ambiguity lingered.
- —The preference-vs-decision failure appeared directly in a test meeting — the one that led to the guardrail above.
Every candidate action maps directly to verifiable transcript tokens.
Nothing becomes an authoritative record without affirmative operator signoff.
No confidence percentages. Unclear owners, missing deadlines and conflicting statements are flagged as named review states instead of guessed.
Conflicting statements prompt human arbitration rather than arbitrary AI tie-breaking.
Superseded points remain logged rather than overwritten.
Jira and Notion outputs are copy-ready text, not live integrations — the MVP tests whether the extraction and review workflow is useful before adding integration surface area.
- —Full benchmark not rerun with the later guardrail; the 41/41 regression tests show it works on known cases, not that overall accuracy improved.
- —Ambiguity handling over-flags: 61.5% precision in the production-style run.
- —Supersession detection is brittle: 0 of 4 in the production-style run.
- —The frozen gold set has no risk-type items, so risk extraction is unevaluated.
- —Audio transcription is not included in the core MVP scope.
- —Jira/Notion outputs are copy-ready only, not native integrations.
Full documentation lives in the repo — PRD, AI system design, evaluation methodology, user research notes, and product decisions log.