How Teamcast Measures
How scoring works
Two layers: an LLM extracts and cites evidence; a deterministic 13-stage pipeline turns that evidence into coverage. No LLM ever sets a score.
Scoring is split so that judgement and arithmetic never mix. A language model is used in exactly one place — reading the transcript and extracting cited evidence. Everything after that is pure, deterministic math: the same transcript and the same ontology version always produce the identical result, bit for bit.
Layer 1 — evidence extraction (the only LLM step)
For each target signal, the model decides whether the transcript provides evidence and, if so, extracts a citation — the candidate's exact words — and scores two sets of components:
Quality dimensions — specificity, quantification, causal_clarity, structural_completeness: how substantial the answer is.
CRISP confidence — epistemic_weight, cit_completeness, explanation_stability, anchor_density, attribution_clarity: how confident we can be that the evidence really demonstrates the signal.
The deterministic 13-stage pipeline
The scoring engine turns those evidence components into per-signal strength and rolls it up to coverage. Every stage is pure math — no LLM, no randomness, bit-reproducible.
| Stage | What it does |
|---|---|
S5 · quality | Combine the four quality dimensions into a quality score for the evidence unit. |
S6 · dedup | Drop near-duplicate evidence (cosine > 0.85) — retelling one story cannot inflate a signal. |
S7 · CRISP | Combine the five CRISP components into a confidence for the evidence unit. |
S8 · typing | Tag the evidence as one of the 45 evidence types (which signal it speaks to). |
S9 · contribution | contribution = confidence × quality × mapping_weight. |
S10 · signal strength | signal_strength = max(contributions) — the single best demonstration establishes the signal. |
S11 · sub-dimension | Σ(signal_strength × signal_weight) × rubric × (0.5 + 0.5 × coverage). |
S12 · role rollup | Σ(sub_dim_score × sub_dim_weight) × role importance (bounded 0.7–1.6). |
S13 · output | Produce the per-competency detail and the coverage_summary. |
S14 · highlights | Surface the strongest behavioral episodes and the thinnest sub-dimensions — with citations. |
Stages S1–S4 are the extraction and typing front-half (Layer 1 above); the deterministic engine owns S5–S10 and the rollup + narrative stages S11–S14.
Best demonstration wins
Aggregation is max(), not an average: one strong, well-evidenced demonstration establishes a signal. A candidate is not penalized for not repeating themselves, and padding an answer with weak restatements does not help — deduplication removes it first.
From scores to coverage
The rolled-up numbers are converted into coverage — sufficient, partial, or insufficient — per sub-dimension and overall, with the gaps named. This is the deliverable a recruiter reviews.
{
"overall_coverage": "partial",
"gap_competencies": ["leadership"],
"covered_count": 5,
"total_count": 6
}See Coverage & the deliverable for the full shape.