Merit Engine - Output Specifications
Merit Engine: Output Specifications
Candidate Report, Agency View, and FSP Platform Dashboard
Prepared by: Fairlawn Strategy Partners Date: June 29, 2026 Version: 1.0
Overview: Three-Tier Output Architecture
The Merit Engine produces outputs for three distinct user roles. Each role sees a different slice of the data, governed by candidate consent and platform role permissions.
| Tier | User | Data Access | Primary Purpose |
|---|---|---|---|
| 1 | Candidate | Own data only | Personal readiness development |
| 2 | Agency (Chief, HR, Training Officer) | Anonymized aggregate of their candidates; individual data only if candidate opts in | Succession planning, training resource decisions |
| 3 | FSP / Tonya R. Dawson | Full platform visibility across all agencies | Product quality, e-learning outreach, client management |
Data flows in one direction by default: upward from candidate, only with explicit opt-in. The agency never sees an individual candidate’s data unless that candidate shares it. FSP sees all data as the platform operator, governed by the Master Service Agreement terms with each agency.
TIER 1: CANDIDATE FEEDBACK REPORT
Design Principles
- One report, generated after every session, available on demand at any time
- Language is developmental, never punitive. The engine is a coach, not a judge.
- Framing: “Here is where you are. Here is where you are going. Here is what to do today.”
- Confidence intervals on predictions are honest - wide early in the 60-day cycle, narrowing as response data accumulates. Never display a false precision the data does not support.
- E-learning recommendations are invitations, not referrals. The candidate chooses.
- Source materials surfaced to candidates are verified before display. A candidate is never shown a source that has not passed the three-point verification check.
Report Section 1: Readiness Summary (The Headline)
What the candidate sees: A single readiness level translated from their overall theta (θ) estimate, with a trajectory statement and a days-to-target projection.
Readiness Level Scale (θ to plain language translation):
| θ Range | Readiness Level | Description |
|---|---|---|
| Below -1.5 | Foundation Building | Core knowledge gaps need attention before timed practice is effective |
| -1.5 to -0.5 | Developing | Foundational understanding present; targeted drilling needed in multiple domains |
| -0.5 to +0.5 | Approaching Proficiency | On track; specific domain gaps are the remaining barrier |
| +0.5 to +1.5 | Proficient | Strong candidate; refinement and oral board simulation are the priority |
| Above +1.5 | Command Ready | Exam-ready; focus shifts to stamina, strategy, and peak performance |
Example display text: > “You are currently at Approaching Proficiency. Your target for exam readiness is Proficient. Based on your session pace, you are projected to reach Proficient in approximately 11 days.”
Confidence interval behavior: - Days 1-14: Wide interval displayed. “Your projected exam score range is 62-85. This range will narrow as the engine learns more about your ability level.” - Days 15-40: Moderate interval. “Your projected exam score range is 68-79.” - Days 41-60: Narrow interval. “Your projected exam score range is 72-77. Your department’s historical passing threshold is 70.” - Never display a range narrower than 5 points before Day 40.
Report Section 2: Domain Snapshot (The Heat Map)
What the candidate sees: One row per domain. Displays current θ per domain, a trend arrow (improving, flat, declining since last session), and a color status indicator.
Color logic: - Green: θ at or above the target threshold for this domain at the candidate’s target rank - Yellow: θ within 0.5 below the target threshold - approaching but not there - Red: θ more than 0.5 below target threshold - needs focused attention
Example display (Sergeant candidate, Law Enforcement track):
| Domain | Your Level | Trend | Status |
|---|---|---|---|
| Constitutional Law and Criminal Procedure | Approaching Proficiency | Improving | Yellow |
| Criminal Law (Alabama Title 13A) | Proficient | Stable | Green |
| Supervisory and Personnel Management | Developing | Improving | Yellow |
| Traffic Law and Enforcement | Proficient | Stable | Green |
| Report Writing and Documentation | Proficient | Stable | Green |
| Community Policing and Problem Solving | Approaching Proficiency | Declining | Yellow |
| Budget and Resource Management | Foundation Building | Improving | Red |
What the candidate does not see: Raw θ numbers. The IRT math stays under the hood. Candidates see readiness language, not psychometric notation. This is intentional - theta values without context cause anxiety and misinterpretation.
Report Section 3: Today’s Priority (The Action)
What the candidate sees: One recommended focus area with an estimated time commitment and a direct link to start a targeted session.
How it is generated: The engine multiplies the gap between current domain θ and target threshold by the domain criticality weight. The domain with the highest product of gap and criticality becomes Today’s Priority.
Gap score = (Target θ - Current domain θ) x Criticality weight
Example display text: > “Your focus today: Budget and Resource Management. This domain has your largest gap relative to its importance for the Sergeant exam. A 20-minute targeted session is ready for you.” > [Start Today’s Session]
Design note: Only one priority is shown. Showing a list of what needs work is demotivating. One clear action is actionable.
Report Section 4: Decision Latency (Oral Board Track Only)
Displayed only after the candidate has completed at least one simulation session. Absent from the report until that data exists.
What the candidate sees: Average response time by scenario type compared to a target benchmark, and a trend over simulation sessions.
Metrics displayed:
| Scenario Type | Your Average Response Time | Target | Trend |
|---|---|---|---|
| Use of Force | 4.2 seconds | Under 3.0 seconds | Improving |
| Subordinate Discipline | 6.8 seconds | Under 5.0 seconds | Stable |
| Citizen Complaint | 3.1 seconds | Under 3.0 seconds | Improving |
| Tactical Command | 8.4 seconds | Under 6.0 seconds | Needs Work |
Framing note: Response time is described as “decision clarity” not “speed.” A candidate who pauses to think through a Use of Force scenario is not slow - they may be deliberate. The narrative framing matters. Benchmark targets should be validated against actual oral board expectations during the beta pilot, not hardcoded assumptions.
Report Section 5: Three-Stage Learning Funnel and E-Learning Invitation
The engine does not jump directly to an e-learning invitation when a gap is detected. It works through three stages in sequence, giving the candidate every opportunity to close the gap independently before a paid resource is offered.
Stage 1: Targeted Practice (In-Platform, Included in subscription) Gap detected. Engine serves additional adaptive items in the weak domain. θ is re-estimated after each response. If θ reaches threshold, no further action. If θ remains below threshold after one full targeted session, Stage 2 triggers.
Stage 2: Source Material Review (In-Platform, Included in subscription) The engine surfaces the specific verified policy, statute, or SOP section that the missed items are testing - not the full document, the exact paragraph. The candidate reads the source, then answers 2-3 comprehension check items drawn from that passage.
Source material is only displayed after passing the three-point verification check: 1. Source URL resolves and content hash matches the ingestion record 2. AI confirmation that the current source text still supports the correct answer 3. Source verification date is within the re-verification window (90 days for statutes, 30 days for agency SOPs)
If any check fails, the candidate receives extended targeted drilling only. No broken or outdated source is ever shown.
Candidate display: > “The items you missed are drawing from one specific section of policy. Here is the relevant passage. Review it, then try a short check.” > [Review Source Material]
If θ reaches threshold after the comprehension check, no further action. If θ remains below threshold, Stage 3 triggers.
Stage 3: E-Learning Invitation (Paid upsell - FSP) Only reached after the candidate has drilled, reviewed the verified source, and still has the gap. The invitation is now a credible recommendation, not a sales prompt.
“You have reviewed this material and the gap persists. A focused 90-minute session with Tonya R. Dawson is available to walk through this concept with you directly.” [Learn More] [Schedule] [Not Now]
Framing across all three stages: No language suggesting the candidate “failed,” “was referred,” or “needs remediation.” The engine observes a gap and offers progressively deeper resources. The candidate chooses at each stage.
What happens when they click “Not Now” at Stage 3: Invitation disappears for 3 days, reappears if the gap persists. After three “Not Now” responses, the invitation moves to a passive “Resources” tab and stops appearing in the main report. The gap remains visible in the domain snapshot.
Report Section 6: Opt-In Sharing
At the bottom of every report, a clear and simple consent option:
“Share this report with your agency’s Training Officer?” [Share] [Keep Private]
If the candidate selects Share: - The Training Officer receives a read-only copy of this specific report snapshot - The candidate receives a confirmation that sharing occurred and can revoke at any time - Sharing one report does not create ongoing automatic sharing
Design principle: Consent is per-report, not a blanket permission. Candidates retain control every time.
TIER 2: AGENCY VIEW
Who Has Access
Agency access is granted to designated contacts only, established at contract signing. Typical roles: - Training Officer (full agency view) - HR Director (full agency view) - Chief (executive summary view)
Agency users never see individual candidate names, badge numbers, or identifiers unless a candidate has opted in to share their report.
Agency View A: Training Officer Dashboard (Daily)
Purpose: Operational management of the current promotional cycle. Updated after every candidate session.
Panel 1: Engagement Summary - Total candidates enrolled - Sessions completed today / this week - Candidates who have not completed a session in 5 or more days (flagged for follow-up, no performance data shown - just engagement) - Average session completion rate across the cohort
Panel 2: Domain Error Clusters Which policy areas are generating the most incorrect responses across all candidates. Updated daily.
This is the highest-value panel for the Training Officer. If 70% of candidates are missing questions on Use of Force - Updated Policy (effective March 2026), that is a training bulletin waiting to be written. The engine tells them before the exam reveals it in failed scores.
Display format: Ranked list of top 5 error-rate domains with percentage of candidates missing more than 40% of items in that domain.
Panel 3: E-Learning Invitation Status - Number of candidates who have received an e-learning invitation - Number who have enrolled - Number who have completed - No candidate names unless they have opted in to share
Panel 4: Cycle Timeline Where the cohort is in the 60-day cycle. Days remaining. Phase indicator (Diagnostic, Mastery, Simulation, Peak).
Agency View B: HR Director Dashboard (Weekly Summary)
Purpose: Strategic readiness picture for the promotional cycle. Less granular than the Training Officer view, more forward-looking.
Panel 1: Readiness Distribution A histogram showing how many candidates fall in each readiness level (Foundation Building through Command Ready). No names. This tells HR whether the cohort is tracking toward a strong promotional list or whether intervention is needed at the department level.
Panel 2: Bench Strength by Unit Anonymized view of readiness distribution broken down by organizational unit (Patrol, Investigations, Traffic, etc. - units defined at agency setup). Tells HR which divisions are producing exam-ready candidates and which are lagging.
Panel 3: Projected Pass Rate Based on current aggregate θ distribution and the department’s historical passing threshold: what percentage of the current cohort is projected to score above passing on exam day. Displayed with a confidence interval that narrows as the cycle progresses.
Example: “Based on current performance data, 68% of your candidates are projected to score above your department’s passing threshold of 70. This projection has a margin of error of plus or minus 12 points at Day 22 of 60.”
Panel 4: Comparison to Prior Cycles If historical exam data was provided at onboarding: how does this cohort’s projected readiness compare to prior cycles at the same point in preparation. Available only if the agency provides historical data.
Agency View C: Chief Executive Summary (On Demand)
One page. Four numbers. Built to be read in 90 seconds.
- Cohort Readiness: What percentage of candidates are currently at Proficient or above
- Bench Strength: Which unit has the highest concentration of Command Ready candidates
- Top Policy Gap: The single most-missed General Order or statute across all candidates right now
- Cycle Status: Day X of 60. Projected exam-ready percentage by exam date.
No charts. No drill-down. No individual data. Just the four things a Chief needs to know.
TIER 3: FSP PLATFORM DASHBOARD (TONYA R. DAWSON / FSP ADMIN)
Purpose
This is a business intelligence and product management tool. It serves three functions: 1. Monitor platform health and candidate engagement across all agency clients 2. Identify e-learning outreach opportunities (candidates hitting thresholds) 3. Track IRT item performance and calibration status across the item bank
FSP Panel 1: Agency Client Overview
| Agency | Candidates Enrolled | Avg Session Completion | Cohort Avg θ | Cycle Day | Contract Tier | Renewal Due |
|---|---|---|---|---|---|---|
| [Agency A] | 24 | 78% | +0.3 | Day 22 | Silver | March 2027 |
| [Agency B] | 8 | 91% | +0.8 | Day 41 | Bronze | Ongoing |
| [Agency C] | 61 | 64% | -0.2 | Day 9 | Gold | June 2027 |
At a glance: which clients are engaged, which are underperforming on engagement (a client relationship conversation, not a product failure), and which are approaching renewal.
FSP Panel 2: E-Learning Outreach Queue
The candidates across all agencies who have received an e-learning invitation but have not enrolled. This is Tonya’s business development queue.
| Candidate ID | Agency | Domain Gap | Invitation Date | Response | Action |
|---|---|---|---|---|---|
| #A-7742 | Agency A | Constitutional Law | Day 18 | Not Now x2 | Follow up with Training Officer |
| #B-0091 | Agency B | Budget Management | Day 31 | No response | E-learning intro email available |
| #C-1204 | Agency C | Fire Officer I - Admin | Day 12 | Enrolled | No action needed |
This panel converts the e-learning trigger into a proactive outreach workflow. Tonya does not wait for candidates to find the module - she sees the gap and can reach out to the Training Officer to facilitate enrollment.
FSP Panel 3: IRT Item Bank Performance
The data scientist view. Critical for maintaining calibration quality as response data accumulates.
Item-level statistics (updated after every 50 new responses per item):
| Item ID | Domain | Current b Estimate | Response Count | Calibration Status | Flag |
|---|---|---|---|---|---|
| 2.7-A | Constitutional Law | +1.8 | 312 | Calibrated | None |
| 1.2-A | Criminal Law | +1.1 | 89 | Provisional | Review at 200 |
| 3.6-A | Supervisory | +2.3 | 44 | Pre-calibration | Estimate only |
| 8.5-B | Fire Officer I | +0.6 | 201 | Calibrated | b shift detected |
Flags to watch: - “b shift detected” - the empirical b estimate has moved more than 0.5 from the initial estimate. Item needs review. May indicate the question is ambiguous or the source policy has been updated. - “High error rate at all θ levels” - item may be flawed. Every ability level is getting it wrong at roughly the same rate. Pull for human review. - “Review at 200” - provisional items approaching the threshold for full calibration.
Item bank health summary: - Total items: [count] - Calibrated (200+ responses): [count] - Provisional (50-199 responses): [count] - Pre-calibration (under 50 responses): [count] - Flagged for review: [count]
FSP Panel 4: Revenue and Upsell Signals
| Metric | Value |
|---|---|
| Active Bronze users showing Silver-tier engagement patterns | [count] |
| Silver agencies with candidate volume suggesting Gold ROI | [count] |
| E-learning sessions completed this month | [count] |
| E-learning sessions pending outreach | [count] |
| Agencies in renewal window (next 90 days) | [count] |
A Bronze user showing Silver-tier engagement is defined as: a user completing more than 80% of daily sessions AND requesting oral board simulation (a Silver feature). That is an upsell conversation.
Design and UX Notes for Development
Candidate report: - Mobile-first. Officers will check this on their phone between shifts, not at a desktop. - Session completion confirmation is immediate and celebratory - a brief positive reinforcement moment before the detailed report. - PDF export available for every report. Clean, professional formatting. Candidates may want to keep records or share with a mentor.
Agency dashboard: - Desktop-first. Training Officers and HR Directors will use this at a workstation. - All panels exportable as PDF or CSV for inclusion in briefing documents. - Role-based access enforced at login - a Training Officer cannot access the Chief view and vice versa.
FSP dashboard: - Desktop only. Internal tool. - Includes a direct action button on the e-learning outreach queue: [Send Outreach Email] generates a pre-drafted, personalized email to the agency Training Officer that Tonya reviews and sends. Does not auto-send.
Fairlawn Strategy Partners, LLC, an affiliate of the Institute for Transformative Change - Confidential and Proprietary Contact: Tonya R. Dawson | tonya@fairlawnstrategy.com Document Version 1.0 - June 29, 2026