Merit Engine - KSAO and Competency Framework
Merit Engine: KSAO and Competency Framework
Content Validity Foundation for Public Safety Promotional Assessment
Prepared by: Tonya R. Dawson, Founder Fairlawn Strategy Partners, LLC, an affiliate of the Institute for Transformative Change Date: June 30, 2026 Version: 1.0
PURPOSE OF THIS DOCUMENT
This document establishes the Knowledge, Skills, Abilities, and Other Characteristics (KSAO) framework and competency model that governs the Merit Engine item bank. It serves three functions:
Content Validity Evidence: Documents the linkage between job requirements (derived from job analysis) and assessment content, satisfying the Uniform Guidelines on Employee Selection Procedures (29 C.F.R. Part 1607) requirement that selection tools measure KSAOs critical to job performance.
Item Classification Standard: Every item written for the Merit Engine item bank must be tagged to a KSAO code, a competency, a rank level, and a domain. This tagging drives the CAT engine’s domain weighting by rank and enables the platform to report candidate readiness at the competency level, not just the domain level.
Defensibility Anchor: In the event a promotional decision is challenged through grievance, civil service appeal, or federal monitoring review, this document and the item tags constitute the first layer of the agency’s defense that the assessment measured job-relevant knowledge.
SECTION 1: KSAO DEFINITIONS
KSAOs are defined below by type. Each KSAO is coded for reference in item bank tagging.
Knowledge (K) – Declarative: “Knows That”
Knowledge KSAOs are the factual and procedural information required to perform the job at the target rank. Knowledge items are the primary content of the Merit Engine multiple-choice item bank.
| Code | KSAO | Ranks | Track |
|---|---|---|---|
| K-01 | Knowledge of criminal law and elements of offenses under Alabama Code Title 13A | Sgt, Lt, Cpt | LE, Sheriff |
| K-02 | Knowledge of criminal procedure, constitutional law, and case law (4th, 5th, 6th, 14th Amendments) | Sgt, Lt, Cpt | LE, Sheriff |
| K-03 | Knowledge of traffic law and enforcement authority under Alabama Code Title 32 | Sgt | LE, Sheriff |
| K-04 | Knowledge of department policies, procedures, and general orders | Sgt, Lt, Cpt | LE, Fire, Sheriff |
| K-05 | Knowledge of supervisory principles and personnel management practices | Sgt, Lt, Cpt | LE, Fire, Sheriff |
| K-06 | Knowledge of progressive discipline procedures and documentation requirements | Sgt, Lt, Cpt | LE, Fire, Sheriff |
| K-07 | Knowledge of EEO law and fair employment practices applicable to supervision | Sgt, Lt, Cpt | LE, Fire, Sheriff |
| K-08 | Knowledge of budget processes, fiscal year cycles, and resource allocation | Lt, Cpt | LE, Fire, Sheriff |
| K-09 | Knowledge of labor relations, collective bargaining, and grievance procedures | Lt, Cpt | LE, Fire, Sheriff |
| K-10 | Knowledge of training standards and legal requirements for officer development | Sgt, Lt | LE, Fire, Sheriff |
| K-11 | Knowledge of community policing principles and community engagement strategies | Sgt, Lt, Cpt | LE |
| K-12 | Knowledge of report writing standards, documentation requirements, and legal sufficiency | Sgt | LE, Sheriff |
| K-13 | Knowledge of fire behavior, combustion, fire development stages, and suppression tactics | Sgt, Lt | Fire |
| K-14 | Knowledge of NFPA 1021 Fire Officer I supervisory competencies | Sgt | Fire |
| K-15 | Knowledge of NFPA 1021 Fire Officer II company command competencies | Lt | Fire |
| K-16 | Knowledge of NFPA Fire Officer III/IV chief-level competencies and strategic operations | Cpt | Fire |
| K-17 | Knowledge of ICS structure, NIMS compliance, and unified command procedures | Sgt, Lt, Cpt | Fire, LE |
| K-18 | Knowledge of hazardous materials recognition, ERG, and incident command for hazmat | Sgt, Lt | Fire |
| K-19 | Knowledge of Alabama Code Title 14 jail standards and county detention requirements | Sgt, Lt, Cpt | Sheriff |
| K-20 | Knowledge of civil process procedures, writs, and service requirements under Alabama Title 36 | Sgt, Lt | Sheriff |
| K-21 | Knowledge of court security protocols and prisoner transport standards | Sgt, Lt | Sheriff |
| K-22 | Knowledge of the Sheriff’s constitutional role, county government structure, and commission authority limits | Lt, Cpt | Sheriff |
| K-23 | Knowledge of PREA requirements, ADA accommodation, and inmate rights standards | Sgt, Lt | Sheriff |
| K-24 | Knowledge of consent decree obligations, adverse impact analysis, and federal monitoring documentation | Lt, Cpt | Sheriff |
| K-25 | Knowledge of strategic planning, organizational development, and succession management | Cpt | LE, Fire, Sheriff |
Skills (S) – Procedural: “Knows How”
Skill KSAOs reflect the ability to apply knowledge in operational or administrative contexts. Skills are tested through situational judgment items (Phase 2), oral board exercises (Phase 2/3), and structured interviews (Phase 3). Multiple-choice items can measure skill application at the single-application and multi-application difficulty levels.
| Code | KSAO | Ranks | Track |
|---|---|---|---|
| S-01 | Skill in applying legal standards to real-time operational decisions | Sgt, Lt | LE, Sheriff |
| S-02 | Skill in conducting lawful stops, searches, and arrests consistent with constitutional standards | Sgt | LE, Sheriff |
| S-03 | Skill in written communication – report review, policy writing, documentation accuracy | Sgt, Lt, Cpt | LE, Fire, Sheriff |
| S-04 | Skill in personnel counseling, coaching, and performance feedback | Sgt, Lt | LE, Fire, Sheriff |
| S-05 | Skill in conducting disciplinary investigations and preparing documentation | Sgt, Lt | LE, Fire, Sheriff |
| S-06 | Skill in shift planning, resource deployment, and workload management | Sgt, Lt | LE, Fire, Sheriff |
| S-07 | Skill in budget preparation, justification, and fiscal management | Lt, Cpt | LE, Fire, Sheriff |
| S-08 | Skill in training needs assessment and training program planning | Lt, Cpt | LE, Fire |
| S-09 | Skill in emergency scene size-up, command establishment, and resource requests | Sgt | Fire |
| S-10 | Skill in incident action plan development and operational period management | Lt, Cpt | Fire |
| S-11 | Skill in executing civil process, managing contested service situations, and documenting execution | Sgt | Sheriff |
| S-12 | Skill in oral briefing, media communication, and public presentation | Lt, Cpt | LE, Fire, Sheriff |
| S-13 | Skill in data interpretation – reading readiness distributions, pass rate projections, and trend analysis | Lt, Cpt | LE, Fire, Sheriff |
Abilities (A) – Cognitive and Applied
Abilities are underlying capacities that enable performance across multiple tasks. They are typically assessed through structured interviews, assessment center exercises, and the adaptive difficulty progression of the Merit Engine CAT session (higher theta candidates encounter items requiring greater inferential and analytical reasoning).
| Code | KSAO | Ranks | Track |
|---|---|---|---|
| A-01 | Ability to analyze complex situations and apply policy to novel circumstances | Sgt, Lt, Cpt | LE, Fire, Sheriff |
| A-02 | Ability to prioritize competing demands under time pressure | Sgt, Lt | LE, Fire, Sheriff |
| A-03 | Ability to synthesize information from multiple sources and reach a defensible conclusion | Lt, Cpt | LE, Fire, Sheriff |
| A-04 | Ability to identify and navigate competing principles or policies without a clear dominant rule | Lt, Cpt | LE, Fire, Sheriff |
| A-05 | Ability to manage organizational units, delegate effectively, and hold subordinates accountable | Lt, Cpt | LE, Fire, Sheriff |
| A-06 | Ability to assess risk and make high-stakes decisions with incomplete information | Sgt, Lt, Cpt | LE, Fire, Sheriff |
Other Characteristics (O) – Dispositional
Other characteristics are assessed primarily through structured interviews, behavioral anchored rating scales (BARS) in oral board exercises, and reference processes. They are not directly measurable through written MC items and are therefore not tagged to item bank items. They are included here for completeness and for use in Phase 3 structured interview design.
| Code | KSAO | Ranks | Track |
|---|---|---|---|
| O-01 | Integrity and ethical judgment under supervisory authority | Sgt, Lt, Cpt | LE, Fire, Sheriff |
| O-02 | Accountability – willingness to accept responsibility for unit performance and outcomes | Sgt, Lt, Cpt | LE, Fire, Sheriff |
| O-03 | Adaptability – maintaining effectiveness when priorities, resources, or environments shift | Sgt, Lt, Cpt | LE, Fire, Sheriff |
| O-04 | Community orientation – genuine commitment to serving the public and building trust | Sgt, Lt, Cpt | LE, Sheriff |
| O-05 | Resilience – sustaining performance under sustained operational and administrative stress | Sgt, Lt, Cpt | LE, Fire, Sheriff |
SECTION 2: COMPETENCY MODEL
Competencies group related KSAOs into clusters that map to observable job behaviors. The Merit Engine platform reports candidate readiness at the competency level on the Candidate Progress Report and Domain Snapshot. The item bank domain structure maps to these competencies.
Law Enforcement and Sheriff’s Office Competency Model
| Competency | Constituent KSAOs | Primary Rank | Domain(s) |
|---|---|---|---|
| Legal Mastery | K-01, K-02, K-03, S-01, S-02, A-01 | Sgt | 1, 2, 3 (LE); S2 (Sheriff) |
| Supervisory Leadership | K-05, K-06, K-07, S-04, S-05, A-02, O-01, O-02 | Sgt, Lt | 3 (LE/Sheriff) |
| Operational Decision-Making | K-04, S-01, A-01, A-02, A-06 | Sgt | 1, 2, 4 (LE) |
| Administrative Management | K-08, K-09, K-10, S-06, S-07, S-08, A-03 | Lt, Cpt | 7 (LE); S4 (Sheriff) |
| Communication and Documentation | K-12, S-03, S-12, A-03 | Sgt, Lt, Cpt | 5 (LE) |
| Community Engagement | K-11, O-04 | Sgt, Lt, Cpt | 6 (LE) |
| Strategic Leadership | K-25, A-04, A-05, S-13 | Cpt | 7 (LE) |
| Detention Operations | K-19, K-23, S-11, A-06 | Sgt, Lt | S1 (Sheriff) |
| Civil Process and Court Functions | K-20, K-21, S-11 | Sgt | S2, S3 (Sheriff) |
| Constitutional Governance | K-22, K-24, A-04 | Lt, Cpt | S4 (Sheriff) |
Fire Service Competency Model
| Competency | Constituent KSAOs | Primary Rank | Domain(s) |
|---|---|---|---|
| Fire Science and Tactics | K-13, A-01, A-06 | Sgt (FO I) | 11, 12 |
| Company Officer Leadership | K-14, K-05, K-06, S-04, S-09, A-02 | Sgt (FO I) | 8 |
| Mid-Level Command | K-15, K-08, K-09, S-07, S-10, A-03 | Lt (FO II) | 9 |
| Chief Officer / Executive | K-16, K-25, S-12, A-05 | Cpt (FO III/IV) | 10 |
| Incident Command and NIMS | K-17, S-09, S-10, A-06 | Sgt, Lt, Cpt | 13 |
| Hazardous Materials | K-18, A-01, A-06 | Sgt, Lt | 14 |
| Fire Investigation | K-14, K-15, S-03, A-03 | Lt, Cpt | 15 |
SECTION 3: RANK-LEVEL ITEM BANK STRUCTURE
Items in the Merit Engine item bank are written and tagged for a specific target rank. The CAT engine uses rank-level tags combined with domain weights to ensure each candidate session is populated with items appropriate to the rank they are testing for.
Target Rank Definitions
| Code | Track | Rank | Description |
|---|---|---|---|
| LE-S | Law Enforcement | Sergeant | First-line supervisor. Primary focus: legal mastery, direct supervision, operational decision-making |
| LE-L | Law Enforcement | Lieutenant | Mid-manager. Adds: administrative management, program oversight, policy application |
| LE-C | Law Enforcement | Captain | Executive. Adds: strategic leadership, external relations, organizational governance |
| SH-S | Sheriff | Deputy/Corporal to Sergeant | First-line supervisor with jail, patrol, and/or civil process assignment |
| SH-L | Sheriff | Lieutenant | Mid-manager. Adds: county government knowledge, budget, labor relations |
| SH-C | Sheriff | Captain/Major | Executive. Adds: constitutional governance, consent decree/monitoring obligations, succession |
| FO-I | Fire | Fire Officer I / Company Lieutenant | First-line company officer. Primary focus: NFPA 1021 Chapter 4, emergency scene operations |
| FO-II | Fire | Fire Officer II / Captain | Company-level command. Adds: NFPA 1021 Chapter 5, budget, discipline |
| FO-III | Fire | Fire Officer III/IV / Battalion/Chief | Executive. Adds: strategic planning, ICS expanded operations, labor |
Item Tag Schema
Every item in the item bank carries the following mandatory tags:
Item ID: [Domain]-[SubNum]-[RankCode]-[SeqNum] e.g., K01-02-LE-S-001
Domain: Integer (1-15 for LE/Fire; S1-S4 for Sheriff)
Subdomain: Decimal (e.g., 1.2 for Criminal Law - Offenses Against Persons)
Target Rank: LE-S | LE-L | LE-C | SH-S | SH-L | SH-C | FO-I | FO-II | FO-III
KSAO(s): One or more codes from Section 1 (e.g., K-01, S-01, A-01)
Competency: One competency from Section 2 (e.g., Legal Mastery)
Stem Type: recall | single_application | multi_application | exception | competing_principles
b estimate: Pre-calibration difficulty estimate (e.g., +1.2)
Source: Citation string and source tag (e.g., Alabama Code 13A-6-132 [FETCHED])SECTION 4: RANK-DIFFERENTIATED SAMPLE ITEMS
The following items illustrate how the same domain produces different items at different rank levels. This differentiation is critical: a sergeant item tests knowledge application in an immediate supervisory context, while a lieutenant item tests policy administration, and a captain item tests organizational judgment.
Example: Criminal Law / Supervisory Leadership – Three Ranks
SERGEANT LEVEL (LE-S) – Single-Application | Competency: Legal Mastery
Item: K01-02-LE-S-001 (b = +0.8) A patrol officer under your command contacts dispatch requesting guidance. The officer stopped a vehicle for speeding and during the stop observes in plain view on the back seat an open container of alcohol with a liquid remaining in it. Under Alabama Code 32-5A-330, the officer should: A) Issue a warning for the speeding violation only; plain view doctrine does not apply to open container B) Issue citations for both speeding and open container violations; plain view provides legal authority to act on the observation C) Arrest the driver for DUI, since the open container establishes probable cause for impairment D) Seize the container as evidence and release the driver pending laboratory analysis Correct: B | KSAO: K-01, K-03, S-01 | Competency: Legal Mastery | Rank: LE-S
LIEUTENANT LEVEL (LE-L) – Multi-Application | Competency: Administrative Management
Item: K01-02-LE-L-001 (b = +1.5) You are a lieutenant reviewing use of force reports for the past quarter. You notice that one sergeant’s unit accounts for 40% of all use of force incidents department-wide, though the unit represents only 15% of patrol personnel. Shift assignment, call type, and geographic sector are comparable to other units. Your FIRST administrative action should be to: A) Refer the sergeant immediately to Internal Affairs for investigation B) Conduct a structured review of the unit’s incident reports to determine whether the pattern reflects lawful but disproportionate force, training deficits, supervisory lapses, or assignment factors not captured in the data C) Transfer the sergeant to an administrative assignment pending a pattern investigation D) Issue a written directive to the unit reducing use of force to align with department averages Correct: B | KSAO: K-02, K-05, K-06, S-05, A-03 | Competency: Administrative Management | Rank: LE-L
CAPTAIN LEVEL (LE-C) – Competing Principles | Competency: Strategic Leadership
Item: K01-02-LE-C-001 (b = +2.0) As a captain, you receive the department’s annual adverse impact analysis report. The report shows that minority candidates are passing the written promotional exam at a rate 22 percentage points below majority candidates, a ratio that falls below the 4/5ths rule threshold under the Uniform Guidelines on Employee Selection Procedures. The civil service board has asked for a response. The legally and professionally defensible approach is to: A) Discontinue use of the written exam and replace it with a structured interview process, which has lower documented adverse impact rates B) Commission a formal content and construct validity study of the exam, implement targeted outreach to identify and address preparation disparities, and document the agency’s findings and actions for the civil service board C) Adjust passing scores for minority candidates to bring pass rates into alignment with the 4/5ths threshold D) Proceed with the current exam since adverse impact alone is not sufficient legal grounds for discontinuation Correct: B | KSAO: K-07, K-24, K-25, A-03, A-04 | Competency: Strategic Leadership | Rank: LE-C Note: Option D is partially true (adverse impact alone does not invalidate a selection tool) but omits the affirmative obligation to investigate and document. Option B reflects the complete defensible response under the Uniform Guidelines.
Example: Fire Service – Three Ranks
FIRE OFFICER I – Sergeant/Company Lieutenant (FO-I) | Competency: Company Officer Leadership
Item: K14-01-FO-I-001 (b = +0.9) During an active structural fire, one of your crew members reports feeling lightheaded and requests to be rotated out of the attack line. You currently have no available rehabilitation station. As the company officer, your FIRST action should be: A) Order the firefighter to remain in place until the attack line advances to a safe withdrawal point B) Remove the firefighter from the hazard zone immediately, assign them to a rehabilitation area, and notify incident command of the personnel change C) Allow the firefighter to self-monitor and report deterioration if it occurs D) Request a medical unit to evaluate the firefighter while operations continue Correct: B | KSAO: K-14, K-05, S-09, A-02 | Competency: Company Officer Leadership | Rank: FO-I
FIRE OFFICER II – Captain (FO-II) | Competency: Mid-Level Command
Item: K15-03-FO-II-001 (b = +1.6) You are a Fire Officer II preparing the unit’s annual budget request. A firefighter in your company has formally grieved the denial of a shift trade, claiming the denial was retaliatory following a safety complaint the firefighter filed three weeks prior. As the officer of record, your FIRST action is to: A) Deny the grievance at the first step and document the shift trade denial as operationally necessary B) Consult with HR and labor relations before responding, and review the timeline of the safety complaint relative to the denial to assess whether a retaliation claim has merit C) Approve the grievance to avoid escalation and the appearance of retaliation D) Transfer the firefighter to a different unit to remove the conflict Correct: B | KSAO: K-09, K-06, S-05, A-04, O-01 | Competency: Mid-Level Command | Rank: FO-II
FIRE OFFICER III/IV – Battalion Chief (FO-III) | Competency: Chief Officer / Executive
Item: K16-04-FO-III-001 (b = +2.2) You are developing the department’s five-year apparatus replacement plan. The plan requires $3.2M in capital expenditures over the cycle. City finance has indicated that the annual capital budget will not support the full replacement schedule and has asked you to defer two front-line apparatus purchases by three years. NFPA 1911 recommends a 20-year maximum service life for pumpers. One of the apparatus proposed for deferral is currently 18 years old. Your most defensible response to city finance is to: A) Accept the deferral for both apparatus to remain within the city’s capital constraints B) Accept the deferral for the newer apparatus while formally documenting that deferral of the 18-year apparatus creates an unacceptable operational risk, and requesting that apparatus be designated a non-deferrable safety capital item with supporting NFPA documentation C) Refuse both deferrals and submit the full capital request to the city council over the finance director’s objection D) Defer both apparatus and request emergency equipment through a county mutual aid agreement until funding is restored Correct: B | KSAO: K-16, K-08, S-07, A-06 | Competency: Chief Officer / Executive | Rank: FO-III
Example: Sheriff’s Office – Three Ranks
DEPUTY/CORPORAL TO SERGEANT (SH-S) | Competency: Detention Operations
Item: S1-03-SH-S-001 (b = +1.1) You are a jail sergeant conducting rounds. A corrections officer reports that an inmate in general population has been refusing meals for three days and has been observed giving personal items to other inmates. No self-harm or suicide watch has been initiated. Under PREA and NCCHC suicide prevention standards, your FIRST action is to: A) Document the observation in the daily log and monitor for one additional day before escalating B) Conduct an immediate welfare check with medical staff, initiate a suicide risk screening, and place the inmate on observation status pending a formal mental health assessment C) Transfer the inmate to administrative segregation to separate them from general population D) Notify the shift supervisor and defer the decision to the next shift Correct: B | KSAO: K-19, K-23, S-11, A-06 | Competency: Detention Operations | Rank: SH-S
LIEUTENANT (SH-L) | Competency: Constitutional Governance
Item: S4-02-SH-L-001 (b = +1.8) The county commission has passed a resolution directing the Sheriff’s Office to reduce patrol staffing in three rural precincts and reallocate the savings to county road maintenance. The Sheriff has determined that the reduction would leave portions of the county without adequate coverage during peak hours. Under Alabama law, the MOST legally accurate statement about this commission resolution is: A) The commission has full authority over the Sheriff’s Office budget and staffing, and the Sheriff must comply B) While the commission has authority to set the Sheriff’s budget, it does not have authority to direct the Sheriff’s operational deployment of personnel – compliance with budget constraints is required, but the manner of deployment remains within the Sheriff’s constitutional authority C) The Sheriff may ignore the resolution since any budget action requiring operational changes is unconstitutional D) The resolution is only binding if ratified by the county personnel board Correct: B | KSAO: K-22, K-19, A-04 | Competency: Constitutional Governance | Rank: SH-L
CAPTAIN / MAJOR (SH-C) | Competency: Constitutional Governance
Item: S4-03-SH-C-001 (b = +2.3) The federal monitor assigned under your agency’s consent decree has requested documentation demonstrating that your last three promotional cycles were conducted without adverse impact on protected classes. Your agency used a written exam and an oral board interview. You have score distributions by race and gender but did not conduct a formal adverse impact analysis at the time of each cycle. Your MOST defensible response to the monitor is to: A) Provide the raw score distributions and represent that no adverse impact occurred based on the outcomes B) Conduct a retrospective adverse impact analysis on all three cycles using the score distributions, document the methodology and findings, and provide the monitor with a written report that acknowledges the absence of contemporaneous analysis while demonstrating corrective action and a prospective plan for ongoing adverse impact monitoring C) Decline to provide the data on the basis that promotional records are confidential personnel files D) Commission a new promotional cycle to generate a prospective adverse impact record before responding to the monitor Correct: B | KSAO: K-24, K-07, S-03, A-03, A-04 | Competency: Constitutional Governance | Rank: SH-C
SECTION 5: EXERCISE-TO-KSAO MAPPING
This section documents the link between each assessment exercise type in the Merit Engine platform and the KSAOs it is designed to measure. This matrix is the content validity bridge between the job analysis (KSAOs) and the assessment (exercises).
| Exercise Type | Phase | KSAOs Measured | Competencies | Ranks | Scoring |
|---|---|---|---|---|---|
| Multiple Choice – Recall | Pilot | K codes (all) | All knowledge-based | All | IRT-scored (pilot: 1PL Rasch) |
| Multiple Choice – Single Application | Pilot | K codes + S-01, S-02, A-01 | Legal Mastery, Operational Decision-Making | Sgt, FO-I, SH-S | IRT-scored |
| Multiple Choice – Multi-Application | Pilot | K codes + S codes + A-01, A-02 | All | All | IRT-scored |
| Multiple Choice – Exception | Pilot | K codes + A-01, A-04 | Legal Mastery, Administrative Management | Lt, Cpt | IRT-scored |
| Multiple Choice – Competing Principles | Pilot | All K + A-03, A-04 | Strategic Leadership, Constitutional Governance | Lt, Cpt, FO-II, FO-III | IRT-scored |
| Situational Judgment Test (SJT) | Phase 2 | S codes + A codes + O-01, O-02 | Supervisory Leadership, Operational Decision-Making | All | Unscored preview (pilot); SME-scored (Phase 2) |
| Oral Board Simulation (Voice AI) | Phase 2 | S-03, S-04, S-05, S-12, A-02, A-06, O-01 through O-05 | Supervisory Leadership, Communication, Community Engagement | Sgt, FO-I, SH-S | Unscored practice (Phase 2); BARS-scored (Phase 3) |
| Structured Interview | Phase 3 | O codes (all) + A-05, A-06 | Strategic Leadership, Constitutional Governance | Lt, Cpt, FO-II, FO-III | BARS-scored by trained raters |
| In-Basket Exercise | Phase 3 (planned) | S-03, S-06, S-07, A-02, A-03, A-05 | Administrative Management, Communication | Lt, Cpt | SME-scored |
Content Validity Chain
The following chain documents how every item in the Merit Engine item bank traces back to a job requirement:
Job Analysis
-> Critical Task Identified (e.g., "Supervisor reviews use of force reports for pattern analysis")
-> KSAO Required (e.g., K-02, K-05, S-05, A-03)
-> Competency Cluster (e.g., Administrative Management)
-> Domain and Subdomain (e.g., Domain 3.2 - Performance Management)
-> Exercise Type (e.g., Multi-Application MC item, Lt level)
-> Item (e.g., Item K01-02-LE-L-001)
-> IRT Parameters (b estimate, later a and c with calibration data)
-> CAT Engine Selection (item selected to maximize Fisher Information at current theta)
-> Candidate Report (Competency-level readiness reported to candidate)
-> Agency Dashboard (Aggregate competency readiness reported to Training Officer)SECTION 6: OPEN QUESTIONS FOR BETA VALIDATION
The following questions should be answered through beta pilot data and SME review before the KSAO framework is finalized for any agency client:
Job Analysis Confirmation: Has a formal job analysis been conducted for the target rank at this specific agency? The public scaffold KSAO framework reflects general U.S. public safety standards (APOSTC, NFPA 1021, NIMS, POST-STD). Agency-specific job analysis may produce different task and KSAO weightings, particularly for specialized assignments (K9, SWAT, arson investigation, jail administration).
KSAO Criticality Ratings: The current criticality assignments reflect professional judgment. A formal SME rating session (minimum 5-7 subject matter experts at the target rank) should produce agency-specific criticality ratings that override the scaffold defaults.
Adverse Impact Monitoring: The first promotional cycle on the platform should produce baseline adverse impact data (score distributions by race and gender). This analysis should be run at cycle close before any recommendations are made to agency HR.
SJT Validation (Phase 2): Situational judgment test response options require SME consensus scoring before SJT items can be scored and used in the IRT model. Unscored SJTs collected in pilot will build the response key validation dataset.
Oral Board BARS Development (Phase 3): Behaviorally Anchored Rating Scales for oral board scoring require a separate SME workshop to develop behavioral anchors at each rating level. This is a distinct engagement from item bank development.
Sheriff-Specific Job Analysis: The sheriff-specific KSAO framework (Section 1, K-19 through K-24) reflects general standards and Alabama training knowledge. A formal task analysis for the specific sheriff’s office (jail vs. patrol vs. civil process assignment mix) should precede any sheriff’s office contract deployment.
Fairlawn Strategy Partners, LLC, an affiliate of the Institute for Transformative Change - Confidential and Proprietary Contact: Tonya R. Dawson | tonya@fairlawnstrategy.com Document Version 1.0 - June 30, 2026