Keep your department EEO litigation free and maintain a balanced workforce. Reducing litigation and achieving a diverse workforce are related goals because departments that have proactive plans to build a diverse workforce also tend to be involved in fewer lawsuits. While achieving one does not guarantee the other, the strategically-managed fire department will strive for both simultaneously.

1. Fire & Police Selection, Inc. Hiring the Most Qualified Firefighters…. … While Avoiding (and Defending) Lawsuits

2. Contact Information Dan A. Biddle, Ph.D. Stacy L. Bell, M.S. CEO, Biddle Consulting Group, Inc./Fire & Police Selection, Inc. 193 Blue Ravine, Ste. 270 Folsom, CA 95630 1-888-990-3473 www.biddle.com | www.FPSI.com Email: dan@biddle.com | s [email_address]

7. The Quickly Changing Climate of Personnel Testing…

9. How Can Testing Practices be Challenged? Title VII Disparate Impact Discrimination Flowchart “ OR ” Practice, Procedure, or Test (PPT) Adverse Impact? YES NO Is the PPT Valid? YES NO Alternative Employment Practice? NO Defendant Prevails YES Plaintiff Prevails END Plaintif f Prevails

19. Firefighter Job Analysis Duties Sample yes yes 4 4.00 8.25 Takes inventory and records and reports equipment and apparatus missing or needing repair or replacement. A3 no yes 4 3.13 8.25 Recognizes the need and makes recommendations for entries in daily log. A2 no yes 5 4.88 8.75 Arrives on time, receives and follows orders, instructions, regarding the work to be done. A1 Assignable Fundamental Best Worker Importance Frequency Domain A Station Duties Supervisor Ratings Firefighter/EMT-I, Firefighter-PM and Fire Engineer Duties

20. Firefighter Job Analysis KSAPCs Sample 3 1 HQ 4.13 6.38 A5, A20, C6, F11 Ability to maintain respect for other’s emotional needs following a crisis. O55 4 1 MQ 4.63 7.63 A5, A20, E1, F8, G2 Willing to remain calm under pressure or heavy workloads, not easily given to hostility, anxiousness, or vulnerability. Wiling to make sensible decisions under pressure. O54 4 1 MQ 4.13 7.25 A1, A2, A24, C5 Willing to be resourceful, goal-oriented, and proactive in bringing tasks to completion. O53 Level Needed Upon Entry Best Worker MQ vs. HQ Importance Frequency This KSAPC is linked to the following duties. Domain O Knowledges, Skills, Abilities, and Personal Characteristics (KSAPCs) : The following knowledge skills, abilities, and personal characteristics affect work behavior and are necessary or important for successful job performance. Supervisor Ratings Firefighter/EMT-I, Firefighter-PM, and Fire Engineer KSAPCs

22. Selection Plan— Sample *The score from this selection procedure will be combined with scores from other selection procedures in the selection process. P/F MQ 3.1 N/A 3.5 1.4 Basic math skills MQ 3.4 N/A 3.9 1.9 Upper body strength P/F MQ 3.9 3.4 3.5 2.6 Knowledge of Fire Codes R MQ 3.7 N/A 3.5 3.9 Verbal communication skills R MQ 4 N/A 3.2 4.2 Interpersonal and team work skills Background Check Structured Interview Written Test Application Form Minimum/Helpful Qualification (MQ/HQ) Importance (1-5) Level Needed (Knowledges Only) (1-4) Level Needed Upon Entry (1-4) Best Worker (1-5) Knowledge, Skills, Ability, Personal Characteristic, or Physical Requirement Selection Procedure and Use ( P ass/Fail, R ank, or C ombine*) KSAPC/Physical Requirement Rating KSAPC/ Phys Req.

23. Job Duties Operationally defined KSAOs Other KSAOs Selection Devices (e.g., application form, tests, interviews) Content Validity Essential #3: Connecting the Test to the Job Content Valid!

28. Comparison Between “Test Use” Options Low Moderate High # Item Requirements Low Moderate High Reliability Requirements High Moderate Low Development Time High Moderate/Controllable Restrictive/Controllable Applicant Flow High Moderate Low Cost Low Moderate High Utility Low Moderate High Litigation "Red Flag" High High Low Defensibility Low Moderate High Adverse Impact Low Moderate High Validation Requirements Pass/Fail Cutoffs Banding Ranking Factor

29. Weighting/Combining Scores Some thoughts... Job Performance Cognitive Ability Experience/ Background Interpersonal Personality Factors

30. Sample Data—Cognitive and Personality Test Scores Total Scores 156 69 W M 175 70 W F 161 70 B M 158 71 B M M M M Gender W B W Ethnicity 63 68 72 Cognitive Score 156 225 175 Personality Score

31. Sample Data—Cognitive and Personality Test Scores Combined Test Scores M M M M F M M Gender W W B B W B W Ethnicity 63 69 70 71 70 68 72 Cognitive % Score 156 156 161 158 175 225 175 Personality Score 73% 79% 81% 81% 81% 83% 83% Combined % Score

35. U.S. Fire Department Input & Opinions Personal Characteristics 55% Cognitive/Academic Skills & Abilities 45% National Fire Select Test™ Cognitive/Academic Component Strategic Decision- Making 14% Problem Solving 14% Map Reading 10% Writing Ability 15% Reading Ability 18% Mathematical Reasoning 15% Mechanical Ability 14% Personal Characteristics Component Thoroughness/ Attention To Detail 9% Public Relations 9% Sensitivity 8% Emotional Stability 10% Proactive/ Goal-Oriented 8% Truthfulness/ Integrity 13% Allegiance/ Loyalty 10% Working Under Stress 10% Teamwork 12% Following Orders 11%

36. Proposed Solutions for Testing Cognitive/Academic Competencies for Entry-Level Firefighters Notes: CV: Content Validity; CRV: Criterion-related validity. CV/CRV Using CV, measure mechanical comprehension skills such as leverage, force, and mechanical/ physics contexts regarding weights, shapes, and distances. Also can measure spatial reasoning (when using a CRV validation strategy). 12% Mechanical Ability CV While a Structured Interview is the best tool for measuring this skill (because the applicant can be asked to apply this skill in firefighter-specific scenarios), some level of this skill can be measured using word problems or other contexts supplied in written format where applicants can consider cause/effect of certain actions. 13% Strategic Decision-Making CRV Measure using word problems measuring reasoning skills in job-rich contexts. 15% Problem-Solving CV Measure using maps and related questions asking applicants how they would maneuver to certain locations. Include directional awareness. 8% Map Reading CV Measure using writing passages or word recognition lists, sentence clarity, and/or grammar evaluation items. 12% Writing CV While a Structured Interview is the best tool for measuring this skill (because the skill includes verbal and non-verbal aspects), some level of this skill can be measured using word recognition lists or sentence clarity items. 15% Verbal Communication CV Measure using either (1) “Test Preparation Manual” approach (where applicants are given a manual and asked to study it for a few weeks prior to taking the test based on the Manual); or (2) a short reading passage containing material at a similar difficulty/context to the job that applicants are allowed to study during the testing session and answer related test items. 14% Reading CV Use written or “work sample” format; measure using a limited number of multiple-choice items. Balance various types of math skills (add/subtract/multiple, etc.). 10% Math Typical Validation Method Proposed Testing Solution Weight Cognitive/Academic (45% Overall Importance)

37. Proposed Solutions for Personal Characteristics for Entry-Level Firefighters Notes: CV: Content Validity; CRV: Criterion-related validity. 11% Following Orders 9% Thoroughness/Attention to Detail CV/CRV These competencies can be effectively measured using either an SJT (using a CV or CRV strategy) or a Conscientiousness (CS) scale (using a CRV strategy). A CS test can be developed using just 20-30 items (using likert- type responses). Such tests are typically successful in predicting job performance in fire settings. 8% Proactive/Goal-Oriented 8% Sensitivity 10% Emotional Stability 9% Public Relations 13% Truthfulness/Integrity 10% Allegiance/Loyalty 10% Working Under Stress CV/CRV Under a CV strategy, a Situational Judgment Test (SJT) can be used for measuring these skills. Alternatively, a custom personality test can be developed using CRV. while these types of assessments can measure if an applicant knows the most appropriate response (using an SJT) or the best attitude or disposition (personality test), they are limited in that they cannot measure whether an applicant would actually respond in such a way. For these reasons, measuring the underlying traits that tend to generate these positive behaviors is typically the most effective strategy. Structured Interviews can also provide useful insight into these types of competencies, as well as background and reference evaluations. However, these tools are time consuming and expensive, so measuring these areas in the testing stage is an effective strategy. 12% Teamwork Typical Validation Method Proposed Testing Solution Weight Personal Characteristics (55% Overall)

38. Selecting the Ideal Firefighter Critical Skills & Abilities Test “A” Test “B” Reading Ability Mathematical Ability Spatial Abilities Mechanical Aptitudes Writing Ability Interpersonal Skills Decision Making Test “C” Map Reading NFST™