This paper was presented in Questionmark User Conference in Miami in March 2010

1. Perfecting the Test through question analysis DELL CONFIDENTIAL

2. About the PresenterNeelov Kar Program Owner for Project Management courses Led the Evaluation Team and introduced QM in Perot Systems Account Manager in Perot Systems since 1998 Worked in other organizations in senior positions and has a total of 22 years of IT experience PMP, ISO 9000 Lead Assessor, Six Sigma and Certified Quality Analyst Certified Instructor for Orientation, Leadership Workshop and Project Management courses. Graduation from Friendship University, Moscow and MBA from BIT Mesra DELL CONFIDENTIAL

3. WELCOME TO DELL SERVICES “Dell Services is an one stop shop for all IT Services delivering its services worldwide in 183 countries with an employee strength of 41000”. DELL CONFIDENTIAL 3

4. Building on years of successful collaboration, together we create Dell Services Serving the public sector, large enterprises and small- and medium-size businesses all over the world More than 41,000 experienced and skilled technology and business-services professionals Approximately $8B revenues from enhanced services and support Best-in-class suite of intelligent, end-to-end services and business solutions to reduce IT complexity and lower costs 4 Global Presence DELL CONFIDENTIAL

5. 5 The Integrated Services PortfolioFrom product support to business solutions, we can help you achieve your business outcomes DELL CONFIDENTIAL

7. Physicians

8. Health Insurance

9. Life Sciences

10. Pharmaceutical

11. Retail Banking

12. Payments and Clearing

13. Insurance

14. Capital Markets

15. Manufacturing

16. Freight and Logistics

17. Telecommunications

18. Travel and Transport

19. Energy

20. Retail

21. Technology

22. Defense

23. Homeland Security

24. Intelligence

25. Civilian

26. State and Local

27. Education6 DELL CONFIDENTIAL

29. Leveraging internet technology, we will be at the forefront of cloud and as-a-service delivery

30. Using standardization to drive down costs and drive up efficiency, from the desktop to the data center

31. Evolving with the industry to bring you the newest technology and more flexible consumption models*By revenue from Gartner’s Industry IT Services Worldwide Market Share Database; August 14, 2009 **TBR's Q209 Service and Support Customer Satisfaction Study DELL CONFIDENTIAL

32. Responsible for Design, Development & Delivery of Project Management and Leadership courses for Dell Services across the globe ISO 9000 certified since 2007 Awarded “2009 Best Place to Learn” by ASTD Dallas Chapter We use QM for Level 2 Assessment SurveyMonkey for Level 1 feedback Adobe Captivate for CBL development PeopleSoft for Training data management Saba as LMS DELL CONFIDENTIAL 8 Learning & Development

33. ADDIE Model 9 DELL CONFIDENTIAL

34. Learn to use the different Perception Report to comprehensively analyze the assessment Understand how item analysis Report helps in identifying the badly constructed questions Learn how Dell Services have done the analysis to refine the test DELL CONFIDENTIAL 10 Learning Objectives

35. P3MM Fundamentals Course 3 Series with 17 VILT Modules Exam after each series Launched in December 2008 Cohort 1, 2 & 3 as Pilot, participants were experienced project managers Used Cohort 1, 2 & 3 Exam results and participant feedback to analyze the test questions DELL CONFIDENTIAL 11 The Case

36. Done through QM Question Statistics Report Item Analysis Report Indices to look for: Discrimination Index Item Difficulty Level Combination of SurveyMonkey and QM Correlation between Test Score and Demography (Role, Account, BU etc) QM and Course Delivery Correlation between Test Score and Course Content Correlation between Test Score and Instructor Test Analysis DELL CONFIDENTIAL

37. Completed Schedule 29 80% or over 12 41% Average time 00:20:05 Completed Cost 21 80% or over 4 19% Average time 00:19:43 Completed Scope 21 80% or over 17 80% Average time 00:09:54 Completed All 20 80% or over 10 50% Average time 00:35:57 Test Results Raised a red flag that there is some problem with Cost Test DELL CONFIDENTIAL

38. Analyzed Distribution of Answers Categories Majority Correct Polarized Scattered Majority Wrong DELL CONFIDENTIAL

39. DELL CONFIDENTIAL 15 Test Analysis Report

40. DELL CONFIDENTIAL 16 Question Statistics Report Scattered

41. DELL CONFIDENTIAL 17 Question Statistics Report Polarized

42. DELL CONFIDENTIAL 18 Item Analysis Report

43. DELL CONFIDENTIAL 19 Item Analysis Report

44. DELL CONFIDENTIAL 20 Item Analysis Report

45. DELL CONFIDENTIAL 21 Item Analysis Report

46. Analyzing the Question and Feedback DELL CONFIDENTIAL

47. DELL CONFIDENTIAL 23 Coaching Report

48. DELL CONFIDENTIAL 24 Assessment Overview Report

49. DELL CONFIDENTIAL 25 P3MM Fundamentals Course has been stabilized after February 2009 and running smoothly since then………

50. Thank You 26