Revamping the CAP Exam: An In-Depth Look at the Process and Why It’s Important

INFORMS, the largest trade association of operations research and analytics professionals that oversees the widely lauded Certified Analytics Professional (CAP®) program, recently announced a revamp of the CAP exam for those pursuing their certification. The new exam will launch April 1 and will be available at all of the program’s Kryterion Test Centers.

The foundational elements of the CAP exam will remain the same. It is still comprised of 100 multiple-choice questions taken in a three-hour time frame. Some questions will be updated and some removed, but the concepts necessary to pass the exam have not changed.

“This type of update is common in test development and needed to keep the exam valid and effective,” said Michael Hamm of Michael Hamm and Associates, an organization that meets the needs of accredited certification organizations like INFORMS.

The update process has been ongoing for the past year by a group of experienced certified subject matter experts (SMEs) in the field of analytics alongside a psychometrician – someone who practices the science of testing. Psychometricians measure the validity, reliability and fairness of an exam. Gerald Rosen of Michael Hamm and Associates revised the CAP examination with the SMEs and was integral in the process of creating valid and reliable questions and answers.

According to Hamm, periodic review and updates of certification examinations are probably the most important quality assurance mechanism for any professional certification program. “INFORMS should incorporate this important process into the ongoing development and maintenance of all of its current and future certification initiatives,” Hamm added.

This is the first comprehensive update of the CAP exam since its inception in 2013. An effective certification program is generally reviewed and updated every 5-7 years because knowledge expands and best practices change.

The process began with the group revising the job task analysis (JTA), which is the formal process for determining what people do in the field – in this case analytics – under what working conditions they do it, and with what knowledge and skills. JTA surveys are used to capture direct feedback from current Certified Analytics Professionals (CAPs) about the knowledge, skills and abilities required for success in the analytics profession.

Using the updated JTA, the team revised the test content outline, which defines the content that will be tested on the CAP exam. “It’s a collaborative effort that requires all parties to have both a comprehensive and accurate test, but also one that reflects the skills and requirements that are currently needed for success in the field,” said Hamm.

Once the test content outline was finalized, item writing and review took place over a period of several months by the SMEs and Rosen. Questions and other items were written using industry resources and reviewed for accuracy and clarity. This extensive process of writing and reviewing ensured that questions are practice-based and assess knowledge that is actually applied “on-the-job.”

Many CAPs are in full support of the exam redesign, and some were even involved in the process.

“The CAP program is essential to increased success in the field of analytics,” said Catherine Truxillo, a CAP and director of Advanced Analytics Education at SAS Institute, Inc. “This review is both necessary to keep the program up to date, but also reflects on the sustainability of not only the exam but the program as a whole. The fact that INFORMS used time and resources to do this speaks volumes about their commitment to the program and the field of analytics.”

For more information about CAP, visit certifiedanalytics.org.