EDAC Turns Three

The distracters—The distracters are the incorrect options on the EDAC exam. The hardest part of writing the exam was developing plausible, while still incorrect, options. Distracters are specifically written to distract less qualified test-takers from the correct answer. They are written to add complexity to the exam, but are not written to be tricky or confusing. As mentioned above, you will often come across distractors that have a degree of accuracy but do not answer the requirement of best, first, or most appropriate. Here’s a sample question:

• A project team is designing a brand new children’s hospice. Before starting the design, the team employed a researcher who conducted a study in three of the city’s existing hospices using the following methods: surveys; in-depth interviews; focus groups. Which of the following best describes the research methodology used? (Stem)

1. Applied research (Distracter)
2. Mixed methods (Key)
3. Quantitative (Distracter)
4. Qualitative (Distracter)

Item types
The EDAC exam consists of four-option multiple choice questions written at three basic cognitive levels. 

Recall—The most straightforward of the item types. Recall items require the candidate to draw from memorized facts, definitions or rote knowledge such as terms or processes. The stem tends to be short with one variable and the correct answer does not vary based on a proposed situation.

Application—The second tier of item complexity, application items present more than one variable and require careful reading, identification of examples, recognition of relationships, and correlation of the answers with the variables in the question or statement.

Analysis—The most challenging are analysis items, requiring candidates to breakdown information into component parts in order to classify, compare and contrast, and/or distinguish between facts, relevant, and irrelevant information in order to select the correct answer.

Each item is written to and classified within a content category, assigned a cognitive level, and validated according to its appropriateness to the certification-level practitioner. Prior to being released, items are reviewed to ensure they are psychometrically sound and legally defensible. 

Test forms
After all exam items have been written, reviewed, and validated, the examination is created by selecting the appropriate number of items from each content area, as specified in the exam content outline. The assembled set of items is called the test form. 

The EDAC exam test form
The process of creating a new test form began with analyzing item performance data collected from EDAC exams taken in the previous three years. During the summer of 2011, Castle Worldwide assembled the statistical outcomes of all existing EDAC test items.

One of the most expressive measures considered when revising EDAC items was item discrimination. Item discrimination is a correlation that describes the relationship between a test taker’s response to a single question and his or her total score on the test.

Item discrimination values helped CHD evaluate how accurately EDAC items identify and differentiate between individuals and their abilities. Ideally, most test items should have a moderate to high discrimination value and provide valuable data regarding overall knowledge of the test taker. 

The effectiveness of item distracters was also evaluated by reviewing the answers: 

Easy key—Did everyone get the question right? The item is too easy and did not distinguish between test takers in terms of their overall performance.

Different key—Did nearly everyone get the question wrong? The item is too difficult and/or may have been keyed incorrectly.

Conflicting keys—Were multiple answers selected as the key? Even distribution of answer options could indicate that a question is unclear or confusing or have multiple answers.