EDAC Turns Three
The concept of evidence-based design (EBD) has been around for decades. Roger Ulrich’s 1984 landmark study “View through a Window” was a catalyst that ignited interest in evidence-based design and the belief that design interventions can foster better outcomes and changes in human behaviors and business performance.
It wasn’t until 2000 that the term was coined and started becoming part of the healthcare design vernacular. To define EBD and document a process and common language, The Center for Health Design (CHD) began creating the Evidence-based Design Accreditation and Certification (EDAC) program in 2006, by gathering a group of volunteers at the HEALTHCARE DESIGN Conference to explore the idea.
We had little idea of the process that lay ahead of us. In the room that day were more than 100 healthcare and design professionals eager to contribute their ideas and experiences toward the creation of a program that would transform the way healthcare environments are designed and evaluated.
It took three years and a concentrated group of content experts from clinical, design, and research backgrounds to collect and assemble the base knowledge, develop the exam content, and prepare the study guide materials.
In the past three years, the exam has been administered to architects, interior designers, physicians, nurses, healthcare executives, product manufacturers, and other individuals in the United States, Canada, and seven other countries. At press time, more than 1,095 individuals have been EDAC certified.
The growth of the EDAC community is helping us make progress toward realizing our vision that all healthcare environments are designed using an evidence-based design process. And more progress is needed.
CHD and its volunteer experts have continued to update and enhance the EDAC program with the development of new resources and tools to assist certified individuals with the integration of evidence-based design into their day-to-day work. Over the past year, we have been working on developing a new examination form, which was released in April.
Why change the EDAC exam?
According to the world of psychometrics, the EDAC exam may have reached a level of overexposure. It is not uncommon for the basic concepts and knowledge content of an exam to be shared between colleagues and friends, when it has been in existence for several years.
As more exam candidates know what to expect on the exam, the outcome results can become skewed and the exam may begin to see a decline in discrimination. Both overexposure and discrimination concerns—in addition to changes in content, procedures, and other information—influence the need for periodic updating of the certification exam form.
In psychometrics, an exam with a high degree of discrimination accurately identifies individuals who possess the knowledge to pass the exam and receive the credential. As exam content is shared, candidates may prepare solely for the exam content and may not actually possess the base knowledge desired for certification.
The EDAC certification signifies to clients, employers, and peers that an individual has a base knowledge about the evidence-based design process. To maintain the quality of the program, it was time to retire the original exam form and launch a new test form that would continue to certify qualified professionals.
Updating the EDAC exam
Although the EDAC exam has been updated, the foundational elements of the program remain the same, including the exam content outline, the evidence-based design definition and process, and the use of the study guides as preparatory materials.
Exam content outline
The exam content outline, provided to all exam applicants, includes five content domains and is used to assess each candidate’s knowledge about the evidence-base design process.
The exam item writers use this outline to identify all test requirements and the number of exam questions from each domain to be included on the examination. The five domains are:
- Evidence-based Design for Healthcare
- Construction & Occupancy
EDAC eight-step process
The eight steps of the EDAC process are integrated throughout each of the content domains, and provide a framework for exam item writing that ensures a candidate has a comprehensive knowledge of the EBD process and how to integrate the process into the design and construction of healthcare projects.
The eight process steps are shown in Figure 1.
Developing a credentialing exam
With the assistance of the original psychometric firm, Applied Measurement Professionals, and the new psychometric firm, Castle Worldwide Inc., the EDAC exam task force worked with the psychometricians and test developers to learn the proper method for writing a statistically sound and legally defensible certification exam.
While all the “trade secrets” of developing a credentialing exam cannot be shared, the following section provides an overview of the terminology, question types, and general guidelines the item writers were trained to use when creating the examination.
Subject matter experts/item writers
A subject matter expert is a person with expertise in a given field or profession who develops the content of examinations. These experts then train as exam item writers to write, edit, review, and validate questions. Each exam question is reviewed and validated by at least three other subject matter experts.
Exam content outline
Before developing the exam, the subject matter experts determined the knowledge and skills that defined the minimum knowledge a candidate should have to become certified and defined the overall performance domains associated with a competent evidence-based design practice.
Each domain was then broken down into specific tasks, knowledge, and skills required. The exam content domains were peer-reviewed externally and then rated for relevance, criticality, frequency, and importance.
These results were then translated and weighted to create an exam that tests candidates on topics/content/processes in the proportion that they would observe and interact with the content on the job. The exam content outline guides the item development and examination process, ensuring that the examination reflects the relative importance of the required knowledge and skills.
The process is designed to evolve over time.
One of the first things the EDAC exam item writers learned was “exam questions” are called items. The EDAC exam includes 110 test items that cover the five content domains. Each item is comprised of the following components:
The stem—The stem is the body of the question that defines the problem, describes the scenario, or provides qualifying information. On the exam, the stem will always be worded as a positively asked question and is void of words such as not or except. However, candidates will often be asked to select the best, first, or most appropriate option in response to the question or scenario being presented.
The key—The key is the correct answer. It is likely that multiple options to some degree could be considered correct. However, the key or correct answer is confirmed by th
e subject matter experts. This works in tandem with the stem language of selecting the best, first, and most appropriate option and requires the candidate to carefully read the stem and select the correct answer. On the exam, candidates will never be asked to identify all except, both A and C, all of the above, or none of the above.
The distracters—The distracters are the incorrect options on the EDAC exam. The hardest part of writing the exam was developing plausible, while still incorrect, options. Distracters are specifically written to distract less qualified test-takers from the correct answer. They are written to add complexity to the exam, but are not written to be tricky or confusing. As mentioned above, you will often come across distractors that have a degree of accuracy but do not answer the requirement of best, first, or most appropriate. Here’s a sample question:
- A project team is designing a brand new children’s hospice. Before starting the design, the team employed a researcher who conducted a study in three of the city’s existing hospices using the following methods: surveys; in-depth interviews; focus groups. Which of the following best describes the research methodology used? (Stem)
- Applied research (Distracter)
- Mixed methods (Key)
- Quantitative (Distracter)
- Qualitative (Distracter)
The EDAC exam consists of four-option multiple choice questions written at three basic cognitive levels.
Recall—The most straightforward of the item types. Recall items require the candidate to draw from memorized facts, definitions or rote knowledge such as terms or processes. The stem tends to be short with one variable and the correct answer does not vary based on a proposed situation.
Application—The second tier of item complexity, application items present more than one variable and require careful reading, identification of examples, recognition of relationships, and correlation of the answers with the variables in the question or statement.
Analysis—The most challenging are analysis items, requiring candidates to breakdown information into component parts in order to classify, compare and contrast, and/or distinguish between facts, relevant, and irrelevant information in order to select the correct answer.
Each item is written to and classified within a content category, assigned a cognitive level, and validated according to its appropriateness to the certification-level practitioner. Prior to being released, items are reviewed to ensure they are psychometrically sound and legally defensible.
After all exam items have been written, reviewed, and validated, the examination is created by selecting the appropriate number of items from each content area, as specified in the exam content outline. The assembled set of items is called the test form.
The EDAC exam test form
The process of creating a new test form began with analyzing item performance data collected from EDAC exams taken in the previous three years. During the summer of 2011, Castle Worldwide assembled the statistical outcomes of all existing EDAC test items.
One of the most expressive measures considered when revising EDAC items was item discrimination. Item discrimination is a correlation that describes the relationship between a test taker’s response to a single question and his or her total score on the test.
Item discrimination values helped CHD evaluate how accurately EDAC items identify and differentiate between individuals and their abilities. Ideally, most test items should have a moderate to high discrimination value and provide valuable data regarding overall knowledge of the test taker.
The effectiveness of item distracters was also evaluated by reviewing the answers:
Easy key—Did everyone get the question right? The item is too easy and did not distinguish between test takers in terms of their overall performance.
Different key—Did nearly everyone get the question wrong? The item is too difficult and/or may have been keyed incorrectly.
Conflicting keys—Were multiple answers selected as the key? Even distribution of answer options could indicate that a question is unclear or confusing or have multiple answers.
Over the past year, CHD staff and volunteer experts worked with Castle Worldwide to rewrite existing items and create new items. The updated EDAC exam form includes 110 multiple-choice items, with items that performed well from the original exam form, revised items that reflect reliable and valid item structure, and new items that cover emerging or evolving content.
The exam continues to test candidate’s knowledge about the evidence-based design process and its application, not specific results of published research studies. However, there are a few key differences in the updated EDAC exam described below:
Item clarity—Both stems and options were refined by streamlining the wording of the questions to improve overall clarity to help candidates understand the question. Test takers should know clearly what they are being asked to identify or answer.
Architecture items—Because evidence-based design is linked to architectural design, there were questions about the architectural process rather than the evidence-based design steps. Those questions have been removed.
Research focus—The core purpose of an evidence-based design process is the use and creation of research to inform and measure the association between design features and performance outcomes. The updated exam includes additional items aimed at testing the candidate’s research knowledge.
Answers—Candidates are required to select the correct key from four options instead of three.
Preparing for the updated EDAC exam
The three EDAC study guides continue to be the primary source of information to prepare for the exam. However, all candidates are encouraged to reference additional books and papers and other published sources that promote the advancement of evidence-based design.
Since the launch of the EDAC program in 2006, several new sources of information have become available, including new research studies, peer-reviewed journals, published books, and online webinars that share evidence-based design best practices and research study outcomes.
The updated exam form was beta tested by individuals who attended the 2012 ASHE Planning, Design and Construction Summit in Phoenix and other individuals throughout the design, construction, and healthcare industry. The performance statistics for the exam items were reviewed and a passing score established.
The online version of the updated EDAC exam was officially launched in April 2012, and all future candidates will be tested using the new examination form.
As evidence-based design continues to transform the way we plan, design, and manage healthcare environments, the certification process will continue to expand the community of industry professionals who apply an evidence-based process to the design and development of healthcare settings around the globe.
Not yet EDAC certified? Go to www.healthdesign.org for the information and tools you’ll need to prepare for the test and maintain your credentials. The first of the three EDAC study guides, “An Introduction to Evidence-Based Design,” is available for free download on The Center for Health Design’s website.
The exam content outline is also available on t
his website. HCD
Julie Kent, B.Arch, EDAC, is the Chair of the EDAC Advisory Council and a Senior Healthcare Planner with Eppstein Uhen Architects. She can be reached at julieK@eua.com.