Creating Standardization for Post-occupancy Evaluations
Post-occupancy evaluation (POE) is one of the most common types of research used to inform evidence-based design (EBD). However, while POEs have been part of the design discourse for many years, there are some significant concerns regarding assimilation and implementation in practice.
In the Third Annual Survey of Research Design in Healthcare Environments (see more on the results here), many organizations state they conduct a POE to measure results (73%), yet this is still a less-used way to gather evidence about design strategies for a project (12.5%). This gap between conducting a POE and using a POE may be aligned with gaps identified in a recent report to the U.S. Military Health System, which include:
- A lack of standardized POE methods and metrics;
- Findings/conclusions that are not widely accessible;
- A focus for technical performance and not outcomes related to patient-centered care; and
- Lessons learned not tied to institutional processes.
With a large number of healthcare facilities undergoing replacement or renovation, a knowledge base of post-occupancy evaluations that examine not only user satisfaction but also healthcare outcomes could be an invaluable asset.
POE initiatives at The Center for Health Design
The RIPPLE database. The Center for Health Design (CHD) has been involved in a major research effort over the past few years aimed at developing such a framework and database for linking healthcare facility design with key user outcomes. This database (RIPPLE) aims to create a ripple effect in healthcare design to support not only facility evaluations but also decision making for future projects.
It’s expected to provide a robust framework that enables all constituents involved in healthcare facility design to optimize design decision-making based on POE and other research evidence, benchmark the design strategies used at individual facilities against other similar ones, and assess the effects of design on healthcare outcomes.
When fully developed, this database will include four key focus areas: environmental sustainability, patient safety, worker safety, and effectiveness and quality of care. The initial development focus, environmental sustainability, was supported by funding from the U.S. Green Building Council and was developed through research conducted in 2008-2010.
As a result of this first phase and ongoing development, the RIPPLE database provides user-friendly, online, self-administrated standardized tools for collecting data about implemented design strategies, user feedback, and healthcare outcomes. It provides organizations a way to compare their outcomes against other similar organizations, and it will generate more rigorous research when a large number of facilities participate in the database.
In addition, there’s an option for a more in-depth, on-site audit and evaluation of inpatient facilities that can be conducted for CHD’s Pebble Project partners by CHD staff who will visit the participating facilities. Integrating an on-site environmental audit for healthcare POEs with standardized metric collection provides additional insight about facility performance through interviews, focus groups, and systematic observations during walk-throughs.
Clinic (outpatient) POE. Despite the recent growth in clinic renovation and construction, there’s very little empirical information specific to clinic design that can be used to inform decision-making. Further, the lack of standardized evaluation tools and platforms for sharing the results of design evaluation has limited the generalizability and usability of the existing post-occupancy evaluations conducted in clinics.
With support from California HealthCare Foundation and the Kresge Foundation, CHD is developing a standardized clinic design toolkit and a database for information sharing and benchmarking. The purpose is to support the ongoing cycle of evidence-based clinic design and improvement by developing resources that enable the evaluation of built projects for their effectiveness in meeting strategic and organizational performance goals through design.
This tool will also be integrated into the RIPPLE database.
The Clinic POE toolkit will include five components, which may be used individually or collectively, based on the organizational capabilities of the clinic. These will include:
- A form to gather general information, organizational goals, and design principles (e.g., the building project details, services, number of physicians, model of care, design intents);
- An audit tool (organized by clinic space), including rating items for a set of design features, which is customizable to include only the design features relevant to the organization’s design intents (identified in the first form);
- An anonymous patient questionnaire to gather perceptions of the clinic environment and service quality;
- An anonymous staff questionnaire focusing on perception of the environmental design and work experience; and
- A form for data collection of outcomes related to the selected goals and design intents (e.g., staff turnover rate, clinic cycle time).
The tool is being pilot tested at five organizations and will be completed by May 2013.
An acute-care facility POE (inpatient facilities). Another tool for on-site POE of inpatient acute-care environments is being developed in conjunction with CHD’s Pebble Project. This initiative will combine aspects of user feedback, technical performance, and EBD features. Working with several pilot sites, the tool has been organized into 10 broad categories of evaluation that link to the categories and outcomes established as part of the RIPPLE database. These include:
- Building envelope (e.g., airtightness, and features to increase daylight and reduce glare);
- Building layout (e.g., circulation, adjacencies);
- Unit configuration and layout (e.g., configuration of work spaces, patient care spaces, support functions, visibility);
- Room layout (e.g., clearances, privacy, control, amenities, visibility, storage, technology);
- Acoustics (e.g., sound absorbing materials, physical separation, alarms);
- Lighting (e.g., daylight, artificial light, task lighting)
- Air/ventilation/thermal comfort (e.g., temperature, humidity, odors, CO2);
- Positive distractions (e.g., audio, visual, access to nature);
- Furniture, fixtures, and equipment (FF&E) (e.g., lifts, adjustable furnishings, hand hygiene devices, grab bars); and
- Interior material selection: surfaces/finishes (e.g., durable, cleanable, attractive).
Once launched, the evaluation would be conducted by CHD staff to minimize potential bias and would include expert opinion (Likert scale questions), quantitative measurement (e.g., distance, area, noise level), digital photography, and annotated plans. Participating organizations will also ask staff to complete the Center for the Built Environment survey, to gather perceptions of indoor environmental quality.
The results will be used as the basis for on-site focus groups to further understand any low-scoring areas, also rated as an important consideration by the respondents. Interviews with leadership will also be conducted to better understand the design goals from the strategic
perspective of the organization. Pebble partners undertaking the on-site POE will have the additional option of entering building data in the RIPPLE database for most robust and comprehensive results.
The inpatient room POE. A tool focusing on room design will be used to collect data to evaluate whether patient room design supports effective work processes, as well as patient and staff outcomes. This project, funded through a grant from the American Society of Interior Designers, will create two linked tools that will address three inpatient room types: a medical/surgical room, an ICU room, and a labor/delivery room.
These tools include a patient room interior design checklist (to support design decision making) and a patient room evaluation tool (to evaluate built projects). It, too, will be structured in the RIPPLE framework and organized by design features. For each design feature, a set of questions will be developed to assess whether performance goals are met. Currently the first phase of the project is under way (creating a design decision-making tool based on the available literature).
Based on the set of criteria developed for the checklist, the evaluation tool will be created. The tool will be pilot-tested in 2013 and will have the following components:
- A checklist for a walk-through to the inpatient room (with rating scales for different design features and performance goals);
- A structured questionnaire for patients, families, and staff; and
- A framework for structured interviews and focus groups to garner qualitative insights.
Collectively, these initiatives will strengthen the ability of both design practitioners and healthcare provider organizations to more effectively and systematically evaluate facility design. More information will be presented at the upcoming HEALTHCARE DESIGN Conference Nov. 3-6, 2012 in Phoenix.
Ellen Taylor, AIA, MBA, EDAC, is the Director of Pebble Project for The Center for Health Design; Xiaobo Quan, PhD, EDAC, and Upali Nanda, PhD, EDAC, are Research Associates with The Center; and Anjali Joseph, PhD, EDAC, is The Center’s vice president, director of research.