Piloting a Building Performance Evaluation Tool
Over the past few years, the province of Alberta, Canada, has seen a dramatic increase in healthcare construction and a radical restructuring of its healthcare system. In early 2000, numerous healthcare capital projects were approved and additional projects have recently been approved. The current capital program is $4.2 billion. Additionally, Canada’s first provincewide fully integrated health system became effective on April 1, 2009, bringing together 12 formerly separate health entities.
While the Government of Alberta (Alberta Infrastructure and Alberta Health & Wellness) provides the capital and operating funding for these projects/facilities, Alberta Health Services (AHS) owns and operates the facilities. The three entities work together to ensure optimal performance and accountability for these assets.
How do we measure success? It was this question that led to a joint initiative whereby a building performance evaluation methodology was developed to assess all new capital projects on the basis of four performance dimensions.
The building performance evaluation
The government’s building performance evaluation (BPE) framework and scorecard methodology was described in the Pebble Report published in the May 2010 issue of HEALTHCARE DESIGN. Using a balanced scorecard approach, projects were to be evaluated according to four performance dimensions to recognize an integral relationship between:
- Physical performance: the design of the physical building and life cycle performance;
- Financial performance: operational factors and the fiscal bottom line;
- Functional performance: employee outcomes, such as satisfaction with the workplace, absenteeism, recruitment, and retention rates; and
- Service performance: patient outcomes, such as length of stay and satisfaction with the healthcare experience.
This framework was piloted in two inpatient units at an acute care hospital in Calgary comparing pre- and post-move measures. The research methods used included:
- Primary data collection techniques
- Patient and staff survey
- Observational study
- Secondary data collection; and
- Operational data acquired from the AHS database.
The pre-move data was collected in September 2009 and the post-move data in September 2010. The research team evaluated seven new design elements. This article focuses on only one of the design elements assessed—the impact of increasing the ratio of private inpatient rooms (Note: Alberta Health Services now typically programs at least 80% private rooms in its acute care facilities—a recent change from the traditional semi-private and ward style rooms). Because of the shift to mostly private rooms, it was anticipated that physical distance of the unit would increase. Decentralized nurses’ stations, and medication and supply rooms were programmed to mediate the increased size.
The research proposition that guided the evaluation of this design element was:
- The increased ratio of private inpatient rooms on the unit will result in an improved healthcare experience for patients and staff.
Travel paths. The diagrams in Figure 2 and Figure 3 provide an example of the change to the layout that occurred with the design of the new units as well as the change in staff travel on the unit. As shown, the travel path in Figure 3 is much more localized and condensed.
Time Spent. Figure 4 provides aggregated data of the areas on the units where nurses spent time. As illustrated, pre-move, RNs spent 31.6% of their time in the patient rooms with 19% of their time at the central nurses’ station and 15% of their time on the corridor of the unit. On the new unit, RNs spent 35.8% of their time in the patient rooms, an increase of 4%.
A comparison of the pre- and post-move physical measurement data reveals that although nurses spent more direct care time with patients, the increased size of the units led to an increase in the travel distance for staff—a finding that was corroborated with statistical significance in the survey data.
Staff and patient survey. In a comparison of the pre-move (M=2.67, SD=.58) and post-move (M=3.52, SD=.64) survey results, staff rated the “increased ratio of private inpatient rooms” on the units favorably. Staff appears to be satisfied with the addition of more private rooms; however, the toll the increased distance is taking on staff needs to be considered. For staff, there was significant variance in the responses between groups depending on length of time spent working on the unit and perceptions of the increased ratio of private inpatient rooms on the units (F=4.37, p<0.00), and perceptions of family involvement (F=7.09, p<0.00).
Patients appeared to view the “increased ratio of private inpatient rooms” favorably (pre-move: M=2.89, SD=.73; post-move: M=3.52, SD=.64). Some of the problems that presented with the design of the old units (i.e., lack of space for family members both in and out of the patient rooms) were addressed in the design of the new facility.
Statistically significant correlations were noted between perceptions of having more private inpatient rooms on the units and general perceptions of the physical space. The layout of the private inpatient rooms was shown to be significantly correlated with perceptions of family involvement in providing patient care. These correlations are significant at the .01 level (two-tailed).
This was the first BPE study conducted in the province. Not surprisingly, it took a significant amount of time and resources. Many lessons were learned that will be applied to future evaluations. One of the lessons is to ensure a focused scope of the study—in this pilot project, the team chose to assess seven new design initiatives. In the future, the scope will be limited to one or two items that will allow work on multiple projects with the ability to report results in a more time-efficient manner.
The research team found that physical and financial performance measures relating to infrastructure were easier to collect than the functional and service performance measures. There were long lag times before secondary data was available, and a restructuring to the single entity, Alberta Health Services, from nine former regional health authorities required significant changes related to consolidation of costing systems during the study period. Now that the systems integrations are complete, access to data should be timelier.
A pre- and post-move comparison involves considerable start and stop work over a long period of time, making it difficult to keep momentum and interest. This can be challenging when working with a team that is both internal and external to the organization and have multiple and competing priorities. A BPE project champion that is committed to the project on a full-time basis is crucial, along with a detailed project schedule and ongoing and frequent communication.
Secondary data for the service components of the balanced scorecard (average length of stay, hospital-acquired infections, transfers) and the financial data is still being compiled at this time, but the conclusions of the BPE so far appear to be positive. Based on the available scorecard information, the increase in private patient rooms has resulted in increased RN time in patient care and improved staff and patient perceptions of the environment. The primary disadvantage has b
een an increase in travel distance. When the remaining metrics are evaluated, a more definitive and balanced conclusion can be drawn. HCD
Bev Knudtson is a Guidelines & Evaluation Specialist with Alberta Health Services. Marie Fontaine is a Healthcare Facility Specialist with Alberta Infrastructure. Claudia Steinke is a Researcher for DIALOG and Assistant Professor at the University of Lethbridge. Lynn Webster works as a Principal Architect on the Health Care Planning and Design Team of DIALOG. Ellen Taylor is a Research Associate and Consultant with The Center for Health Design. For more information on Alberta Health Services, please visit www.albertahealthservices.ca.