Challenge
Mackenzie Health employs over 450 physicians and 2,600 other staff members, processes nearly 55,000 patient medical record accesses every day, and has just one privacy officer to monitor everything. It is critical that clinicians have easy access to records when treating patients, and also that safeguards be in place to prevent inappropriate access to records (e.g., snooping).
Identifying healthcare employees who may be accessing patient records for inappropriate reasons requires an in-depth understanding of healthcare workflows. Commonly used rule-based audit solutions detect specific scenarios, such as employees accessing co-workers’ or family members’ records. These systems are imprecise, and can quickly flag far more accesses for review than can be reviewed manually. Mackenzie formed an innovative procurement partnership – the Privacy Auditing Innovation Procurement Project – with Michael Garron Hospital, and Markham Stouffville Hospital. Together, the hospitals initiated a competitive dialogue process to procure a customized smart auditing solution.
KI Audit was developed via the Mackenzie Innovation Institute’s Privacy Auditing Innovation Procurement Project. Through iterative cycles of enhancement, we refined our solution based on feedback from hospitals to create a specialized audit solution for the Ontario health sector.
KI Design collaborated with the hospitals to define the desired functions, performance and benefits of an access audit solution. KI Design was selected as the preferred vendor. We continued to optimize our proposed solution during a six-month pilot phase, and were awarded a contract with the hospitals participating in the project.
Approach
The privacy laws that govern access to personal health information focus on purpose: why each person collects, uses, or discloses personal information, and whether the reasons are appropriate. KI Design rejected rule-based audit systems and instead developed a smart auditing system that uses machine learning to analyze the clinical and operational reasons that staff access records. The core of this technology is more commonly known as an explanation-based auditing system (EBAS) designed and patented by Dr. Fabbri of Maize Analytics.
To detect unauthorized accesses, the technology identifies an intelligible connection between the patient and the employee accessing the patient’s records. AI changes the fundamental question underlying auditing tools from “who is accessing patient records without an authorization?” to, “for what purpose are hospital staff accessing patient records?” Asking this question helps the technology break down staff workflows and identify common and unique purposes for accessing any given medical record. The technology is able to filter out accesses with appropriate explanations so that the Privacy Officer can focus on the much smaller number of suspicious accesses.
Results
Dr. Fabbri’s explanation-based auditing system is generally able to identify appropriate clinical or operational explanations for 95-99% of records accesses, and prioritizes the remaining 1-5% for review based on risk rankings. An evaluation of the pilot phase of the Mackenzie Innovation Institute project found that the smart auditing system configured by our data science team was able to explain 98% of records accesses. The evaluation study provided recommendations for additional data inputs that could further reduce the number of unexplained accesses. These recommendations have informed our subsequent projects.
Our customized KI Audit solution has been adopted by five Ontario hospitals, and we have begun implementation across one of Canada’s largest health networks.
