Introduction
This document provides procedures for activities required in the AI Tools In Pacific Healthcare Clinics policy.
Approval of AI Tools
The policy states that AI tools must be evaluated and approved before they can be used in a clinical setting. The evaluation and approval process is as follows:
- Requests for evaluation should be brought forward by leadership in the clinic wanting to use the new AI tool, or by the Healthcare Operations Team when new tools are available in shared Electronic Health Records systems.
- First, University Information Services, under the direction of the Chief Information Security Officer, does a privacy and security evaluation of the AI tool. An evaluation can be requested here. The rubric for evaluating AI tools can be found here. If UIS approves, evaluation can continue to the next step.
- Second, if the tool is an internal feature provided within one of our approved Electronic Health Record systems, the EHR Support Analysts of the Healthcare Operations Team will evaluate for compatibility with established EHR workflows, policies, and settings. If the EHR Analysts approve, or if an EHR system is not involved, evaluation can proceed to the next step.
- Third, leadership of the clinic that would be using the tool will evaluate the product on the following criteria, using the evidence provided by the tool's publisher:
- Has the tool been shown capable of contributing to better clinical outcomes for our patients?
- Can the known risks of the tool (false positives, false negatives, hallucinations, bias) be adequately mitigated with clinician training and oversight?
- Is the use of this tool compatible with the development of skills by student clinicians, and if so when should it be introduced in their learning?
- If output from the AI will be given to patients or published, is there a risk of claims of copyright infringement? (Clinic leadership should consult with the Office of Legal Affairs as necessary).
- If clinic leadership approves use of the AI tool, this approval should be documented, e.g. in the clinic handbook. Documentation should include:
- The name of the tool.
- What clinical activities the tool has been approved for.
- Any special risks associated with the tool (beyond those general risks associated with all AI tools).
- Any special restrictions on when in their training student clinicians can use the tool.
Annual Review of the Safety and Efficacy of AI Tools
Leadership of each clinic must annually review all AI Tools that have been approved for use in their clinics. Evidence to be considered includes:
- Any new information provided by the tool's publishers about their safety or clinical efficacy.
- Any reports of error, bias, or hallucinations by clinicians using this tool.
- Any other feedback given by clinicians about the use of this tool.
Based on the information considered, clinic leadership may choose to:
- Continue using the tool as is.
- Continue using the tool but adjust training for clinicians to mitigate risks and/or maximize benefit to patients.
- Discontinue use of the tool.
Training on AI Risks
The policy states that, before they use any AI tool in healthcare, clinicians (including student clinicians) must be given training, consistent with their level of involvement, on use of AI in healthcare that includes awareness of the possibility of biases, hallucinations, and of AI's limitations.
The university Compliance Team (HIPAA Privacy Officer, Chief Information Security Officer, Legal Counsel) will provide training to new clinicians (or clinicians new to using AI tools) on the basic concepts of safe and ethical use of AI in healthcare. Currently, that training is being provided as live in-person training. Contact the Chief Information Security Officer to schedule a training. The goal in the future is to have the training available as an assignable module in our healthcare compliance learning management system for on-demand training and annual re-training.
Training on how to use individual AI tools should be provided by clinic staff in the same way that clinicians are trained on the use of other clinical tools. Any risk mitigations for this specific tool, as identified by clinic leadership in the approval or annual review processes, should be included in this training.
Patient Notice
The policy states that patients must be given notice when AI is used in their healthcare (with the exception of administrative tools, such as scheduling tools).
Patient notice can be included in the signed intake consent forms provided to patients. However, consent is not required, only notice. The following is suggested text for a notice:
The following AI tools may be used in your care at this clinic:
[Text describing the tools in use and what they are used for]
Pacific University Healthcare Clinics are committed to the safe and responsible use of AI in healthcare. All AI tools are thoroughly reviewed for their safety, privacy, and efficacy. All clinicians using AI tools receive training on the safe and ethical use of AI in healthcare. All outputs or advice from any AI tool are reviewed for accuracy by a qualified clinician before they are used in any documentation, decision making, or patient communications. AI tools will never make independent decisions about your healthcare.
Please speak to your provider if you have concerns about the use of AI tools.
Notice can also be given verbally, using a script that contains information similar to that shown in the written notice above. If given verbally, the fact that notice was given should be recorded in the notes for the patient encounter.
Feedback on Potential Inaccuracies and Bias
The policy states that any clinician who suspects error, bias, or hallucination in a clinical AI's output should notify their program. Notification should be made using this form. This form is HIPAA compliant and patient names and medical record numbers can be included in the report.
The results of the form initially go to the Chief Information Security Officer, who notifies the Compliance Team, the EHR Analysts, and leadership of any clinic currently using this tool. Clinic leadership is responsible for any necessary follow-up with the clinician making the report.
See Also
AI Tools in Pacific Healthcare Clinics Policy
AI Evaluation Procedures
Questions?
Contact Support