For my last semester, I have been working on my HCI capstone with fellow students Alex Stern (Information Systems), Salem Hilal (Computer Science), Susanna Park (Business Administration), and Minnar Xie (Psychology & Fine Art). We are working with Dr. Rebecca Jacobson, Professor of Biomedical Informatics at the University of Pittsburgh, to design and prototype a feedback system for radiologists. Our prototype will be used as part of Dr. Jacobson’s grant application, ultimately aiming to support radiologists’ learning throughout their careers.
In the current workflow, radiologists make diagnostic decisions, but rarely receive feedback on final outcomes for the patient. This is partly because existing information systems are separate and disconnected, especially in private practices where radiologists may not be able to access other medical record systems, but also because compiling patient outcomes for radiologist is currently a manual process. Consequently, it is difficult for radiologists to learn from the challenging or ambiguous cases they evaluate throughout their careers on a daily basis.
- Developing the backend for software that the radiologists could use to receive feedback
- Developing experimental methods to test ways to provide more tailored and individualized feedback to practitioners
- Examining and identifying the positive changes that would result from the implementation of the first two parts
For our initial phase of research we conducted interview with working radiologists, literature reviews, constructed an affinity diagram, as well as a customer journey map.
The interviews were critical in providing us insights into the types of tasks accomplished during a radiologist’s workday, the frequency of daily tasks and approaches, as well as the process of bookmarking and saving cases. It also allowed our interviewees to educate us on factors we did not initially consider, such as the differences between private practice and teaching hospital environments.
We examined literary texts in radiology of best practices in radiology, feedback in learning systems, and informatics in healthcare to build context for the project. Through literature reviews, we were able to better understand what types of feedback systems would work, and why previous ones may have failed. Additionally, it provided us with a framework to help drive the development of our initial prototypes.
We constructed an affinity diagram to synthesize and surface findings from our research. By organizing all of our research notes through the affinity diagram, we were able to gather six key insights, discussed more in depth below. For pictures of our affinity diagrams, see Appendix.
Content Journey & Flow Diagram
Based on our interviews and literature reviews of current radiological practices, we created a content journey map that depicts a step-by-step account of a patient’s medical record passing through various stakeholders in the medical process. This customer sequence map and the flow digram showed us the role of the radiologist in relation to the patient’s care. It also shed light on the fact that radiologist have a lack of a structured feedback loop in understanding the final outcomes of the patient’s health.
Insights From Research
Based off the initial phase of research we discovered 6 key insights about radiologists in terms of feedback. These insights provide the guidelines and requirements our solution must meet if it is to be an effective tool in encouraging radiologists to learn and improve.
- Disjoint sources of information cause problems.
- Feedback is useful to radiologists.
- Radiologists need feedback appropriately.
- Radiologists are working at capacity.
- Radiologists are open to and seek feedback.
- Radiologists are motivated to perform well.
Pain Points we Tested
We began the prototyping phase by developing a set of pain points designed to push the radiologists, in order to understand their reactions to certain issues. Our pain points investigated:
- Being evaluated against peers
- The timing of notifications
- The effectiveness of feedback away from work
- Individual vs. aggregated notifications
- The value of immediate alerts on discordant cases
- The usefulness of referencing prior cases
- The benefits of seeing a compiled list of bookmarked cases
Storyboarding & Speedating
This first round of storyboarding led us to see what desired features and functionality might be as well as where not to venture in terms of disrupting their workflow. The initial storyboards led us to form a final set of storyboards that we ran by our client and medical technologists as a proof of concept. The second set were received well and led us to the conclusion of moving forward in the development process. Additionally, we created a service blueprint (shown below) to show how radiologists would interact with the system on a high-level.
1st Round Speed Dating
Second Round Storyboards (proof of concept)
Based of our research and testing, we began wire-framing the initial flow for how our system would work.