For every hour that physicians spend on direct patient care, they spend 2 hours on administrative tasks and EHR documentations. This causes growing dissatisfaction and burnout amount physicians, decreasing the quality of care that they are able to provide to patients.

The goal of the PhenoPad project is to provide convenient and efficient clinical note taking during physician-patient interactions through the use of AI.

PhenoPad is an application that is easy to use and allows clinicians to capture patient notes in realtime. It provides AI-based note taking services which improve the quality of patient-physician interaction through:

Hand Writing Recognition
Transforms handwritten notes into texts in real time. Refines recognition results using medical language model.
Speech Recognition
Processes speech audio using medical language model and identifies speakers using deep learning approach.
Natural Language Processing
Performs medical term recognition on notes and ASR transcripts, detects and disambiguates abbreviations based on local context.
Clinical Decision Support
Suggests relevant phenotypes and provides differential diagnosis on diseases.


Overview

Hand Writing Recognition

 When a doctor writes down a line of script, Phenopad immediately captures the line and recognizes it using medical language model. Once done analyzing, it displays a text bar above the script with the recognized text. Alternative candidates for each recognized word are also provided for the doctor to correct any spelling or semantic mistakes.

  At the same time, the NLP detects any potential abbreviations and returns the extended forms based on local context. NLP will also search for key medical terms and displays all candidate phenotypes found in the text as togglable tags. The doctor can choose to add or delete phenotypes or switching their states as Yes/No, which indicates whether the patient presents the existence of the specific phenotypes.

Auto-recognizes line strokes and displays extended form of abbreviations.
A single "ca" term will result in "cardiac arrest".
Adding "breast" will update the abbreviation to "breast cancer".
Use NLP to do text mining on collected data for medical term detection.

Speech Audio Recognition

 Instead of jotting down the sentences, the doctor can also choose to record the session. Phenopad provides convenient speech audio service which processes and identifies different speakers using deep learning approach, then displays the transcript on side for quick reference. The NLP will also search through the transcript to detect any key medical terms and suggests all candidate phenotypes. All transcripts of past conversations are also saved locally, which allows the doctor to recapture any missed patient information through a single click.


Realtime conversation quick-view
Saves transcripts of past conversations for future reference
Medical term detection from speech transcripts

Clinical Decision Support

 When the doctor changes the state of a candidate phenotype to "Y", Phenopad automatically performs analysis and provides a list of related phenotypes as further investigate suggestion. Meanwhile, it also determines the diseases that are most commonly found based on the collected phenotypes, and list them out as a clinical diagnosis reference to the doctor. The doctor has the option to view detailed information of each phenotype and disease through an external extension where all information is retrieved from authorized database APIs.

Quickview of saved phenotypes and results from clinical decision support
Overview page to show all phenotypes, suggestions and clinical decision support results

Additional Features

Phenopad also provides the add-in feature which allows the doctor to add a widget by drawing a square or a rectangle on the note page. The add-in widget gives three different options that support drawing pedigree diagrams, inserting local images and taking photos for annotations. All add-in widgets also support resizing and repositioning, and can be minimized into side bar if the note page has run out of writing space.

Draw a square to add a control with 3 options
Drawing: Sketch pedigree diagrams
Insert: Add local image for annotation
Photo: Use built-in camera to take photos

Contact

E-mail

jixuan.cs@gmail.com

Send Email

Address

D.L. Pratt Building #286
University of Toronto - St. George
6 King's College Road, Toronto
ON, M5S 3H5