A methodology of error detection: Improving speech recognition in radiology

Date created: 

Automated speech recognition (ASR) in radiology report dictation demands highly accurate and robust recognition software. Despite vendor claims, current implementations are suboptimal, leading to poor accuracy, and time and money wasted on proofreading. Thus, other methods must be considered for increasing the reliability and performance of ASR before it is a viable alternative to human transcription. One such method is post-ASR error detection, used to recover from the inaccuracy of speech recognition. This thesis proposes that detecting and highlighting errors, or areas of low confidence, in a machine-transcribed report allows the radiologist to proofread more efficiently. This, in turn, restores the benefits of ASR in radiology, including efficient report handling and rcsource utilization. To this end, an objective classification of error-dctcction methods for ASR is established. Under this classification, a new theory of error detection in ASR is derived from the hybrid application of multiple error-dctection heuristics. This theory is contingent upon the type of recognition errors and the complementary coverage of the heuristics. Inspired by these principles, a hybrid error-detection application is developed as proof of concept. The algorithm relies on four separate artificial-intelligence heuristics together covering semantic, syntactic, and structural error types, and developed with the help of 2700 anonymised reports obtained from a local radiology clinic. Two heuristics involve statistical modeling: pointwise mutual information and co-occurrence analysis. The remaining two are non-statistical techniques: a property-based, constraint-handling-rules grammar, and a conceptual distance metric relying on the ontological knowledge in the Unified Medical Language System. When the hybrid algorithm is applied to thirty real-world radiology reports, the results are encouraging: up to a 24% increase in the recall performance and an 8% increase in the precision performance over the best single technique. In addition, the resulting algorithm is efficient and modular. Also investigated is the development necessary to turn the hybrid algorithm into a realworld application suitable for clinical deployment. Finally, as part of an investigation of future directions for this research, the greater context of these contributions is demonstrated, including two applications of the hybrid method in cognitive science and machine learning.

The author has placed restrictions on the PDF copy of this thesis. The PDF is not printable nor copyable. If you would like the SFU Library to attempt to contact the author to get permission to print a copy, please email your request to summit-permissions@sfu.ca.
Document type: 
Copyright remains with the author
School of Computing Science - Simon Fraser University
Thesis type: 
Thesis (Ph.D.)