Join Advance-CTR and the Data Science Initiative at Brown for this 5-part series exploring machine learning, its methodology, and application in biomedicine and health. The purpose of this series is to serve as an introduction to machine learning for researchers, clinician scientists, and others who may be interested in using these methods in their research.
Wednesday, October 20, 2021
Ruotao Zhang, MSc: “Role of Calibration in Uncertainty-based Referral for Deep Learning”
The uncertainty in predictions from deep neural network analysis of medical imaging is challenging to assess but potentially important to include in subsequent decision making. Using data from diabetic retinopathy detection, we present an empirical evaluation of model performance and the impact of uncertainty-based referral, an approach that prioritizes referral of observations based on the magnitude of a measure of uncertainty. We consider several configurations of network architecture, method for uncertainty estimation, and training data size. We identify a strong relationship between the effectiveness of uncertainty-based referral and having a well-calibrated model. This is especially relevant as complex deep neural networks tend to have high calibration errors. Finally, we provide evidence that post-calibration of the neural network can improve uncertainty-based referral.
Dilum Aluthge, MD, PhD student: “Supervised Machine Learning Workflows for Electronic Health Records”
Supervised machine learning can be used to develop clinical decision support systems for use in electronic health records (EHRs). The first portion of the talk will provide an overview of the supervised machine learning workflow. The second portion will present an example application of classification using EHR data, specifically the problem list and medication list from a patient’s chart.
1. Rajkomar A, Dean J, Kohane I. Machine Learning in Medicine. N Engl J Med. 2019 Apr 4;380(14):1347-1358. doi: 10.1056/NEJMra1814259. PMID: 30943338.
2. Sinha I, Aluthge DP, Chen ES, Sarkar IN, Ahn SH. Machine Learning Offers Exciting Potential for Predicting Postprocedural Outcomes: A Framework for Developing Random Forest Models in IR. J Vasc Interv Radiol. 2020 Jun;31(6):1018-1024.e4. doi: 10.1016/j.jvir.2019.11.030. Epub 2020 May 4. PMID: 32376173.
About the Speakers
Ruotao Zhang is a PhD student in the Department of Biostatisticsunder the supervision of Dr Steingrimsson and Dr Gatsonis. Before coming to the US, he worked as a data scientist at China Resources Holdings. Ruotao graduated from University of Oxford with a MSc in Applied Statistics, and before that he obtained a BSc in Mathematics from Imperial College London. His research interests focus on statistical machine learning methods with application to biomedical data.
Dilum Aluthge is an MD/PhD student at the Brown Center for Biomedical Informatics, Center for Computational Molecular Biology and the Warren Alpert Medical School. His advisors are Dr. Neil Sarkar and Dr. Liz Chen. His research focuses on the theoretical concepts of learning health systems as well as the practical considerations of their implementation. Specific areas of interest include machine learning, clinical decision support, health information exchange, standards and interoperability, and physiologic reserve. Dilum earned his Bachelor of Science in Applied Mathematics at Brown. He is the co-creator of the PredictMD machine learning framework, which is implemented in the Julia programming language. He is also the founder of the JuliaHealth open source organization.