Join Advance-CTR and the Data Science Initiative at Brown for this 5-part series exploring machine learning, its methodology, and application in biomedicine and health. The purpose of this series is to serve as an introduction to machine learning for researchers, clinician scientists, and others who may be interested in using these methods in their research.

Wednesday, October 27 

Joe Hogan, ScD : “The Role of Machine Learning in Predictive and Causal Inference”

Many machine learning methods can be formulated as complex but flexible statistical models. A common and important use of the models is to generate accurate predictions from a large set of covariates. When prediction is the goal, machine learning methods can learn prediction rules that involve interactions, nonlinearities, and other characteristics of the prediction function that would be difficult to know in advance.

In this talk I demonstrate the utility of machine learning for generating causal inferences. For large-scale observational data, causal inference typically requires the correct specification of one or more component models for the purpose of confounder adjustment. This applies to propensity score methods, inverse probability weighting, regression adjustment, and standardization. The component models are not usually of direct interest, but in cases where there are many potential confounders they can be difficult to specify in advance.

I will first describe the key differences between predictive and causal inference. Then, using a couple of examples from HIV and infectious disease research, I will illustrate how machine learning algorithms that can be formulated as ‘proper’ statistical models play a key role in the process of generating causal inferences.

About the Speaker

Professor Hogan’s research concerns the development and application of statistical methods for large-scale observational missing data. He is interested in causal inference, missing data, and quantifying uncertainty associated with untestable assumptions. Nearly all of his work is motivated by applications in HIV/AIDS and infectious disease. For the past several years he has co-led an NIH-funded international training program designed to build research capacity in biostatistics at Moi University in Kenya.

Dr. Jean Feng, Assistant Professor, Department of Epidemiology and Biostatistics, University of California, San Francisco (UCSF)

Safe approval policies for continual learning systems in healthcare
The number of machine learning (ML)-based medical devices approved by the US Food and Drug Administration (FDA) has been rapidly increasing. The current regulatory policy requires these algorithms to be locked post-approval; subsequent changes must undergo additional scrutiny. Nevertheless, ML algorithms have the potential to improve over time by training over a growing body of data, better reflect real-world settings, and adapt to distributional shifts. To facilitate a move toward continual learning algorithms, the FDA is looking to streamline regulatory policies and design Algorithm Change Protocols (ACPs) that autonomously approve proposed modifications. However, the problem of designing ACPs cannot be taken lightly. We show that policies without error rate guarantees are prone to “bio-creep” and may not protect against distributional shifts. To this end, we investigate the problem of ACP design within the frameworks of online hypothesis testing and online learning and take the first steps towards developing safe ACPs.

The Advance-CTR Translational Research Seminar Series showcases clinical and translational research from across Rhode Island. Presentations, followed by feedback, allow presenters the opportunity to refine and strengthen their research. Seminars are held virtually on the second Thursday of each month.

This month:

• Ami Vyas, PhD: “Are We ‘Choosing Wisely’ and Improving Value in Cancer Care?”

Dr. Jonathan Bartlett PhD, Reader in Statistics, Department of Mathematical Sciences, University of Bath, England

Hypothetical estimands in clinical trials - a unification of causal inference and missing data methods

In clinical trials events may take place which complicate interpretation of the treatment effect. For example, in diabetes trials, some patients may require rescue medication during follow-up if their diabetes is not well controlled. Interpretation of the intention to treat effect is then complicated if the level of rescue medication is imbalanced between treatment groups. In such cases we may be interested in a so-called hypothetical estimand which targets what effect would have been seen in the absence of rescue medication. In this talk I will discuss estimation of such hypothetical estimands. Currently such estimands are typically estimated using standard missing data techniques after exclusion of any outcomes measured after such events take place. I will define hypothetical estimands using potential outcomes, and exploit standard results for identifiability of causal effects from observational data to describe assumptions sufficient for identification of hypothetical estimands in trials. I will then discuss both ‘causal inference’ and ‘missing data’ methods (such as mixed models) for estimation, and show that in certain situations estimators from these two sets are identical. These links may help those familiar with one set of methods but not the other. They may also identify situations where currently adopted estimation approaches may be relying on unrealistic assumptions, and suggest alternative approaches for estimation.