Brown University had just released its Virtual Degree Conferral program when graduating senior Logan Cho learned he received the inaugural Carney Institute for Brain Science Reproducible Paper Prize. The $750 award honors innovations in reproducibility of publications. 

“The timing of the award allowed for a rewarding culmination of my academic work on an otherwise unusual commencement weekend,” said Cho, who received his bachelor’s degree in neuroscience on May 24. “I was so grateful for the privilege of learning and working with such extraordinary people, both at and away from Brown, over the last four years.”

Cho received the award for his contributions to a study published in the Journal of Clinical Oncology. He was part of a team that developed an open-source algorithm which could be applied to radiology reports in other institutions. 

A light-bulb moment 

Jason Ritt, scientific director of quantitative neuroscience at the Carney Institute, and Andrew Creamer, scientific data management specialist at Brown University Library, teamed up this year to update the existing library’s Innovation Prizes and create new awards — including the Brain Science Reproducible Paper Prize — to reflect Brown’s recent focus on open access, research rigor and transparency, and to highlight innovations in student research. A full list of awardees can be found on the library’s website.

This partnership, Ritt said, is the result of an informal conversation during a Carney Coffee Hour, a community-building event held weekly at the Carney Institute during the academic year.

“I was at a Carney Coffee Hour with Andrew Creamer, and I told him we were thinking of a thesis prize specifically for use of a computationally reproducible paper format,” Ritt said. “He told me about the library’s Innovation Prize, which had a more general scope, but had not been offered recently. I think a light bulb went off for both of us, and we quickly worked out a plan to combine efforts for a pair of prizes highlighting reproducible research.”

The Brain Science Reproducible Paper Prize was focused on computational reproducibility, which means the ability to recreate the results of a study using the code and data used in that study. The judges reviewed submissions based on novelty, effectiveness for reproducibility, openness, and technical implementation. 

“Different fields have followed very different trajectories in building norms around reproducibility with, for example, computer science often leading and life sciences often lagging,” said Ritt, adding that he hopes to continue to hold and expand these awards. “This desire to expand within Carney the use of already existing technology was high on my mind when I made the connection with Andrew.”

Automating clinical reports

The goal of Logan Cho’s project was to develop an open-source algorithm for text mining of medical information from clinical reports of patients with glioblastoma, an aggressive type of cancer. Cho manually tagged glioblastoma radiology reports so that the algorithm had a reliable “ground truth” from which it could learn.

“I saw firsthand how heterogeneous and complex narrative radiology reports can be,” Cho said. “This publicly available pipeline allows for the efficient extraction of crucial medical information from otherwise clumsy and verbose glioblastoma radiology reports.”

The Reproducible Paper Prize judges, David Sheinberg, a professor of neuroscience, and Ph.D. student Abdullah Rashed Ahmed, said Cho’s work was innovative. It “nicely illustrates how open source tools for natural language processing can be used to mine information from clinical reports of patients with glioblastoma,” Sheinberg and Rashed Ahmed wrote in their evaluation. “The approach was innovative and, between the publication and the associated Python notebook made available through GitHub, the analysis pipeline was clearly presented.” 

This summer, Cho is working on remote research and volunteering with GetUsPPE, an organization that allocates personal protective equipment to frontline workers amidst the COVID-19 pandemic. He will join the M.D. program at Icahn School of Medicine at Mount Sinai in August.