AI and crowdsourcing to help physicians diagnose epilepsy faster

Epilepsy, a chronic neurological disorder, always causes unprovoked, recurrent seizures — but the experience can be very different from person to person.

A highly individualized condition, epilepsy is extremely difficult to diagnose uniformly or at scale, which is further complicated by the fact that disease expressions change over time. One development in helping us better understand epilepsy is that researchers have been collecting electroencephalography (EEG) data about patients for quite some time.

In a new paper in The Lancet’s EBioMedicine journal, we describe the design and implementation of a new open hybrid cloud platform to manage and analyze secured epilepsy patient data.¹ We also present the findings from a related crowdsourced AI challenge we launched to encourage IBMers across the globe to use Temple University epilepsy research data and our new platform to develop an automatic labelling system that could potentially help reduce the time a clinician would need to read EEG records to diagnose patients with epilepsy.

The results indicate deep learning can play an extremely important role in patient-specific seizure detection using EEG data, gathered using small metal discs—called electrodes—attached to a patient’s scalp to detect electrical activity of the brain. We found that deep learning, in combination with a human reviewer, could serve as the basis for an assistive data labelling system that combines the speed of automated data analysis with the accuracy of data annotation performed by human experts.

The challenges within the Challenge

The IBM Deep Learning Epilepsy Challenge, as described in the EBioMedicine paper, asked participants to develop AI algorithms that could automatically detect epileptic seizure episodes in a large volume of EEG brain data collected by the Neural Engineering Data Consortium at Temple University Hospital (TUH). For this application, operating at high sensitivity (~75%) while maintaining a very low false alarm rate is crucial. IBM researchers who participated as competitors were provided with an ecosystem that allowed them to efficiently develop and validate detection models. Importantly, competitors did not have direct access to the dataset nor were they able to download the data.

The IBM-TUH challenge organizing team processed participant responses through objective and predetermined evaluation metrics. One of our goals was to lower the barrier of entry in using AI model development platforms. We turned to crowdsourcing because it let us draw from a larger pool of talent across the company, essentially turbocharging the discovery process.

As organizers of the challenge, our big test was finding a way to exploit the “wisdom of the crowd” while keeping highly sensitive medical data secured and private. IBM’s hybrid approach to cloud infrastructure played a pivotal role in meeting that challenge, enabling the broader research community to participate in crowdsourced model development, all while keeping patient data secured and preventing it from being downloaded or directly accessed by participants. Our challenge platform infrastructure was housed and hosted data behind a secure firewall, allowing participants to test and submit models and then to receive feedback about the performance of their algorithms.

Close to one hundred IBM researchers participated in the challenge. The criteria for evaluation of submitted models were fairly straightforward—detect a seizure when there is one, without producing a lot of false positives that would undermine confidence in the model being judged. The best performing model in the challenge would have, if used in the real world, decreased the amount of data a doctor would have had to manually review by a factor of 142. That means that instead of having to manually review 24 hours of raw EEG data, using the models developed in the challenge, a doctor would only have to review 10 minutes of data. The key wasn’t just speeding up analysis, but also accurately labeling and reducing the amount of data a doctor would need to review.

After close to two years building the platform, the challenge served as a showcase for its capabilities. The platform facilitated the use of Temple University’s data to develop an effective detection system that we hope can one day assist neurologists to improve the efficiency of EEG annotation. Ultimately, this lays the foundation for clinicians to develop more accurate, personalized and precise treatment plans for epilepsy patients.

We’ve continued to develop our deep learning platform and have already made an updated version available for a public crowdsourced challenge project with MIT. The platform will be eventually open sourced, a move we anticipate will open the door to even more exciting deep learning projects.

Helping physicians to improve patient care

Our work is part of IBM’s larger mission to build a digital health platform that can analyze a range of biomarkers—including sleep, movement and pain—and use those metrics to help physicians better understand, monitor and treat diseases. Deep learning—and the AI models the method creates—can potentially complement doctors’ clinical assessments to help them provide faster and more accurate diagnoses and treatments.

Additionally, IBM Research and Boston Children’s Hospital will soon publish joint work in which AI is used to study epilepsy. The paper showcases AI models that can detect the largest range of epileptic seizure types yet in pediatric patients—including seizure types that have never before been able to be detected automatically using technology. The AI algorithms use temperature, electrodermal activity and accelerometer data from commercially available wearable devices (such as smartwatches) to detect and identify epileptic seizures. This work was also showcased recently at the American Epilepsy Society Annual Meeting (AES) and through PAME (Partners Against Mortality in Epilepsy) Recognition in the Clinical Research Category.²

This work is part of IBM Research’s use of AI to better understand a range of diseases and conditions through the analysis of natural, minimally invasive biomarkers such as speech, language, movement, sleep, pain, stress levels and mood. This includes work to use these data points to help better monitor, measure and predict events for conditions such as chronic pain, Alzheimer’s, Parkinson’s and Huntington’s diseases, as well as psychiatric disorders such as schizophrenia and addiction.