PROVIDENCE, R.I. [Brown University] — When North Korea conducted its recent nuclear weapon test, the blast had been detected by a global seismic sensing network operated by the Preparatory Commission for the Comprehensive Nuclear-Test-Ban Treaty Organization (CTBTO). The network, called the International Monitoring System, aims to “make sure that no nuclear explosion goes undetected.” Software designed in part by a Brown University computer scientist is helping to do just that.

The most recent North Korean test wasn’t terribly difficult to detect. It was a fairly large blast, it occurred in a place where a test wasn’t surprising, and the North Korean government made no effort to hide it. But clandestine tests of smaller devices, perhaps by terrorist organizations or other nonstate actors, are a different story. It’s those difficult-to-detect events that VISA — a machine learning system that Brown’s Erik Sudderth helped to design — aims to find.

The International Monitoring System includes 149 certified seismic monitoring stations around the globe. Those stations send data to the CTBTO’s Vienna headquarters, where analysts compile all seismic events into a daily bulletin supplied to nations around the world. The vast majority of events detected by the system are natural — earthquakes and seismic tremors of various sorts. But occasionally, like recently in North Korea, an event is triggered by a large explosion.

Analysts can easily pick out unnatural events from the characteristics of the seismic waveforms they create, but before they can determine whether an event is unnatural, they need to know that an event has occurred.

“You have hundreds of stations all over the world producing high-dimensional data that’s streaming in 24-by-seven,” said Sudderth, assistant professor of computer science. “[People] can’t look at all the data all the time. They need the help of automated tools.”

Those automated tools keep a constant eye on every station and create a log of potential local detections. They also combine data from multiple stations to hypothesize the time, location, and magnitude of plausible seismic events. Analysts then look at those data to determine if indeed each detection was from a seismic event or just represents random noise. Once an event is confirmed to be real, analysts review it to determine whether it was natural or human-made.

The Vertically Integrated Seismic Analysis (VISA) project began in 2007, when the CTBTO was looking for an upgrade for its software system. The older software was making lots of mistakes, Sudderth said. It was wasting analysts’ time with false positives. But most critically, it was missing lots of smaller events and making errors in triangulating the exact position of events. “It was the job of the analysts to clean up the results of the automated system to some acceptable level of accuracy,” Sudderth said. For him and his colleagues, the raw automated data, combined with data that had been cleaned up by experts, was a goldmine.

“This is what we in machine learning think of as training data,” he said. “We can look at an event that we have reasonable confidence was real and look at its relationship to what the station actually measured. What do the errors look like? What are the biases? We use this historical data to get calibrated models of these things.”

Out of that came a system that models the multiple layers of uncertainty that occur in the processes of generating seismic events and in detecting them. Seismic waves travel in different ways through different types of rock. That introduces uncertainty because there is no high-resolution map of rock types across the entire surface of the Earth. On the detection side, each sensor works a little differently, and all of them are subject to many types of random noise — from wave activity in marine sensors to vehicle traffic on land.

Combining these statistical analyses leads to a comprehensive generative model of the rates at which seismic events of various sizes occur in various locations and the ways that energy from these events propagates to seismic sensors. Sudderth and his team devised an efficient inference algorithm that can scan incoming data to find events that likely represent an actual seismic signal.

VISA has been under testing at the CTBTO’s headquarters in Vienna for the last four years or so. It currently offers the analysts a kind of backstop for the organization’s original automated system.

“As it is now, the analysts have kind of a VISA button [on their computers],” Sudderth said. “They can press the VISA button and it says, ‘Here’s a bunch of stuff we think you missed.’” The analysts can then decide whether or not those events should be included in the daily seismic bulletin.

Sudderth and his colleagues have shown that VISA can reduce the number of missed events by 60 percent compared to the original system. It can also provide more accurate location information in many cases. For example, VISA did a better job than the older system in pinpointing the location of a prior North Korean nuclear test in 2013, Sudderth said. He and his colleagues at the University of California recently published a paper on their work in the Bulletin of the Seismological Society of America. The International Society for Bayesian Analysis awarded this paper the Mitchell Prize, which recognizes an outstanding paper that uses Bayesian analysis to solve an important applied problem.

Sudderth was able to confirm that VISA did detect the recent North Korean test, but that blast was large enough that most traditional systems would have caught it as well.

“If all you cared about was finding events in North Korea, there are simpler, more targeted things you could do,” Sudderth said. “Where this system would potentially be a lot more powerful would be catching someone trying to do a clandestine test of a smaller device somewhere that you don’t know is a test site.”

Sudderth hopes VISA’s ability to detect more events might eventually aid in gaining full ratification of the Comprehensive Nuclear-Test-Ban Treaty. To be enforced by international law, the world’s 44 nuclear nations must ratify the treaty. Eight of those nations, including the United States, have yet to do so. President Clinton signed the treaty in 1996, but Congress refused to ratify it.

“One of the things that was raised as a concern about ratifying it was the difficulty in verification,” Sudderth said. “If the technical systems for validation aren’t good enough, then countries aren’t going to be willing to ratify. So that’s one thing this work is trying to address and remove as an obstacle.”