Breaking new ground: ESnet streams science in real time, simplifying workflows

In April 2024, researchers at the Thomas Jefferson National Accelerator Facility (Jefferson Lab) in Virginia held their breaths as they clicked a button. Moments later, cheers erupted. The monitor showed a steady flow of raw physics data streaming at a blazing 100 gigabits per second3,000 miles across the country to California.

The test didn’t just demonstrate high-speed transmission; it marked a new era of real-time data processing at a scale previously unattainable. The raw data streamed seamlessly from Jefferson Lab to the National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory, over the Energy Sciences Network (ESnet6), with no temporal storage, buffering, or latency issues.

Once the data reached NERSC’s Perlmutter supercomputer, 40 nodes comprising more than 10,000 cores processed the stream in real time and returned the results instantly to Jefferson Lab for further analysis. This could be a game-changer for the big-data scientific community.

A technological leap in science

While we’ve become accustomed to streaming video, live events, and meetings without interruptions, doing the same for massive scientific experiments is an entirely different challenge. Scientific data streams from instruments like particle accelerators and nuclear physics detectors are orders of magnitude larger than any Netflix movie. The April 2024 test was the culmination of years of collaboration between Jefferson Lab and ESnet to solve this problem.

Their solution? A hardware prototype known as EJFAT (ESnet-JLab FPGA Accelerated Transport), which uses advanced field-programmable gate arrays to load balance and stream massive amounts of data in real time over wide-area networks. This breakthrough could pave the way for a revolution in how scientific data is handled, enabling researchers to process information as it’s generated, regardless of the distance between experiment and computation.

The future of data streaming in science

The Department of Energy’s (DOE) vision for the Integrated Research Infrastructure (IRI) is to seamlessly connect research tools, infrastructure, and user facilities to accelerate discovery. With innovations like EJFAT, the future of scientific research is moving closer to real-time processing and analysis, eliminating the bottlenecks caused by traditional file-based methods.

The test in April was a significant step forward. The team replayed data from Jefferson Lab’s CEBAF Large Acceptance Spectrometer (CLAS12) to simulate the real-time streaming environment. Future experiments will focus on connecting live detectors to computing resources, a necessity for upcoming projects like the Electron-Ion Collider.

As the DOE and research institutions continue to develop these groundbreaking technologies, the potential to harness real-time data streaming across multiple labs could radically enhance scientific discovery, transforming how research is conducted and analysed.

What’s next for EJFAT and scientific streaming?

The EJFAT prototype has demonstrated its potential, but there’s much more to come. By leveraging hardware-accelerated processing and advanced networking, EJFAT could be deployed across multiple scientific instruments, enhancing their ability to process real-time data. Collaborations are already underway to connect instruments such as the Advanced Photon Source and the Facility for Rare Isotope Beams to the Oak Ridge Leadership Computing Facility.

As the project expands, the goal is to create a flexible, scalable network that can support the growing data demands of the scientific community. With EJFAT leading the way, the future of science may lie in the power of real-time, continent-spanning data streams.

A new era for scientific collaboration

This experiment at Jefferson Lab in collaboration with ESnet is more than just a test—it’s a glimpse into the future of how science will be conducted. By eliminating data bottlenecks and allowing real-time analysis, researchers can make discoveries faster than ever before. And as technologies like EJFAT continue to evolve, the possibilities for scientific breakthroughs are limitless.

Watch the video

Read the original media release.