Integrated Research Infrastructure: Argonne Combines HPC And Experiments To Speed Discovery
insideHPC, Wednesday, January 31st, 2024
When the massive upgrade at the Advanced Photon Source (APS) at the U.S. Department of Energy's (DOE) Argonne National Laboratory is completed later this year, experiments at the powerful X-ray light source are expected to generate 100-200 petabytes, or 100-200 million gigabytes, of scientific data per year.
That's a substantial increase over the approximately 5 petabytes that were being produced annually at the APS, a DOE Office of Science user facility at Argonne, before the upgrade. And if you consider the DOE's four other light sources, the facilities are projected to yield an exabyte, or 1 billion gigabytes, of data per year in the coming decade.
'An exabyte of data is equivalent to streaming 1.5 million movies every day for a year,' said Nicholas Schwarz, Argonne computer scientist and lead for scientific software and data management at the APS. 'But we need to do a lot more than simply move a lot of data around. For the X-ray experiments carried out at the APS, we need to use advanced computational tools to look at every pixel of every frame, analyze the data in near real time, and use the results to make decisions about the next experiment.'