13,464-core cluster of Intel® Xeon® Gold 6140 processors deliver 5X the computational power of previous system
1Toulouse University’s New Olympe Supercomputer Reveals Insights in Industry and Oceanography
High Performance Computing (HPC) Intel® Xeon® Scalable Processor
Olympe Highlights:
• 374 compute nodes with 192 GB memory, delivering 1.37 petaFLOPS
• 748 Intel® Xeon® Gold 6140 Processors, 13,464 compute cores
• Two large-memory nodes with 1.5 TB memory
• 12 nodes with four GPU accelerators per node
Executive Summary
CALcul in Midi Pyrénées (CALMIP) is the Toulouse University computing center located in Toulouse, France. Serving a range of researchers across academia, science, and industry, CALMIP needed to extend their computational capacity to offer resources that help maintain its competitive abilities in the region. In 2018, the institute added Olympe, a 13,464-core cluster built on Intel® Xeon® Gold 6140 processors. The system delivers 1.37 petaFLOPS of performance.
Challenge
Established in 1994 in Toulouse, France, CALcul in MIdi Pyrénées (CALMIP) is the computing center of the University of Toulouse, serving the entire scientific community and others. CALMIP enables access to High Performance Computing (HPC) resources for researchers in a wide range of fields, including materials, fluid mechanics, astrophysics, chemistry, and many more. In 2008, they began engaging with industry, providing computing resources for large-scale simulations that could support their innovative activities. Ten percent of CALMIP computing resources is dedicated to industrial research.
As scientific researchers seek deeper understanding into various phenomena and behaviors across their fields, they need more powerful computational resources.
CALMIP’s existing supercomputer could not support these large calculations in a reasonable amount of time. In 2018, CALMIP acquired their next-generation, 1.37 petaFLOPS supercomputer named Olympe.
CALMIP provides programming experience and knowledge to help industrial and academic researchers solve their problems quickly using Olympe.
CASE STUDY
Case Study | Toulouse University’s New Olympe Supercomputer Reveals Insights in Industry and Oceanography
The simulations use computational fluid dynamics (CFD) calculations on a 1 billion nodes model. Plastics engineers have accumulated 150 terabytes of data to work with and are running simulations on Olympe’s 13,000 cores. The results present the details of the heating in the reactor at very fine resolution. Scientists are still studying the results of this simulation, and research continues.
Another project helps scientists understand the move- ments of water masses in the Mediterranean Sea. They are using the Sirocco coastal ocean modeling system to simu- late the currents in the Mediterranean at a resolution of as little as 500 meters. Olympe’s 13,000 cores run an empiri- cal orthogonal function (EOF) analysis on data gathered between October 2012 and November 2013 throughout the Mediterranean basin.
In this close up of one of the analyzed regions (left side), the Levantina basin at the northwestern tip of Egypt (see position in the box) exhibits winter surface cold zone temperatures indicated as RG (Gyre de Rhodes). It is surrounded by warmer water organized in semi-permanent intermediate-scale eddies (e.g. Iera-Petra (IE)) of 80km in diameter.
The high resolution analysis allows scientists to see more detail about the surface current vortices of the currents in the basin (right side, in blue). Current vorticity is organized in
“submersoscale filaments” (in red) of only 4 kilometers. It is the role of these small scales that scientists are studying to understand, and Olympe gives them a view of these struc- tures at a scale they have not been able to study in the past.
The work will help oceanographers, such as Oceanographer researcher Claude Estournel, to better understand and rep- resent the development of the marine ecosystem, beginning with planktons at the bottom of the food chain, which affect migrations and health of fisheries.
“Understanding the evolution of the entire marine ecosystem in the Mediterranean is particularly important since marine ecosystems are expected to change as a result of climate change,” commented Estournel.
Solution Summary
Large HPC clusters, such as Olympe, allow researchers to gain deeper insight into many areas of science and industry.
With their 13,464-core supercomputer built on Intel Xeon Gold 6140 processors, CALMIP delivers the computational resources to academia, industry, and laboratories in the region around Toulouse. Olympe is already enabling new Reactor heating model of 1 billion nodes run on CALMIP Olympe
with 13,000 cores. Courtesy IMFT.
Solution
Built by Atos, the BullSequana X1000 cluster with 13,464 cores of Intel Xeon Gold 6140 processors, is designed to solve very large-scale scientific problems. Olympe gives CALMIP five times the computing power of their previous system.1
“Olympe is our 5th generation of HPC resources at CALMIP,”
stated Nicolas Renon, Operations Manager for CALMIP. “We are currently working with industry on a large simulation that consumes all 13,000 cores of the cluster. Running the same computations on our previous supercomputer would take about six months to complete compared to 15 days on Olympe.”
Industry and research labs turn to CALMIP and Olympe for both the HPC resources and the expertise to get the most benefit from the codes that researchers run. “We have close relationships with Academic users and Innovative industry,”
added Renon. “It is a win-win situation. They know their data and the models, and we provide the programming experience and knowledge to help them solve their problems quickly using Olympe.”
Result
CALMIP currently expects 200 projects to run on Olympe between 2018 and 2022. Two major projects began work in June of 2018.
One is working with researchers at the Institute of Fluid Mechanics of Toulouse on behalf of manufacturers of plas- tics. In a gas-particle fluidized bed reactor used to manu- facture plastics, many processes are occurring that affect heating within the reactor tank, a structure some 30 meters high and five meters in diameter. Overheating can cause a runaway reaction. This results in solidification of the reactor and shuts down the reactor for months, inhibiting continued production of materials.
Engineers in the industry have not understood the processes in the tank in enough detail to take steps to control overheat- ing and runaway. Scientists from the Institute and program- mers from CALMIP are working together to run a massively parallel numerical simulation of the reactions in the tank with very fine detail.
Analyses of the Levantina Basin (left) and surface current vorticities (right) using Sirocco Coastal Ocean Modeling System on CALMIP Olympe with 13,000 cores. Courtesy Sirocco
2
Case Study | Toulouse University’s New Olympe Supercomputer Reveals Insights in Industry and Oceanography
Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation.
Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at https://www.intel.com/content/www/us/en/products/processors/xeon/scalable.html.
Intel, the Intel logo, Xeon and Phi are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others.
© Intel Corporation 042019/RJMJ/KR/PDF Please Recycle 340107-001US understandings of the processes in plastics manufacturing to
help material producers maintain productive facilities. And the new supercomputer brings more detailed knowledge to oceanographers about the currents of the Mediterranean Sea and their impact on the ecosystem within the basin.
Where to Get More Information
Find out more about Intel® Xeon® Scalable Processor family at https://www.intel.com/content/www/us/en/products/proces- sors/xeon/scalable.html.
Find out more about CALMIP at https://www.calmip.univ- toulouse.fr/.
Learn more about Olympe at https://www.youtube.com/
watch?v=pzw9BOVq49Y (French) and https://www.youtube.
com/watch?v=9mvPH6ZEWuk (English).
Read about the work of Olympe and Sirocco at http://sirocco.
omp.obs-mip.fr/.
Solution Ingredients
• 374 compute nodes with 192 GB memory
• 748 Intel® Xeon® Gold 6140 Processors, 13,464 compute cores
• Two large-memory nodes with 1.5 TB memory
• 12 nodes with four GPU accelerators per node
1 New system configuration: Compute: 374 two-socket nodes of Intel® Xeon® Gold 6140 (18 cores), 12 nodes with four NVIDIA V100 GPUs; Fabric:
InfiniBand EDR; Storage: Lustre 1.5pB + NFS 60TB. Previous system configuration: Compute: 612 two-socket nodes of Intel® Xeon® processor E5- 2680 v2; Fabric: InfiniBand FDR; Storage: Lustre 780TB + NFS 111TB. Micro-code level and workload performance not included as this reference is only about peak performance. The theoretical performance of the new supercomputer is 5X the theoretical performance of the previous system.
3
