Blog

<link rel="canonical" href="https://resources.rescale.com/state-of-cloud-hpc/" />

Cloud Adoption is Driving HPC Toward Digital R&D

Post by
Edward Hsu

High Performance Computing is a massive industry ($55 billion by 2024) that is the lifeblood of research and science innovation. From supersonic jets to developing cures for pandemics - research engineers rely on HPC for algorithmically complex workloads running on massive datasets.


Today this HPC industry is still running mostly on-prem in supercomputers and Linux clusters in private datacenters. But the rate of specialized architectures, the diversity of options for simulation software, and the general pressure to bring more agility to research engineers to rival the “developer experience” their enterprise software developers face - has created a massive surge in moving HPC workloads to the cloud.


Big Compute sponsor, Rescale, analyzed the aggregated data of more than 300 customers to generate insights from leading high performance computing practitioners. This inaugural Big Compute report reveals the drivers behind the massive cloud transformation wave that is overtaking the HPC domain.


Big Compute is Driving the HPC Industry to the Cloud


Big Compute is the use of high performance computing (HPC) hardware and specialized software at scale to achieve breakthroughs in science, engineering and business. The report -- “Big Compute: 2021 Cloud HPC Ecosystem” -- reveals key trends in how HPC hardware and specialized software are accelerating cloud adoption across science, engineering and business.

State of Cloud HPC Report - View


An Explosion of Simulation Software and Specialized Architectures


The simulation software market is on fire - an industry expected to reach $21 billion by 2025. Across our library of 600 software packages that Rescale supports, we're seeing most organizations typically run three or more simulation software packages. ANSYS Fluent is the most dominant simulation software provider across most HPC use cases, but within every industry, domain-specific challengers are ever- growing. For example, CFD++ (Aerospace), LS-DYNA (Energy), CONVERGE 3.0 (Automotive). It's a great time to be a science engineer, in terms of the diversity of simulation software choice, and this report catalogues some of these emergent simulation softwares in specific industries.

Source: Big Compute 2021 State of Cloud HPC Report - Download Full Report


Multi-Cloud is Uniquely Valuable for HPC and an Operational Reality


The combination of a fast growing catalogue of HPC software packages, up against different public cloud providers offering different advantages in underlying architecture is driving a multi-cloud trend. This report shows that more than 50 percent of HPC users are using more than one public cloud provider, up from 35% in 2018.

Source: Big Compute 2021 State of Cloud HPC Report - Download Full Report


HPC Leaders are Seeking to Close Talent Gaps and Deliver on Digital R&D


Business leaders understand how cloud changed software development and are now looking to apply cloud transformation to R&D. Research engineers are specialists in the science domains they serve. Cloud providers are specialists in cloud infrastructure. But the universe of engineers who are specialized in both the applied science domains and cloud infrastructure is very small, exacerbated by a talent shortage in the HPC industry. This report found that for most HPC execs, they are looking to automation to deliver more holistic R&D solutions, including workflow automation to help their organizations get the most out of the cloud.

Source: Big Compute 2021 State of Cloud HPC Report - Download Full Report
Source: Big Compute 2021 State of Cloud HPC Report - Download Full Report

See the full report.

Edward Hsu is Chief Product Officer at Rescale, driving intelligent computing for digital R&D. Prior to joining Rescale, Edward ran product and marketing at D2IQ (formerly Mesosphere), as well as product marketing at VMware. Earlier in his career, Edward worked as a consultant for McKinsey & Company as well as in engineering in Oracle's CRM division.