What’s Happening in the Race to Expand Big Compute, Big Data, and AI

Speaker:

Please welcome to the stage, CEO of Hyperion Research, Earl Joseph.

Earl:

I’d like to spend just a few minutes [inaudible 00:00:16] a couple interesting topics on unlimited computing. And by unlimited computing, which I was asked to talk about, I am referring to the combination of big computing at a reasonable price, big data AI, but also the ease of use access like Rescale and other companies provide. And the combination of those are providing new ways of solving science engineering projects and different types of research.

Earl:

So my agenda for my presentation today is starting out with some of the thoughts processes as far as these new approaches. And then the second topic I’m going to cover some of the new on-prem investments being made around the world. As far as unlimited computing, on-prem investments are growing dramatically. The cloud investments are growing even faster than that. The third topic has all the new different hardware that’s coming out. Right now, we’re in a situation as far as in technical computing and HPC, where there’s just a tremendous amount of innovation taking place in both the hardware, software side. And fourthly, then cloud computing. Can cloud computing actually offer this unlimited computing resource? We’ve heard some talks this morning about people where they’re doing snapshots in time. In other words, using a million cores for maybe 20 minutes. That’s something that was never really possible in the past to do that. And then also how fast are people moving in that direction? And as I mentioned yesterday, we had to change our forecast because people are moving much faster than we were expecting. And then I’ll wrap with a few conclusions and predictions.

Earl:

So first of all, new approaches to solving hard problems. If you take the combination that I mentioned, massive compute, massive data, AI deep learning machine learning, and ease of use interface to that, can we actually give up on the scientific method now? Of course, I would say, no, we’re not going to give up on it, but can we do something different in addition to the scientific method? And the answer to that is I think is yes. And there’s many different examples.

Earl:

Let me just toss out one in astronomy is right now determining how old galaxies are, how far away they are, what if you took all the data that’s available from astronomy, every picture, every diagram, everything that’s ever done put in one data structure and just let a computer do deep learning and machine learning to figure out what the correlations are? That could be an entire new way of determining distances of galaxy, size, motions and everything else, but may have nothing to do with the actual science behind it. It’s purely the machine doing that. That can generate tremendous new breakthroughs that would have been impossible in the past. And we are seeing some of those happening right now today.

Earl:

But right now I want to suggest that AI is in the low IQ stage. Most problems today are bounded problems. There’s lots of observations if you chose choices that are made within those. But each one of the solutions generally are one trick thing. So in other words, if you want to do 10 different tasks, you need ten different AI solutions in that package, either from a software perspective, in some cases, even the hardware perspective. But there’s some many really good successes already. But in the future, both in the near future and next few years and five to 10 years, we think AI is going to be moved to the stage of being high IQ AI. And that means when there’s lots of observations, a lot of relationships and many choices for the system itself to make.

Earl:

But for example, discerning human motivation. We think that’s something that’s within five years right now. Fully automated driving, again, that’s a two to five year zone. Politics may come in and make that take longer, but the capability will be there. The third one I have though is kind of the holy grail for a lot of folks. At what point in time can you actually give a physics book to a computer and have the computer understand the underlying theories and then actually apply them? And our hope is that we’re within the 10 year timeframe for that one. So that’s the progression that we’re tracking and seeing right now.

Earl:

Now there’s both the positive side and negative side. As Stephen Hawking said, “Success in effective AI could be the greatest event in human civilization, but it also could be one of the worst events.” Because there’s both the national security side of it, cybersecurity. One of the talks yesterday said cybersecurity may be totally broken at some point in time because of AI. So there’s two sides of what needs to be looked at there.

Earl:

So where do we stand right now on deep learning? Well, one of the problems with teaching a computer how to think like a human is, we actually don’t know how humans think in detail. So that’s a serious, serious problem. And I suggest we just give up on trying to do that because that’s going to take so long to get to that point. So deep learning machines can learn on their own way beyond the instructions of humans give it. And so that’s the key part is to let the computer learn in a different manner in the manner that computers work. They’re also able to talk with each other and learn from each other. And again, as I mentioned yesterday, NVIDIA has some tremendous examples of where they have farms of different machines doing the same task, which ones do better. They share that data. They do a reset and continue onward. But they’re also capable of creating a culture between all those different systems.

Earl:

The other part about deep learning, and we have a whole different area of research we do related to what we call transparency, is we can’t understand right now how the machine actually is making the decisions in many cases with deep learning. Now, some people have suggested let’s put a super computer next to the AI machine to track what the AI machine is doing so we can tell what it actually did. Well, that’s good for maybe a few things, but that’s not a longterm solution by any means.

Earl:

So it means two different parts of it. One, once the machine does deep learning, it needs to be able to tell us how it made the decisions in so many cases. And in some cases we have to give up and just accept the responses. Now until the machine can tell us how it made the decision, there’s lot of illegal implications and like that before they can actually go full speed. But those are the kind of the big technical problems that many people are trying to address right now.

Earl:

But with everything I just said, this is going to be leading to other new paradigms on how to do things. And I’ve got just an example here. This was a major insurance company, and the way the insurance company worked is you’d call them for a quote on the phone for any one of five types of policies: automotive, life insurance, home policy, and that, and the person who answered the phone literally would type in your information. And by the time they typed in the information and sends it to the system, the real time system had a hundred milliseconds to make a decision. So that’s based on your profile. Now, what happens if they priced it too high? Well, if they gave you too high of a price, you’re not going to sign up for it. They lose the business. What if they price it too low? While they’re losing straight profit on one side, on the other side, the amount of risk could be very high, some different issues.

Earl:

So what did they do? And this is really a shifting paradigm. They said, “Let’s just calculate all the policies for every adult in the United States.” So they just simply did turn this real time problem into batch processing problem. Now it turned out it needed a fraction of the mono compute problem. So this is a case where they actually did a larger problem with less compute because you weren’t doing real time. But what they did is since they flipped this, they were allowed to access multiple additional databases, data structures. So this insurance company right now for everyone in this room has already calculated your price for car insurance, life insurance, home insurance, and five different policies. And by the way, they update that every week. Every week and it’s updated.

Earl:

Now, to some people that’s a scary thought. Now, effectively, this literally had a massive financial impact to this company as far as success in the marketplace. But again, that’s kind of changing the paradigm of how you think about doing work. So using this path of instead of processing each event once at a time doing it for an entire audience, it can be dramatically cheaper, but most important, it creates new and more powerful capabilities. In other words, you can have an AI system actually doing your entire pricing on things.

Earl:

So two examples. For home loan applications, and we’re working with a bank right now, who’s already implemented this and going to allow us to share the success story soon, is they calculated for every adult in the US and every home in the US. We talk about common rhetoric problem with just massive calculations there. But by doing that, if you want a quote, they can give you the most optimal quote they can that fits their financial portfolio they want to go with.

Earl:

Another one that we’re working with the US government on is health insurance fraud. So how could you use AI, big data, and everything else? Now, of course, there’s some personal security issues around personal data and some other issues here. But the ultimate goal here is to track every procedure done on every US person by every doctor and facility and find where the issues are. Now, this is already being done for all the VA systems and anyone who’s been in the military because you’ve signed off. The government gets your personnel file and medical files. A number of insurance companies haven’t joined this too. But again, this allows us to actually change the nature of how we’re doing things using massive data and AI.

Earl:

So with that as background, I’m going to talk a little bit about our predictions on when the next gigantic set of computers are going to be installed around the world. We think in the next five years, there are going to be about 26 Exascale or near Exascale class machines. Exascale just happens to be the peak performance of a system. And around the world, the governments are going to invest roughly $9 billion for these systems. And where are they going by year? So this is showing China, EU, Japan, the US, just those three regions, roughly the number of installations, the type machines and so on. Oh, I should also mention. You notice there’s a note there. The UK is also joined with Brexit and everything else. The UK is no longer part of the EU sharing there.

Earl:

So the US, here are the first three systems. Congress has already said okay to 600 million each. So the price tag on these machines are on the order of $600 million. The delivery of the first one will start in late 2021 and El Capitan, 2022. In addition, there’s a fourth machine. That’s also hasn’t been contracted yet, but in the budget for this.

Earl:

In China, there’s three different major research programs that have been doing prototype experiments. Originally, there was going to be a contest as far as which one gets selected. Most likely all three are going to be given full funding. And so these are three different organizations that are working on the Exascale machines in China. And by the way, if you want more information on any of these, just send me an email because we have entire reports on every one of these systems.

Earl:

In the EU, the EU was planning to build two Exascale systems a little bit later than the rest of the world by about a year. But the reason it’s a bit later is Europe wants to build their own processor. And they’re doing a lot of development work and actually building their own processor for these systems, and then they’ll have additional ones. But the other one interesting thing about Europe, it’s the first country to announce that they’ll have a hybrid HPC and a quantum infrastructure in Europe by 2027. So a lot of the discussion I’ll talk a little bit more about quantum computing, but they actually planned to do a hybrid system by 2027.

Earl:

The UK, as I mentioned, has also decided that they’re not going to be left behind in Europe, and they hope to have their system up and running before all the systems in Europe. So that’s where that global contest is. The cost assessment here is to staggering, again, 700 million pounds to maybe even a billion. Now, in this case, this assessment includes the building, the infrastructure in that. That’s not just the cost of the computer.

Earl:

Now, the most expensive computer in the world though we feel will be the Japan Post-K system, the Fugaku system. It’s already being installed right now. So it’s got a fairly long installation period. They’re expecting it to be about 37 times faster than the previous case system. It won’t quite be an Exascale system, but the price tag right now is 1.1 billion. So this will be the first billion dollar supercomputer being sold or created in the world.

Earl:

So as I mentioned then, all the new hardware from AI to quantum computing right now, this chart was shown yesterday as far as tipping points for technical computing. The reason we like to show this chart is that if somebody does come out with a new solution, new hardware, new software, or whatever, in HPC the market can change very dramatically. Also, when you make it better for HPC, more cost effective, the ultimate computing, AI and big data, a lot more people spend a lot more money. Because if you can do it a little bit better in HPC, there’s always a demand for much more. So this new hardware, new solutions and approach from an investment viewpoint can be tremendous.

Earl:

So here’s just a few of the AI hardware companies that are developing an entire ecosystem around what’s going on. I’m going to drill down in one of them, Cerebras, just because they had done something very unusual. They created the first wafer scale computer. And this wafer is about this size. In the picture here I’m comparing it to the next largest processor out there, the NVIDIA processor, the Tesla V100. Amazing thing that has more than a 1.2 trillion transistors and 400,000 compute nodes. And this right now is at two DOE labs under test. I assume they built maybe a dozen or so of them, but they’re additionally already constructed, working and actually in two customer beta sites.

Earl:

So another approach is the ARM approach. This is our forecast that we just announced a few months ago as far as our sizing of how many systems will be built with ARM processors. The Japanese system being built by Fujitsu is all ARM based with the very custom and very large processor. And so right now we’re expecting the ARM processor market to grow by about 60%, 65% a year. Now, again, that’s starting from a smaller number, but still very strong growth here.

Earl:

As I mentioned, Europe, Europe is developing its own custom processors, and here’s the layout of what they’re doing. Basically their concept is to have a lot of different chiplets in the whole design. And what Europe is hoping to do is to have an organization which they’ve created that would actually do the design work for a company that comes to it. And then that company would take that design work and have a processor made in the builder system. And Atos Bull, for example, is probably going to be the first one that takes advantage of this.

Earl:

But then I’d like to cover the other parts of the system. So Sugon has come up with a new way to cool these very large supercomputers. We have a little innovation award that we gave to them last fall. And what they basically are doing, they’re taking a computer board, they’re putting into a small cage that’s filled with liquid. So this is the 3M phase change fluorinert. And by doing it the way they’ve done it is they aren’t actually using many pumps or anything else. If you’re familiar with phase change cooling is at room temperature, the liquid goes from a fluid to a gas. So basically they just pumped the gas off the top of it, go through chiller and then just pour it back into the thing. But every board has its separate little miniature aquarium holding each board. So this interesting innovation allows them to have a liquid cooled machine where you can change each module for repairs or expansion and everything else.

Earl:

It also lets you overclock, which is really nice. And if you overclock stuffs, let’s say one part of your system is running really intensively and your normal thermals would go up on it, because of this phase change cooling, it actually can take out the heat immediately in that section versus another section because it’s actually immersed inside of the liquid itself.

Earl:

So in quantum computing, the promise is just absolutely substantial. And so on cybersecurity, I think a lot of people have heard about possibilities there, material science, chemistry, pharmaceutical, machine learning, and various optimization areas. The optimization area D-Wave has already sold a number of machines for doing optimization and really hybrid use where you have the quantum computing finding optimal area, then putting it into the HPC system to actually do the calculations and going back and forth.

Earl:

But there are tremendous challenges still. Quantum hardware is not really available. Outside of the D-Wave approach, which isn’t a universal quantum, there’s still some significant issues, but breakthroughs are being made all the time. The performance gains that everyone talks about are theoretical. The actual performance gains that people are experiencing are long ways from that. It’s still a Wild West as far as how you program and everything else on that part.

Earl:

But when I mentioned a lot of investments go, and this is just one snapshot of a report as far as publications, the US and China are almost locked in sync as far as if you use a crude metric like publications and Germany is number three. So this is the heaviest places. As I mentioned yesterday, we are attempting to track every R&D program on quantum computing around the world and AI. And we’re just in the first phases of this. So if you’re interested in those, we would be glad to share that with you. Fortunately it’s funded by the US government. So that means everything’s available to everyone for free. It’s a very much open science research we’re doing there.

Earl:

So here’s the snapshot of one part of the studies we’ve done. We’ve asked a lot of the quantum experts what are their key technical challenges that they see right now? The first thing is obviously the qubit itself needs to be made more reliable. There was an announcement yesterday by an Australian company that has made a step forward in that, and the gates need to be more accurate. Right now when you actually use qubit or quantum computer, the amount of time it stays stable is in fractions of a second. So your whole calculation has to happen very quickly, but you can see there’s a lot of issues here. And the last one’s my favorite. People just don’t think in quantum algorithms or quantum terms right now. So requires a whole change in paradigm as far as thinking.

Earl:

So how do we think quantum computing is going to play out in the near term? We think there’s a lot of ramp-up as far as companies exploring it, investing in it. Part of the reasons you don’t want to be left behind. When the quantum computing started to work, we also think the cloud access model is where it’s going to come from. And this is a case where the user interface I mentioned at the beginning, making them easy to use will be the holy grail of quantum computing because people are just not going to have quantum experts and quantum programmers all around the place.

Earl:

With that, then I’d like to shift gears to cloud computing. So as I mentioned yesterday, we’re declaring 2019 as a major growth period for cloud computing. We’re calling it a tipping point. By tipping point, what we mean is things grew with just a dramatic spike, and you only see this sometimes in the marketplace. For us, we can say, hey, we think a spike is coming, but it’s very hard to tell when it’s going to happen.

Earl:

So this is a chart showing our original forecast line, the blue line. And then the orange line is our new forecast. Now we’re still working on this because if you notice, we showed the big tipping point, but we expect there to be another one or two tipping points in the next few years. So we think the actual orange line will be much higher we’re showing. But the change that we just announced recently is we actually found that there was a billion dollars more being spent on the cloud. And what the spending is just to be real clear. This is not what the cloud providers are buying to put it in the cloud. This is what end users are reaching their pocket books to spend in the cloud. So this is a cloud spending. And so roughly for 2019, it’s about $4 billion was spent by HPC users for cloud capabilities.

Earl:

So which users are those? What sectors are there? What verticals? So the bio life sciences for various reasons you can see is very strong sector. The manufacturing, as everyone’s been talking about, is a very strong sector here. Relative to their overall market shares, you’ll see that the government labs and academic, even though these numbers look healthy, these are relatively small to what they’re doing on-prem. And I would suggest the government users may be the slowest to actually shift to the cloud as far as a major portion of what they do on-prem.

Earl:

And so this is what really, really happening from all our surveys to users. It’s moving from a historic model where everything’s on-prem, fairly inflexible. I mean, three to four year buying cycles moving to really the hybrid emerging model where you have a fair amount on-prem, but then you use the cloud also. And if you’re interested, we have a whole list prioritizing what are the most important parameters like access to new hardware, access to large data, faster queue times. I mean, there’s just a whole list and it varies by industry.

Earl:

Here’s a comparison for all sites and government only why they use cloud. So all sites, extra capacity surges is still number one. For government sites, it’s access to new special hardware software that they don’t have. For government site to get its hands on new software in that takes quite a long time. And so being able to just pull out your credit card and go to the cloud. And by the way, a government credit card can go up to $25,000 with no contracts. So anyone here who’s trying to sell to the government cloud stuff in that, keep it chunks under 25,000 in your goal as far as contracting.

Earl:

And then some conclusions and predictions about what we think is going on. First of all, the Exascale race and the reason I spent time on this. This is where some massive investments are being made. We think the Exascale technologies are going to trickle down to many other parts of the market. Originally, we always thought that the Exascale technologies, are trickled down to medium sized systems, small ones. We think they’re going to trickle right out to the cloud. We think anything that’s developed in Exascale world and all these investments, everything from the hardware to parallel file systems, the amount of software work going under Exascale is just phenomenal. And almost all the software work is about scaling your codes. So it’s taking all this existing, old codes, old applications, software in that, and figure out how you can make it run on a giant machine, which is exactly the same trajectory as how you’d move it into the cloud and take advantage of the cloud.

Earl:

They’re also being designed, not just for HPC as they would traditionally. They’re being designed for three primary pillars, the traditional modeling simulation, AI machine learning, deep learning, and then also dealing with big data, business analytics and then various types of analytics. So there’s different types of processors in those areas. But again, a lot of things that will fit both ways.

Earl:

The other part though, is the amount of money, the importance, because the return on investment national security in that the supercomputers are now becoming massive national assets. But so far that many countries are saying they have to build it themselves too. I mean, in other words, they have to own the actual hardware. And that’s why you see in China, Europe and Japan, you see the processors being developed, memory technologies and everything else. So quite a change there.

Earl:

As I mentioned earlier, there’s lots of new processing accelerators on the way. I don’t know how we’re going to ever figure out how to program them all, integrate them into the entire systems in that. So the hardware vendors that we talk to, each one of them are developing their own software department because they know they have to integrate them because the rest of the world’s not going to do it for them. The beauty here though is there’s going to be a lot more solutions. And we think the future supercomputers and HPC systems are being a broad mix of processors. It’s no longer going to be an accelerator and a core processor. We think we’re in multiple ones. NVIDIA is the dominant accelerator today, but we think there’ll be a host of them in the future.

Earl:

And then the last big prediction I have here is artificial intelligence is growing faster than the HPC market as a whole and faster in almost every IT market. From our numbers right now, we’re looking at roughly a 30% a year growth rate in AI, and that’s across where there’s machine learning, deep learning, all the different attributes. And when we speak to AI, we always kind of lump in big data along with AI because the two of them are kind of go hand in hand right now, or at least moving more.

Earl:

And the last bullet we do think this trust transparency issue will be solved over time. And with that, I’d like to just leave you with this. If you want to learn more, and that we do have this thing, the HPC User Forum. We’ve had I think 75 meetings. But where I’ve got it circled in red there, you can download the meeting presentations from the last decade. So if you’re interested in what’s going on, supercomputing with different sites are doing, whether it’s Boeing, Ford, Department of Defense, DOE, then Europe, China, and Japan. And we actually make that available for the whole community. And with that, I think I have maybe time for one question.

Earl:

There has been a lot of talk around AI and ML today and yesterday, which seems to me it’s a little bit more oriented for ML itself. But I’m more focused on ML for scientific workflow or assimilation workflows, which I don’t see any production level work is out there. And almost everything is in either in the national labs level or academia level. So there are the researchers there, there are the hardware companies, there are cloud providers and the companies like Rescale, aiming to bring them together for the HPC workflows. How do you think what kind of a model can bring that research AI for CAE kind of research to production level feature?

Earl:

Right. So we are tracking all the AI programs around the world. We’ve only started that eight or nine months ago. It will actually take us four years to get there. So you’re absolutely right. The really heavy work that’s being invested in done is in national labs and somewhat academic, but I’m going to argue mostly national labs around the world are doing the heavy lift at the moment. But every one of them wants to have some influence with industry. So they are trying to start working with industry right now. There are a number of industrial concerns though, that are investing heavily in AI. And primarily the German automotive companies, quite frankly, they want to own the future cities as far as smart cities. They feel that the smart cities are going to be based on autonomous vehicles. So if you’re going to make really autonomous vehicles work, you not only want smarts in the car, but you want to have the smarts and the cameras, smarts throughout the whole city and have an integrated system. So they’re investing very heavily because that’s the market. They want to go after.

Earl:

The other place on the industrial side, quite frankly, is China is investing massively as far as building out their infrastructure for the cameras, the video processing real time things, the companies that are building the equipment for that usage. But I think the best way right now, and many of the company larger concerns right now, I think are on a path of exploring. So not on a path of implementing anything right now. And I think most of the implementations will start in the cloud. And they definitely need help as far as figuring out how to do it. There are not a lot of AI experts in the world. And that’s the biggest shortcoming when we do our surveys of industrial people. Why haven’t you done more? Why haven’t you taken advantage of this? And the first response is, “We barely have anyone who even knows what it means.” So there’s a lot of work that needs to be done. Great. Thanks. Well, thank you everyone. I appreciate your taking the time.