Name: Brainstorm AI Singapore 2024: Cutting Data Center Energy Consumption By 50%
Uploaded: 2024-07-31
Duration: 5 min 33 s
Description: Presenter: Tim ROSENFIELD, Founder and CEO, Sustainable Metal Cloud

Presenter: Tim ROSENFIELD, Founder and CEO, Sustainable Metal Cloud

Brainstorm AI Singapore 2024: Cutting Data Center Energy Consumption By 50%

Conferences

I've got a short amount of time, about five minutes uh here to describe um a little bit about what we do and um how we fit into the A I and energy ecosystem before we have a little chat on stage. Um So I'm Tim co-founder, Co CEO of Sustainable Metal Cloud. I was trying to think of a way to describe what we do in a very short amount of time and I guess I'd, I'd characterize it as we seek to solve A I's energy problem. Um A I has AAA substantial energy challenge and problem um which will unpack over the course of the next few slides. We started life in Australia as you can tell from my accent, I'm Australian um about seven years ago. Uh our mission and our vision myself and my co-founders was as outsiders to the data center and A I or as it was known then HPC industry. Um How could we apply engineering to turn energy into knowledge more efficiently? Now, what I mean by that is for most folks, the data center is a place that data is stored. It's now becoming a place that data is processed these A I factories are actually large engineering challenges. And our, our process was, could we start a refresh? Could we think of what is the best way to do this and to accomplish this? So we started in Australia where we built a 20 megawatt um engineering center. The problem that we're trying to solve for how to turn energy into knowledge as cost effectively and as efficiently as possible starts with heat, heat and cooling. And that is one of the drivers of A I's energy challenge, roll it back one step. And the graph I've got on the screen here, you can see a very clear trend. And what this is showing is that over the last decade or two, we've had real substantial improvements in chip density and chip efficiency. But as we've come to now, two nanometer three nanometer process nodes, we're getting less and less efficient with every step. And this has coincided with the rise of A I which unfortunately uses the most energy intensive and hottest chips out there. So behind me, on, on the graph here, you can see in, in green, we have the power limits of nvidia's GP U series. So this is ramping up at a very quick rate and this is driving a massive challenge in the data center. So more heat in the data center means you need more cooling to remove that heat. And that is why our solution it's quite different. So we use liquid. Um We've developed a liquid platform that you actually take the servers, you put them in. It's called single phase immersion. A full server in oil and use the properties of oil which has a heat coefficient 1000 times greater than that of air to remove the heat from the servers. Our system was designed to be modular to go into any data center and to host commonly found workloads in the A I ecosystem like generative A I training using H 100 omniverse using L 40 s and other types of platforms in a nutshell. What it is is we want to solve for taking any DC like the uh like the strip diagram here. And coming up with a platform that would fit into that data center, whether that was built for liquid or not. So whether it was designed 15 years ago or whether it was designed this year, either way we wanted something that was agnostic to the data center that it goes into. And then on top of that, we wanted to improve the efficiency of the servers themselves. So by integrating the data center with our new type of A I factory with an NVIDIA GP U based server, we could achieve breakthrough efficiency which we'll get into in more detail in the next session. Long story short, by vertically integrating the solution, we cut around 45 to 50% of the energy to run NVIDIA GP U based workloads today, over standard airbase configurations. We've designed this to be containerized so that we can deploy it in an existing data center or we can build Greenfield data centers. What we're doing here in Singapore is we've deployed seven of these systems. We're deploying into India. We, we've deployed into, into Australia and we're deploying into Thailand and we're now looking to other regions and around the world. And the final slide before I leave you is how does this get used? So we're actually trying to take this to um as many places as possible. And one of the ways we do that is installing the infrastructure in the data center, making it an A I factory, but then making it easy to use by allowing others to consume this service as a managed service, whether it's a cooling as a service solution or whether it's a GP U as a, as a service solution with partners like NVIDIA. Um and customers around the world are able to access this today.