Name: Brainstorm AI Singapore 2024: It Ain't Easy Being Green
Uploaded: 2024-07-31
Duration: 19 min 11 s
Description: Seth DOBRIN, Founder and CEO, Qantm AI Tim ROSENFIELD, Co-founder and co-CEO, Sustainable Metal Cloud Moderator: Clay CHANDLER, FORTUNE

Seth DOBRIN, Founder and CEO, Qantm AI Tim ROSENFIELD, Co-founder and co-CEO, Sustainable Metal Cloud Moderator: Clay CHANDLER, FORTUNE

Brainstorm AI Singapore 2024: It Ain't Easy Being Green

Conferences

Brainstorm AI Singapore 2024: Will AI Help Combat The Carbon Crisis— Or Make It Worse?

So, um I want to just kind of follow with a couple of questions for you, Tim, because these were really uh impressive. So you can, you can basically get carbon emissions, electric power consumption uh and, and spatial requirements down by about 50% for each of those three things. And I couldn't quite hear you back. But did you mention the recent kind of metric that, that uh kind of benchmark this? Yes, I didn't actually mention that on the presentation, but indeed, no, it, it's one thing to come out with a system that um you know, is very energy efficient. Obviously, we live in a world of um of intense commentary around sustainability. And there's obviously the propensity for greenwashing. It's just a fact of life. So it was very important for us that we could demonstrate to our users and frankly to folks that we're in dialogue with who might use our technology, that this system despite being generationally further ahead in terms of its consumption of energy was as performant as it could be. So to your point, Clay, um we worked with a industry association called ML Commons, ML Commons um For those that don't know, is um I'd kind of describe it like the Olympics of A I uh for, for geeks. Basically, it's an open source uh forum. It was founded by NVIDIA and Google and Microsoft and there's about 100 and 25 members now. And every six months there is a competition to run different A I models and see how fast your system runs them and forever and a day that's been the main metric as to how this Olympics are judged moving forward. Um And this submission that we just did ML Commons said, look A I has a sustainability problem. There's a green problem with A I. So why don't we start tracking power? How much power does it take to run these models? The end output is we've been the first in the world to submit, not just the performance, but the power benchmarks and have that verified by the industry. And it's, it's a very exciting moment for us. It's great. You know, um when we were visiting this data center, you made some comments that I thought were really interesting and I thought maybe I might ask you to share with the group here. Um That the reason you came to Singapore was kind of the of the uh you know, the, the, the paradox of plenty. You know, we often feel, you know, hear the story about how uh countries that are blessed with lots of natural resources actually don't develop that much because they don't have to. Singapore is the opposite of that and maybe you could say sure why, why that influenced your decision to come here. As I mentioned, I'm, I'm, I'm from Australia um originally and Australia is a big country with a lot of energy. So that's sustainability and, and uh efficiency has never been high on the agenda. Singapore is a, is a, is a perfect environment to put, to put, to work. The thesis that we need to do better. Singapore has power constraints, it has land constraints, it has a very strong drive at government policy level and within in within industry to do better on the sustainability front. So for us moving our business to Singapore and setting up Singapore as our global hub um was a good test because we've been able to deliver into Singapore more A I capacity than um has ever been available before in a private cloud like ours. Thanks to this ability to use half the energy. So that's one of the ways that we've actually been able to showcase in a constrained environment like Singapore, this type of technology is, is needed. So it's, it's interesting to me because it's, you know, this observation that it's in precisely an energy and, and land and resource constrained environments that you, you have to radically innovate. You can't just tweak the traditional air cooling system and get, you know, three or four or 5% increases. Uh a year, you've got to do something that's going to radically cut, uh, resource uh, requirements. Uh, so it's kind of like, you know, if you can make it here, you can make it anywhere. Right. Exactly. And, and, and, you know, I, I absolutely love Singapore. I live here. It's a, it is very much that, that ethos, if you can, if, if you can prove it in Singapore, it's hot, it's constrained. They want to do better on sustainability. Um You can prove it to the world. Seth, let me get you uh involved here. Seth. For those of you who don't know is the former uh A I officer at IBM. He's written a wonderful book which is available, I think in the lobby if you get a chance to pick one up. Um And uh is really one of the most uh kind of far reaching A I thinkers in the world. We're delighted to have him here. He has also, he is also the founder of a start up that is investing in sustainable A I technology Seth. Maybe you could say a little bit about that and why you have chosen that path. Yeah. So in addition to quantum A I, which is a consulting company, we also have a venture fund called One Infinity, which is part of the reason I I came here is to, to raise for that and we invest in responsible safe and, and Green A I. Um And investing in green A I is important. Uh A because there's a lot of greenwashing going on. Uh You can get on, on quote unquote, green clouds that's moving on green CRE carbon credits, which is multiple counting of it's not really green. Um And so just like a lot of things aren't really A I. Um And so we really thought it was important. My partners and I to look at how can we support true investments in things that are green, like really green new hardware, uh new paradigms and how you support A I uh and, and workloads around A I. So some of them like, like with Tim's building uh completely new ways of thinking about how you operate on silicon. So uh moving away from traditional semiconductor technologies to new silicon technologies that are 1000 times faster than today's GP US or new mass compression algorithms that are 50 times faster uh than traditional uh matrix multiplication, which is how uh language models run today or even things as simple as I'm going to have GP U as a service. And some percentage of those GP US are going to be available as certified green clouds where they run on wind or solar or oh my gosh, if you put it in Iceland, it can all be geothermal and not only is it completely carbon neutral in the wintertime, it's carbon negative because they use it to heat the housing and the environments around them. And so there's lots of very simple things that start ups are doing that are really truly green. Uh And so that's why we decided to, to start investing in, in platforms like this and, and by the way, you can make a lot of money on them. OK? One of the things that Tim and I discussed was I hadn't kind of fully realized is that part of the issue here is the people that operate data centers. Um they're not engineers for the most part, right? Their, their uh the real estate investment trusts uh and private equity funds. That kind of think of these structures as office buildings or more or less an Amazon warehouse or something like that. But as Tim was pointing out, you really have to think about the engineering and think of them as factories in the way that an engineer uh uh would um how, how much I mean, if we can switch to these kind of liquid immersion cooling type technologies and away from the traditional air conditioned uh uh model, what do you think Seth about that? Is that, is that going to be a solution that will really move the needle in terms of power and space consumption? You know, I think there's multiple ways that you can, you can tackle this, right. One is, you know, there's a lot of, you know, traditional data centers are not just air cooled, they're water cooled, water pooling creates problems of itself. Right. If you go and use chat GP T today or an, or claw, you know, I don't know how many people, you know, this, but for every 25 to 50 prompts, depending on how the, how big they are. You use about a half a liter of water just through evaporation. Right? I mean, so, you know, they're liquid cooled more efficient than a and then air, but you're still harming the environment. Um And so we absolutely need new paradigms of cooling. You know, people say we're gonna, you know, consume 70% of the world's electricity. You know, we've been running out of oil for 50 years. We're not going to run out of electricity. People like Tim are gonna invent new technologies where uh and we're gonna hopefully fund some of them uh that are gonna get us to where we're gonna continually find new ways to make sure that that doesn't happen. So it's gonna be things to like new cooling. It's gonna be new hardware paradigms, it's gonna be new math paradigms, it's gonna be moving away and people talked about this earlier today. You know, these massive large models are not what the technology was built for. They're cool, they're fun, but that's not the solution. That's also gonna help the power, it's really small models, it's more efficient technologies. And so we're not gonna have this problem of these massively large models used to all these powers because that's not what the solution. Yeah, that's very interesting. So, a couple of things I hear there, um, one is, um, well, so on this, on this uh problem of, of kind of water consumption, you, you sort of figure out how to recycle the water to make that more efficient. Well, you don't use water, right? You use, we, we use wastewater so we use wastewater and, and, and I'll describe a little bit, you know, for, for a little GG for a second here on how it comes together. But this con of that, that you talk about clay, which is that, you know, traditionally the data center, the end to end energy into knowledge, whether it's cloud, whether you're using, you know, whether you're doing a Google search or whether you're using, you know, whatever Apple Maps, there's all these things in between, which is this, this point, you know, and, and you just said it too. There's this point behind it, the actual factory that most folks don't see. We're, we're using phones now and, and today and you just don't, you don't feel it and you don't touch it but its real world impact. So this notion that there's actually been a silo, there's a data center who builds a building, there's a cooling engineer who builds a cooling system which involves chilled water. There's a computer manufacturer who builds a computer, there's a chip designer who designs a chip and each one of those people says to the other one or you go figure out your part about this, you go figure that out. You, someone will figure that out. You know, when Jensen comes up with a chip from NVIDIA, someone else figures out the cooling and that's, and it's been fine up until recently. The problem is that graph that I put on on screen before. Um There's this insane amount of energy consumption now going into these chips and a vertical system like ours is the way that we get over that to your point. Clay and question about water. Yeah, most data centers, whether they're using it for direct chip cooling or whether they're using it for air, they use chillers and that a real energy hog, you know, especially in this part of the world where it's hot and humid. So when you use a chiller, you're really over cooking the amount of energy you need our system, our system takes the building's effective waste water. So it's called condenser water. So after the, after the water has done its job in the rest of the data center, it goes out into an Evaporative cooling tower and it gets recycled. We take it at its hottest point before it goes into that cooling tower, we bring it in to our hypercube which is a self contained system and then it does its thing. So in that sense, technically, we use water, but we use water that was already in the loop and we don't use chillers. So that's the, that's the energy saving. What this kind of suggests is that um there's been driving the chips, you know, Moore's law, which we talk about all the time is kind of well at work in making them faster and better and smaller. But that there hasn't been up until now, a real Moore's Law type dynamic in the way that we think about data centers, we just use the, we build the big ones out, keep on building them now. And that's what we need is we need to start applying engineering and science to, you know, condensing those I I think to a certain extent. So data centers got smaller for a long time, for a long time, data centers got smaller and smaller, so much to the effect that, you know, there are a lot of places where they were trying to figure out what to do with the space in data centers. Um But now that A I is really taken off data centers, there's in some data centers are still a lot of room, but they're consuming more power because the chips are more power hungry. So in a lot of places, it's not a space issue, it's a power and heat issue. And so we shouldn't talk about a space. It's really a power and cooling or slashing issue. And it's not that we can just put data center, all the data centers in Iceland or or Sweden or, or, you know, Denmark? Well, I, I would challenge that actually because do we really care about latency when we're having a conversation? Every, and, and we need to really think about, when do we care about latency when we're training models? We don't care about latency when we're using models. When do we care about latency if I'm having a conversation? So, if my employees are having a conversation with the system, do I care if it takes a half a second versus 1/10 of a second? So I really care is that worth impacting my company's carbon emissions? Right. Right. But that, that's a really interesting insight because it, it uh it's a focus, it shifts the focus away from the, the operator of the data center or the manufacturer of the chips, you know, to the actual user of the data. And what you're suggesting if I hear you correctly is that we all need to figure out a way to be more thoughtful about the trade offs involved in, in, you know, using it's part of the decision, right? And getting back to what you said about my book, right? That's part of my conversation. Look, it's part of the architectural decision, the conversation you need to have when you're designing each use case and it's part of the human conversation, right? When you talk about Human Centered Design, right? Panos was talking about the human, you know, human A I, you got to think about the human, who's the humans going to interact with it? How is the human gonna, you know, how is the human going to interact with it? What is the human decision? How important is that human decision? All of these things play into the architecture which has implications on we're talking about human centered. When we're talking about green, it has implications to how you make that, how green that, that, you know, how you, how would you, you would greeny that, that, that, that, that A I application? Yeah. So it's looking at the kind of demand side and the, and the psychology of the user, I mean, I use, you know, chat GP T all the time and I've gotten really lazy about it. I mean, you know, it's so good, it's so efficient that if I can't think of the right word, I'll use it as a thesaurus when I've got an actual thesaurus that's probably on the bookcase behind me. And I don't, there's nothing about the way that, that, you know, is delivered to me, that service is delivered to me that makes me stop and think actually, you know what, for the amount of resources I'm gonna, I'm gonna burn to just do this. I, I should actually pick out a real book or use Siri on your phone. OK. 11 interesting unfortunate headwind to that though is the um the uh growing um narrative around data gravity and data sovereignty, which is, which is, which because I'm of the same opinion. In fact, you know, a system should be designed for its best use case. You know what we've designed. Our A I factories are not necessarily what you'd use for storage or what you use for general purpose. CP US, there's different, different horses for different courses, but there is a very real and present. I mean, we see it with the Chips Act but you know, it's a lot more out there. There's very restrictive laws heading towards parliament in India. There's a whole lot of countries that are very considerate about making sure that d doesn't leave the country. Yeah, and it might not always be the choice of the end user unfortunately. So there's a policy piece of that as well. Well, it sort of came up in our conversation with Minister C about data sovereignty and if you know, if you are Singapore, for example, you can make the argument. Well, why don't you just move your data across the border to Johor? It's cheaper, lots of space. Um But there's lots of data that you really like. You Singapore want to keep on shore, you know, your national health insurance data, you the defense data, the military data, uh anything that's proprietary commercial data, the big banks, you know, these kind of companies, there are a lot of multinationals who want set up a, you know, a data center here but not have it across the border because they're not sure that it's as stable as, as Singapore is, which has a kind of better international brand. Well, if you're multinational, you can't control data sovereignty laws, there's data sovereignty laws in the Middle East, there's data sovereignty laws in Europe, there's data sovereignty laws in South America and in Africa. And so yes, Singapore can control it, Singapore, but you can't control it for the rest of the world. Um And, and so even outside of individual countries, they're gonna exist. Um And, and some of them are for good reasons. Some of them are just because so popular. Yeah, but, but there are technologies now that you can actually, you know, Federated learning actually works. Now, uh you know, we talked about it for a long time. It didn't really work now. Now it actually works. So if we were to think of just kind of one or two things that we could really focus on and prioritize to fix this problem of energy consumption and carbon emission of data centers, what in your view would that, would that be? I think, you know, when you're, especially when you're looking at generative A, I get back to the root of what the foundation models, which is the technology underneath generative A I was actually built for, look at, look at using small task specific models, so focused models uh those use a lot of energy because less blender energy because they're smaller uh focus on the use case, what problem you're trying to solve and then determine what type of latency you can, you can, you can tolerate. Um And then I think, you know, focus on, do you really need generative A I? What's the right tool for the job? Uh And I think those are the things that you can do today uh Before we develop new technologies. Really interesting. Ben, a very sophisticated. I, I feel like I've learned so much from this uh from this panel. Uh Tim, I know you all are looking at bringing some of your manufacturing capa capabilities to Singapore uh fairly soon. Uh And, and we'll be rolling out, as you mentioned to Thailand, India. And uh what was the other one in Australia in Australia field? Great. So it looks like it's a, it's a promising market for that kind of technology. Um We are out of time. I think we'll have to leave it there, but I want to thank our two panelists for a fascinating conversation about what really is a, is a crucial topic uh for, for artificial intelligence. Gentlemen. Thanks very much. Thanks. Bye.