Name: Brainstorm AI Singapore 2024: Keeping Up With AI’s Computing Demands
Uploaded: 2024-07-31
Duration: 19 min 40 s
Description: Vinay AWASTHI, Managing Director, Greater Asia, HP Dahai LI, CEO, ModelBest ST LIEW, Vice President, Qualcomm Moderator: Jeremy KAHN, FORTUNE

Vinay AWASTHI, Managing Director, Greater Asia, HP Dahai LI, CEO, ModelBest  ST LIEW, Vice President, Qualcomm Moderator: Jeremy KAHN, FORTUNE

Brainstorm AI Singapore 2024: Keeping Up With AI’s Computing Demands

Conferences

I think it's going to be interesting panel because all three of our panelists are in some way involved in the attempt to kind of bring A I to the edge as one potential solution for the hardware dilemma that A I poses. I'm actually going to start with, with ST I'm curious what strategies you guys have in place at Qualcomm around this. Um And, and how important it is for you to sort of integrate A I at the edge into to the solutions that that Qualcomm is working on. Well, thank you for the good questions and it's really nice to be here. Um We believe that to harness the full potential of A I and A I has been adopted in all industries in almost all walks of life increasingly so, um it needs to be intelligence everywhere. Um by having intelligence everywhere, we use the terminology of hybrid A I. So you will share the A I workloads whether at the edge or at a cloud and as such, um the full potential of A I can be gathered. And also there's a lot of uh there are all the good benefits that come from it because for instance A I that have been performed at the edge can show a lot of personal security and confidentiality, not to mention uh you know, it free up the traffic going up the cloud all the time. And on top of that, there's a huge economy to this as well because edge devices, edge devices are many, many things, cell phones, smart cars, P CS these days and thousands and millions of IOD devices. And it helps to make sure that that distributed way of processing A I will also tailor for the specific A I where it is needed. So, so we are, we are we are we are committed to that. And that's why one of the things we have done recently is we actually launched our Snapdragon X elite chip set platform for the PC. We believe 2024 is when the PC industry, the PC is being reborn A PC is a pivotal portion of the entire um edge A I and not to mention of course, smartphones and XRVR and so on and so forth. So we believe this is the right thing and the important thing to go. And there is a big need for a lot of the consumers enterprises, small medium size companies as well to upgrade a lot of the devices to be able to do these uh A is at the edge. So I think business opportunities there is also helped to, you know, um involve more people into the uh digitized economy of the future. So that's what we are well suited, you know, Qualcomm is well suited to do that. Excellent. Um Do can you tell us a little bit about model best and and about mini C PM, which is this model you develop and how this kind of represents a step in the direction of these kinds of models that can actually run on on edge devices? OK. Thanks Jeremy. Uh uh I'm very happy to be here to representing uh model best. It is a premier A I start up in Beijing model best is all about creating uh the top level big model and we really value efficiency. Uh We believe that efficient model means better performance, uh smaller uh size and low less cost. But with the same capabilities. Yeah. Um and uh even some folks in the media class Chinese Mistral because we uh because the automated pursuit with high efficiency is the thing we have in only in common. Um uh instead of just, just comparing uh the parameters and the performance, we focus on the knowledge density of the big model. Uh uh That's uh first thing we, we very, very uh very focused on. Yeah. Um uh OK. The star product, our star product is a mini C PM. It is a bunch of uh uh super light uh models that you can run on your phone. Uh um But they pack a punch with bigger ones such as GT 3.5 GT four V or even so future, maybe GT four. these models have been popular all over the world in the past half years. Yeah. Yeah. And you said we were speaking before and that he was saying you, you've released five different versions, updated version C PM just in the last six months is that we, we have, we, we move very fast. Uh We have released 45 versions of mini C PM in the past half years and uh we will uh release uh two versions uh additionally uh in fuel in risk. Oh, wow. So this is technology moving very fast and I think what you can do on devices rapidly catching up with workloads that before you could only run in a in a data center. Vinne, I want to turn to you and, and HP you're known for, for P CS, you're known for printers. Where do you see A I as you know, kind of fitting in with, with that kind of hardware? Well, I think first and foremost, great to be here and thank you for the opportunity. Um If you look at the A I story so far, I mean, I mean, it has mostly been data center and cloud centric right now. We do know that while it's a very powerful story, it is it has limitations in terms of how much it can reach um what we see is the proliferation of A I in a massive way over the next three or four years, every person in this room and everywhere in the um in the world who is using a personal computer right now will find that in order to use A I over the next 34 years, as many of the use cases emerge and perfect themselves, your current PC generations, they will be all obsolete. And for us to really take advantage of A I to every individual, every small and medium business, every large business government. Um you know, this is going to be a massive effort to provide um A IP CS because at the foundation of A I is the need for driving a very order of magnitude higher computation. So, you know, today's P CS, there is a term now we call trillions of operations for second tops, right? Today's P CS are anywhere from, you know, 6810 and we are not talking about 40 tops, 55 tops even, you know, higher than that. So our drive is to be able to provide um a huge portfolio of products P CS, printers, communication equipment that will allow people to be able to use A I models at the edge in whatever use cases they have. Now, it's interesting um when you arrive at Microsoft released this early version of an A I PC, um but it had to sort of record everything you did on the PC and some people were worried about that even though they, they were trying to sell it as more private than having to upload data to the cloud constantly. Um, there was still this concern about security. How are you thinking about this at HP? Well, I think security is going to be extremely important if you look at it, I mean, why is security important? Because as any computational power, any software, it's a tool, you can cut it both ways, right? Um If good guys are not going to use security and A I and the latest tools for the bad guys are going to do. So, so what we are doing is developing models that actually learn from the various attacks that continuously happen and those models will be A I trained. So we have a suite of security products from, you know, bios all the way up to the OS, we call it food security. And that is really based on A I. So it's just one example about how we have to be as responsible technology providers, provide technology that can keep people safe first and then allow them to do all the, you know, enjoy the benefits of the new technology that's going to come. Great. I'm going to ask for questions from the room in a minute. So please think of your questions and raise your hand and I'll get a mic to you. But first I want to, I want to talk here, you talked about how popular mini C PM has been. You also have this interesting case where some researchers at Stanford plagiarized the model or sort of built something that seemed very based on mini C PM. Can you talk a little about what happened in that case? And I'm curious whether this is sort of a risk in general as we move to smaller open source models that, you know, there will be this proliferation, it will be very hard to control what people do with them. OK. It's really interesting. Uh We released a new multimodal version of mini C PM uh that comparable to GP four V on the uh multimodal site uh in on May 20. Uh It's, it's very cool and the feedback from the open source community has been aware. But then on May 29 some people on the github homepage point out to us uh that there was a Stanford project has copied us. Uh And the team claimed to have built a better model with fewer parameters, which was 101% of GD four V, which is true. And the little training training cost only $500 which is not true. And the uh smaller as, as small as smart as the GP T four V which is true. Um After comparing uh we found these models are almost the same, the American one could even understand the Chinese Asian text, which was merely found in the public between data. So as you know, what happened next, uh The Stanford team apologized publicly after uh getting caught out of the plagiarism and they deleted the program uh the project on the github. Um This incident, uh We think it's not uh does not represent uh Stanford officially because it's just uh initiated by a few undergraduate students. Um And we forgive them and uh uh remind everyone to give credit to where it's due in the open source community. And the incident uh just make us uh appreciate the open source community even more. Uh We are thankful for the support to resolve the issue so quickly and fairly. Um And the model best has always been active in the global open source community. Um We have created created uh the open BMB community which has been the biggest open source community uh in China. Uh And we continue to work on projects, open source projects and uh cooperates with other communities to uh advance model technologies, right? I want to get for a question from the audience in a minute, but I'm going to ask a slightly awkward question, which is right now, there's a lot of concern about running these large A I workloads in data centers on the energy consumption. And a number of companies have Microsoft and Google in particular have said, you know, they've been thrown off their, their track to net zero because of the data center growth that they've had. And so some people are looking at on devices, oh, we can avoid all that. But if you, if you look carefully at where some of that additional carbon footprint has come from. It's not actually the energy used to run computations in the data center. It's all the energy that went into the production of the chips that are in the data center. And then there's also concern about, well, where are you running these models? And so if we're pushing everything onto device, my question for Qualcomm is what's happening in terms of the energy intensity of carbon intensity of the manufacture of Qualcomm chips. And then also if we're pushing everything out to consumers and the energy is being used, you know, when they charge their PC or their phone, isn't there an issue there? Because a lot of those consumers might not have access to renewable energy as their source of, of home power or office power? Well, I think first of all, I, I think that we shouldn't say that everything is pushed to the edge. There is a very close collaboration between the cloud and the edge. In fact, if you look at the entire eys I, I would say that uh you know, Qualcomm has always embraced new challenges and new technologies from three G, four G five G. And because of the Connectivity foundation that five G has brought, it has enabled us to provide the options of processing some of these A I at the edge, but you still need the cloud to do certain, you know, it's very efficient to do like massive learning development stuff on the clouds. But then there are minute little things that make sense to do it at the edge. I think the, the, the tricks and the intelligence and the innovations is what to do where. So you, you have things that will happen on the edge, on my little cell phone, my P CS in my car that are not even an option not to do it there because you need that latency, you need that short latency, like smart cars, you need to have that response immediately, right? So there are these things that are just absolutely make sense to do it there. And then as you grow it bigger, the cell get bigger and bigger, you go do the intelligence of the edge and then more edge and then to the clouds. So I think there is that intelligence that with the with the liberalization of the technology and know how and empowering so many smart people to do the right thing, there will be a good balancing point where I think the the industry and the uh the developers and companies will write the will find the right balance to where it is. That's why I think that the model that will really reap the benefits of A I is something called the hybrid A I which is going to be cloud devices and so on and so forth in a, in a network questions from the audience. I have a, have a question, if not, I've got more. So don't worry, I'm going to ask, I'm actually going to turn to Vin A also and ask on the sustainability question. Um It, I mean, you just said everybody is going to going to need a new PC uh in the next few years and maybe we're all going to need new phones too because we're going to need that. That latest elite X shift from Qualcomm. Doesn't that create also a sustainability issue? You're going to have all these people discarding these old devices, you're going to have an E waste issue. How is HP kind of looking at that? Because I know you have a, you have a firm commitment to sustainability, but does this throw you off of that? It does not actually if anything, we are accelerating of a commitment to our sustainability goals. And because you know, our sustainability uh framework is very broad, we, we look at how to make devices more efficient, which means today we are working on devices that can actually have truly 24 hours, 48 hour battery life, you make them more efficient, you know, because there is less loss of power when you are using those devices. That's one part of the framework. The second part of the framework is using sustainable materials as we build our devices. So every PC, every printer today now has recycled material, not just plastic but recycle um material that goes into the chips, recycled material that goes into the casing. We are also making our factories more efficient. So zero waste factories is our goal. Last but not the least. It's a very important part for us to also work as what we call renew supply chain, which means we can take devices back, renew them and put them back into the market, which is because there are a lot more people in this world today who need computational devices than those who actually have computational devices. So for us to work on making our products more sustainable and also our supply chains, more sustainable, making renewed products back into the market. I think that's our framework and we are very committed to that. And does A I have a role to play in doing that making your operations more efficient or find, you know, energy savings throughout your operations? Absolutely. I mean, if you look at today, we are talking about lights out factories, those factories will run because they are enabled through A I. We are talking about devices that will be optimized, whether it is, you know, eliminating the processes that are unnecessary and thereby increasing the battery life. It's all being driven by A. So it has a lot of use. Now, we are the same thing. Well, you know, the, the chip that we just introduced. And because of our heritage in in the very efficient computing and very very efficient power consumption, you are going to be able to do more things in shorter period of time. And imagine that you, you can use these A I models after you really allow the thousands and thousands of developers to think about how to solve some of the problems you just posed, you know, so I think A I with the right tools with the right liberalization and uh you know, open up of the uh of the tools will enable a lot of solutions to solve the problem. Some, some of them you just mentioned, that's fascinating. And while we have Dahi here, I want to ask it's slightly off topic. But one of the amazing things about mini C PM and some of the models you've worked on is their ability to, I guess translate these ancient Chinese calligraphy, these Tsinghua bamboo strips. I don't know if you can talk a little bit about that because that was also one of these tests for the model. And I think it's also one of the ways they caught that, as you said, the Stanford plagiarism because their model turned out could do this too. Which was very strange if it hadn't been using your training. But yeah, the the Tsinghua bamboo split is just uh Chinese Asian uh rep uh that uh uh t economy, uh a t friend uh just uh sent to Tsinghua. Uh So uh there is no uh digital information on the internet. We just take a photograph and use it as the uh fine tune um picture uh to build it into our model. So we very make sure that any other model should have not ability to identify the, the Tsinghua, the character on the Tsinghua. So that's the reason we uh quickly identify the model of the excellent. Well, we're out of time, but I want to thank uh ST and Dahi and Vin for being here. It's great to have you. Thank you so much. Thank you. Thank you.