Kevin Scott, Chief Technology Officer and Executive Vice President, Technology and Research, Microsoft 
Interviewer: Jeremy Kahn, FORTUNE

Brainstorm A.I. 2022 - A.I. And The Promise of Large Language Models

Conferences

A.I. And The Promise of Large Language Models


Brainstorm A.I. 2022 - A.I. And The Promise of Large Language Models


Welcome back everybody. Welcome to day two of Fortune brainstorm AI. We have got a great program ahead. We're going to explore the amazing advances made by large language models, and then we're going to get a firsthand account of the Ai focused digital transformation at a snack food giant. We're going to close out our day with a keynote conversation with Reid Hoffman at the end. So many of the people who have joined the conversations here at our gathering offer great examples of the kind of purposeful leadership that we want and need more of in the world. And that's exactly what led Fortune to create. Our newest membership community called Fortune connect. It's designed to help prepare those who aspire to become leaders of the future and we're really, really proud of this program. It's seen tremendous growth since its inception just 2/2 years ago. The goal is to create truly purpose driven leaders and to help them understand the dynamics of stakeholder capitalism. What it means, how it works when society is demanding so much more from companies such as yourself. The program is designed for mid career professionals who are rising fast on the executive track. I'm sure you have folks like that in your organizations and I would encourage you to consider putting them in this program. Um please check it out at fortune connect dot com when you can. Okay, before we start, I'd like to thank our partners for their work in bringing us all together here are presenting partner partner accenture and our partners C three ai and intel, we greatly appreciate your work in making all of these conversations come to life now let's get to it. They're called foundational models giant Ai systems trained on vast amount of text sometimes combined with images or even video. The results are systems that have achieved breakthroughs in a range of natural language tasks, from writing to summarization, to question answering as well as generating new visual content. Many see the models ushering in a revolution as these systems are deployed to accomplish knowledge work that was previously previously reserved for us. Lowly humans. But does big tech have a blind spot when it comes to the limitations of large language models? How do we assess these risks? We are really really fortunate to be able sit down with a veteran engineer at the top of the industry to share his insight into the promise of foundational models and his views on responsible ai. So please welcome kevin scott, Chief technology officer and executive Vice president of technology and research at Microsoft for a conversation with fortunes, jeremy Kahn. Hello everybody. Thanks for joining us this morning. Um Kevin, I want to start out by asking you why are these large language models and large foundation models? Such a big deal. Why do you think they're so important? Well, I think a bunch of people are getting clued into why they're a big deal. If you just look at what's happening on twitter with chat GPT over the past handful of days. Um so I think the thing that's really interesting about them is uh this very idea that they are foundations that they are platforms on which you can build a whole bunch of other things that they have just become powerful enough, where you're no longer in this mode that you have to do machine learning, where you have a narrow slice of data, your model, your training pipeline, and you're building something that is narrowly focused to a particular feature or product. Like we are now able to invest in building these these really big things and then thousands, tens of thousands, hundreds of thousands of people can use them to build all sorts of incredible things uh which is great because one of the things that it does, it really opens up the aperture of who can actually use Ai now. Uh so you uh will need new sorts of specialties in the future, but you don't need to have a PhD in computer science anymore to build an AI application, which I think is really, really exciting. Yeah, fantastic. Um I know that Microsoft uh you own GIT hub and with GIT hub you created something called co pilot, Can you tell us a little about what co pilot is, how it works and how you see that as a model for other products and features that you're hoping to roll out going forward. Yeah, so co pilot is the thing that we built in uh collaboration with open AI. so um we had this epiphany pretty early in the development of GPT three, that these large language models were actually able to write code. Um And so we uh invested additional you know energy and focus and built this product called Get Up Co pilot which is a coding assistance, so it allows a developer to express in natural language, a piece of code that they would like help writing and it admits it. Um and it it is you have been one of those things where the productivity gains are high and maybe more importantly the amount of love that developers who use it express for it and like how quickly it's become an essential part of their workflow has been just sort of surprising and wonderful to see that the you know, the thing that I think is interesting is that uh it is like get up co pilot for coding assistance is just 11 sort of co pilot that will exist in the world. Like I think in the limit you will have lots and lots of these things helping people with a pretty wide range of tasks, whether you are a video editor trying to pull together a piece of content or you're a journalist doing research or writing an article if you are a doctor trying to deal with your caseload, Like almost everything, I think that you can imagine where you're doing cognitive work, you will be able to see things like co pilot assisting with that work for everything, correct. Um How do you respond to customers who are really interested in this? Think this sounds you know like something that might help them in their business, but they're basically kind of afraid of using these models because they didn't train them themselves. They don't really understand the biases. I mean how do you sort of respond to customers who have those concerns? Yeah, like it's a thing that we hear all the time and I think the way that we get past this is we just sort of see how you you know, you build these dependencies like if you look inside of an organization, like any of you who run engineering teams, like this idea of taking a dependency on something that someone else built is just a hard thing to do in general. Um And yet we we do it all the time. Like people don't think about uh when they need to write a piece of software, do I have to write my own compiler or do I have to build my own operating system from scratch? Uh And so what what I think will happen. What we'll see over the coming years is that some part of the functionality of these big foundation models will start to look more like an operating system, a thing that, you know, it's just so complex where your organization probably won't be able to make the investment in building the most powerful possible thing um, and where like a small number of folks will be able to just because they are building a platform where the cost of building the platform gets amortized over a whole bunch of different things. Uh, and then we will have to figure out what the business model for this stuff looks like because you know, I think Bill Gates said once or at least it's been apocrypha li attributed to him that the true measure of success of a platform is when most of the economics and the value created on the platform, go to the creators, not to the person who built the platform, wow. Um, now I think we'd be remiss if we didn't ask you a little bit about this issue with co pilot, which is that as you know, has now been sued by a developer who says, well, my code was used in the training of this, um, but I'm not getting any compensation and I don't think, you know, is the proper use of the license of that. Is there anything you can say about that lawsuit? Well, obviously like, I can't say much about it. Um, but you know, the thing that I will say is like a bunch of this stuff will get figured out over the, over the coming years. Um, yeah, we they're, they're a bunch of different philosophical ways of thinking about how you build these models, uh, and I think we'll be okay as long as, you know, the world is building things that create value for lots and lots of people versus building things where you're sort of concentrating things into fewer and fewer hands, like that's where we will get into like real, real trouble. Right? Obviously, there are other concerns about safety around these models. You know, what are they gonna, will they produce biased content? How how have you at Microsoft? I thought about trying to mitigate those. Yeah, we we started many years ago before uh like transformers and deep neural networks were uh were even things thinking about how do you build a responsible AI practice? And it started as a thing in Microsoft research. We have now built a real responsible Ai discipline where we have an office of responsible ai that sits under brad smith and our legal team and we have a uh ethics committee that is composed of not just computer scientists, but social scientists and uh you know, folks who are experts in different disciplines that are very relevant to deciding, you know, what's safe, what's responsible, what's ethical when you are thinking about applications, these models, we have published a thing called the Responsible AI Standard, that's in its second version, it's like available for folks to have a look at. And like we have rigorous processes inside of the company, the try to evaluate uh what we're launching against that standard to make sure that at the very least we're being consistent with our own uh stated guidelines. And we're more than happy to have uh public debate. That's the reason why we published the standard about like where the lines ought to be and like we've drawn, you know, uh a bunch of very clear lines about like what a sensitive uses and like in in the cases of some of these sensitive uses, uh you know how you should deploy machine learning or AI systems, if at all. Right. Um Are there questions in the audience? I want to I want to let you all get a chance to to ask kevin anything that's on your mind. So please think of think of questions. Um there's one there go first. Please stand up and state where you're from. Hey, I'm Jill from capital G. Thank you so much. Um Obviously with co pilot, you guys had a really clear example of using the innovation from open AI sort of tightly coupled with the distribution of GIT hub. How do you think about other examples of that where open AI and Microsoft might work together. And I think you mentioned sort of other co pilot examples maybe, but a little bit more about how you expect that relationship to continue forward over the next several years. Yeah, I mean we we've got a really awesome partnership with open AI that's multifaceted. So like the biggest thing that we do together is we build ai computing infrastructure for training big models, so GPT three? GPT 35 which just got released chat. GPT like all of these things, like we're not only trained on Microsoft built infrastructure but uh the imprinting parts of what they're doing, so when you make an api call like that is going into like highly optimized infrastructure to support some like pretty technically complicated things that a lot of work went into. Um but we also collaborate with them on, you know, hey, here's an interesting application for these models. We have a Azure version of the Open AI A P I that runs inside of our cloud that is enterprise hardened and has all of the things that some customers both expected need from, from an enterprise version of something like this. Um And you know, we we even do a bunch of work together on safety and responsibility. Um So we've we have an awesome partnership with them. I just can't say enough good things about, about the people at open Ai they're just extraordinary folks. Other questions, there's a question down here in front, we can get a mic to this gentleman, please stand up and broad Meadows from the AI Foundation. I'm curious based on these new foundation models has the profiler skill sets that you hire for changed in the last couple years. Yeah. Um one of the big transitions that we've had and this has been going on for, you know, four or five years now, is that um what are machine learning engineers do uh in their job day to day has changed pretty dramatically. Um So yeah, I referenced it earlier, like it used to be the case that a machine learning applications team, inside of a product or feature team at the company, like had their own data, like had their own model, like they decided what the model architecture looked like, they would train it, they would uh like do the evaluation on the model that they trained, they would deploy it into some testing infrastructure and see how it performs and then they would sort of run their own like feedback cycle on top of it. Um More and more of what we're doing right now is like we have a very small number of foundation models that we use inside of the company that people use either directly like zero shot or one shot where there really is no fine tuning for a particular application or that you do a few patterns of like fine tuning to get it to perform in a particular way. Sometimes it's cost, sometimes it's uh yeah, you needed to have some performance characteristics in terms of quality that the base model doesn't have uh with, with no fine tuning. Um And so that like that's just a different mindset than the one that we had just a few years ago and like it's an interesting transition for folks in the organization to go through. Um and like oftentimes, it's a interesting thing and the teams who have little machine learning history, uh like can just go faster because they don't have a bunch of things, they have to go unlearned. Uh and they're just sort of open to a new approach rather than, you know, digging in and saying like, oh no, but my way is better. Um so it's a big transition happening. Um I think for folks who don't have their own specialized machine learning teams, like all of the foundation model stuff is almost nothing but goodness because it basically means that you have a more approachable set of expertise that you need to like get going. Um and that like, we, we still need that deep, deep expertise on like how to do all of the optimization, but like we can again, you know, focus that on foundations and then like that work gets, you know, advertised across the gazillion applications, kevin in 2020 you wrote a book called reprogramming the american dream where you kind of argued that, you know, Silicon Valley is this great place, It works really well, but we, we need to create this opportunity everywhere, not just in Silicon Valley. I'm curious if there's a, I have a role to play in that vision, I think it absolutely does. Um the the reason that I wrote that book is there, there's a vision of the future where um where these foundation models don't become platforms and where you see further concentration um into a smaller and smaller number of companies and you know, hands of experts that are mostly in like urban innovation centers in the United States and a couple of other places globally. And I think, you know the interesting thing if you concentrate on making these things available as platforms to lots of people and like you try to uh it's not even that they're free, but like if you get lots of people using it, you can drive the unit economics of what they cost to access even down. Um and it just creates a bunch of opportunity. Like if I look at the part of what I did in the book is I went back home to rural central Virginia where I grew up and I talked to a bunch of people who had entrepreneurial mindset and given these tools like without computer science degrees, like these folks will be able to see the opportunities that they give them and like they will absolutely be able to incorporate them into the things that they're doing and their businesses and to create new businesses that they hadn't imagined were possible before. And to me that feels really exciting, right? So I mean the idea is you could use machine learning in your business even though you have no background actually in it, you don't know how to code, you just instructed, you know, here's what I'd like to have this function do this app to and it is built for you. Yeah, I mean a lot of and and again you all are sort of sort of seeing it play out in real time. Like we we have uh we've been on this journey for many years now just because we we sort of had confidence that this is the way things were going for a while, but with chat gpt like what you're seeing is you can via prompt engineering with like expressing what you want to have accomplished a natural language, like you can get these models to do very very powerful things and so like that's just a much more approachable way of being able to harness the power of digital technology and like I've been talking about this for several years and like this is actually cribbed from Peter nor big, who gave a talk about it like seven or eight years ago where you know the thing that machine learning and ai does is it teaches, it transforms the task of Harnessing the power of your machine from programming to teaching. Uh and programming is an activity that's like hard to learn. Like you have to go to school, you have to practice like it, you know, believe Malcolm Gladwell or not like you probably need to put in your 10,000 hours to get really, really good at it. Whereas a toddler can teach someone how to do something like it is a thing that's inherent to us as humans. And so I just think it would be a more much more accessible way for people to get real power out of their technology than than this paradigm that we've had since Ada Lovelace, you know, 170 something years ago. And right now there's still quite a lot of art to kind of prompt engineering. And there's people saying, oh well we should start teaching kids how to prompt engineer. But I've heard other people say, no, I don't think we're gonna need to do that in five years. What's what's your view of? Oh I I think you are going to have to. I mean I was chatting with one of the teachers at my kid's school who saw chat GPT and they're like, oh my God, like now I'm not gonna be able to get my kids to write essays anymore because they can, you know, cheat, use chat Gpt to do the work for them. And I was like that is it's a pedagogical mistake to think that the essay like the artifact that you get at the end is the important thing about what it is you're trying to teach a student. Uh And so in a sense like nothing really is changing here other than you have this tool and like the student themselves like has to become the teacher to the model. So like the first thing they ask it is gonna be crap. Like it'll have factual mistakes that will the models. Like the hallucination rates like everybody should just sort of be tracking or like going down like you know chat gPT hallucinates way less than GPT three. And it's just getting better and better over time. But like still like the thing will make mistakes and so your job is like be the editor for this thing, be the teacher like coax it into getting you the output that you really need. Like that's the important thing. That's the special thing about us. Like the thing is just a tool so won't won't know much about history but we'll know a lot about prompt engineering. Like um No kevin thank you so much. We're just about out of time but thank you so much for joining us. This is a fantastic discussion.