will cut through the noise and reveal the sector’s immediate opportunities and most pressing challenges. Join C-suite executives, leading academics, and prominent policymakers as we assess the industry and examine new business cases for AI. This year’s program, presented in association with Accenture, will feature intimate one-on-one conversations, engaging panel discussions, and lively roundtable sessions–all designed to address how AI is changing the ways we do business.

Brainstorm AI 2023

Vinod Khosla, Founder and Partner, Khosla Ventures Moderator: Jeremy Kahn, FORTUNE

Brainstorm AI 2023: Vinod Khosla’s Views on Investing and More

Conferences

For the last two days, we have heard from organizations leading the charge in A I innovation and implementation. We're, we've demoed new technologies, explored some of the most pressing concerns around responsible A I and what it all means for the future of work. Our next speaker has predicted that A I will replace 80% of jobs in the next 25 years. He was the first start up investor in open A I and he's invested in organizations like doordash headspace and block. Currently, he's on the final investing stages of a $3 billion funding round. We are thrilled to be joined today by Vinod Kola, founder of KLA ventures to discuss what's next for the future of A I investing. He's going to be joined by fortune, senior writer Jeremy Kong. Please welcome to the stage. Great. Thank you so much for being here. Oh, happy to be here. Great, fantastic. So you were the uh the first four for profit investor in open A I invested even before they had the partnership with Microsoft. Um obviously, they've just had this huge blow up with the board. What, what's your take on? What happened? And what lessons do you think there are, maybe are to draw from it? Well, not a lot of lessons to be drawn. You should get the right board members. Uh, they were, uh, there were a bunch of misinformed board members applying the wrong religion instead of making rational decisions. The company is much better off today than it was a month ago. So that's great. Right. Uh, and one of the, the splits that became apparent, uh you talk about sort of the, the religious view of, of the book or it is the split between uh you know, the so called doomers and the, the, the people who are saying that they're effective acceleration is um what I mean, some people are thinking that split itself is, is, is not helpful. What's your view on, on that split that seems to be developing in the community? Look, every new technology has its pros and cons there huge upside with A I, I've been talking about it for probably a dozen years now and then there are some disadvantages, but the tumors are focusing on the wrong, less uh risks. Uh nuclear had a risk. It could be used for good or bad biotechnology had a risk could be used for good or bad. The same is true of any technology that's going to be powerful by far. What does the magnitude higher risk to worry about? Is China not sentient A I killing us all? I mean, it's so silly and frankly, it's people in the press who amplified that narrative because it's so sci fi, you know, it resonates with the movies we've been seeing. So it, it gets this artificial amplification from the press, but it's sort of not worthy of a conversation to be honest. Alright. Well, I'm sorry, I asked, I hope we at Fortune have not been part of that application process, but I want to pivot a little bit. You mentioned China. I know you're very keen to talk. You really think that there is a split between the US and China and that there should be that there. You know, we should not be collaborating on this technology. What's your view of the kind of uh technological arms race between the US and China? And do you think the Biden administration has played this right? Uh in their strategy? Let me be clear, the the risk of 10 and A I killing us exists, but it's about the same as the risk of asteroid hitting our planet and destroying us all. Also, you, you have to characterize risks properly. The risk of China is humongous first next year. I'd be surprised if there aren't 100 million or more bots with persuasive A I one on one engaging with every voter trying to influence our election for their purposes. That's very likely to happen. I'd say probably a 95% probability it will be influential in the election and we should be worrying about it. Um The longer term over the next 25 years, whoever wins the A I race will win the economic race. So economic wins and power will derive from uh technology events. And I think it's up for grabs who wins in the next 20 years and whoever wins, the economic race will have social influence. And I think whether the world believes in western values in Southeast Asia and Africa in Latin America everywhere or believes in chief philosophy, which is some combination of Tiananmen Square type tactics, dictatorship and a reasonable philosophy that I don't care about the bottom 5% we abuse. But the, the society as a whole is better off to give him credit. That's sort of his political philosophy. I don't like that one, but which one wins is what's at stake over the next couple of decades? That's the really important. Do you think? And do you think the Biden administration has done the right thing by cutting off exports of advanced chips and A I software in some cases to, to China? Absolutely. Yeah. Interesting. When, when maha introduced you, I think, I think she got this wrong again. She said you had predicted that A I would eliminate 80% of jobs, but I think you actually said it's 80% of 80% of that. Exactly right. Maybe you can explain a bit more about what you meant there and which jobs do you think will, will be eliminated. So, uh probably a dozen years ago, I wrote a blog called uh the A I Doctor. 80% something like 20% human included. Right. Right. And there will be a human role and we can't fully define it in all these and A I will need humans to learn from. So there's always a learning model. Um but it will be able to do most jobs or most parts of most jobs. So 80% of 80% of jobs is like 64% of all the labor, economically valuable labor that's done. I think that's a very good model. But let me give you the example of what happens if we replace 80% of what a doctor does, he can have five times more interaction with the patient and the number of interactions in the USA doctor has with their patients, say primary care is 1/5 of or 1/6 of what happens in Australia. For example, should your pa your doctor check in with you every month and have A I check in with you every day on how you're doing. Especially given how important the disease burden chronic disease, diabetes, heart disease hypertension. You should be checking in with a patient every day. My son runs a company called uh CC I that is building an A I primary care doctor. The idea is for chronic disease. You will do 100 touch points in the 1st 90 days, like very simple idea can't be afforded without a human, still a role for the doctor. He's really overseeing the A I and maybe changing things, but much more interaction is needed to avoid things like diabetes. And we've all read the book nudge that nudging is so important to healthcare. In fact, it may be the most important drug we can invent and, and would, does the human do the nudging or you have the A I do the nudging A I will do the nudging but under human guidance, got it. Uh And you were telling me just backstage, you're, you're sort of taking this thesis to other professions as well. You've invested in a company that's trying to do this for it professionals. Is that right? Well, we just invest in a company called Cognito. You know, business users shouldn't have to make a request for a new report or a new query of their data warehouse. They should just say in English what they need. And the goal of this thing is to directly write the code to access, say your snowflake system or your database system and get your answers. Why do you need to then go to it and somebody then deal with that and there's a lot of overhead. So one of the predictions I made uh probably 11 predictions uh in the last couple of weeks, one of those is there will be a billion people on the world programming of the 7 billion people on the planet. What do I mean by programming, writing in English language? What you need? If you have an incomplete specification, the A I will ask you clarifying questions like what do you want to do? If, if the number of customers is zero, do you divide by zero? Uh you know, humans can't think of all that. So will there be a ball programmers? No question that will happen in the next 10 years. And I think you said that you think going forward expertise will be less important and actually knowing to ask the right questions will be more important. Maybe you can talk a little bit about that. Well, you know, so A I is very good at expertise if, if I have breast cancer, for example, hopefully I don't get that. But uh I, I hope my oncologist has read the last 5000 papers on breast cancer, which is probably 2023 publications on breast cancer. Like the amount of research is so large that no human can keep up with it. Uh And most of the time they can't and they don't remember it if they've read it and you want that sign in your care. And that's what I'm talking about. Expertise will be much better and broader. Maybe there's the human elements of care that the A I that the human provides. Maybe there's other parts. Great. I want to take some questions from the audience if you have a question, please raise your hand and we'll get a, a mic to you. Um, and can I see any questions? Yeah. Right here. A woman right here. Yeah. Hi, Suzanne. From Invisible Technologies. Thank you for that. Do you think that people know what they want people to do and what they want machines to do? I think people make assumptions, but I actually don't, I'm not seeing it. I think the thing to remember about the future. Uh It's, it's very hard to forecast. You all know the Yogi Berra sign. Um It's hard to predict the future. What you ha what happens is as technology develops and how it develops, changes, how it's used. So uh use cases will evolve and people will adjust to the use cases and, and I think this this process is very messy. So any long term forecast, the only thing you can be sure of is it's wrong. So when people ask me for forecasts, I say, especially in technology, anybody who makes a forecast is not smart enough to know they can't make a forecast. And it's been universally true. Uh For those of you interested uh in fun books. Uh There's a book by a researcher called Phil Tetlock Expert Political Judgment. He basically said, experts have the accuracy of dot throwing monkeys. And this is based on following 20,000 experts over 20 years on like 84,000 predictions. He said, what's the accuracy and it start throwing monkeys was the exact phrase after 20 years of research, uh very scientific conclusion. But we had to realize you can't predict everything that happens. You can't predict when chat GP D gets to charge GP D capability. We invested in open A I five years ago. I couldn't have told you when that capability would emerge. I could tell you in the long run A I is going to be really, really pivotal and placing bets and it's going to be very important. Great. I want to get you another question just here. Thanks. You know, I'm Anita Vata with Iona Ventures, uh synthetic biology and pathway of G I. How do you see that any relevance to humans at all that are living almost to the point that when a child is born in this decade, it's an entirely different wire framing. How do you see that pathway over the next couple of decades? Uh You know, every child could be, should be fully sequenced, single point genetic defects. You can design uh a protein sequence to go address a specific uh defect very specifically A I can design proteins that fold in a particular way that address a particular single point mutation. The same thing will happen for uh multi uh gene sort of polygenic kinds of diseases. Uh So I don't think we are very far from when you will have one drug for one patient, for example, enabled by A I Wow. Another question is in this table here. You know, it's sort of silly. Let me take a very trivial example, 7 billion people on the planet, they get one dose of aspirin. Even though 30% of the planet humans can't even metabolize aspirin. Is that silly or what, you know, critical drug like blood thinners like Warfarin or Plavix, depends on genetics. You should get a different dose based on the, your genetics, not the same dose for everybody. But that's what we do. We do crude medicine. Let's get to this question before we be not. Uh John just up. I'm the CEO of 1440 based in Park City, Utah. Um my favorite place I know, you know, it's been great so far this year we met at Sundance last year. Um So my question for you, uh you said the English language. And so, but if you think about the world population, a lot of people don't speak English. So what do you think of like language translation? Because you know, somebody could ask in Spanish or French, you, you, you have a view that there will be sovereign LL MS. So there will be sovereign L la Las. We just announced last week, I think A LA LL LA just for India because all the different languages there are so different, they don't even use the same uh script. And so a fundamental thing you hear in A I is tokenization tokenization in TGU should be different. A tokenization in Kanji should be different. So you will see this localization of these models. But beyond that, it is such an important technology, I think from national security, national defense, national control uh point of view, countries, important countries will develop their own A I. And will this be done by government or will it be done by private industry? My bet is it will be governments encouraging private A I, this is not something governments can do. It's moving too fast to plan. You sort of have to be on your, on your reactive feet most of the time. Right? I think it's like skiing. You just have to keep your balance and you can't sort of like, fantastic, we're out of time. But thank you very much for being with us, Beno. I really appreciate it and thank you all for listening. Thank you, everybody.

Andrew Bindman, MD, Executive Vice President, Chief Medical Officer, Kaiser Permanente Lidia Fonseca, Executive Vice President, Chief Digital and Technology Officer, Pfizer Sean McClain, Founder and CEO, Absci Moderator: Michal Lev-Ram, Editor-at-Large, FORTUNE

Brainstorm AI 2023: Revolutionizing Health Care through AI

Andrew Bindman, MD, Kaiser Permanente, Lidia Fonseca, Chief Digital and Technology Officer, Pfizer, Sean McClain, Absci, Michal Lev-Ram, Editor-at-Large, FORTUNE

COVID-19 pandemic brought a major digital transformation to the health care industry. Organizations were tasked to bring life saving vaccines and medicines to the market. At breakneck speed, hospitals were inundated with patients and doing their best to collaborate with labs and research teams to provide the best possible patient outcomes. In many cases, artificial intelligence is what helped these organizations get through the most challenging of times. We are thrilled to have three leaders with us today who are revolutionizing health care with the help of A I. Please welcome doctor Andrew Binman, executive vice president and chief medical Officer for Kaiser Permanente who created an A I tool that helps detect and alert medical staff if there's a patient decline, Lydia Fonseca pfizer's first chief Digital and Technology Officer with the help of A I pfizer was able to review clinical data in real time which helped them get the COVID-19 vaccine to market quickly. And Sean mclean founder and CEO of A BCI A drug discovery company that's using generative A I to help cut the drug creation process from five years to less than 24 months. Please welcome them to the stage right there. OK. Um Thanks for being with us. Thank you. Um I wanna start with just getting a, a kind of big picture look from the three of you. Um Obviously your respective organizations have been working, you know, in and, and with A I for, for some time, but over the last year specifically, um what would you say are the biggest shifts and biggest impact and Andy, I'll start with you. Well, I would just say, you know, like everyone else, we've noticed that A I is a big deal. Uh And so, uh you know, we're looking uh and trying to establish the right kinds of structures to be able to bring these kinds of tools into our environment. You know, one of the things that we're very fortunate about at Kaiser Permanente, we for a long time, have developed really robust data systems, curated our data and so forth. So I feel like we're kind of we're at the launching pad ready to take full advantage of the tools related to A I. And that's going to come up in so many parts of our organization, whether it's at the front end of clinical care or some of the work that goes on behind the scenes related to uh coding, diagnoses, billing all all all those activities. So it's a really robust set of uh of interest that we have not to mention all the patients facing things member facing things that we can do to help our members be able to navigate our system more successfully, so tremendous energy, tremendous excitement about the potential of these tools with a hint of caution. Of course. So we want to make sure we're responsibly implementing them to make sure that it's they're safe while we're making sure they're effective Lydia. You know, we believe that health care is being rewired and that rewiring is happening across the entire patient journey from the payments to the care, access to the to the care delivery experience itself. Uh So that's one major shift. I think another major shift. And I think many of us have been using A I for a number of years. We know that A I is not new. I think one of the big shifts is that we're not applying it at scale. I know for us, you mentioned at your opening, right. Part of the reason why we had the COVID response that we had was because we harness technology A I and digital to really accelerate a process that normally takes 8 to 10 years and we were able to do it in 269 days. It was 269 days from when we announced that we would develop a COVID vaccine together with biotech to the day that we receive the emergencies, the FDA S emergency use authorization. So the shift is that we're applying these at scale and then the other major shift as well is I think, I think most of us out there are looking for ways not only to be more productive, to be more innovative, but also to be more efficient. So, and I think that's a, that's an important perspective. So it's not just cost savings. That's one part of it. I think it's even more important if you can accelerate your business. So those are the kind of key shifts. Me. How do we see? Ok. And Sean, maybe you could also before you dive into the question, explain a little bit more about what aide does because I probably didn't give it as much justice as you can. Yeah, absolutely. So we're using generative A I to be able to design antibody based therapies really at a click of a button where you take, let's say an oncology target, you feed that into the model and you can have then the model design antibodies that have the specifications that you want. What is the potential efficacy in the clinic, the develop ability, manufacturability and what we're really seeing and we're seeing in our own pipeline that we're developing. Our first lead asset is a target in IBD. And we're going to have an ID filed with that particular drug in 18 to 24 months and it took us roughly $3 million to get to a drug candidate. Plus, you know, the I and D enabling study cost there. And so you're able to really transform how you get drug into the clinic in a very rapid and cost effective manner. And I think that that's one item we're really excited about and it's not even being able to get it there faster and cheaper, but also being able to kind of unlock new biology, you couldn't have had access to before. And I think one of the things like we're really excited about is the emphasis on the data. I think if you look at Chad GPT, the large language models that are out there, they're trained on the whole internet. And if you compare the internet compared to the available biological data that exists out there, it really fails in comparison. And you know, we really see the need to generate new technologies that can scale these biological data sets for particular applications and then also the need to to unlock the new, you know data. How do we get patient data, you know, readily available? How do we create consortiums? How do we aggregate all this data and create new data to train these generative A I models? Because we don't have the data like, you know, you know, open A I and chat GBT does you know when compared to the internet. And so I think that's one of the things we're really excited about is just the emphasis on data and how important that is to biology and health care. Um From AAA patient perspective, would you say that anything has shifted over the last year or so. Well, absolutely. I mean, we've been using A I tools for a long time, but they've been mostly the type of A I tools that have been more kind of predictive analytic tools. So, uh for example, at Kaiser Permanente, we've had a very exciting tool and alert monitoring system that basically a I predictive tool that's capturing laboratory data that's coming in assessment made by nurses. What's going on with our hospitalized patients? That's very good at predicting the likelihood of a patient who might decompensate and require ICU care in the next 12 hours and then have an intervention team ready to act on that. So it's not just an alarm going off that no one's paying attention to. But in fact, we've aligned the workflow with the predictive A I tools and this of course, has an enormous impact on patient experience because it's a lot better not to have to go to the IC U, but to have your care, you know, taken care of in a regular hospital ward and be able to go home more quickly. So there are experiences like that already. But I think what we're anticipating is more and more opportunities for uh tools that can help because we have, you know, full capture within Kaiser Permanente of the whole patient with our system tools that can help our members when they log on to our website and so forth to be able to navigate and those tools to be able to anticipate the kinds of needs because of notes that were generated by the physician or others as part of the care team that can help the patient navigate more quickly to, oh, I see you have a pharmacy need and we're going to take care of that or you were supposed to be referred to the cardiologist. Let me help you set that up. So you can imagine this kind of assistance going on that will directly interface with the member in a way that will make it feel very personal in terms of how, how the member will interact with our health plan. Um Lydia from your perspective. So look, um we believe that digital and A I have the power to drive early diagnosis to enhance adherence to treatment and to really enhance uh health outcomes. So, you know, for example, some of the things that we've been doing, we partnered with fitbit, you know, to proactively identify uh patients that could have atrial fibrillation, we monitor their, their heart rhythm and if we spot any irregular heart rhythm, they get a signal and those that get a signal are then encouraged to go and have a conversation with their health care provider. We have 7 million people around the globe that have signed up, you know, for this capability. Another example, together with Walgreens, we launched AMBA and AM A is an A I APP for Metastatic breast cancer patients that have been prescribed I brands. And what Amber does is that it really provides them with tips, personalized education, medication reminders, and it even will send an alert to that patient. Should they need outreach from their pharmacist? So, you know, pretty powerful stuff, right? And then another thing that we are pretty, that's pretty important to us because we, I think the pandemic uncovered that people need help to be able to understand and access health information. And so one of the other tools that we've developed is an A I capability called clear and clear, makes health information more personalized and easy to access. So consumers can take greater control of their health. So clear, simplifies health information to help advocates communicate with patients in an easy and understandable way. So, you know, to us, as we think about the patient and how we can impact consumers and patients. Absolutely, we believe that digital data and A I um are helping us discover and develop new breakthroughs that, you know, the faster than ever and that's gonna benefit all patients. And the last thing Michel that I'd like to say is that, you know, we're very excited about our, we have a, a proposed acquisition of F and F is a biotech that focuses on cancer therapy. And you know, we are very excited to bring their innovative antibody drug conjugates to more patients with cancer faster. Our big hope is that we're able to do for cancer. What we were able to do for COVID and that's really, you know, what drives us and we think about the impact that that will have on patients is, is what drives us every day. Um I wanna go to all of you for questions, please raise your hand. We'll bring a microphone over and we've got a few if we can get a, a mic over. Um but, but just real quick um Sean, I know you're not necessarily a a, you know, consumer patient uh facing brand but, but, but from where you sit, you talked about the, the speeding up that you're able to do. Um at what point does that kind of start having real impact for, for patients? Yeah, absolutely. So we used to be a pure play platform company where we would use our technology to develop assets for our partners, you know, such as Merck and Astrazeneca. But we've recently started developing out our own pipeline and that pipeline we started building out this, this year in conjunction with Andreas Busch, who came on to our team, the head of R and D at Shire and Behr previously with his help along with, you know, our generative A I tools, we'll be able to put an antibody in the in the clinic in roughly 1818 months and it has best in class properties associated with that or I should say potentially we'll see ultimately what happens in the clinic. But I think that these are big changes, being able to get a drug into the clinic in 18 months instead of 5.5 years with potential best in class attributes to it. I think it's really exciting. Now, these technologies, you know, still have a ways to go to ultimately proving it out in the clinic and we have to be, you know, data driven with any new technology that's out there. But I think what we've seen so far is, is really promising and exciting. And I think it is going to ultimately start to increase that success in the clinic. And ultimately shorten the time it takes to get these new cutting edge therapies to patients. Um Take a question back there, please introduce yourself. Hi, my name is Adam Kaufman. We have a venture fund in Ohio, the up to fund great topic uh for the pfizer or maybe for the Kaiser executive. Uh How do you think about innovation because I have so many founders who want to sell into huge institutions like the both of you. And I know how guarded appropriately healthcare innovation is. So, is it more in-house technology pursuits or how do you even think about scouring the marketplace for new technology, Andy? Do you want to take that? Maybe you can tell? Sure. Uh great, great question. And in fact, uh you know, something that I think, um uh I'm, I'm part of a AAA group at the National Academy of Medicine that has brought together uh leaders from several health systems, brought together ethicists, biotech people and so forth to discuss essentially what you're bringing up. What I brought up from the health system perspective is, you know, what is good hygiene with regard to when some vendor comes up, that I can know that, that, that the essence of this tool a has been developed on data that is diverse so that it's going to in fact result in answers that are accurate, not just for a segment, the population, but for a broad swath of the population that we take care of that. And that it's going to be a safe tool in the sense that it's not going to create risk in terms of privacy and releasing information in ways that our patients and members are not comfortable with. So I think we need to have some common ground rules that can help health systems like Kaiser Permanente be able to rapidly evaluate different kinds of products from vendors and so forth so that we can more quickly adopt them. Because as you say, if we're doing it one at a time, very slow process, if we're not working together on this, there's likelihood of it'll be fragmented in terms of how the different marketplaces will deal with this and the risks that will be introduced. So I think it's in our shared interest to develop some clear ground rules about what good practices are with regard to evaluating these tools and we are trying to develop in house some capacity to do that because we are aware that a lot of the tools are developed on a very narrow segment of the population. I think we probably all have seen some of the data that suggests that almost all the health, their tools are developed from California, Massachusetts and New York. And we don't really, you know, there's a lot of people in between there, it turns out. And so we need to make sure that the tools are in fact working on other parts of the population. So we're looking into our own ability to be able to test out tools to see how well they work in our population in terms of their predictive capabilities and so forth. Look at the end of the day, it's not about how shiny and cool the tool is. It's about does the tool make health care more effective, more efficient, more equitable, more timely, more patient, safe, centered and safer? That's what we look at across the board in all aspects of this. And we want help by working together of thinking about what are good, safe practices with regard to vendors, to be able to do that, that will help everyone speed up this process if I can, if I can give you the Pfizer perspective. So first of all, we believe that health care is a team sport and as much as we have a lot of talent in health and we do, we're never gonna be able to keep up with the latest and bleeding edge innovation. So we've taken a, we've taken a dual, a dual prong approach one, we're always developing our talent and, and we have great talent and we invest in them. In fact, I run learning and development at pfizer as well. So we definitely invest in our talent, the other thing that we've done and, you know, we put in place what we call the Breakthrough Change Accelerator. And it's basically think of it as a sandbox. And what we do is we bring our data to the sandbox and then we throw out a challenge and we invite partners and potential partners to come and answer our challenge. And we again, we provide the environment, we provide the data and just to give you an example, um the COVID for the clinical trial, we threw out a challenge because we needed to be able to process clinical trial data much faster than any of us had seen before. So we threw out that challenge in our breakthrough change Accelerator. And there was a company that I hadn't heard about before, came in, you know, proved to us in our breakthrough change Accelerator sandbox that they had a solution. So not only were we able to do a lot faster. Like six weeks, we ran the, you know, we ran the challenge for six weeks we need and then we could tell within the six weeks who had the goods. Within eight weeks, we had signed a contract with them and we actually applied that capability and that capability we applied into our COVID program and it processed and analyzed the clinical trial data from the COVID trial 50% faster than any other method that we have seen before. So we, we continue to do that. We throw our challenges and, and then that invites, you know, partners and then we prove it out and then the contracting in exchange, we have to make sure that we, that contracting is easy. And you know, we don't put people through a lot of hoops because this is about speed. It's about innovation. You know, Adam Zips over at Amazon web services. He likes to say that speed is a choice and we believe as well that speed is a choice. So anything that we can do to simplify it, if the practical approach that we take. Well, speaking of speed, this conversation flew by and I'm so sorry because I know there were other amazing questions out there. But I hope the three of you can stick around and uh you know, chat with people during the, the upcoming break and later. Um and I just wanna thank the three of you for coming out here Andy by the way, rode his bike to the conference. So how hot is that at this time? But thank you so much.

Yasir Anwar, Chief Technology and  Digital Officer, Williams-Sonoma Inc. Sravana Karnati,  Senior Vice President and CTO, Walmart International, Walmart Moderator: Verne Kopytoff, Senior Editor, FORTUNE; Co-chair, Brainstorm AI

Brainstorm AI 2023: Shopping for New Uses of AI in Retail

Yasir Anwar, Chief Technology and  Digital Officer, Williams-Sonoma Inc. Sravana Karnati,  Senior Vice President and CTO, Walmart International, Walmart, Verne Kopytoff, Senior Editor, FORTUNE

Hello, everyone. And thank you, Yasser and Shabana for joining us. Good to be here. Thank you. And a reminder towards the end of our talk, we'll be taking questions from the audience. So please think up some good ones uh for our pan to start with. Uh Could you set the scene for, for the audience with a couple of examples? Your favorite examples of A I you're using at your company. We start with you, Mr Yeah, thanks. Um Yeah, I I represent Walmart. I work for Walmart International and the CTO for Walmart International. Um You know, we have uh you know A I is, is behind everything that we do. We've been using A I for a very long time including the transformer model that is core of chat GP T. Um You know, a couple of really good examples I can, I can highlight for this holiday season or in the way we are improving. Search for our customers. You can today, you can natural language type in the search box and um the the tool automatically understands the query and gives you the most appropriate search results. That's one example and the other one I can give you is in the way we are predicting demand at what granularity we are doing that and how we are disposition the demand, the inventory in our network across the supply chain. Those are two examples. Yeah, for us and for me specifically, it's about moving the right needle for the business. So besides doing a lot of R and D, we want to deploy A I to move the real business business outcomes. So two examples, recent ones were what is a personalized recommendation and personalization engine we built in house leveraging A I and uh parts of gen A I. And it has allowed us to generate a tremendous amount of uh incremental revenue and conversion lift, which has made a big dent in the in the in the digital revenue. At the same time, we also use personal uh we do a lot of personalization products in our brand like Mark and Gram. So monogramming and stuff, we get returns, we used to get returns and it's difficult sometimes to find the accurate cause of those returns because either there's a thread or there is a location of the monogram or A or or uh the machines that are using that. So we leveraged an A A I based solution, we built it, which is image based and we pass our returns to that solution and it accurately identifies what could have been the cause why a customer returned a particular product that helps us in closing the loop with our production manufacturing and tells us this has been a very good, very recent um exploration and deployment of A I in our company. And there are many more examples. Well, you may have heard it's we're in the busy holiday shopping season. Um When do you start preparing for that? And do you do anything special? Simply because it's the holiday season in terms of A I I go first. So in terms of the way I, so first of all, I would say at Walmart, we have every day is a holiday type of uh you know, operation. So uh we are super focused on our customer all through the year. Uh But we all know, so the holidays uh time frame is a very special one. We are sitting on a lot of inventory. We are, we are, we are bringing in a lot of inventory from overseas and, and domestic channels as well. Uh So being able to, you know, accurately predict the demand and uh and uh store to store the inventory at the right place at the right time is, is, is the most important thing we can do. And then really, you know, scaling our systems is another one that we do. Um You mentioned A IA I, like I said is in everything that we do. One of the things that we do in our infrastructure is enamel detection, you know, whether it is for fraud fraud detection or anomalous behavior in the infrastructure to see if workloads are not behaving the way they are supposed to behave and they need to be shifted around. So that's another example where we use ea I are, there are many such examples and you also holidays anything. Yeah, I think for the A I, it's also important to understand that A I is, you know, we talk about governance and it could also go rogue. It could also build biases over the period of time. So holiday when you are, you know, 40 30% of retailers, business happens in the Q four. It's important to kind of revisit your business rules, revisit your testing strategy on A I itself and make sure it's not going into other directions. So that's a general A I governance that is we focus on for preparation of holidays due to high volumes, prepare better data for it for its training of the MO models. And uh one other example from a functional perspective is uh you know, the promised date for the customers and finding the optimal path and supply chain to get the goods delivered to their homes. So we also leverage it there and with holiday with the different uh you know, allocation of the inventories in different locations and the and the holiday products. Um it has a major role to play to make sure we are serving to the customers in the most accurate manner and in terms of uh demand forecasting? What kind of data goes into that? Are you using like weather? And is there any kind of strange data you're pouring in there? Yeah. So obviously, you know, one of the things we did was to look at our like, like uh you know, most people look at past demand and then predict, you know, future demand. Uh But we found that over time, we need to incorporate all other signals as well, whether it is uh you know, weather data, social media channel, uh you know what people are searching on our site. Um And whatever else we are, we are seeing in, in, in the, in the market, in social media and other channels, incorporating all of that and being able to uh predict demand at a Q level by store level as well. So we've been able to apply the uh you know, deep neural network strategies to predict demand. And we, we, we did a I before I would say in this year, uh the, the fidelity of the data and the fidelity of the res results that we're seeing is much, much better compared to what we've seen in the past. So I think kind of the answer is kind of same and common because I think the dimensions of data are increasing day by day. So even if you have the models that we're running, you need to fit it more dimensional data. One you were talking about season one, it could be weather, it could be the supply chain answers that have been evolving from last year to now, things have changed. The cost of uh supply chain has changed, it has changed the dynamics of things making available. Your stored locations. Are there the events that are happening? So you try to tap into as many data insights you can bring in to be able to do better forecasting. Um Now, now yo your your supply chain is particularly uh so challenging because you do a lot of custom products. Uh you order a couch with 100 different options. There's no couch sitting in a warehouse with those options available. How how does that make uh sort of predicting delivery? Uh How do you go about predicting delivery under those circumstances? Well, I think um you know, a delivery from a order from a click all the way up to the uh houses the customer requires uh goes through multiple segments. One of them, as you mentioned is manufacturing. So we can talk about all of the segments, but we look at the different segments of our supply chain. But in this case to your specific to your question, we look at the within the four walls, productivity of that particular plant or a unit. Understand based on the volumes that are coming in uh what has been the historical production and what they can predict based on the raw materials that are available there. And that becomes a good data point into calculation of the delivery points for those uh custom products. And how about you? Uh Servan, you're not creating necessarily custom products, but you do have millions of products. Yeah, we have millions of products and you know, several notes in our supply chain, whether it is distribution centers or D CS or stores and every one of these things we call that is a node you could ship from either to the uh to the customer or if it's a DC to a store or directly to the customer as well. Um So one of the things we we look at is, you know, what is the optimal location for inventory? Right. So can you delay that as much as possible? So that you can actually have risk pooling and then hold the least amount of material in stock in your, in your network. Uh And then that, that also should consider, you know, your, your catchment areas for a, for a store, uh availability of your drivers, you know, when you're talking about, you know, uh distributing, you know, end products to customers. Uh And all of these supply chain signals are used not only in, in predicting what is the best place, but also what is the most optimal place to put it, right. So I make a distinction between, you know A I strategies um and you know, highly optimal mathematical strategies. So I think, you know, solving supply chain problems really involves blending both. It is not just A I or JA I, but it's also mathematical programming techniques such as operations research and, and so forth and, and the author uh obviously A I has an impact on jobs. Um How is William Sonoma uh thinking about that? Yeah, I think we, I mean, we are very excited about the advent of A I and especially the JN A that is coming and people talk and worry about and scared of losing jobs. I would put a question for ourselves and to everybody that I think I would be personally more worried and scared if in every five years, we don't have a big technology disruption that is coming because that actually brings jobs and it's not me again. We all love data. We are here for data. Data tells us historically, no matter whether it was from uh from horses to cars, to airplanes, to electric vehicles, you name it all the advance of technology came in. People might have been scared back then. But the new ecosystems, these technologies generate, generate millions of jobs. So if the key is about adoption of A I in a sustainable manner for the business outcomes, not just for, you know, shiny object running after that and also safe and secure A I. If you produce that and you scale it, I believe in the next five years, we'll have multi million jobs generated and a new ecosystem that will emerge, that will require jobs. Of course, it will require some training and transforming how we achieve our work and also redefining our priorities where we spend time uh in doing meaningful work. Uh Well, let's see if we have any audience questions out there, we'll have people uh with paddles coming with the microphone to uh we have somebody up here on front here. I am. Uh could you uh give your name and identify your affiliation? Sure. Um John Jessup, I'm CEO of 1440. Uh Yes sir. You guys are a big sales force shop. So I'm curious your thoughts on like how like data cloud like commerce, personalization and then Einstein, it's in the kind of overall shop we have partnership with sales force. I wouldn't necessarily call we are a sales force shop. We leverage some of their great technologies, especially their email marketing program and we build a lot of custom in house solutions and leveraging the open source A I and we continue to evaluate with them whatever they're bringing. I strongly believe that it cannot be just you're doing your own uh little A I and 27 solutions and broken different parts because in few years, if you do not uh integrate with the larger ecosystem, whether it's open A I, whether it's C Force Google uh API S, then uh the work we will do and solutions we will build. After a few years, you have to abandon them because they will not be able to catch up. So key is how do you leverage the ecosystem but also have the expertise in house to build and drive your own destiny versus depending too much on one vendor or a company. Another question over here. Thank you. I'm Sammy from a ARP. Um Could you, as you've, as you've ramped up your use of A I and to maximize your investment in it, Can you talk to us about the kind of roles and jobs that you've increased or you've created in your organization? Yeah. So, you know, uh one of the things we have done is to um educate all of our uh associates around use of gen A I and uh roll out, you know, incremental tools. For example, we rolled out a tool called my assistant that allows our associates to ask questions around their benefits or what have you. Um And so we are in the, in the stage where we are getting our, our, our personnel or associates trained on A I and advanced computation technologies. Uh When it comes to our supply chain, the kinds of work our supply chain uh associates do is quite different now than what they did a few years ago. It is not that, you know, we created new jobs from that, but it is the kinds of things that those personnel do that is quite dramatically changed. And we see this, you know, continuing this trend continuing um with our associates, you know, doing more of creative strategic work and less of mundane sort of manual type work. So anything that can be automated can be obviously automated. So using systems with A I, what we can do is to actually embed some intelligence, you know, in the systems as well. So they can mimic, you know, human like decision process. But when you are solving the problem accurately, as I said, you don't need to sort of emerge it with the mathematical sort of foundation. And many times our associates actually are asking for those solutions. And so we believe in designing the systems and solutions with our associates and with our customers as opposed to for them if you will. Yeah, I think a lot of the thing is about adoption and training and yes, making your, first of all, it starts with the top. So your CEO your board, I think it's very important to bring them along and explain them how you're gonna approach A I, what are the relevant use cases that you go? And then even in the in the building cycle, it's important. The role of a product manager in technology has dramatically changed leveraging A I. So they have to understand to be able to go to the outcomes, the way they are going to define the KPIS are very different. So there is training going on. So new you can transform a role of product managers have emerged. Then of course, you have to have A I engineers, data scientists, more and more of them. So they are attacking the problem in a way. So it has generated newer titles, newer jobs with newer skills, but it also requires the entire supporting ecosystem for them to come to fruition. Yeah, great. Well, I think we're out of time. I'd like to thank my Panelist, Servan and Yasser. Thank you for the uh fascinating chat.

Jim Cathey, Chief Commercial Officer, Qualcomm Kyle Daigle, Chief Operating Officer, GitHub Madhav Thattai, Chief Operating Officer, Salesforce AI Moderator: Rana el Kaliouby, Co-founder and CEO, Affectiva; Co-chair, Brainstorm AI

Brainstorm AI 2023: Implementation in an Organization 

Jim Cathey, Chief Commercial Officer, Qualcomm Kyle Daigle, Chief Operating Officer, GitHub Madhav Thattai, Chief Operating Officer, Salesforce AI, Rana el Kaliouby, Co-founder and CEO, Affectiva

Sal Khan, Founder and CEO, Khan Academy Moderator: Jeremy Kahn, FORTUNE


Brainstorm AI 2023: How AI Is Transforming Education

According to a sales force study, more than 70% of Gen Z report having used generative A I with such a rapid adoption rate. It is no surprise that the education system has had to address concerns around the use of gen A I for cheating and plagiarism. Our next speaker believes that instead of fearing the misuse of A I in education, we should embrace its potential. And with the right guard rails, an approach, artificial intelligence can become a personal A I tutor for every student and an A I teaching assistant for every teacher here to talk about his latest project. Khan Migo. Please welcome Khan Academy founder Sal Khan. He'll be interviewed by Fortune senior writer and brainstorm A I CO chair, Jeremy Kah. Thank you. So, thanks for joining us. Thanks for having me. Um Why don't you start by explaining a little bit about what Conmigo is for those who aren't familiar with it. Yeah, I was just introduced at a very high level, think of a tutor for every student, a teaching assistant for every student. You know, at first we just, if you think about what Khan Academy is a lot of folks know it started with me tutoring cousins almost 20 years ago and I start with one cousin, words, friends of my family, free tutoring is going on before I know it. I'm working with 1015 cousins. I started writing software for them to work at their own time and pace. I start making videos for them and, and if you think and you fast forward to even pre generative A I it's really been about how can we scale what I was originally able to do with Nadia. And so when we saw what was possible over a year ago, you open A I showed us GP T four summer of 2022 we said, wow, this could take us that much further. And so our original view was, wow, this, if a student is struck stuck on something on Khan Academy, it can help them with that if they're watching a video and they're like, why do I need to learn this? It could help them with that. We started to realize it's valuable for a teacher to do things like lesson planning, grading papers, writing progress reports. And the more we've worked on it, we realize it can do things that would even transcend a traditional tutor. It could emulate. We have simulations where you could talk to you or the donkey or Jay Gatsby um simulations you, you could talk to Alexander Hamilton or, or Thomas Jefferson. Uh We just launched last week where it won't write your essay for you. But uh you can get feedback on your essay and it's like working on a Google Doc with, uh with your editor. Uh when, when you're trying to make a paper better, uh So, so big picture, we're realizing almost every day this can go do more and more of the things that one, we were always trying to scale that a good tutor, teaching assistant would do. But we're starting to realize it can go pretty far beyond that. And how do you make sure the students aren't using it to cheat? Because I think that was one of the big fears and, and a lot of people were using chat EPT to cheat when it first came out. Yeah. And, and this is a, you know, that first weekend we had access, we're like, this could be used to cheat. Um, but it has all this other power. And so from the beginning, we had all the debates inside the organization. We're like, ok, well, first of all, we can set it up so that Conmigo itself does not cheat. So if you go to Conmigo and you're a student, there's two modes. If you're a teacher, you can put it in teacher mode and then it will help you learn how to explain it and things. But if a student says, tell me the answer, it will say I'm your tutor. I'm not going to tell you the answer how do you think you should approach the problem? On top of that, there's fears around just students getting into unproductive conversations. You know, we, we've already deployed this with tens of thousands of kids in real schools and I mean, just a stereotype 1/7 grade boy, uh will immediately. Right. Let's call it Red Hat test it in ways that very few of us would imagine. Uh So we have guardrails and the A I itself, we have a second A I that's moderating it. All of the conversations are transparent to parents and teachers. If the, if the moderating A I thinks that the conversation is shady, it'll shut it down and it'll actively notify the teachers and the administrators. So there's a lot of guard rails there. And then for the, you know, like this essay tool where we already launched the essay tool where we're going is roughly by back to school. Probably sooner a teacher is going to be able to make an assignment through Conmigo, probably develop it with Conmigo as well. Conmigo will then tell the student, hey, let's work on this paper together. I'm not going to do it for you, but I'm gonna be your coach. I'm gonna give you feedback and then when the students and then the student can submit it and, and maybe even the teacher can get some preliminary grading and things like that and what Conmigo is going to do and is already doing to some degree. It won't just give the teacher here's the essay that Jeremy wrote. It'll say here's the essay we worked on it for 4.5 hours. Jeremy had a little trouble coming up with the thesis statement. We eventually got there. And by the way, in your class of 30 about 15 of the kids had trouble coming up with the thesis statement. But icon Migo a confident it is Jeremy's work because we did it together. The writing that was produced is consistent with the other writing that I've seen of Jeremy, especially the writing that he's done in the classroom. And if you were to go to chat GP T and even before generative A I, there's plenty of websites on the internet that you could pay $5 a page and they'll hire someone in Kenya to write your paper for you. This is real. Um If you did that and you copied and pasted it in Conmigo, tell the teacher, like, I don't know where this paper came from. It just showed up, it looks shady and it's completely inconsistent with what Jeremy's been doing in the classroom. So instead of being the end of the college essay, this can actually preserve it potentially and, and extend that process and make it interactive and even go, potentially, even go beyond the, I think the essay will always have real value. Uh But often it could be a simulation. Now, it could be a conversation, an interview that you have with the A I instead of just a static. How do you make sure the information that Kono is giving the student is accurate? Because again, if you're using an LM, this has been one of the big concerns is that they'll invent information, they'll hallucinate I think for step one, anyone making an app, especially for students. But really for anyone, you have to be completely up front and say like this is not an infallible source of information. So even on Conmigo, there's things like Conmigo can sometimes make mistakes now with that said, especially who we are and what we're trying to do, we are trying to minimize that as much as possible. A big way we've been able to do that and we have a whole bunch of evaluations to measure the hallucination rate and the math error rate, anchoring it on Khan Academy content. When you anchor it on content that already exists, your hallucination rates almost go to zero. We have a couple of use cases where it isn't quite as anchored. But even there a lot of work to make sure it gets better and better on the math side of things. And this is stuff we worked on October of last year. Prompt chaining, being able to do math in the background. We, we worked a lot with a lot of the open A I researchers on ways to do multiple calls to the A I to significantly bring down the math error rate. So already we feel pretty confident and we're seeing every successive generation of models are getting better and better and you know, some of the what we're, what you're now starting to see with models, you know, with, with Google's Gemini, we'll start seeing what Gemini Ultra can start doing in, in about a week's time. Um You're going to have, you know, there's rumored models we all know about open A I. So I think that's, you know, by the end of this year it'll probably be even, it will be a lot better. Now, you guys were built on open A I GP T four when they had a sort of fiasco as Altman was fired, you know, no one knew what was going to happen. All the employees were going to leave. Were you worried? I mean, what, what was going on at Khan Academy at the time? Yeah, I mean, I was intrigued by what was going on. Um I uh very quickly we got reassured. We got, I'm sure they reassured many folks, but very quickly we got reassurances that there would be no interruption of service, etcetera, etcetera. At the same time, we did reach out there are GP T four end points you could get directly from Microsoft. So we did reach out to, you know, the highest levels of contacts we had at Microsoft to make sure that we wouldn't have an interruption of service. So I, I didn't lose a ton of sleep over it and thank God it's all kind of come back to we think normal now. But um yeah, it was, it was an interesting, I mean, for everyone here, it was an interesting couple of days. I've got some more questions. I wanted to get some questions from the audience. If there are any, anyone have a question for yourself from the audience, please raise your hand and we'll try to get a mic to you. There's one right here, this woman here, if you could please stand up and identify yourself when the mic gets to you. Hi, I'm Shanaz. D. I work at the post venture. I used to work with Sal too. Um Quick question with Con me go. Does this mean you're gonna have less videos generated on the con side or is it gonna be complimentary? How do you view that? Yeah, I think it's a very fascinating question. Long term, you know, in the, in the short term, I think short term, medium term, I think they're extremely complimentary. I think as we mentioned, like the hallucination question, you know, what can a Khan Academy video do? Well, it's, it's able to, you know, talk about it the way I did it with my cousins, etc, etc, what's its limitations that people have always known about? Well, what if a student says, why do I have to learn this or they have a follow up question, etc, etc well, now the generative A I has the context. It could either refer you to other content or it'll in that moment answer the question. We've actually now done it. So it has memory on you and, but that memory is completely transparent to the user, to teachers so they can reset it, they can delete it. They're like that inference isn't right. But if you go on to Conmigo and you say, why should I have to learn this? It'll say, well, what do you care about? And you say, oh, you know, I want to be a doctor one day and so it'll be able to connect it and then it'll use that information in the future to hopefully try to keep you engaged. Now, a big conversation we've been having internally at Khan Academy and I'm someone who's personally and I made videos yesterday. I personally made 7000 videos. So I don't want them to be all irrelevant in five years. But um I've come to terms with they, they might be a transition step uh in three years, four years, five years, it's very plausible that Conmigo is going to essentially create a video on the fly for you that it's not really a video anymore. It's an interaction and you're going to be able to talk to it. It's just gonna feel like you're blackboard together like a Khan Academy video. So we'll, we'll see where that goes. We're we're definitely trying to, I tell the team at Khan Academy every day, like we should, we're canceling projects and adding projects on a daily basis. We're canceling the ones that we think are going to be irrelevant in three years because of this dynamic. And we want to work on the ones that will only become more and more relevant as the models get better. Great other questions. Do you have any other questions? I've got a question for. So which is, do you think this technology ultimately will help inequality? Because you're going now really be able to put a personal tutor in every student's hand or is this going to be something where you know, the better off kids, they're going to know how to use this technology, they're going to have, you know, a mobile phone that they can run it on and, and essentially they're just going to pull ahead and it will actually exacerbate inequality. So yes and yes, I think you're absolutely going to see all of our Children adopting this first. Now with that said we are, you know, on the drive up here, I was talking to some states who are talking, who are talking about adopting Conmigo for the state. So we're talking to a lot of districts, I think by this time next year, we're going to be looking at many hundreds of thousands of kids in mainstream public schools, if not millions using this as part of their day to day. So that will help drive. But with that said, I think everyone in this room, you and your Children are going to be on the bleeding edge of that. But what I always tell folks is and look, some of this is true of even classic Con Academy. There's much more awareness in this room of it than if I were to go into more underserved parts of the country. And we've obviously been trying to work on that. But I always tell people, you don't look at whether it's helping or driving inequality by looking at the proportion who's using it. You think about it by saying what happens if it disappears. So I could probably, you know, we're right next to the Museum of Modern Art right here. If I just observed who's going in and out, it's probably 80% people who are educated, people who are affluent who are using it. Does that mean that it's driving inequality? Well, I would say what happens if that museum disappears. If that museum disappears, everyone with resources will fly to another museum or look at art books or find some other way to get exposure while that 20% is going to have nothing. And so we say the same thing about something like Conmigo, if Conmigo didn't exist, people are going to find another tool, you're going to get a paid tutor, you're going to do whatever. But there's that 40 50% of kids. If it disappears, they're no longer gonna have something that can advise them on college, something that can help with their essay, uh, something that can tutor them. Great. I want to get one more question and, uh, this gentleman here, thank you. First of all, thank you, have a parent and uh, we thank you for all that you do for our kids. And uh, thank you very much for all the math. He help that you've given me over the over the years. Um Quick question on Yeah, you good. Your brand is built on trust. We send our Children to Khan Academy, whether it's the app or the website, um teachers, et cetera. So how do you, what advice do you have for brands who are in the information business and the trust business? How do you think about the risk reward? Why did you jump in so early um in an environment where there's a lot of trust for you to lose if you don't get it? Right. Yeah. And it's fascinating because I, I even before a I, I used to point out to our team and even, you know, our philanthropist, I was like, look edu education. It tells you something about the space that there aren't a lot of brands in the space. No one not to pick on anyone but like no one says, oh, that's a Pearson textbook. It's going to be awesome, right? Um Some of them might be but I, I'm just saying, you know, there's not, it's not, it's known, but it's not a brand in that way. And, and I, I hopefully we have built that people say, oh, it's approachable. It's quirky, it's connected, but it's high quality. It's conceptual. We had those debates inside our, our own organization this time last year. Um You know, and what I just told the team is, you, you know, we can't just run from this. Our brand is also about innovation. And so let's put the guardrails in place every one of our fears about brand or otherwise, let's write it down and let's turn them all into features. And, you know, I talked a lot about them and I remember when chat GP T came out that day, I immediately slacked Greg Brockman and I'm like, what's going on? You guys have us under an ND A. We're not supposed to release until March of 2023. You just launch something. And Greg's like, oh, this isn't a new product. It's just a chat interface on GP T 3.5. It's been out for months and the world has just exploded. And I was like, oh, people are going to throw out the baby with the bathwater. We're working on all this stuff. All these school districts are starting to ban it because of cheating. In hindsight, it was a blessing because everyone allowed them to process a very imperfect tool and frankly, by about March. They start saying if only someone we could trust puts the right guard rails around it, avoids cheating, makes it transparent, helps solve the bias the data. And then we're like, here you go. And honestly, we have, that's great. I'm going to have to stop you there because you're right out of time. But I thought fears to features is a, is a great takeaway. Thank you so much Saul for joining us and thank you all for listening. Thank you.

Aaron Levie, Co-founder and CEO, Box Moderator: Michal Lev-Ram, FORTUNE  

Brainstorm AI 2023: The Big Picture: Silicon Valley and AI

Brainstorm Tech 2023: The Big Picture: Silicon Valley and AI

This next founder has been operating in Silicon Valley for more than half of his life with such experience comes a deep understanding of the power players in the evolving technology landscape at a time when many organizations are calling for efficiencies. Box has already been there and done that in 2021 Aaron Levy successfully defended himself against an investor's attempt to oust him from his CEO position. And as of last week, Fox reported an operating margin of 25% here to talk more about the future of Silicon Valley and regulating A I. Please welcome Box's co founder and Ceo Aaron Levy. Ok. Aaron says it wasn't exactly half. I I think um uh I was trying to do some math. I think it's like one year, maybe less than that. So you should feel very, very young. I I do although I'm feeling very old right now. So I apologize, I don't make a lot of eye contact. Um I have some like allergic face uh issue. Um So we're trying to get to the bottom of it like shampoos and um quite a bit. So I apologize if I'm not like staring right at everybody. But, um, uh, it does make you feel very old although I've been, um, and also I say chat G BT is decent at helping me. I was going to say yes. Are you using it? Well, so the doctors are extremely defensive when I come in and I bring in Chat G BT recommendations. I found that they, it really dramatically changes the course of the conversation and they kind of want me out of there, maybe don't lead with that. That would be my suggestion. Ok. So we're going to talk briefly about Open A I and kind of where I should say the Open A I saga that enthralled all of us. Um One of my favorite uh tweets posts on X from you during that crazy uh few days is what you wrote when everything ended and it was ok. So can we all now agree to never do that again? Um And I'm, I'm curious, I just wanna hear, you know, you're very prolific when it comes to pontificating on a whole slew of topics and, and including what we all witnessed. So what are the, what are the big takeaways for kind of Silicon Valley and, and tech large, I think. Um this is gonna be kind of annoying. So we might need to jump to the next topic quickly. I actually don't do that many takeaways. Um I, I think literally if you, if you rate uh if you kind of did just, you know, looked at the ratio of the amount of drama to the amount of takeaways. It's like a, it's, the ratio is really off. Um, uh, and, um, and I, I like, like, I think the main takeaways don't have weird corporate structures, like it's just like it never ends well. Um, and so that's probably the biggest takeaway. Um, I think if there's maybe another subtlety in this and I'm sure this is what, what's been talked about quite a bit is, is we do have some factions emerging in Silicon Valley around A I. And so I think probably maybe the bigger takeaway would be, um, would be, you know, how do we land the plane as an ecosystem on this topic of A I safety A I regulation and, and these debates and clearly there was, there were real, you know, sort of factions emerging on the board and within organizations, um, more and more on this topic and it's, it's hard to kind of reconcile the two different groups. Um, it's not obvious what, what, uh, what you would land on that would make everybody satisfied because you have one group that, that is, is sort of increasingly believing that A I could end the world. And you have another group that, that sort of believes that's not gonna happen. And, um, and in fact, if anything, we need to accelerate, you know, innovation and progress in A I to get all the benefits of it. Um, and, and those are, you can't square those two things. Um, because one sort of imagines a nuclear winter, I mean, a nuclear, um, uh, you know, bomb going off in the other, um, imagines that, you know, would be able to solve my, my face issue. Um, if we had just better a, I, so like those are like, hard to bring together. I love how you're calling more attention to it. I know, I, I, when no one would have noticed. Ok. Ok. Well done. Um So, but, but, but talk about this because this was gonna be my next question. The two camps, you know, it's not binary, is it? How do you, how do you bring the, the nuance into the? Uh Yeah. No. So the reality is, is that probably 80% of people are somewhere in the middle of that bell curve that, I mean, as with every political, you know, topic. Um, and this is a political topic at the end of the day. Uh You're mostly going to hear about the edges and, um, and, and the reality is, is that, you know, the middle is we, we probably need a lot of progress because there's a lot of benefits to A i, whether it's education or health care or, um, or across the economy. At the same time, we need to ensure that we have a, you know, a decent degree of safety and, and protection and maybe like, like, you know, some asterisks around this, but some degree of regulation, I tend to sort of believe that the regulation should fall more in the category of regulating the use cases of A I across the existing agencies that tend to regulate those, you know, use cases of any technology. As opposed to imagining there's some central, you know, centralized monolithic regulator that perfectly predicts all the things that we should, you know, you know, prevent from A I doing A I from doing. And so that would mean that the FDA should regulate A I in its, you know, area of expertise in the FAA and so on. And that's probably the best outcome because, you know, if you have the centralized regulation of a model, one person might view an output of that model as being dangerous. Another, another might view that as actually being extremely productive and the law really needs to be more about how do you put that output into practice and in, into, you know, some degree of production that's gonna be hard to essentially me to sort of manage. Ok. Um Please raise your hand if you have a question for Erin and we'll get to you a little bit later um on, on, on box, more specifically, um you've been trying to make knowledge workers more effective for many years now, as we heard half your life, almost half your life. That actually is true. It was the silicon Valley thing. We, we, we moved to the Valley about a year into the company. So, so, but, but half my life has been, you know, working with files. Very weird. Yeah. So when I met you at Fry's Electronics in Palo Alto, that was a year and a couple years in. Yeah, exactly. All right. Um R IP prize, by the way. But, um, so, so my question is, um with generative A I, what's different, what's changed over the last year in the product and then how you're engaging with customers and how you're working internally? Yeah. So uh for, for us, I mean, we, we kind of landed on, on, you know what I think of as, as one of the best opportunities in A I because we work in the land of unstructured data and unstructured documents. And so what's inside of a document? It's lots of text. What's, what's a large language model really good at understanding and working with text. So if you imagine like, like where is there a large corpus of information, you know, in any kind of enterprise that's in your documents? And that, that could be your legal documents, your, you know, medical information, um equity research contracts, just anything that you have that is full of text A I models are increasingly good at summarizing synthesizing extracting data from searching across. And so that, that was a really kind of profound opportunity for us. And we tried a bunch of things with A I over the past seven plus years. And what we found is you needed very narrow models for each use case because that was the only way that we could get the right level of quality and efficacy from A I. And so the real breakthrough recently was GPT 3.5 and then GPD four, maybe now, Gemini where you have these models that can work across a large variety of contexts. That's what makes it actually more practical for, you know, kind of any given random enterprise to be able to use the technology. Because previously, when we tried to bring A I to your documents, we would have to say, OK, so you're a law firm that's based in the UK working on these kinds of contracts. Here's a specialized model just for you, that was the only way to really kind of solve these problems. Now, you can do it in a generalized way, which means that you, you actually can scale this as a, as a product. And so we're in the very early stages with box A I, it kind of does exactly what you'd imagine we would do. It takes all of your data plugs in an abstraction layer that connects to open A I or Google and other partners over time and then lets you work across your data with, with those A I models. And you said maybe now Gemini a lot of conversations about vendor lock in and fear of that. How are you guys approaching that question? Yeah, I mean, I think it's prudent probably for everybody to have as much optionality and a future proof strategy with, with your, with your A I, um, you know, certainly if you're a smaller start up, you know, speed the market and thus sort of, you know, probably picking a vendor right now, you know, matters a lot. That was actually why I think the open A I, you know, moment was so dramatic. Um It was not your classic sort of leadership struggle um or dynamic. This was a technology that's been embedded in tens of thousands of products at this point. And so it really had a full ecosystem. Um you know, it was, you know, it was consequential across the ecosystem, which is why you saw so much um uh you know, uh of the ecosystem rallying around the issue. Um But for us, you know, we, we wanna work with multiple vendors because we know there's gonna be different specializations, different costs, um different approaches to the technology. And so we'll plug in um you know, any, any vendor that our customers want to work with. OK. We have a question back there. Hi, Amy Lemire from WXR Funds. So one of our portfolio companies does A I for rashes. Oh my God, thank you. What is it called? Health? I was gonna say any questions just not about his, I'll take any recommendation. If you know an A I app for face rashes, please let me know. Yeah, so they do. But related to that like at what point, how do you assess uh you can trust that data? Like going to the doctor afterwards with the recommendation? So, I mean, actually, I mean, I'm not being facetious. I actually think this is why I'm extremely optimistic about humans and A I and I think this is what gets missed and there's, there's, you know, plenty of good research and literature on this. But, but in my own personal life and I think probably most people's personal lives A I is a rapid accelerant to getting information, getting, you know, insights and expertise and then you almost instantaneously go and ping a person who, you know, as the expert in that, in that domain and you say, hey, does this sound right? So this idea that A I replaces lawyers is just completely impossible. A I will accelerate our ability to get advice that we then go ask a lawyer about. And so because at the end of the day, we're going to need liability, there's going to be liability in the system. You want a person able to apply some degree of human judgment on your specific circumstance and, and the amount of context you have to have about that individual circumstances is at this point, you know, probably 3 to 5 orders of magnitude more than what the A I model is getting when you just do a quick chat back and forth. And so all of that context, all of the situational awareness, all of the understanding of, of, you know, just like what is germane to this particular person's issue because of the, you know, jurisdiction or the place that they're at, like the A I model is never going to really, I mean, anytime soon is not going to have that level of depth. And so, so, you know, whether it's, whether it's coding or recommendation systems or um you know, health care or education, I think it actually increases the ultimate sort of usage of a lot of the surrounding products and services of those industries. So I'd be, I'm mostly, you know, extremely optimistic across the board, even things like designers. Um I, I, you know, my, my, my quick, my quick belief would be that actually the design consumption and thus design services goes up as a result of A I because what it does is it lowers the bar to experimentation dramatically as you experiment and do try new things you instantly say, oh, actually, like let's turn that into a real thing. And then all of a sudden professionals, you know, you go out to some kind of professional for, for whatever that product or service is. So whether it's, you know, again, legal reviews making a web page, health care advice tutoring, I think this is good across the board for the economy. Other questions for Aaron, raise your hand, please. Ok, we go. We've got one back there. Thank you. Hi, Jennifer Fonstad Al Cell. Good to see you. Um My question is about um privacy and around the idea of using proprietary data. So a lot of folks are thinking about um a lot of, a lot of companies are thinking about how to bring in their proprietary data in a way where they're not risking that um the privacy issues around their customer data. And then also thinking about proprietary uh I'm sorry, privacy from uh an application in the cloud such as yours. So I wonder how you're thinking about this and in particular, if there are any vendors that you are finding are useful in those areas? Cool. Yeah, good. Uh Thanks. Um So, so I, I mean, I like, I, I don't wanna turn this into an advertisement but um but unfortunately, like, but I will um and, and, but like your question is exactly dead on um To why, why we think there's a big opportunity is um is uh like, like probably the most boring part of our service um that like literally like I shouldn't even be allowed to mention on stage because it's so boring. Um Access controls and permissions is going to become like a 10 times larger problem with A I because if you just said, hey, you know, here's this, you know, A I system that works across your entire enterprise and accesses everything across your organization. And somebody asks a question of that, the likelihood of that, you know, sharing information that the user actually is not supposed to know about because they're not actually supposed to have the source information accessible to them is very high right now because we've all dealt with, you know, information security through obscurity essentially where we sort of obfuscate the access to the data as the way of securing it. But all of a sudden A I wants to go look at everything, it wants to consume everything and, and so that's going to be this big problem, which is, you know, when Bob asks for Sally's hr you know, data or salary is Bob supposed to get an answer back or not. And a lot of our, you know, typical enterprise systems are not prepared for A I going out and trying to ask that question and getting the answer. So this is where things like again, like just like understanding who has access to what data, the controls around it, the privacy around it matters a ton. Obviously, that's, you know, the business were, but there's gonna be thousands of products that do that. Um On the training side, this is, this is where I think I, I I'm concerned a little bit that, that sort of training your own data set right now is not a complete panacea because um you know, a lot of companies, I think initially want to go and train a model on their data, that, that, but then instantly you ask the question of like, well, what's your like, what data like, like? And then you have to get pretty, pretty narrow, pretty quickly. Um Because anything that is sort of generalized across your business, probably not everybody in the company can get an answer back and, and that would be appropriate. Um And so you can't just like train your company A I model on hr information or on product information because can anybody in the organization ask a question of that A I model and get an answer back? Like would, would that be appropriate? And, you know, all of a sudden you'd have corporate secrets getting revealed. So things like kind of, you know, data protection in the A I, you know, model itself is like a complete, I mean, there's a lot of start ups that I think are working on that. It seems to be like, you know, a relatively either difficult or intractable problem at the moment, unless you put an application layer, you know, between the user and the A I model that is sort of watching for all of that. Um But I would say just super early days, somebody will make a ton of money, um you know, solving that problem. And um but, but these are the things that I think we still have to explore. Erin. Thank you so much. I'm gonna let you go and look at that app. Ok? Thank you. Ok. All right. Thank you, Erin. Appreciate you being here next up. We are going to take a 30 minute break. We'll meet back here at 415 and when we come back, we're going to kick off with a demo from hour one on how to create text into generative A I video content. See you back here shortly. Thank you.

Natalie Monbiot, Head of Strategy, Hour One Moderator: Ellie Austin, Deputy Editorial Director, Live Media, FORTUNE

Brainstorm AI 2023: Lights! Camera! AI! Harnessing the Power of Generative Video

Hello, everyone and welcome back. We have some really interesting conversations ahead this afternoon. So let's get started. Now, generative video is reimagined, reimagining even the way we create content, simple texts can be turned into video presentations, news content and product demos in multiple languages within just a matter of minutes. And that's just the beginning of this, this spring hour one, an A I video generation company launched video wizard, a feature that automates the process of script writing and video creation by having the user simply submit a prompt here to show us how to use generative video and to discuss what the future holds for the technology. Please welcome our one head of strategy Natalie Moo. Hello, Natalie. Thank you so much for being with us. Now, I alluded to it just then in the introduction, but I'd love you to briefly run us through our one's mission in possible use cases. Yeah. So our mission hour one is to re invent video production and make it uh very easy for anybody without even having the skills to produce video uh to do so in just a few clicks. Um And so maybe what we can do is jump into a quick example. Let's go for it. At hour one, we're on a mission to replace cameras with code for the next generation of video creation. We replace the camera layer with A I and combine all the generative A I tools to enable professional video creation at your fingertips. So you might recognize this as someone that looks a bit like me. This is my virtual twin. Uh She is entirely generated using A I after a little bit of uh footage was submitted to our algorithm with my permission uh to generate her. And that's my, also my generated voice. And um I can also show you a little bit about kind of yeah, how that this was made. So once that's done, you can then find A I NAT uh within our platform. And, and so the platform has a selection of a whole range of different people that you could or virtual avatars that you could choose. Exactly. So you can see here, it's almost like a library of virtual humans there on the left. And for this video, I picked my own and I picked one in a style of expressiveness that I felt was fitting to an introduction at the Fortune A I brainstorm. And in this case, I entered the text because I knew what I wanted to say. But as you said in your intro, Ellie, it's also possible to just generate a video from a prompt and that includes backgrounds that are relevant. And in this case, you can sort of drag and drop your logo, your brand assets, pick your voice. In this case, I'm picking my own A I generated voice. And then just in a few clicks, you're able to sort of create this video. And with this panel on the left, you can see sort of a range of virtual humans and for each so you can create your own virtual human or use a stock character from the platform. And what we see this as being is kind of the home for your virtual human because we're constantly enabling new functionalities and features to not just create and protect your virtual human, but to actually activate it in different levels of life likeness and expressiveness appropriate to the context in which you're presenting it. And so these stock characters that we see in the library, are they the avatars of actors that you've hired? Could you talk a bit about that? And then I'd also love to know how you can go to bed at night and sleep easy knowing that your virtual avatar is only going to be used in context that you feel comfortable with. Absolutely. So every single one of the virtual humans on our platform, and there's been hundreds that have been captured over the last 4.5 years that our one has existed and each of them is backed by an actual human. And each of these humans has an agreement with our one that allows first of all with permission for their likeness to be generated. And then also the agreement determines use of their characters. So for stock characters on the platform, they basically agree to being featured in content from our commercial customers and they get paid micro payments or a passive income for their appearances in the case of my virtual twin and how I can sleep easy at night, I have sole use and the team at our one, I've given them permission as well to generate content using my likeness. But through agreements, we protect and sort of restrict who has access to your virtual twin. Now, I understand Avatar nap could speak in multiple different languages. I think our one supports 80 languages. Is that correct? And if so can you explain how you pick the languages to integrate? Given that I think there's between six and 7000 languages in the world. How do you choose which ones to focus on? Well, we aim to enable all of them. So if so if there's any demand or we can imagine any kind of future demand for these languages, then then we do enable that and just a note on that. So the fact that my avatar is there right now, I can create infinite amounts of content featuring my avatar and or a stock character if that's more appropriate. But what gets me really interested and excited about the future of the technology and actually say the future, but it exists today is the ability to kind of augment my skills. So I can now have my, I can be communicating in different languages that I don't know or don't know so well and I can show you how that looks. I never thought I'd be able to speak Mandarin spoken by over a billion people. How the world has Hua Tao and the mou sh sh the mo the sh sh sh and so presumably you have quite an international client base at the moment. Yeah, so um we can uh so yeah, so basically where you can make your, we can let you talk and then come back to the drive and drop your visual asset and the text you wanted to say and boom, here we are the Benito Avatar, activate the circus visual Ae Aras is Squire Riga. I A So I have a very keen avatar activate that wanted to jump in before we talked about our customer base. Uh But just to explain what's happening here, is that uh so in, in my original avatar, the one that you just saw, I was captured, captured in a professional setting. And it's kind of, it's designed well for kind of static scenarios and static context where I'm presenting. Uh But what we're working to do now and kind of help fulfill our mission of sort of reinventing video production to enable anybody to be able to start in videos is that we want to remove that friction point of having to kind of be in a professional setting to create your avatar. So now what you can do as my avatar activate, actually just sort of explained is that you can take any data asset that already exists or you can just create shoot a little bit of fit on your phone, which is what happened here And I can take that visual data and I can basically drop in any text that I want and just suddenly bring it to life. So we're kind of lowering the barriers across the board to enable this type of video creation at large. We're kind of coming to the end of the session, but I've got a few more questions I want to get in. What's the cost of this? If I'm a client and I come to you and I want to use this technology, what's the entry point? So the entry point is really low. Um It's kind of like single digit dollars. If you want to just kind of get started, you can even make a free video if you want to um just to kind of get a taste for it. Uh And then it escalates depending on kind of volume of video, how many custom avatars you want to create uh the, you know, uh sort of enterprise packages we work with um a lot of companies uh uh sort of enterprise uh size including um J and J as Astrazeneca Berlitz, um A number of different uh companies of that scale and we do kind of custom packages. What's your biggest concern about the technology? So I think it's really important that we kind of proceed in a way where we're being very thoughtful about how to protect um and be ethically create ethically sourced avatars, which is kind of a new term that we bandy around uh the office at hour one. Uh First of all that avatars are actually based on real people that actually consent to having that avatar created and activated. Um And so that is something that we are really do doubling down on and sort of related to that is uh privacy enabled avatars whereby you can kind of opt out of being in the data set if you want to. So we're kind of building in different levels of being able to do that. And very finally, where do you anticipate this technology being in five years? And what use cases might that be that we can't envisage right now perhaps? Yeah, so maybe I'll just flip to this one quickly, right to the end. And so we can go to the final video and I can just explain. So, so with all of our avatars, as I've said, each one is actually a real person behind it. And what we're seeing is people like this, this is actually a guy called Ian Becraft. He's a real world futurist and he is, his virtual twin has actually been employed by a broadcaster, uh AAA 24 hour news broadcaster that by the way, doesn't have any studios, doesn't have any anchors on real anchors on staff. And this business is allowed to exist because of this technology I heard earlier uh in part of the intro to this conference, like, what does this mean for, you know, real world like jobs, journalists in this, in this case, how we see the future is that people will be able to augment themselves, they'll be able to scale themselves to create uh to um to augment their presence in this case. Um He Ian is a person that trades off his appearances. He can actually be in more places at once. He can speak languages that he doesn't speak ordinarily. And so uh we see this as the ability to kind of like scale, scale yourself and also to create new businesses, business units, whole new ideas with this new medium of generative video. Nay, this is fascinating. Thank you so much for being here today. Thank you.

Erik Brynjolfsson, Jerry Yang and Akiko Yamazaki Professor and Senior Fellow, Stanford Unstitute for Human-Centered AI (HAI) Shane Luke, Vice President, Product and Engineering, Head of AI and Machine Learning, Workday Atif Rafiq,  Founder and CEO, Ritual.work; Author, Decision Sprint Moderator: Jeff John Roberts, Crypto Editor, FORTUNE

Brainstorm AI 2023: Economic Impacts of AI and ML on the Workforce

Erik Brynjolfsson, Stanford Unstitute for Human-Centered AI (HAI) Shane Luke, Vice President, Workday Atif Rafiq,  Founder and CEO, Ritual.work; Author, Decision Sprint, Jeff John Roberts, FORTUNE

Anastasis Germanidis, Co-founder and CTO, Runway Kylan Gibbs, Co-founder and Chief Product Officer, Inworld AI Ely Greenfield, Chief Technology Officer, Digital Media, Adobe Moderator: Ellie Austin, FORTUNE


Brainstorm AI 2023: Entertainment at the Speed of AI

Anastasis Germanidis, Co-founder and CTO, Runway Kylan Gibbs, Co-founder and Chief Product Officer, Inworld AI Ely Greenfield, Chief Technology Officer, Digital Media, Adobe, Ellie Austin, FORTUNE


Matthew Prince, Co-founder and CEO, Cloudflare Jennifer Tejada, Chairperson and CEO, PagerDuty Moderator: Michal Lev-Ram, FORTUNE

Brainstorm AI 2023: The Infrastructure of AI

OK. So on to the next session, when your organization's role is to keep your customers safe and protected. Implementing new technologies requires a secure foundation. It's incredibly difficult to unwind data once it's been integrated. And it's therefore very important for companies to protect their IP and trade secrets while putting guard rails in place to protect against bad actors as we were just talking about. So how do leading players deploy and harness the power of A I safely joining us now for a conversation on the challenges, strategies and solutions needed to build a strong A I infrastructure, please welcome Matthew Prince, co founder and CEO of Cloud flare which helps protect its clients by serving as a reverse proxy against web traffic. And Jennifer had chairperson and CEO for the incident response platform pager duty and they'll be interviewed by fortunes editor at large Nihal Lavra. Welcome. Um OK, so lots of talk about, um you know, concerns about ethics, philosophical kind of disagreements, um uh about A I but we're going to talk about nuts and bolts, which I know sounds very, very sexy. Uh but it is very important. So I, I wanna hear from both of you. You have so much, um you know, different perspectives, both internally your organizations, but also customers because the customer ba base that you support is, is very, very broad. So when it comes to infrastructure and deployment, what is, what are the top concerns that you're hearing and that you have today? And Matthew, maybe we'll start with you. No, I think the first is just access to it. Um We are still in a place where access to GP US is limited, although that limit is somewhat artificial. Um If you look at the big cloud providers, one of the things that the decisions they made early on was to make it expensive for you to move your data from one region to another or from one cloud provider uh to another AWS for example, marks up their transfer costs of data by almost 4000 times what their underlying costs are and what that is doing in the A I space is actually further artificially constraining access to GP US because there's GP U capacity across the different clouds at any given time. But getting your training set to those places is often very difficult. I think that's the first thing that we're hearing. I think the second thing which is starting to come up more and more and especially in the last 72 hours with the starting to internalize what the EU is doing around regulation is A I today is very much the infrastructure, 95% of GP U capacity for A I is being deployed in the United States. And I think that the rest of the world is waking up and saying, we don't want to make what they see as the mistake of the first generation internet where basically all the data cap back to the US, they want to not repeat that mistake going forward. And so whether you agree with that regulation or disagree with that regulation, I think more and more you're going to see that regulators are going to say, how do we make A I local and how do we make sure that we have some peace and some control over all of this innovation that's going forward? Sure, I think a lot of our customers are working through trying to find a path between research and operationally delivering products and services and changing the way their people work, looking for ways to keep strategies simple. But having to assess the capabilities you have within the organization, understand the uh data and security parameters with within which you need to work, training uh both your people uh and making sure that they're executing within guardrails. Uh but also not losing sight of the opportunity to really change the way they work. And even uh preparing your board members for dealing with some of the new issues that arise as a result of using generative A I in your work today. So there's the technology infrastructure. And then there's also how do we, how do we learn, how do we operate, how do we iterate whether you're a large company that is super conservative or a company that sees this as a game changer and is going to move really fast. So good segue to something that you've brought up before Jen, which is kind of this, you know, FOMO fear of missing out versus fear of jumping in or getting in. I'm not gonna say the acronym because it sounds too stupid but, but, but, but talk a little bit about that um that balance and sort of the pressures for, for both sides. How are organizations looking at it? Yeah, and we, we hear different things from different customers. Some customers are taking a path of um trying to enable every employee in their organization to get some practice and start finding ways to create efficiency and productivity within their teams to really embrace a culture of automation. Uh And then we see other customers who are saying uh you can't use generative A I within your team, you can't use it within the, we, we won't allow our employees to use your platform if you have it. And it's that um fear of the risk that is keeping them from some of the benefits of generative A I. And what's interesting in that case, when you take that risk, adverse stance, your employees can use it as an excuse it gets in, it becomes a convenient excuse for not moving as fast as maybe your competitors will move. And so, you know, what we advise our customers to do is find pockets where you can start test and learn and, um, understand some of the things that are going to work for you and some of the things that will not and then apply those against different use cases across your business. Matthew, you, uh, always have a, an interesting take on things. Um A and, and you've, you've talked about um sort of the, the share, the concern that a lot of the experimentation may not work out. So what's your approach to experimenting and, and what advice do you have for people in the room? You know, I, I um I think if we're all totally honest with ourselves, most of the people in this room, if they're doing anything in A I are effectively just lighting money on fire. Um And, and that's, are you guys happy to hear that? Um Sorry, uh Some of it will work, but the real challenge of A I is that the demo is easy, but production is incredibly hard. And so you're seeing all these incredibly flashy demos that people are, you know, selling their boards on selling V CS, on raising money on. But actually turning that into something that generates real value. That's where the gap is, what's here. And what I think is really tough about this space is we are so used to as product managers, as founders, as engineers working in incredibly deterministic systems, machines. When you tell them to do something are supposed to do the same thing every time you tell them to do it. And what I don't think we've gotten used to is that these are incredibly non determinist. In fact, they're by design non deterministic systems. And as a result, you know, in the security space, I try to understand all four corners of every application that we're securing. I want to understand what every input and every branch on the tree of the output that is impossible in A I. And so I actually think that as we think about this space, we're going to have to have different product managers that think about the space in a different way. We have to think about regulation differently. And how I've started to think about it is actually how we put the guardrails on A I look a lot more about how we put the guardrails on our people within organizations rather than the machines within organizations because A I systems actually behave unpredictably in the same ways that most of your employees behave somewhat unpredictably. Isn't that one of the most exciting aspects of this though? At the same time, the opportunities that that can present again, it depends on what you want. If you want, uh you know, a system where when you turn it on, it does the same thing every time and it's very predictable and it's solving a particular problem. A I is not very good at that. If on the other hand, what you want is a creative agent, then I can do that. Now, I think that then starts to get us down the path of what is a, I really good at A I is really good at replacing people and that's a very uncomfortable thing for us as a society to get used to. But that's what it's good at in a way that traditional deterministic machines haven't been as good at. They're much more tools whereas A I is in a lot of ways replacing it, potentially a lot of the creativity that we today rely on human beings to do. And as a result, it has different guard rails that you can put in place because it can say completely crazy things. Would you like to? I, I would, I would, I mean, I, we, we hear a lot of concerns about, oh my gosh is A I is A I gonna replace all the developers in the world is A I going to um take control over the world uh in, in some surprising and, and unknowing way. I mean, I'm generally very positive about the opportunity that, that generative A I and a more broadly um creates because a lot of what we see today and you heard Aaron Levy talk about it earlier that you still need a human in the loop to apply judgment and empathy and deep layers of context that a large language model can't today produce for you. And so um our belief is that as, as generative A I advances, yes, it's going to replace a lot of the toil and a lot of the meaningless less fulfilling tasks and jobs that we don't really like doing. But at the same time, it's going to create capacity for us to do more interesting things with human effort and human capacity and human context that we might not have been able to get to for many, many years. And so there's an acceleration uh that I think is really, really exciting, it needs to be monitored. So it's, it's another one of these technologies that you've got to have uh systems and automated instrument, instrumentation, tracking and managing. And you've also got to put your businesses in a position to respond to issues as they arise in real time. And so I think we also have to change the way we operate in order to support our teams and our people and our businesses and having success with A I. Um please raise your hand if you have a question for Jen and Matthew, um which we'll get to in just a second. But on the, on the, you mentioned toil and on the kind of experimentation side and trying to move forward with some, some innovative approaches, um At least when it comes to productivity maybe internally. Um you've talked about a a toil bucket and encouraging employees not to be confused with a toilet bucket. Very, very different. What is toil? So what's the bucket list of all the crap you hate doing as an employee? And can you apply some form of automation or a I to move that off your plate? Right? Like that's one interesting place to start. We started having this conversation because we found that employees that are closest to the business problem that understand the most about the business problem often come up with the most practical use case automation or for generative A I or A I more broadly, it's not necessarily the phd that, that does that it's somebody who actually is feeling the pain of the problem. Most founders start their company by having exposure to pain and agony that they want to make go away. So it's, you know, what is your bucket list for the toil and you know, the unfulfilling work that you'd like to never have to do again and start there? And there's remarkably a lot of it still around. Are you moving forward with? I mean, are any of these generating, like actual new processes, new ways? For sure. I mean, like some of the things that even we're doing within our own um products, one of the things that we're using generative A I to do is to draft the first version of the post mortem after a major incident which today takes multiple teams across multiple systems many, many hours times days to just capture what actually happened. What did we do? What did we learn? What remains to be fixed or changed? So this won't happen again. It's a very painful manual process that requires a lot of um you know, integration, coordinations between teams and now gender V I captures everything that went on in our platform and gives you the first draft within a few minutes. Questions for Jan and Matthew raise your hand if you have one. OK. I'm gonna ask you about uh another concern that's come up multiple times, which is vendor lock in. Can you talk a little bit about Matthew uh Matthew about your approach as a company? I know you've struck partnerships with so many different companies in the space. Well, you know, I think that um I think the good news is, this is a new space and that there's a lot of innovation. I think that right now um access to GP us is, is a lot of what's driving this, especially if you're trying to do things in the training space. Uh And the cost of that are just so, so, so enormous that it's very hard to compete with that if you don't have the capital uh to do it. I think over time, um You know, the great news is GPS are just silicon and over time silicon, we just watch time and time and time again comes down, down and down in price. I think the thing that we're trying to do is across the space, make it easy for data to move to wherever the resources are that need it. What I think is the biggest thing holding A I back is somewhat counter intuitive. I mentioned it already but the fact that you have to pay to take your data out of the big cloud providers, but you don't, they don't charge you anything to put it in even though that's a 4000 times mark up on what their underlying costs are costing the same amount to take the data out is put it in. And yet you have to pay that. That's actually restricting and locking people in. If I were a regulator, I would have one very simple regulation, which is the cost of taking data out should be the same as the same as the cost of putting data in. If you have that, we have a lot more competition in the space on different companies being able to compete with their individual models with their individual systems. And we wouldn't sort of have the sort of mediocrity of every different Amazon product which is out there. We'd have competition on every model. We'd have competition on every data storage layer. We'd have competition on all that space and you'd be able to mix and match and create better products. That's the best thing that we could do for the industry. That's what we're trying to drive forward. And I'm hopeful that more cloud providers will follow in our footsteps. And from your perspective, Jen, I think it's important to take a platform approach to this and, and create optionality for yourself in the future. Because to Matthew's point, we're very early and you can already see how one version of an LLM you know, surpasses another and then the next one leap frogs and the next one after that leaves, we're gonna learn a lot in the coming days, weeks, months and years. And we also learned a lot from cloud locking. And so creating optionality for yourself and taking more of a platform approach to this, I think gives you the most flexibility to apply the right technologies to the different types of problems you're going to try and solve because they don't have equal cost cases, they have differing degrees of difficulty in usage. Your people are gonna need different things to uh solve different problems. And so I think taking that platform approach and, and giving yourself the opportunity to sort of future proof uh where your business is going by working with a number of different partners. OK. And we have a question over here. Hi, I'm Anne Michael American Institute of Physics Publishing. And I really want to thank you both Matthew something that you said though really hit me and that was the idea that Oh, you're just setting your money on fire. And it was about scalability and I totally get that. Especially if you're seriously along the path, maybe where you are, where you're developing products, we're 100 and 50 people. And I think our primary use case right now is actually learning. So we are spending small amounts of money on, not what I think is on fire because we're going to ultimately have to pick between different providers of different services. And we can't be stupid when we do that. And so I'm just curious about where you see learning and education and being a smarter consumer because we're not going to build everything ourselves. Like where does that fit in? Yeah, I think again, I think that um if that's what you acknowledge you're doing, there's incredible value in figuring out how these systems work, figuring out where the edge cases are building those, those systems. But when I talk to some, you know, agricultural company and they say we're building our own LLM model. I'm like, mm probably not the best use of time. I, I do think that there is real value in education, getting people trained up on this. And I think especially making sure that we understand that the engineers, the product managers, that the people who are going to be using these systems in the future, understand that the difference between deterministic machines and non deterministic machines and both of them have a place and both of them have a use but they have very different parameters and one is actually, I think much more close to being like a huge manager of humans. I mean, I think A I systems in their non determinism almost act like human beings in a weird way. Whereas deterministic sort of the things we're much more familiar with as engineers have a different set of skills and a different set of constraints. My guess is that the education system is going to play a big role in making sure that we understand how to work with both of these tools in order to get the most out of them. Yeah. And I would just add to that, don't limit your ability to learn and the pace of your learning to just what you can do inside your own organization, apply a growth mindset. I mean, one of the things I've loved about being in Silicon Valley and being able to collaborate with people like Matthew, we compare notes on things all the time and we learn faster together, then we will learn individually on our own. And yes, you've got to be aware of competitive pressures and challenges, but at the same time, like, do you have to make every single mistake yourself? Do you have to learn everything the hard way? Can you apply that growth mindset and try and learn faster by learning together as a community? Um I, I'm guessing most of the people in the room. The, the investors here are not investing in agriculture companies that are building their own. Llm si, I think, but any other words of, of advice real quick, you know, I think it's, again, it's, we're in such early innings of this space. It's going to be interesting to see how it develops. I do think that fundamentally, we've got to think about whether it's the policy questions, whether it's the internal questions, thinking about how these systems work. They're not the solution to every problem, but they are a solution to a different set of problems that we haven't had before that we've often had to throw people at. And so I think that's going to open a bunch of possibilities, but it's going to require very different skill sets in order to make them 100% going, going to require very different skill sets and going to require a lot of practice. So you have to keep experimenting over and over again. You've got to give people the space and time to practice with different technologies and learn what they do and figure out the best path to take. And I think it's really easy to get somebody to try something once. But are they in it every day? Are they in it every week? Are they, are they really building a skill set around it? So do that without setting money on fire. Thank you so much, Jen and Matthew. Thank you for having us.

Stuart Armstrong, Co-founder and Chief Technology Officer, Aligned AI Tulsee Doshi, Director and Head of Product, Responsible AI, Google Brian Patrick Green, Director, Technology Ethics, Markkula Center for Applied Ethics, Santa Clara University Moderator: Rana el Kaliouby, Affectiva, Brainstorm AI

Brainstorm AI 2023: A Look at Accountability and Best Practices 

Stuart Armstrong, Aligned AI Tulsee Doshi,  Google Brian, Patrick Green, Markkula Center for Applied Ethics, Santa Clara University, Rana el Kaliouby, Affectiva, Brainstorm AI

The fear surrounding A I extends beyond job replacement and productivity concerns. There are many who believe that in the rush to commercialize A I safety and ethics are not being prioritized appropriately. President Biden's executive order stated that responsible A I has the potential to help solve urgent challenges while making our world more prosperous, productive, innovative and secure. As we all know, implementing secure and responsible A I has proven a little bit more difficult than we thought here to discuss best practices for mitigating biases, ensuring safety and building responsible A I. Please welcome Stuart Armstrong, co founder and chief technology Officer at Aligned A I A start up focusing on teaching A I to align to human values, Tulsi Doshi, director and head of products for Responsible A I at Google where she has led more than 30 launches to make their products more inclusive. And Brian Patrick Green, director of the Techno of Technology Ethics at Marula Center for Applied Ethics at Santa Clara University. Brian is also the author of Space Ethics where he explored how ethics relate to space exploration. They're going to be interviewed by Fortune Brainstorm A I co chair and co founder and CEO of Afic PVA. Doctor Rana, uh L KUBI. Uh Before we dive right in, please watch this video from one of our partners checkpoints. All right. Thank you all for being here. Uh We're gonna be discussing how to make sure our A I is responsible and safe and Tulsi. I want to start with you exciting news from Google this week launching Gemini. Um Tell us a bit more about, I mean, Gemini is one of the first multi truly multimodal A I models out there. So tell us a bit more about that. And then how do we ensure that it's built in a way that mitigates data and algorithmic bias? Yeah, it's a great question. So yes, we are very excited about Gemini. And when we say multimodal, really what we're talking about is the ability to engage across modalities. So things text, image, video, audio, really being able to bring these efforts together, right? So think about being able to take an image and ask a question about that image and get an answer, right? And these combinations of things open us up to a wide variety of use cases that we're really excited about. And that has a lot of potential that we're really excited about, but introduces a bunch of new responsibility considerations. And so a lot of what we've been thinking about is we're building Gemini is how do we take everything that we have learned in responsibility and adapt it to this new structure in these new capabilities, right? So we released the A I principles, Google's A I principles five years ago now and those A I principles cover things like safety, fairness, transparency, security, and safety. And so now we're saying, well, what does that mean in the context of multimodal? And what does that mean at every stage of the development process? So how do we think about safety in our training data? And what does it mean to ensure that we have really high quality training data? How do we then translate that to the model? And what are the different use cases that might come up in the context of multi modality? So for example, you know, an image by itself might be a perfectly safe reasonable image text by itself might be perfectly safe. But now the combination of the two is actually offensive. And so how do we develop new metrics, new ways of evaluating that account for those concerns? And then also then the various kind of downs use cases and applications? And so for us, it's really been, how do we build those A I principles in how do we build the practice of the work that we've been doing every step of that way? And that's really what we've been working on and are going to continue to work on as we keep expanding Gemini Hours. It's amazing Stewart, you're on a mission to align A I algorithms with human values. Yes. What does that mean? And also are human values universal. Um Let's not get into that. The answer is universal enough for A I purposes. Um Well, it's just the outgrowth of, from the very first program. It's get the machine to do what you want it to do and not do what you don't want it to do. It sounds simple. But that's the failures of that cause all the disasters today. Nobody wants their algorithm to be biased. Nobody wants their algorithm to go crazy when it goes off distribution. Nobody wants adversarial images to break your classifier. Uh Everybody. And the thing is that alignment is a, is a capability. It's not just sort of a safety measure like 1/4 airbag uh on the roof or something. It's why are the consumer service bot so terrible. For instance, it's, you can have powerful ones, you could put a GP four behind it, but GP D four is not reliable enough and no, none of the models today are reliable that so you can't use them for that because then they'll start insulting the customers, this will blow up. So because this chatbot is not aligned with the designer's intent, you can't deploy anything but the most simplistic safe lockdown models. Brian, what concerns you as it relates to A I? There are also comment on like the intersection of space and A I ethics. Lot of time. There, there are lots and lots of things to be concerned about all the way from safety, all the way to what does this mean in terms of human self conception? Because if we externalize what we think makes us human, if we're, if we think that we're human because of our intelligence and then we externalize it from us and it becomes better than we are at thinking, then what does that mean about us? What's left with, with what human beings are? And we can see this in the way it's going to affect labor because people identify with their jobs. There are all sorts of these kind of effects. Once again running the full spectrum of anything that we can apply intelligence towards is going to be, you know, potentially an ethical issue related to that. And so if you want to talk about space, there are, there are plenty of issues related to that in terms of how is A I going to be used in the design process? How is it going to be used in, in how, you know, tracking debris around the earth or tracking satellites or coordinating large satellite constellations? There's, yeah, there's just a lot of things to think about. So you teach ethics and I'm curious, do you see young people interested in, you know, in issues related to ethics and inclusion? Yeah, this is, this has actually been really fascinating over the last four or five years, my class has gone from having just a few interested students, you know, maybe, maybe a half the class fill up, half way, all the way to having a long waiting list. So I think that people have recognized now that there's a lot happening. That is, I think that's great news for the world. So I wanna ask about um your views on Biden's executive order and the EU A I act. So to see what, what do you think about that? I mean, I'll start by saying like, I think it's amazing that we are having this conversation, right? Because we do need really good regulation. And I think that is actually a really critical part of making sure that we are building systems that are safe and consistent across the industry. And I think there's a lot to value in both the EU A I Act and in Biden's executive order in terms of how we think about high risk systems, in terms of the way. For example, the executive order is setting up an appropriate hub and spokes model that actually really takes into account all of the amazing work that organizations like Nist and other organizations have been doing for years to really build out and understand what risk means and how we develop strong systems. I think there's nuances now that we need to figure out, right? So does a dual use foundation model mean? And how does that definition apply across companies and organizations and across the industry? And so I think we're at a really good point to now be able to have those conversations, I think and really figure out what are, what does it mean when the devil gets into the details? I think I'll be taking audience questions soon. So tee up your questions. But Stewart, what do you think about um A I regulation in general and specifically the executive order? Um Well, there's an expression that you can't manage what you can't measure. And that's a big problem with this. So we want to get rid of bias. But what is bias? And that's one of the reasons that today we're launching a product for measuring uh gender bias is the first one and removing it from language models and other generative models. Can you talk a little bit more specific? Give us an example of how it would work. Um Well, I I deployed it in creating stories and it was quite interesting that female protagonists tend to have very passive roles. Uh male protagonists tend to have very active roles. So it was uh there was a princess who was beautiful, found a cuddly dragon who became her protector. There was a prince who found a fierce dragon but wasn't afraid and they became friends. Those sort of biases creep in all the time. We actually found that fiction has a lot more bias than say, even professional things you asked for top professions, for men or top professions for women. That's less biased than asking for a romance story. What does the protagonist do? But the problem is these large language models are trained on all of this data, right? And that's another problem is that you can't just tell them be less biased to some extent, you can tell to a human. You have very biased training data account for this, but you can't tell that to an A I because they don't understand, they don't generalize concepts in the same way we do. And that's another thing that we're working on to allow models to use concepts in the way humans do and present this to humans in a human, understandable way and say which of this approach is better than that? That's, that's another product we're launching today. We're launching lots of products today, Brian, what's your view on regulation? I think that there's more than enough out there to be regulated, that would be the way to look at it, right? There's so many things happening. But the real question is how to apply this regulation. So one of one of the things that I think of as an ethicist is that, you know, if everybody just made the right decision in the first place as an individual within their organization and the organization was had, you know, different ways to verify this or facilitate these good types of decisions. If the organizational structures were set up right within the organization, nothing bad would ever get out and then the government would never have to get involved with regulating it, but that's the world that we live in. So the question is, how do we set up a situation where the people who are closest to the technology and the technological development have the resources that they need in order to make the decisions that they need to make and do the best that they can knowing also that not every organization is going to be as good as some organizations and also some individuals are not going to be as good at this others. So we really need to have a structural solution. You know, in engineering ethics, there's something called the Swiss Cheese model where you line up the piece of the Swiss cheese and some of the holes line up in the problem, you know, get stopped by some layers, but it gets through to others. If you have enough layers, then the problem doesn't get through. But you really need to have the layers and if you don't set those up, then the problems will get through. I want to quickly follow up on that. So um in your classes, like do engineering and computer science students are they required to take your ethics classes. So they are required to take an ethics class. They don't necessarily take my ethics class. OK. That's good. OK. Questions. Do we have any questions? Questions here? All right. No questions yet. So let me follow up on the idea of how to align your organization to be bought into implementing ethics. How do you do that at Google? Yeah, it's a great question. I think there's a few different pieces you have to pull together if you really want to drive organizational cultural change. Right? I think the first is actually having a vision in a North Star for the organization to align on. And so at Google, the A I principles is that vision for us, right? Really setting a North Star with seven principles, we really want to make sure that our products hold themselves accountable to and then valuing that progress, right? Rewarding really great work that drives that work forward, measuring ourselves across that. We've established an A I Principles review process that actually reviews, we review all of these A I launches coming through to actually make sure they're meeting that criteria. Um And we're also building tools and resources to actually make it easier for teams to do this work, right? I think starting and really thinking about how do you measure bias? How do we provide the tools and the resources to help answer those questions? Because responsible A I isn't one size fits all and we don't want to make the mistake of saying, hey, there's a button you can press, right? We want to actually build that in to the design process every step of the way and that comes with building the tools and resources at every stage of that process and raising the right awareness, our teams for how they can actually do this work such that, you know, when they do go through a review, there's no surprises because you've actually been building it in, from the beginning. So for, for, for some of our audiences here who are like either ceos or leading A I efforts within their organizations, do you advise that there be a kind of an ethics team or how do you integrate it across the entire organization? Yeah, it's a good question and it's something we've, we always struggle with that polarity, right? Because on one hand, you want building systems to be everybody's job and it should be everybody's job. On the other hand, you also need to have individuals who really have the space and the time to dive into the hard questions and build out the right expertise and the right best practice. And so that's kind of why we've built a combination of both. So we have central teams like mine that really focus on what are the gnarly, hard questions that are coming out in responsible A I, what are the new challenges we need to understand? How do we answer questions? Like how do you measure bias? And then we work across teams where we have individuals embedded across the organization who are thinking about what does inclusion mean in the context of youtube or what does it mean in the context of search and we partner directly with those teams to kind of share insights back and forth, to make sure we're learning. What problems are they seeing in the products that we should be thinking about and driving more research, what are insights that we have that we can bring back into the product? And I think honestly, that is a very much a setup I would very much start with, right, eventually, as we build more and more state and more and more insight, I often joke that I'll work myself out of a job because we really will have built responsibility into the entire organization. But there's so many new questions that are coming up that I think there's a, there's a certain amount of work to do before we get there, Stewart. How about you? How do you work with organizations through aligned A I? Well, there is a lot of truth in what you're saying and you do want to sort of get in the habit of, of bringing ethics. But we're also, and we have our organization set up so that it requires unanimity amongst the shareholders to pivot it away from AAA I alignment organization. So that's never going to happen. Um But we want, we don't want safety and ethics and alignment to be a separate thing. We think that if your product is unsafe, you have failed, you have not built a successful A I, it's, I think Stuart Russell's expression was we don't have one group of engineers who build bridges and a second one that goes around and points out that they might fall down. So you should change that there. You just build bridges. So you want to build A I that is successful, that is align, alignment is a capability. So the, we want everyone to be doing it because that's their job. If they, if their A I is racist, they have failed. This is um so that's the there is, of course, you're going to have to have some people looking over the ethics and other parts of the team as you describe it. But the fundamental thing that we want is that people understand alignment is success. This is a capability. Yeah, I think actually, you know, building on that, we've been talking a lot about how if you really wanna build bold successful A I, you have to build responsible A I and I think the more we talk about responsibility, the same way we talk about quality, the same way we talk about accuracy, right? These are building a responsible product is a better user experience. Uh And so you actually are building a better experience for your users. And so actually baking it into the core fundamentals of how you build the product, I think is actually a really critical point. And I think it's more of a question of how do you instill that? How do you give teams the resources to do that work such that it does start becoming a part of that in the same way. And just to build on that, that's what the Marla Center tries to do with many of our materials on our website are just free materials we're putting out there to give people the tools that they need in order to make the good ethical choices that they already want to make. So we have, we published a book that June, which is called Ethics in an Age of disruptive technologies and operational road map. And it goes through five steps, it starts off and it's set up like a road trip. First, you decide you want to go. Second, you figure out where you are right now. Third, you figure out what your destination is. Fourth, you figure out what you need in order to get from here to there. And fifth, you make sure you're tracking your progress. So, and all of these things and it's all from an ethical perspective, right? To make sure that you're thinking about how to really implement these, these tools, these ways of thinking into the organizational structure to make sure that you have everything that you need. Yeah, this is great. Well, thank you so much. My takeaway from this panel is that we're at a time where implementing responsible A I and Safe A I, it's actionable, it's very doable and there's a way to implement that across your entire organization. Thank you. Thank you.

Sissie Hsiao, Vice President and General Manager, Google Assistant and Bard, Google Moderator: Jeremy Kahn, Senior Writer, FORTUNE; Co-chair, Fortune Brainstorm AI

Brainstorm AI 2023: Future of Google Bard and Generative AI

Hello. Hi, everybody. Welcome. Welcome to the 2023 Fortune Brainstorm A I conference. Great to see you all back here again in San Francisco. Um This is the third year of Fortune's brainstorm A I conference. Yes, we did get started before chat. GP T long before. Uh And many of you have been here with us from the beginning, artificial intelligence as we all know is not a new concept. In fact, it's been around since at least 1956 when Dartmouth hosted a summer research project and conference. And the concept was not all that different from ours here today to gather some of the greatest minds in the field and come to uh more or less of an agreement may be on a shared vision that computers can be made to perform intelligent tasks. I think we've established at least that much. And now with the explosion, of course, of open A is chat G BT and other technologies. Um Last November generative A I has garnered the attention of regulators as well, the general public and businesses alike with A I's newfound attention and capabilities comes a wave of both fear and excitement. On how it will be deployed in the coming years and what effect it's going to have on our society as a whole. A I as well as on its way, revolutionizing fields like health care, entertainment, manufacturing, education and a whole lot more. But with power comes great responsibility. A I has the potential to perpetuate, perpetuate biases and in the hands of bad actors spread misinformation and open the door to major security risks. Many fear it's going to replace jobs. Some have even argued that A I could even replace CEO S and journalists. Oh, no. Um With so much unknown and unregulated, we look forward to insightful conversations over these next two days as we examine new business cases for A I and explore how it's already changing the way we live work and do just about everything. We have a stellar lineup of speakers. But remember that this event was really created for your participation and is meant to be an interactive discussion among speakers and attendees. The most important conversations are those among all of you. Um As we navigate the innovations and the challenges of implementing new technologies and new ways of collaborating. Before we start some important Acknowledgments. I want to recognize the incredible work of our brainstorm A I, co-chair fortune, senior writer Jeremy Kahn Fortune, senior editor Vern Kito and co-founder and Ceo of Afecta. Uh Doctor Rana LK Calli. Thank you to our co-chair. We are also pleased to be joined by additional Fortune colleagues who are going to be moderating some of our discussions, Alan Murray, CEO of Fortune Ellie Austin, Fortune's deputy editorial director for live media and Jeff John Roberts, editor of Fortune crypto. Uh Before we get started, I also want to extend a tremendous thank you to our partners for their contributions to this event. Our presenting partner, Accenture, our partners, a ARP Amgen checkpoint hand show Sales Force Workday and our supporting partner Felicia. Before we get into our sessions, an important reminder about our conversations, you probably get sick of us saying this, but it's important. What's said on the main stage or during the strategy sessions uh is on the record unless otherwise noted. And what's said over a break or over dinner or a walk is of course private unless agreed upon otherwise by all parties. OK. Let's get started. As mentioned earlier. The release of open A is chat GP T has brought worldwide attention to the uses of generative A I chat bots and the ability to release large language models to the public. A few months later, Google released its conversational generative A I chatbot known as Bard. Since then, Google merged the Bard teams with the Google assistant team to create assistant Wood Bard, a new personal assistant chatbot powered by generative A I. Google also unveiled a generative A I search experiment that may alter how we all find information and possibly disrupt digital business models across industries. Last Thursday, Google announced the release of their newest model Gemini, which you all probably know all about. Uh And that is now going to Power Bard here to talk about the future of Gemini powered Bard. What's next for the chatbot? Please welcome General manager for Google assistant and Bard Sissy show. She's going to be interviewed by Fortune, senior writer and brainstorm A I co-chair Jeremy Kahn Sissy, thanks for joining us and thank you all for joining us. Um We're going to go to questions from the audience uh hopefully towards the end of the session. So please start thinking of your questions. Um Sissy Big News last week with the unveiling of Gemini and the announcement that Gemini was now going to be powering barred. Um There was also a lot of uh disappointment, I think over the video of the, the demonstration that came out with uh that announcement um in, in Heinz, do you regret sort of how that that video was handled? Well, maybe let me start by saying thanks for having me, Jeremy and thanks for having me for start there. Um Very excited to be here to talk about generative A I and, and bart, of course, I mean, let me start with the last part of your question. The video is completely real. All the prompts and the model responses are real. We did shorten parts for brevity which we put in the video as you know, informational on the making of the video. But we're also looking forward to releasing the API in A I studio this week so that developers can play with it. But you know, that's really addressing the video. I think the main, you know, headline story is we've entered the Gemini era. This is the best set of base models in the sort of medium size range and the large size range on a variety of industry benchmarks that all researchers benchmark their large language models off of. And so I've been really excited to take that expression of that power in those Gemini models and put them in Bard, which we saw with our side by side blind evaluations, right? So Bard is the best most preferred free chat bot now um in the market in blind evaluations with third party Raiders. Great. And what has been the most interesting use cases you've seen so far with Bard or, or things that have really surprised you that people are doing with it? I think what surprises me is the just the breath of things that people do uh with the chat bots. It's really hard to say it does just one thing or the other thing because it does so many things. So I'll give you an example um you know, for my personal use um at my home life, like my daughter just got her ears pierced. And so I was looking for a particular type of earring that she was looking for that wouldn't cause allergies. I brainstormed with Bart about that. Uh Bard gave me some brands. Bart also told me how to take care of your ears to not get an infection. That's a very like personal use case that was able to, you know, take that problem and, and solve it for me. But um at work in productivity cases, we have people using it for, we have, I have a coworker who's uh English is a second language uh for him. And so he uses Bard to rewrite his emails to be more professional. So really these a is have just such a broad, you know, use set of uses because they can role play, they can generate content, they can brainstorm with people. So I'm really excited to see just like the breath of uh things that people are trying and also, you know, what really resonates with users in the world. And when we look ahead to Gemini, what really excites you about what you're going to be able to do with Bard now powered by Gemini that maybe you couldn't do six months ago. Yeah. Well, I mean, first of all, Gemini is just a stronger base model which means like everything that comes out of it is more high quality. So we see it in text generation, we see it in code generation, we see it in instruction following. So uh typically complex prompts have many, many instructions embedded and the model is able to follow those instructions. Um But I think the most like sort of visually stunning is of course multimodality, right? This is where the model can ingest, you know, not just text but image and audio and video and also communicate back out to the user um in not only text but also, you know, in other sort of means of communication like video and image and, and audio. Um a very simple example. Um I just used bar to uh I went to a dinner, I had a menu that was like, choose a course from the 1st, 2nd, 3rd course. And then I had a wine list and I took a picture of the two menus side by side and I said, choose me a dinner for tonight, one course from each section with a wine pairing to go with it. And it actually did fabulously. And so like even something like that as simple as just taking a picture of something and asking a question or for the bot to do something for you um will just become second nature as like these multimodal models uh become, you know, as easy to access as your phone. Yeah, interesting. Um And a lot of people are looking forward today when, when these models, um you know, you can prompt them to go out and do something for you to take an action on the internet on your behalf. Uh How far away is that? Kind of agency do you think and will Bard be for Google the thing that, that carries out that your mission in the world for you? Yeah, I mean uh getting things done for people is definitely within reach of these uh agents or we call them agents because they have this agent of capability. So already Bard can summarize your emails, right? We've connected Bard to gmail, youtube. I can summarize youtube videos for you. You can ask questions. So already like this action ability is starting to emerge when you connect different tools that exist in the world into the large language model and foundation. The reason why it's able to do this is you're teaching the model to speak another language, which is essentially code, right? When it can seek API S, it can actually translate the user requests and convert it into an a a set of API calls with reasoning on top. So I'll give you an example. Um One thing I want Bard to be able to do for me personally is uh book my entire kids summer camps. Yeah, all the, all the parents are happy. OK? I have two kids, they have different interests. I need to find camps every week of the summer that are within driving distance and actually I can pick up and drop them off, right? And so this is really hard, right? So you have to think about all the reasoning that goes into that. And so I've gotten barred to the point where it's giving me three options per child per week. Um But wouldn't it be awesome if I could just say, please now book this. We're almost there. How, how far away do we think? Mm. Things move really fast in this industry. I can't say exactly when, but I think using tools and not only using tools but the logical reasoning on top is very powerful and we're definitely working on it. One of the things that people have been concerned about with all of these LM powered chat interface has been hallucination. And I remember being here last year, maybe some of you were also here last year. And there were definitely people saying, don't worry, hallucination is going to be solved within nine months. It's going to come back next year, it'll be solved, it's not solved really. But how are we doing on making progress on hallucination? And do you think that this is a solvable problem? I uh look, so hallucinations are inherent to all large language models. Now, we even know this word in the lingo of this industry, we all are aware of this. And before we didn't, you know, use this word very much, um It's two pronged approach. One is, can you actually just improve the issue, right? Can you make the bot hallucinate? Yes, uh uh uh hallucinate less the answer is absolutely yes. And that takes hard work, fine tuning evaluation. And we've seen the hallucination rate um decrease in bard over the last eight months. We've been working on it really hard and definitely improving. And I would also say the smarter the base model, the more it tends to understand how to use search as a tool to really reduce its hallucination. So that's like sort of one step. The other step is we're, we're first to market to launch something we call double check. Um in the industry, it's called an entailment model. But really what it is is checking all the sentences that come out of the bot with search. And if you think about it, if you're writing an essay or doing something important as a human, you're going to be doing a lot of searching to corroborate your facts, right? You're going to be reading all of the different maybe dissenting views and really, you know, deciding for yourself, what is the right way to write any particular piece of content? So we put double check into Bar to help people really quickly get access to like search corroboration on top of the model's output. And that's yet another way that if you're, you know, doing something like really, really important that it must be factual you can use double check is another tool to help you um really understand what Bard is saying? Interesting. And do you think that uh Bard and search, which right now are very distinct products? Will they remain distinct or are we going to see increasing kind of merger between the two. You know, I think of um you know, search and ste which is the application of large language models inside search and Bard is really complementary. Um A I is a, a raw technology that has application in lots of products, not just search and a stand alone chatbots, but I'm sure many of you in the room are thinking about how can this transform like my business operations and the way we do work. And, and so I think that um the way I think about it very simply is you go to search because you mostly are wanting information, right? And I think she does a fantastic job sort of summarizing what uh the web is really saying about your question. Whereas Bard is like you are directly tunneled into the A I, right? And it has this full amplitude of expression, you can have it write poems, you can have it to critique your code. You can, you know brainstorm ideas for a birthday party with the bot, you can have a conversation just for fun. And so it's just kind of a very different expression um of what underlying is, is basically a similar uh base model technology, right? I wanna try to get questions. Uh I want to have a question from the audience is uh if we can get a mike to the gentleman down front uh here at the first table. Um please. And if you could please state your name and where you're from? Hi, uh I'm Edward Hall. Uh I teach A I at NYU and I run in TABS and I'm uh behind the big parts of data commons project. Um So, uh question is uh how does this transform Google's business model? Opening eyes, business model is charge subscriptions to consumers. And so, um while of course, the internet is still advertising, um how does consumer get clean information out of this? And what's the transformation of the model business model? You know, um really, we're really focused and barred on making the A I very, very helpful for people. You know, monetization is something that, you know, we're, you know, we're always looking at, but it's not really our focus right now. It's really about like, how do you make this technology really great and help people with their everyday tasks and other questions. Um Who else has a question? Raise your hand. Oh, the gentleman here at this, there we go, wait till the mic gets in. There you go. Yeah, hi. This is Sar I go on with fresh works. And um you know, one of my question is around security for the models because people are doing prompt hijacking uh and being able to understand what's coming out. So is there going to be the need for firewalls for these models? Well, I mean, security has like different facets, right? One is, for example, if you have um specific data that you're grounding the model against, you don't wanna leak that data, right? So you need to make sure that any um uh data that's, you know, used to flavor your model in any particular way is protected. There's also and uh there's also ways, for example, when you use tools to make sure that you know, the the chat bot is operating tools in a sandbox, right? So there's different like layers of security that we add to uh avoid prompt hijacking, avoid leaking grounding data or training data or um make sure that like its execution is safe. So at in Bard, we do a lot of red teaming, we have adversarial testing to make sure that like we're constantly, you know, looking at the model and making sure that it has different layers of protection around all these potential exploits that um might occur. Next question just here, stand up on a table cloth. I am Michael, the American Institute of Physics Publishing. And um so I, I wanna go back to what you were saying about the distinction between search and the use of an A I because fundamentally we searched because we had to break down what we wanted to do into these obscure pieces to find something. But if we could have just asked, we never would have searched, we would have just gone right to, you know, it's almost, it's kind of a, it's a choppy way and I guess I'm thinking back about the gentleman's question about business model. Can you see a time when we don't search, we ask? And then how do you get ad revenue from that? Um, like I said earlier, I think, you know, there's the two parts of the question one is as the A I gets smarter. Can it take tasks off your plate? I think the our endeavor is to make that. Yes. Right. Because we think that a is have such incredible aptitude like solve my summer camp problem, right. That's a very complicated problem, right? With a lot of steps to that. But um in terms of, you know, is it ads is other forms of monetization, it's just very early and it's not something that like I could say, you think we're gonna see, are we gonna see with Bard a kind of store? I mean, uh open eyes, talked about trying to allow people to create their own models and maybe have some kind of a, a kind of app store for a model store. Are, are you thinking about that for Bard? You know, um I think it's great that there's so much innovation, right? Like customizing uh uh an LLM into a particular type of behavior is something that I think some people wanna do, especially power users. So I'm interested in how people wanna use that, you know, right now we're really focused on making Bard really, really helpful. Um And why I have a distinction is because Bard can play act many different flavors of behavior, right? Because you can just prompt it into being an interior designer or a statistician and you can actually prompt your way to many different uh flavors of it if you will. So I think um you know, whether you need to really split that bot into many little sub bots or whether the main bot is really helpful at many things like these are just different models of expressing the, the core technology. And I'm very interested in how this, you know, will, will work for people. Well, sissy, we're out of time. Thank you so much for joining us. Uh And uh thank you all for listening. All right. Thank you so much.

Jon Chorley, Chief Sustainability Officer and Senior Vice President, Supply Chain Management, Oracle Rudina Seseri, Founder and Managing Partner, Glasswing Ventures Rebecca Yeung, Corporate Vice President, Operations Science and Advanced Technology, FedEx Moderator: Verne Kopytoff, FORTUNE

Brainstorm AI 2023: Supplying Solutions for Industry

on Chorley, Chief Sustainability Officer and Senior Vice President, Supply Chain Management, Oracle Rudina Seseri, Founder and Managing Partner, Glasswing Ventures Rebecca Yeung, Corporate Vice President, Operations Science and Advanced Technology, FedEx Moderator: Verne Kopytoff, FORTUNE

Jon Chorley, Supply Chain Management, Oracle Rudina Seseri, Glasswing Ventures, Rebecca Yeung, Operations Science and Advanced Technology, FedEx, Verne Kopytoff, FORTUNE

Welcome back to the second day of fourteen's brainstorm A I conference. We hope you enjoy dinner. We hope you enjoy breakfast. And now let's get down to business today. So before we get started, I want to extend a really big thanks to our partners for their huge contribution to this event, our founding partner, Accenture and then our partners who are a ARP Amgen chat point hand show sales force and work day. And then we have our supporting partner, Felicia. Thank you so much to all of our partners. So a quick reminder before we kick off about our reporting and social media rules, anything that said here on stage or in the strategy session throughout the day is on the record and that includes the Q and A s but what's said over the table during a meal or on a break in a corridor that is private unless agreed otherwise by all of the parties involved. OK. So with that out of the way, let's get started with today's first session since its inception A I has been a helpful and powerful tool in making supply chain predictions with the help of A I organizations are now focused on optimizing the supply chain and ensuring it can be tailored to the needs of clients and customers. With the holiday season, fastly approaching supply chains, technologies and cost management abilities are as ever being put to the test. So here to discuss how A I is tackling the challenges of the supply chain. Please welcome John Chorley, who is the chief sustainability officer and senior Vice president of supply chain Management at Oracle, Regina Sri, who is the founder and managing partner at Glass Wing Ventures. Rebecca Young Corporate Vice President for Operation Science and Advanced Technology at fedex where she also oversees the company's innovations in robotics and they will be interviewed by my colleague, Fortune, senior editor, Vern K. Yep. Have a seat. Hello, everybody. I hope you had a fantastic dinner last night and thank you panelists for joining us. Let's first off, let's set the stage for uh the discussion and, and just give us a maybe start with you, John, uh an example of your favorite A I for the supply chain. Well, that's that's a tough question long continuum of A I being applied to supply chain, obviously the latest. So as it were is Ja I and you know, there's that, but the history of A I and supply chain is really looked at a lot of optimization problems and trying to help solve those. One of the things we've worked out on most recently is predicting your estimated time of arrival, more accurately using A I technology and in including in that other causal factors, weather, seasonality and so on. So that one, I think it is a great example of how we can continue to evolve the use of A I in supply chain. Well, I should go to Regina. I think I know what you're going to say. We'll get to that. Thank you and good morning everyone. So selfishly helping from one of my portfolio companies because we're very active investors, early stage investors in the application of A I for supply chain. And one of those companies being Verison based out of Atlanta, where they're focused on inventory optimization and particularly in direct parts. So not the parts that actually get embedded in the production of a finished good, but the parts that go into the machinery um there is an average of 60 to $80 billion waste pure, poor tracking um over stocking under stocking emergency ordering. And what this team has done is Jena is the wrapper, not the differentiator. They're actually leveraging recurrent neural nets with deep learning and some other architectures and technologies to fundamentally both harmonize the data. So you know exactly what you have in stock, harmonize it even if the data is inputted in perfectly, even if it's off off prem rather than just in the cloud. And then also give you visibility into optimization for when you order because they see the entire supply chain so super excited about that one. And Rebecca. Yeah, it's really hard question for me to answer because uh we our operations cover both the physical and digital side. And so putting the perspective of fedex operation, we just celebrated our 50th anniversary. Today, we move on average 14.5 million shipments on daily basis to 220 countries and territories. So as John said, for supply chain, it takes a lot on the digital side, actually optimize the movement of goods. And also we have think about we have 700 planes, we have 210,000 vehicles and 5000 locations. And so in order to um move the shipments in the most optimized manner, we use a lot of the A I tools, whether it's the JA I or like predictive analytics, etcetera. But what I most excited, I think we that's my answer is on the physical side. I mean we talk about Ja I always like joke with my colleagues like this is the language part of the A I. There's also a math part of A I, right on the robotics and auto technology side. And um we really see the convergence of underlining technologies from a sensor perspective. The the sensor gets a lot better. So robots actually see better today and also the computation gets faster. With GP U, the robot can think faster. And then the ultimate A I algorithms enable robots today to have the touch have the force control, have ability to actually move in efficiently in tight space. So in the past, robots are very much doing precision work and the secret is the imprecision, like more dynamic like movement is more difficult. So for our operations, you think about 14.5 million shipments on daily basis, it all vary in size, shape, weight, packaging material. And it's also randomized because they come down from sort and also have very high like high throughput needed because you all want your Christmas gifts to be on time. You don't want us to hold inventory. And so if you combine the randomized shipment, um real time decision making and also high throughput, we require very high advanced A I. So that's what I'm most excited about. And let's be specific. You have a robot that packs trucks with boxes, random sizes, random weights and what and what are the limitations of a robot like that? It's just an experiment, experimentation phase. They're not quite as good as humans yet, right? Yeah, we're still developing that because like, uh if we look at the history of robots like uh um like in the past, it's more like a very precise decision work and now we move towards making real time decisions. So the it's a lot of decisions to make because in the inside the trailer, you can have hundreds to thousands of packages and for robots to not only actually sense how heavy is the package. But also um you have to be cognitively playing the Tetris game where to place the package. So that uh a few factors you have to think about is put the heavier boxes at the bottom because you don't want the heavy boxes to be on the top, which to make the world less steady and also easy to cause damage because uh in our operations reliability, highest customer service are all important factors. So the robot basically is doing both the heavy lifting of moving the boxes as well as playing the 3D te at the same time. So that's the complication. And John um we had the uh the, you know, the COVID pandemic which upended the supply chain. What did we learn from that? And are we prepared for the next major disruption? Are we better prepared this time around? Yeah, I I think we learned that the tremendous amount we learned that you can become too lean, you can become too just in time. And that essentially injects risk risk into your supply chain. We learned a lot of breakdowns in communication between different parts of the supply chain. That information flow wasn't as smooth as it could and should be. Uh we learned that flexibility is important. And I think now that we've sort of come through the acute phase, the fever phase. If you like of the of the pandemic, we're now into the hangover phase. Maybe we've oversupplied inventory in certain cases and it's in the wrong place and we're trying to move that through the supply chain. But I think the longer term lesson is going to be around resiliency, how to build resiliency into your supply chain. And that I think means looking at flexible sources of supply, looking at different ways of hedging your inventory, multi enterprise inventory optimization techniques and those kind of things where mathematics can really and A I can really help, help give you an optimal solution. Um And I think we're also learning about planning that, that you know, a single answer plan isn't, isn't correct. And that you need to look at um a scenario planning where different factors possibly those more likely to change you per those and understand how that might affect your plan and get a range of outcomes with probability uh helping direct uh how you make your decisions. So these things are all evolving. Obviously A I has an enormous part to play in those processes. Computational power lets you do these things now. And I think as we emerge from this, that core aspect of resiliency is going to be is going to be central and Regina, you had a little midi announcement to make that not only deals with supply chain but all A I just go ahead. Sure. So basically this morning, we announced what we call the A IA I pallet. So my firm were an A I Originalist Fund and started investing in the application of A I in 2010. And through the years have developed proprietary model where one can start with the use case and try to figure out what are the best techniques, architectures ways to train the algorithms or you can start with, you know, I have this algorithm at hand, what kind of data do we need? So we provide a framework which has been proprietary to date and we have open sourced it as of this morning for whether it's a software founder that is trying to recruit, you know, a co founder, that's an ML expert. And he is trying to figure out what technology or what capabilities should they have or whether you're an executive and you're getting the three letter acronyms from all your techies and are trying to make head or tails and everyone in between. And we think this is important because even in our chat right now, two things have happened. We use A I as if it's a, you know, monolithic term. It is not, it's like saying technology behind it, there are literally hundreds of architectures and within those architectures, hundreds of techniques. And then we combine them in ensemble models and it's basically thousands of possibilities. So how do you and us as executives and investors know that we are making the right decisions in what we are building. Bless you. Furthermore, far more though even the sort of beyond that point, the second point is gen A I and A I are not the same thing. May I repeat that? Gen A I and A I are not the same thing. Gen A I is incredibly transformative. We have crossed the chasm to mass consumerization. Therefore, we all latch on that is what drives the, the optimization is we to mention it or we won't get any traction. But it's really because grandma now can hug the computer because it's cute in the responses. And my 10 year old why just get some help for a book report? And I'll tell her mom. So when it touches all sides of the spectrum, you know, that is mass consumer that is based on a very fundamental underlying architecture called transformers. Happy to talk to you about that that knows them. And I repeat it is very important because we will never interact with software in the same manner. Having said that there are many, many, many other architectures and you know, machine learning techniques that will drive a lot more value in the enterprise, whether it's on the central loan or on a combined basis. So I just want to make sure that in all these conversations, we don't miss the forest for the trees. OK, John um to sort of put on your sustainability hat does A I or generative A I also have a role in terms of improving sustainability in the supply chain. Absolutely. I mean the enemy of sustainability is waste. And clearly, if you can run an efficient supply chain where you're just making what you need, you're not wasting either fuel because you're having to expedite things because you made a mistake on your planning or you're putting moving material into the wrong place at the wrong time. If you can eliminate all of those wasteful activities, you know, better routing. For example, where A I has an enormous part to play. If you can use those techniques, you can significantly drive down the waste in your supply chain, the fuel, the carbon intensity in your supply chain. And we have many customers doing exactly that. So you know, you can obviously move that upstream and start thinking about network design, which can help you improve the structure of your network to be more suited to how your business is moving and changing. Beyond that, you can look at transform transformations in your business where you're moving from being a product company into a service company. How can A I give you insight into how your products are being used? Therefore, you can fold that back into the way you operate your business and design and, and support your products. So, you know, at the end of the day, we are going to solve this existential sustainability, environmental crisis through the clever use of technology. And A I is clearly a key component of that Rebecca at fedex, not only are you doing robots you're doing? Um autonomous trucks. Uh could you tell us a bit about that? Yeah, I think this is another form. I totally agree. Like for A I, it comes in different format and the key is to use the right A I technology to solve the strategic problem. So it's more like lead with the business values and they match the best technology. So in the A B space, we are actively testing with Aurora here in the Valley. Um um we actually have been testing since 2021. So over the last five years, it's actually correction. Last two years, we have been working together with Aurora and the pack up because we believe that for A V technology, the OEM redundancy is really important to lead with safety testing, the lanes and really focusing on the like safety and um the reliability of the technology. So, so far, we have been had a sound load and 250,000 miles. And uh uh literally like we move about half a million pounds autonomously every month. So it very much help us balance the business needs, which is imbalance the length as well as like length. We have driver shortage and Regina, what's the future of autonomy in the factory and on apparently on the roads as well? Big question. Um So I think I would articulate it as a two part, perhaps, perhaps process one, this notion of the co ploting dynamic that we will have with A I, whether we have it with the agents in our everyday lives as private citizens. But also in our professional setting, you can think of it as you know, it's ambient A I, it's everywhere and we have so much data that therefore our performance is, you know, intricately tied with what we do around A I and we may or may not need neuralink, we'll figure that out, but fundamentally, it will drive productivity and then there will be certain areas where we can have complete autonomy or at least one of those five stages of autonomy. But there will be other areas where I think the human given our creativity and inherent ability to reason will dominate but will be enhanced by our co ploting and ambient relationship with A I and uh to, to wrap things up, what, um, how do you choose which projects to pursue they're out of so so many, you mean as, as, as me as a developer or considering there's dozens of options you could, how do you choose? Yeah, I mean, I think I'm a believer in crawl walk run. I suppose that, you know, clearly solving supply chain problems is a journey. It's requires continuous focus. So, you know, begin, begin with a project that is tractable and which addresses a specific need and then gradually grow based on your experience with that. I think that, you know, obviously A I has changed enormously, uh with the advent of chat GP T um it's sort of, as you said, it's the sort of the, the, the liberal arts side, liberal arts side of A I, as opposed to the more mathematical algorithmic side that we've been used to up until now, actually, it is a mathematical just manifests as a Exactly. So, so look, I think, you know, you need to look at your own business, understand where some of the challenges are there, Look to see if there's A I techniques that can be applied to that that are approachable. Our, our approach within oracle is to deliver solutions, not, not just technology, we want them to be easily adopted. There's obviously a learning curve associated with getting data scientists on board, the extent to which you can find problems where you can minimize that you can move forward quicker. Uh So you know, get some experience, get some results and then iterate and move forward. Well, thank you to the panelists. You've had a fascinating discussion and uh well, the next panel coming up next. Thank you.

Paul Daugherty, Chief Technology and Innovation Officer, Accenture Moderator: Alan Murray, Chief Executive Officer, FORTUNE

Brainstorm AI 2023: Using AI to Reimagine the World of Work

So an ongoing conversation that we're having at this year's event is of course, what A I means for the workforce and how we can train employees to work alongside artificial intelligence in a way that also improves their own experience of the workplace. Um We're going to be discussing this in a second with our next speaker, Paul Doherty, who is the Chief Technology and Innovation Officer at Accenture, and he'll be interviewed by Fortune Ceo Alan Murray. But before we hear from those two, please watch this video from our founding partner Accenture Reinvention. It's how retailers can use the power of A I for the benefit of all personalizing consumer experiences without bias and giving employees new capabilities to supercharge what they do, where could reinvention, take your business accenture? Let there be change. Paul, thank you for being here and thanks for supporting this event. I think it's, it's been uh uh I I everybody here has been really engaged in the conversation for a great event, a great buzz, a lot of great uh dialogue going on there. So thanks for being part of it. Look, uh uh we've heard a lot of examples over the last uh uh the last two days of companies that are using A I and generative A I to make uh extraordinary productivity gains. But they're anecdotal. You've deal with accenture, deals with more customers than anybody on this stuff. You have thousands of customers that you're working with. Uh where are you seeing the biggest gains? Is this really revolutionary or is it, is it more hype than uh reality? Now, the, we believe it's clearly more reality than hype. But of course, there's always a lot of hype with any new technology. And I think the exciting thing and the thing that creates this hype perception around, around generative A I is, is the fact that it's so human like it's like it's the first technology where it's like looking in the mirror, you know, no other technology has really been like that. And I think the and technology really is going to continue to become more human like as you look at what becomes beyond generative A I as well. And I think that creates a lot of misperceptions around individuals and work and how work happens. And one thing I've been saying is we might be well served by taking the word artificial out of A I and just focusing on intelligence because at the end of the day, that's what we're, what we're really trying to create is a greater intelligence by combining the human and the, and the machine capability. And we as you said we work, we see a lot across a lot of industries around the world. We've had thousands of conversations with, with clients on generative A I. You can't have a conversation that's not about generative A I anymore. And we've done hundreds now of projects with clients on Generative A I. And there's 33 things that, that we believe based on that work we're doing. The first is that, that this will lead to the reinvention of companies. And you saw a little bit about that in the video, but it's things like in a telco where we're working on a customer care example, our customer care implementation, generative A I in the call center. In this case, 30% increase in productivity and 65% better experience as measured by the customer in terms of what they get back in drug discovery, looking at dramatically impacting timelines. I think that there's been some other sessions here talking about that in places like energy, looking at how you can take tremendous cost out of the industry overruns on capital projects by by bringing in new forms of generative A I that can do things you couldn't do previously so that we see this reinvention of companies, we believe that will redefine leaders. So as we look at the Fortune 500 as one benchmark going forward, more change going forward that even the tremendous change looking to the past because we're seeing a big gap, you know between how companies are pursuing it that's going to lead to disproportionate performance. And then the third takeaway is uh is the fact we really believe this is about human potential and human potential, both on the productivity side and the creative side. And those are the, the three, you know, key things. Uh you know that we're seeing in the, in the early days when you talk about, you, you mentioned the telco example with 30% improvement in productivity. When you talk about productivity numbers like that, inevitably, that raises the question of is that, does that just mean that you're gonna have massive layoffs in the uh customer service area or in the other areas where this is being implemented, what you've seen in terms? Or is there reason to be worried about lack of jobs? I think the biggest worry is, is uh is, is the jobs for the people who don't, who won't be using generative A I, you know, the, I think that's the big transition that we're in and most, you know, for our company, the way we look at it is is that we don't, we don't really need less people, we need more people who can work with generative A I in different ways. This impacts our company as, as much as any company. When you look at software engineering and coding, the things that we're doing and that's the same discussion we have even with this particular company, I was talking about, this wasn't about workforce reduction, it was better outcomes and more growth and more productivity uh and impact that they could get to their workforce. Now, there will be some consolidation. Productivity means you need less people to do the same thing. The question is people want to do more things as well. And uh that's where we'll continue to see a lot of focus and the big, you know, the things I'd say you need to focus on are, are three things you need to think about work and how work changes which and I again, think about intelligence rather than artificial versus human intelligence and how it changes that you need to think about that really dramatically reimagining the work in the way it happens rather than just applying technology in this into what how work happens today. The second is the workforce, which is what you're getting at. Where do I need more people? Where do I need less people? What skills do I need? We have something called our R 12, our required 12, which are the 12 things we need at scale to continue to drive gender of A I in our business. I suggest every company developed the same, you know, in terms of what does that look like R 12, I mean, these are the skills you need specific skills in this, in a variety of different skills across the 12. 1 of it is, is around the understanding the change skills that you need to, to drive, you know, processes across human machine. And other is prompt engineering, you know, which is probably a basic one. But there's, you know, this set of, but, but talk a little more about that. This phrase, prompt Engineering is kind of an interesting one. I mean, three or four years ago, what everybody was saying was, oh, you know, you've got to go to coding school, you got to have technical skills. If you don't have technical skills, you're not going to be employed. Now, we've had so many conversations here over the last two days about human in the loop. You talk about prompt engineering. People who understand how to use the tools smartly, who have the judgment to be able to uh uh manage the guard rails when the, when the technology goes astray, how do you train people for that? The uh you need to do it on multiple levels. So the training, there's a, there's a level of cultural training that you need to do that we believe touches everyone. We have something called TQ technology quotient, which is that for all of our 700,000 people at accenture and A I has been a part of that. So all of our people have been trained in A I now they're all, you know, learning about generative A I, no matter what their role is in the company because everybody needs to understand it to some extent, because they need to think about and understand how it's impacting our jobs and help shape how it's going to impact their jobs. Then there, then there's breaking it down into uh the specific skills you need in different professions in, you know, as you mentioned in coders. So how is the job of coder changing again? We believe we need more, not less going forward but the what, you know, but you need them to be doing different things. We talk about shifting left and you know, doing different things in the software development life cycle. And some of when you look at legal professionals or paralegals versus lawyers, you look at you in the call center, you look at plant engineers in refineries. It's about looking at that work change and then modeling what are the new skills that people that, that are required? So that's what we're doing in detail for every job category across every industry. And you know, kind of looking at what that change in profile looks like and how do you prepare people? So what's your advice? I want to open it up by the way to questions. So if you have any actually raise your hand now so we can get the microphone uh to you. Uh There's a question over here but what's your while we're, while we're setting up for that? What's your advice? This is a big journey. Uh You're talking about reinvention. Uh companies are saying, OK, how do I get ready for this? How do I get my workforce ready for this? What is your advice to companies that are uh beginning this journey? The first piece of advice I'd start with is uh invest more in the people than the technology. I think those that, that really focus on the people are going to be the ones that are successful. And uh we, we have the underpinning of what I just was talking about with these roles across industries is that we've done a comprehensive research that shows that 44% of working hours, 44% of working hours across industries will be impacted by generative A I. Uh it's up to, it's close to 80% in banking. It's, it's less in some industries and you need to prepare for that future. And there's five steps, you know, we think you need to take to prepare for that future. They're not all around workforce, but this is the key to driving success. And the first is having a value led approach that sounds like a consultancy thing to say and sounds obvious but not not many companies doing that by value led. We mean looking at the at across your whole value chain at everywhere where you need to apply generative A I and building that in a systematic manner of the business case in mind rather than just doing experimentation on use case and shifting from this use case mindset to value chain is really a big change. We need to see in 2024 as companies shift out of the use case mode of 2023. The second thing is build a gen a ready digital core. We believe geno of A I requires a fundamentally different and new enterprise architecture. There's new intelligence layer, there's a new gen a backbone that you need, that can do things like isolate you from changes and isolate you from training differences across models. So you can get the results you need in the business and you need to invest in that. The third is this change around talent and workforce and developing your change competency and the learning capabilities around that. The fourth is responsible A I done intentionally because uh there's a big a big gap there as well. 95% of executives say they, they believe in responsible A I and they're going to manage the risks. 95% 6% have responsible A I programs in place in the company. It's a big gap between, between intent and execution. And then the fifth point is it's a continuous uh reinvention. I think there's a lot of people that think this is a find my big use case and go and it's a big shot success. We believe this is a multiyear reinvention. You need to again, look across the value chain set that architecture carefully prepare your workforce and it's a multiyear change program. Wow, that's good advice. Thank you for those five question right over here. Uh Good morning. My name is Raquel Rodriguez. I'm associate general counsel at the A S Corporation, a renewable uh energy company. Something that we're struggling with is um the cultural integration of A I at our company. Uh People are afraid people are resistant. They wanna keep uh doing their work the same way. They've always done it. So what have you seen work in terms of motivating and encouraging people to adopt? A I, the uh I think there's a gap a lot, but there's a gap also, I think between in companies of understanding, I think the main thing I'd encourage is back to what I said earlier around the education and cultural, you know, sharing and learning across the organization. There's some research we're going to release shortly. That's where we asked executives about the disruption they expect to their workforce. And 80% said there's not going to be a lot of disruption. There's not going to be a lot of reduction in their workforce. There's gonna be a lot of change in learning and education, but the workers don't believe it, you know, they're, they're afraid. So 50% of workers are really afraid for their jobs and what this is going to mean. So there's a disconnect between the executive mindset and intent and the employees. And I think that's where, you know, the training and understanding and learning and sharing the mindset of where you're going is really important to bring people along because there, there is no A I ready workforce you can hire for a year from now or two years from now or three years from now. If you need a gen A I enabled plant manager for your refinery, you're not going to be able to hire that. You need to bring your workforce with you and develop it, which is why I can't remember who said it yesterday. Someone on the stage yesterday talked about, you know, have celebrate successes, get a team, have a success, then show them to the rest of the organization. So people can be less afraid more. Just if I could add one other point, I think the other thing is, is thinking is educating the educating people where the gains from an individual perspective are gonna be there's productivity and doing work more effectively, which is, which is good, there's creativity, people producing different results. We see this in our, when we're starting to apply in our sales profession, they, they're getting different results and you know, in coming up with different shapes of things than they would have. Otherwise there's outcomes that they produce people boost, they can get more output, do more generative design in product design as an example that and then the final one, which is a surprising one is people like their work better when they're using generative A I, we're finding this in a number of scenarios where you're taking some of the drudgery and tension out of their work. And one of the things we're finding in a lot of use cases, people just like what they do. It's a great point. We have two questions over here. Why don't we take both questions? And then let Paul respond to both. OK. Hey, Paul VJ from U I on the Telco example that you spoke about the productivity lift. Did you advise the Telco on which um conversational A I product to use? Did the client of the Telco know which uh product to use? How, how did, how does the decision get formed? And you are the trusted advisor, how does that decision get formed? How much is the client know about Ja I and which product to get? And how do you shape that decision? A really important question? And part of that those five things I talked about what you know is, is, is in that digital core is, is needing to get into those decisions. I believe the the foundation models you select in the architecture you build is strategic and will you know dictate whether you're going to be successful? So in that case, in particular, some of their existing products, they had led to a, you know, pretty clear decision on which products were the right ones to build around in that case. But it's not always so obvious and you have to decide, do you do you use some, you know, from your company, the products that you have as an example, do you use something from uh your enterprise software companies SAP or uh or uh oracle or salesforce who are bringing out their own capabilities or do you use a model, uh you know, and build your own, you know, pre train or customize or prompt, engineer it. And in that case, is it going to be open source or proprietary? A lot of decisions to be made? And we believe these are, you know, senior level strategic decisions that uh that people need to be educated on because the wrong choices today will limit you going forward. And we're seeing is similar to the early days of cloud is proliferation where companies are going down a path that's gonna lead to proliferation of different models for different purposes connected to different data, producing inconsistent results all over the organization leading to the need for both uh dissatisfaction and a disenchantment with, with uh generative A I but also a lot of clean up work to be done. And you can head that off by taking the right approach. Paul, a lot of really great advice crammed in a very short period of time. Thank you so much for being with us and, and sharing those insights. Thanks Alan, thanks to all of you.

Sarah Hinkfuss, Partner, Bain Capital Ventures Chintan Mehta, Chief Information Officer and Head of Digital Technology and Innovation, Wells Fargo Prem Natarajan, Executive Vice President, Chief Scientist, and Head of Enterprise AI, Capital One Lúcia Soares, Chief Information Officer and Head of Technology Transformation, Carlyle Moderator: Jeff John Roberts, FORTUNE


Brainstorm AI 2023: AI’s Impact on Fintech

Sarah Hinkfuss, Bain Capital Ventures, Chintan Mehta,  Wells Fargo Prem Natarajan, Capital One, Lúcia Soares, Carlyle, Jeff John Roberts, FORTUNE


Martin Adams, Co-founder, Metaphysic Moderator: Jeremy Kahn, FORTUNE

Brainstorm AI 2023: Cruising to a Fake Out

Welcome back, everybody. We've got some great sessions ahead. So let's get to it. You probably saw the Deep Tom Cruise videos online, the ones where he's hanging out with Paris Hilton backstage or dancing with Diplo on New Year's Eve looking ever younger than before. Well, guess what? They aren't actually him but an A I generate video from metaphysic A I A start up utilizing A I to create hyper realistic visual effects here to show us how A I can help us build immersive content. Please welcome co founder of Metaphysic A I Martin Adams. Good to see you. Hi, how are you doing before we get started? Can you tell us a little bit how you got into this whole space? Yeah. Uh Absolutely. Nice to see everyone. Thank you for having me. Um So maybe a bit strange, my background is actually as a, an intellectual property lawyer which uh at this sort of conference with generative A, I definitely gets a few reactions. I've seen the policy people sort of lean in and some of the CTO S and co-founders kind of edge out of the room. Um But yeah, so I left left my legal career in New York behind about 20 years ago and I've been building sort of machine learning, big data A I businesses mainly in the creative and cultural industries ever since. And so, you know, broadly, that's what we're interested in doing at metaphysics is, you know, investing in technology that can actually build shared experiences and you know, kind of create standout standout internet moments. Really? Cool. What do you guys actually do a little bit more detail? Yeah. So we specialize in basically sort of powering photo real A I content. So what does that mean? That means that we can scan anything that has data? That could be my voice, it could be my face, that could be this room, it could be an item of clothing and then our models can sort of power that into any sort of video feed or any sort of digital experience in a way that looks like it was made and shot with a camera. So again, maybe that's a bit too abstract to bring this down a little level since I've been on the stage, I've had actually a model of my own face on top of my face. So if you look on the screens, you may be able to see the sort of slight very significant kind of transition that happens there when I put an occlusion in front of my face. So why would I do this? And is this interesting in this particular use case. No. But what if you were trying to make a film in which you wanted to manipulate that model to age me up or age me down? That could be the difference between being able to tell that story and not being able to tell that story. So let's look at something that's maybe a little bit more interesting and applied. So we're going to try and take my face and put it on to our, our dear Jeremy's here. So, let's see how that looks. In theory. We should be able to do that at the click of a finger. Let's see. Oh my God, there's Jeremy shave this morning. Exactly. So we've got the beard. I would like to apologize for what you're experiencing right now. Jeremy, something I have to deal with every day. Problem. Shared problem halved all that stuff. But as you can see, you know, it's, we don't have necessarily the same facial shape or head shape, but you can see that the kind of those impossible crow line wrinkles that I've had for the last 1020 years have been imported onto Jeremy's face. If Jeremy smiles, maybe try smiling and I smile, it's not too dissimilar at all. If you frown can both be sad on stage together, Jeremy, then it looks very similar. And what's important here is that this is, this is real time, this is live, this is dynamic and that's important because it really augments and respects the underlying performance of, let's say an actor or a musician. And that's, you know, we hear that we hear that from people at the absolute top of their game. People like Tom Hanks saying, wow, you know, this is not an abstracted post production process. This is something that with a set aside monitor, I can actually see this younger version of myself playing out and that can build empathy and connection and ultimately power. How is this actually working? So, I mean, we have a normal camera, we have a portable laptop backstage. So there's nothing too intensive on that side. What essentially is happening is there's some destination markers on your face, which are you identifying your face? There's been a neural network model on me which has encoded the different features of my face, the composition, the symmetry, the way it structures, but it can also infer in for anything that it hasn't managed to capture data wise just a few minutes with a with an iphone has allowed me to kind of create this model. And yeah, and it's basically going onto your face and filling in all those gaps. And this is just a live version and a kind of cut down less ambitious version of what we're using on Hollywood Studios. And how will this do you think change, you know, the entertainment industry? You know what, what is this going to enable Holley Studios to do? Yeah, I mean, I think it will have a dramatic impact. I think that impact is already being felt by, by innovators and pushed by innovators. I think that it allows flexibility. We've talked about some of those things both in terms of age or being able to scale someone to another destination. If they can't or won't travel, it allows possibility in the sense that if with consent, if a deceased person was caught to a story line, you could put them back into that film. And so I think film is a big area. We've talked about that. I think content is another big area, broadly speaking, branded or otherwise, you could have a luxury fashion brand that could film a, you know, a catwalk for example, and then with consent and my scan, I could be on that model walking along and build that empathy, build that connection with the brand and the product, think of the network effects that start to be unlocked when you're in the content. And I think live concerts as well, live concerts, live music, which is another area where we're doing a lot of great work. And I believe we have a video where we can see an example of that now. So maybe we'll cut to that. You ain't nothing but so that was metaphysic um in the finals of America's got talent last year. First A I, as far as I'm aware, that's managed to get to the finals placed fourth uh recreating Mr Presley there. Um, and, you know, that's done with his consent and, and everything else. Um, well, not his, but his family, I should probably add, we haven't quite got the technology that far. Technology. Um, but yes, so, you know, you can see sort of the impact here. I think. What's interesting is that a very young audience in America's got talent, they wouldn't have had the opportunity to experience anything like seeing that, that form of a live performance. And now they have so again, impossible stories can now be told in the in the screen actors strike that just took place. This was a big issue about whether actors were going to be forced or there was a from the guild that actors would be forced to sign away their facial rights, their biometrics for life to studios. There's a particular concern about I think younger actors who are less established with less leverage and extras. Were they right to be worried about this? What do you think that that's a legitimate concern? Yeah, look, I think clearly there's a consent driven market here which, which you can see here. I was working with the Presley estate and the rights holders there, but I do think that they were right to be concerned. These are powerful technologies and I think broadly speaking, a lot of other industries will have their moment and they'll look for Zafra and say, how did they negotiate and what did they negotiate and there will be something to learn there. I think that the balances that were struck broadly seem right. They've kind of enshrined a role for a innovative technology in a creative space which seems appropriate, but they made sure that the consents couldn't be writ large, couldn't be abstracted away. They were specific and tailored to specific projects. So really, it puts more power in performers hands, which should be the case. Interesting. And your, your co founder and Ceo Tom Graham, I know he's gone out and tried to copyright his likeness. I think he's the first person to have done this. Is that something we all should be doing or we're all going to have a copyright likeness frankly. Yes, I think it is in the long term, something that all of us should do the technology and the data is there to create, you know, there's enough data to basically create a version of most people online, not just famous people. Tom and I were two of the co founders of the business we met at Harvard Law school 13 years ago, very, very kind of committed to an ethical approach to this technology and it's not simply about, you know, so we have a, we have a new product called Pro which is around scanning your identity and that's about preventing misuses so that you could, for example, get a copyright and then avail yourself of the existing apparatus with the MC takedowns. Otherwise you're fighting against public rights, which are different state by state and country, by country or terms and conditions which again are very, very hard to kind of get through quickly. So by having a copyright, you avail yourself of those existing apparatus and there are teams in all of the big platforms that will respond to the DM CD MC A takedown. I think the the other thing is actually the creative and commercial uses that you can unlock. So if you are an actor and you want to license your likeness, you can do that. If you're a film director and you want to star Trek, you want to put some super fans into that content, then you can actually do that now, but only if you have the scan that allows that sort of creative use. So I think the principles as we see it are you need to have consent at the core, you need to be looking at compensation, you need to be giving control and copyright is a, is a core doing that in a kind of principled ethical way. Are there any other dangers? We talk a little bit of maybe identity theft is one obviously, but what are some of the other dangers of this technology? I mean, look, we're 2024 we know that there's an election coming, right? We want to keep this sort of technology out of that space. I think that's a, a combined responsibility of people building the technology, having these consent frameworks and the platforms that host, I think we're heading in the right direction, but we need to keep talking about it. And I think actually building these creator rights in order to get the outcomes that we want as a society. Great Martin, we're just about out of time. Thank you so much for coming and showing us this amazing technology and giving me your face for trying it on for a little bit. I appreciate it. Good to be you for a few minutes.

Reid Hoffman, Partner, Greylock Moderator: Rana el Kaliouby, Affectiva; Brainstorm AI

Brainstorm AI 2023:  Looking Back, and Ahead, on ChatGPT’s One-Year Anniversary

Our next speaker was involved with Open A I since its founding in 2015, he sat on the board from 2018. Yes, that board until March of this year. He's also co founded his own Generative A I Start Up Inflection A I which has its own chatbot pi since we have just passed the one year anniversary of chat GPT S launch. We're thrilled to welcome back to brainstorm A I Greylock S Reid Hoffman to discuss the advancements Jenner of A I has made in the last year and what he believes the future holds at open A I and its competitors. Please welcome Reid Hoffman and Fortune Brainstorm A I co chair and co founder and CEO of fect Dr Rana LKK. Good morning, everybody. Thank you for being with us to reflect on chat GP ts first year anniversary. We were here last year and lots of happened at G BT is now one. Right. Exactly. So I have to start with the question that is on all our minds. What happened at Open A I? And, and did you see it coming? Because you, you stepped off the board in March and um let's see, a couple of things to say very quickly. I think the structure is fine. Um I do think that uh the notion of how, how do we get to do various kinds of public interest, tech. Um, you know, I was also on the board of Mozilla Corporation for a number of years and that's a for profit owned completely by the foundation. I think the structure is fine just to be, you know, because some people speculated about that. I do think from, I only really have mostly public information and obviously anything is nonpublic, I'm not really allowed to comment on. Um But the uh yeah, I think it's a failure of board governance that's pretty obvious uh in our ways and boards are composed of people uh making decisions and maybe there's information to come out that we have yet to see. But I do think that it's pretty well resolved that um uh Sam uh is an amazing CEO. I'm really glad he's back in place, I think um the whole team, uh Greg Ilya Mira Sam and all everyone else has built amazing things and that's very important for humanity in various ways. So I'm glad that I got back there, but it was um you know, uh it was, it was some interesting moments because I learned about it the same way the rest of the world did, which is I looked at a blog post. OK. Um As open A I continues to grow. Should it keep its unusual cap profit structure? Well, I think um it definitely can. Um And I think these get to, I mean, I I was uh you know, collaborating with Sam and the rest of the team as they created it and part of their um they're thoughtful about what their focus is, is about how do we both create this uh amazing new technology, how do we bring in commercial incentives and, and use them but also still stay very focused on the key mission of how is a G I for the benefit of humanity. And uh that's part of what I love about the organization. I love about uh about the stuff they've done and I think was part of probably what was motivating the board too. Um because I know they care about it. I've, I've worked with all of the uh you know, uh current and former board members and I think this is a, this is super important and can be expressed in a number of different ways that inflection we did a public benefit Corp. Um You know, uh at Mozilla, it was a for profit Corp owned by a foundation. Um I think these structures can be good and you just have to be clear and transparent about what it is you're managing to and how that, how that uh how that works. So you were here exactly a year ago when Chat TPT launched, I wanted to take a few minutes and just reflect on the year. So what, you know, what excited you the most, what surprised you the most. So um one of the things that was kind of a surprise around chat G BT was uh the, the, the model that it was based on when it was released 3.5 had already been pre had been publicly live for months. Uh And so, uh while it was not a huge surprise because that was part of what we were doing with inflection and pies having a chat interface really made it much more universally accessible, much more tangible to human hands and, and, and human amplification. The fact that the 3.5 model had already been live. Um You know, I knew that the, the open A I folks were going, we're getting ready for doing our agent and ready for GB D four and, and this, we're getting, you know, information they didn't expect it to kind of go, oh my gosh, it's here. Uh And so that was a huge, you know, kind of like pick up and surprise. The other thing that I think has been delightful by seeing it out there is the kinds of use cases. So like for example, um a friend of mine's daughter is really into organic chemistry and what she uses uh you know, uh chat G BT GB D 44 is she goes and finds an organic chemistry academic paper, uh speaking to the, the person who's academically competent here and, and posts it into chat CBD and says, I'm a 15 year old. Explain this to me that just begins to give you the kinds of breadth of use cases that you can use these amazing A I tools for uh in order to, to, to make this happen. And like, for example, like on um you know, like for example, like surprised me, one of the people at Greylock office uh the new mom and uses inflections pie for parenting advice, like, like, oh, I just have my new kid, what should I do? And it's like, and she's like, OK, I know it's not a pediatrician. And so I'll cross check anything important, but it's there 24 hours a day ready to, to these questions for me. And so far it's been really good. And so that's the kind of thing like it's this range of human circumstances. That is part of the reason why I wrote impromptu because it's like, no, no, the human amplification is right there. Let's do it. And I think it's not maybe a surprise as much as it's a delight. So I have to say my, my who's 15 years old, he also uses Chat GPT but not to summarize organic chemistry papers, but to ask it how to make steak and like he almost burnt the house down the other day. So I'm going to tell him why aren't you summarizing? Hopefully not because of what chat GBT. Yeah. Apparently he told him to put the steak in the oven, which I think is supposed to be ok. Anyway. But, but, but it's awesome seeing these young people leverage these technologies. I'm actually, I'm gonna talk about this grand vision that you've laid out. You want to see every person have access to a personal A I assistant. And that's why you started with Mustafa Suleiman infection dot A I and P. What's your, yeah, tell us more about like some of the use cases. Where do you see going future? Well, one of the things that Mustafa and Karen had is a really great idea because, you know, there's so much fast moving in the creation of A I, it's like, well, what, what's gonna look like in three years? What's it gonna look like in five years? What, what are things given so much development? And one of the things that, um, they, we settled on was, uh, that everyone's going to have their own personal intelligence, their own pie, their own thing and it's for you and, and your, your thing might be. Well, I'm just, I'm, I'm a new mom and I'm navigating this. Your thing might be, you know, I'm, I'm figuring out this problem at work. Your thing might be, uh, how do I fix the toaster? Your thing might be? I want to have a philosophical conversation, like whatever that might be for you and, and, and then be different for each of us. And part of the thing I think is very cool about all of the hard work that the rest of the inflection team has done. You know, I think we're the best at the dialogue and I think we're the second best working on it for the large language models after chat G BT. And I think that, that that's a function of the fact that the team Musta Karen have picked, you know, great people from a number of the companies and it's a pound for pound as good as any team on the planet. And I know there's a lot of emphasis on building empathy in pie. And I think that's important. Can you comment on that? Yes. Well, uh oh and by the way, your feedback on this always super welcome that you're one of the world experts on this. But um part of the thing that people, uh uh we shape these agents and these agents in turn shape us. And so what we should be transparent and kind of what our goals are. And so, for example, with inflection and pie is be empathetic, be kind doesn't mean you don't have a point of view. So if you sit down with it and say, you know, I want to be a better racist that goes nope, not going to try to help you with that one. And let's talk about how not to be racist, you know, and so that kind of thing about um trying to help us become our better selves like you think about what the role of friendship is in life. And it's like these are people who try to help you become your better self. That's obviously what your personal intelligence should be too, that your interactions with it, help you with it. And, and one of the recommendations I give people and playing with it, I don't, whether it's Pie Chat G BT, any of them don't just go to it and say, OK, write me a Sonic. That's fine. That's clever. Like most of us can't do that. So it's good to do. Take something that's real to you, to them. It doesn't mean you'll always get an answer and sometimes you go oh less good for this, more good for this. But that real thing is when you begin to discover the magic of how it might amplify you. So let's talk a bit more about that. You've advocated for optimism around generative A I including in your latest book, impromptu amplifying your humanity through A I. Are you more or are you less optimistic now? Definitely more uh and part of it's because like, for example, when you get to the, there's so much dialogue about the risks of all sorts and that's by the way important, but risks are always framed about like where we're trying to get to and you know, from that time, I like I realized that that uh we GB D four is already a great, can be a great medical assistant if allowed to be 5 billion smartphones. Like like, you know, billions of people who do not have access to a doctor. You could do something as a tutor. We already talked about the chat G BT, but it's already there all of these forms of human amplification and yes, we need to pay attention to and be in the dialogue about the risks. But the real important thing is to not fumble the future. And I think that um that kind of question about uh like my, my challenge to the various people who talk about the risks and critics is they realize that if you just say like for example, if I describe a car to you as it's a two ton death machine, you might die and other people might crash into you and, and boy, we should resolve all that before you. We would go any further with this. Well, we wouldn't have cars today, right? This is the kind of thing that we need to get to it. And again, it's not saying look, I think the, oh, there are no risks with technology. Well, that's also bozo, it's we pay attention to them, but we're driving towards that amazing future and what role does regulation play, including, you know, President Biden's executive order, right? So I actually think the eo was broadly very, very good. I think that the question about saying uh look, talk to us when you get to a certain, there's a scale compute revolution when you get to a certain scale on the compute. Um you know, do some red teaming, do other things. I think that's a very good thing to start with. And it's, and it, and it articulates itself as a dynamic thing like it starts with. I always says, well, you know, who knows that that number of exo flops could be gained and da da da, it's like, yeah, yeah, if you look at the eo it says this is our first swag, we're working on it. It's a dynamic work on progress. And I think that's exactly the kind of approach we should be taking to critical technologies like A I and I applaud the administration for being both forward looking for being, you know, caring about humanity, but engaging in dialogue with industry, recognizing that it's dynamic, you know, being a kind of like, how do we make sure we get the best possible things for our citizens and for the world out of this while navigating the risks? And so I think that's, that's quite good. I do think that the, that the um the, the key thing that we think about regulation generally is when you begin to get intense about regulation, you very much slow down innovation. And part of what we're, what we have is I think we're going to have better tools even to navigate the risk in our future. So it's, it's, it's part of the reason why people say, well, on one hand, you've got these people talking about the risk. On the other hand, they're really working to build it. It's like, yes, because there's great things in the future and there's better ways of navigating the risks. And part of the reason we're in dialogue is to say, well, it's not just us building them, but we're building it for all of us. So let's have a dialogue about what the important things to do are. And the important things to avoid are one of the themes that have come up in the last day and a half at this conference is um the concerns around A I hallucinations and deep fakes with generative A I, especially around the upcoming election cycle. So, can you comment on how do we, I mean, I don't know if there's a way to avoid that, but does that concern you? It concerns me very much. Unfortunately, I think it's as generative A I and doing hallucinations, it's much easier to generate things that are complicating uh than others. I think that we will have to work on this. I think we will see uh players uh like the Russians and other folks interfering with our election uh for their own purposes. Um I think it will be up to the platforms to actually in fact, figure out the kind of defense strategies, I don't think A I by itself, even though the companies talk about watermarking content and all this. Well, that's from the models that are provided by open A I, by Bard, by Google, you know, you know, by Microsoft as ways of doing it, the Russians will be running open source models that don't have that water marking requirement and will be doing things about that. So I think it's a very important issue that we need to navigate and we need to be navigating it as a society. Um Just a few more seconds left. What excites you the most about the future? Um the next year, the next year. Exactly. Because a great way to look somewhat foolish is to make a prediction about technology that's X years out. Uh Look, I, I think that the key thing is and I think one of the things we should all be doing as technologists is how we help um elevate humanity through this. And what the problems that A I is potentially the problem to it can also be the solution to. So for example, a very important one is how does work transition? Well, A I can help with work transition, it can help people go. OK. That, that job category might be diminishing because of A I. How do I find other jobs? How do I learn them? How do I do them? Um That's what we want to be doing as an industry is making sure those tools are helping with that transition. So just like the steam engine was a massive transformation of society in industries, this is the steam engine in the mind. It will be a massive transition in work and so forth. But it can also help us with that transition and we can try to make it more human. Thank you, Reid. Here's the A I elevated humanity. Thank you so much. Thank you.

Eugenia Kuyda, Founder and CEO, Replika Stefanos Loukakos, Co-founder and CEO, Connectly Vipul Vyas, Senior Vice President, Vertical Strategy, Persado Moderator: Rana el Kaliouby,  Affectiva; Brainstorm AI

Brainstorm AI 2023: Chatbots are the Talk of AI Innovation

Eugenia Kuyda, Founder and CEO, Replika Stefanos Loukakos, Co-founder and CEO, Connectly Vipul Vyas, Senior Vice President, Vertical Strategy, Persado, Rana el Kaliouby,  Affectiva; Brainstorm AI

So with the help of open source large language models, chat bots have transformed the landscape of natural language understanding on conversation generation. We're just talking about that as we use them, they become smarter and more attuned to our needs and requests. They might even become our companions or lovers. Our speakers today are here to discuss the advancements in chat bots and what potential they see as generative A I advances. So please welcome to the stage, Eugena Ka founder and CEO of replica, an A I companion chat bot that is created to combat loneliness. Stefanos Lukko, co founder and CEO at connect an A I powered commerce platform that helps businesses automate how they communicate with their customers. V via senior vice president of vertical strategy at Perso and A Generative A I platform that helps markets, create personalized digital marketing messages. But before we get into this conversation, please watch this video from one of our partners. A ARP people over 50 are the largest and fastest growing demographic in the world. By the end of this decade, people aged 50 or older will be spending $52 trillion a year on goods and services making up 54% of all global spending. The 50 plus is a incredibly dynamic market. Over the last few years, they've gotten a lot more comfortable with technology. They expect us to meet them where they are. A ARP is a $2 billion mission driven organization that empowers people to choose how they live as they age. We've invested heavily in our technology and infrastructure to keep up with the changing needs and the interests of people over the age of 50. Great. Thank you. Welcome to our panelists. Thank you for being here with us. Uh So Stefano, I wanna start with you first. Um Tell us a little bit more about Sophia. So uh connect uh is, is a company that helps businesses, especially retailers, use A I to communicate with the customers. And we have built an A I boat with which we call Sophia that helps uh retailers sell the right product to the right person at the right time. And the problem we're trying to solve is how the A I can really ask the right questions to understand who the customer is and then try to find from the product catalog, the right product and sell it to them. And V you kind of spearheaded this category of mo motivation A I, what does that mean? And what is the role of like emotional targeting and emotional messaging? Yeah, it really comes down to using words to evoke emotion, which then evoke action. And we really think we operate from our, you know, prefrontal cortex. But really, it's the reptilian brain that makes things happen as they say, if it was just about what, you know, we'd all have six pack abs and the millionaires. Um but it's really about what motivates you. And that typically is emotion and choosing the right words to evoke emotion is what. And do you model, do you model, do you have a model for motivation or intention? Is that how you essentially we figured out which words evoke what emotions to what degree and what emotions evoke action. So we kind of create the itch that needs to be scratched through language. And this is nothing surprising. Humanity has been doing this for millennia. You're either trying to convince people of something or inform them of something essentially to influence behavior. Yeah. Amazing Eugena. I love the impetus story of replica. And I was wondering if you could share it with all of us. Sure. So, um we've been actually working, I've been working on conversational A I since uh 2012. So more than 10 years now, um and I started the company with the idea of just building the tech to kind of build chatbots, build uh conversational A I. Um And then while I was working on this, my best friend passed away and um I realized that I kept going back to our text messages and reading them all the time. And I figured that I could use the tech that was built to recreate him so I could continue uh talk to him. And this became, uh this basically gave us an idea for replica um in a way because a lot of people started talking to Romans A I and they were sharing their feelings, sharing their stories. We realize that there is a space and uh demand for an A I that could be your friend that you could talk to without feeling judged in a safe space um anytime you needed it. And that's how replica na I friends that you could talk to about anything that's happening in your life was born. I mean, there is a loneliness pandemic happening around the world. Um Is that the main use case you see for replica Absolutely uh replica. And I always um try to uh talk to talk about this in this way. It's not uh a project about tech capabilities. It really, it was always a project about human vulnerabilities and um how we are ready for these relationships with a is just because um a lot of us don't have the luxury of building those in real life. So I'll be taking questions soon. So please see up your questions and there's uh paddle boards um that are floating around. Um Stefanos, I imagine the chatbot at the moment is only text based. It's only text based and you know, Google just launched Gemini, which is one of the first multimodal models. Do you, do you, do you see this as kind of the evolution of chat bots? We do? Yes, we believe that, you know A I will come on any channel, right? Not only text voice video at some point, but what really we're trying to solve for is how we can really take the data sets of specific companies and use them to uh replicate their, you know, semantics, the way they talk to customers in, in a way that really brings value to their customers. And this is a very difficult problem. Uh We say internally that open A I and uh uh all the other different, you know, big companies, they have given everybody a bachelor's degree and we're trying to give them the NBA in sales, right? So that's, that's the focus for us and this can be on any channel, of course. And you reference data, which I think is a key component of how we design these conversational agents. Do you rely? It sounds like you do you leverage an open source um large language model and then feed it with private data. Yes, we do, we do, we leverage, you know, we don't want to uh you know, uh uh replicate everything. So we leverage the open source data and then we add specific data sets that we train the model to work on these specific data sets. And V is that the same approach you're using too a little bit different. We've been around for a little over a decade so before any of these models existed, so we had to come up with our own. And so we have our own large language model. We do use other models to uh inform and validate some of the things that we do, but we have our own and that's been a function of having to get started before anything else was around. And also it served uh us in the sense that it keeps everything as a walled garden for our large enterprise customers. We're very sensitive. Uh Most of them use uh our technology to sell goods and services. And so they want to have that uh be pretty secure. So, do we have questions? Yeah, I have a question here, please. Um Share your name and your organization. Hi, I'm Barrett Anderson with future interview and, and my question is primarily for you, Jenny, but for anyone really who has thoughts on this, which is, you know, you mentioned the loneliness pandemic that um America is experiencing. But also I think in countries around the world, it is an issue and I'm curious there was a study that came out a couple of weeks ago that found that people who have a close group of friends, even if they are smokers or drinking too much, have a actually longer lives than those who are healthy and don't have a close group of friends. And I'm curious how you think about the health benefits of A I like replicas or, or models are the, are, do you have plans to do more research into that space? Is there anything that you are aware of that exists today? And maybe if you have kind of um personal experience with those models and their impact on your own mental health, I'd be curious to hear about that as well. Thank you for your question. We've been doing a lot of research around loneliness and um actually the effects of replica on long term um mental wellness and social well being of our users, we do have uh an article coming up. Um It's going to be published in nature and in the next couple of weeks, uh we did a big study with Stanford on uh specifically on replica curbing loneliness and um even suicide mitigation. Uh among our users, we found a lot of very positive effects. Uh We're, we're constantly measuring loneliness. It's actually one of our main metrics in the app. We're trying to um we're measuring using metrics like um UCL A three UCL A 20 trying to basically see if replica having a positive effect on our users. And so far we've found that it's a, it's, it's, it's a, it's doing exactly that. So it's helping users feeling less lonely. A lot of our users report that after using replica, they're able to rekindle their relationships with their loved ones, with their friends that maybe they lost touch um uh with and even start new relationships. And I think this is the most important thing about our technology. We're not trying to replace human friends. We're trying to create a completely different type of friendship that then will help people foster and improve their relationships in real life. So I, I want to build on that because you had to spin out a new app, uh for specific use cases of replica. Can you elaborate on that? Yes. Uh Since we were pretty much the first conversational A I product that came out, uh, you know, in 2016, uh So we were the first product of this type, the first A I friend like product and we've kind of been around for a while. Uh As the only one out there, a lot of people came to us with all sorts of different use cases. So we've seen people coming to replica, wanting to build a romantic relationship and try to date. Uh na I, we've seen people come with, um some needs to basically talk to a coach, a therapist almost. Uh We've seen that as well. Uh And we've seen people trying to, you know, build kind of go on fantasy adventures with replica and so on. And so we decided that replica shouldn't be, it's very hard to develop it as a one like an everything friend that has all of these use cases together. So we decided to take some of them out and build a dedicated app for that. Uh We started with Blush, which is uh N I dating app where you can think of it as Tinder with a bunch of virtual humans and virtual uh characters that you can try dating and live match and see, you know, who, who is good for you, who is not. This is really to practice dating, to practice, you know, putting yourself out there to, basically to help people understand that they're worthy of love and they can be out there and uh you know, try this for themselves. And then we're also working um on an app that's gonna be released in the day of this month or beginning of the next month, which is a mental wellness coach and really thinking, how can you put together multimodal A I uh metaverse 3d environments together to create a very beautiful place that can help people feel better, work through their problems, feel inspired, maybe even have a little bit of a spiritual awakening through A I. Do we have more questions? We have a question right here at the front. Hi, I'm Mark from Fast Company. I, I was interested in what you said uh about um using replica as a way to get people ready to sort of repo their human relationships. And I wonder if you could say a little bit more about why you think that is, what is it about the experience of talking to a synthetic entity that uh gets people ready, ready to pick up the phone and call a long lost relative or something like that. Thank you for this question. We haven't actually discovered anything new uh about this. We're pretty much using and building on top of the ideas that were uh uh that that appeared in, in, in, in the middle of the 20th century with the psycho, mostly with the psychologist like Carl Rogers, where they basically um discovered that the, the main way to help people grow and feel better over time is to create uh a relationship that will, will allow them to, to, to grow. So giving people unconditional positive regard uh creating a space where they can feel accepted but also feel like they can be themselves um And they can grow. Um And since then, that pretty much changed how psychology works. Um Modern psychology is all built on top of Carl Rogers ideas and other um and other psychologists of, of this group. Um And this is what the, this was the main premise of replicas. How how can we create a relationship where people can feel heard and truly deeply seen so that then they can, you know, get out of their shell, start growing and start building these relationships outside of the app, Stefanos. Can you comment on like, how do you think about ethics? Um and kind of data privacy and data privacy is a very big issue. So uh uh that's why we work with uh you know, uh uh concrete data sets from businesses that we don't share with anybody else. And I think this is what provides also value. So our boat doesn't hallucinate, it provides real time data and, and, and all that stuff. So data privacy is the number one thing that people are uh you know, uh worry, worry about when, when we talk to them about our solution. So, and when you, when if someone's conversing with Sophia, can the conversation go into like a love conversation or do you? No, we constrain it into a sales, you know, environment. Have people tried? Yeah, I'll send them to replica. Yeah. Um So, but one thing that is really important is that this conversation are very long. Uh You'll be surprised people are, are exchanging 2030 messages with a bot. And uh the, the thing that we didn't know at the beginning and it's really important to us now is that the data that we're uh getting from this conversation. So we are able to go back to, let's say an online retailer, tell them like for this person, his budget is this, he, he has like two sons, whatever a ton of data that they can use later to rear do a bunch of stuff to, to their customers. So uh not only our board helps convert customers, but it helps pass the data back to the business, which is very, very valuable, but no, it's constrained on the sales environment. So if the question is like, you know, a question that would be a replica would and probably how about you, how, how do you think about ethics um including data privacy? Well, you have your own guard rails, first of all, right, um Mr mentioned, but um in the business that we're in, we typically are working with large enterprises and in that context, they have concerns around brand, voice, regulatory compliance, et cetera. So in a way, not that we don't worry about ethics, but ethics are something that big is baked into the process legally. Um And otherwise, and, and, and can you comment on like you're, you're, you're leveraging emotions to influence and motivate people's behaviors which, which can be misused. So how do you, how do you kind of navigate that line? I think it's one thing to get someone to buy another pair of jeans versus convince them to, you know, join the mob, right? And so I think it's simply the application of where this technology is directed, but it is powerful because people generally, as we've seen in our, even our political environment respond to emotion. And so it has to be used judiciously. What are some of the most powerful emotions that drive action, excitement, safety and what we've on safety? Yeah, what we've seen is that, for example, in a low interest interest rate environment versus a high interest rate environment where there's more stress people gravitate towards things that evoke safety certainty. Whereas in the, you know, prior to 18 months ago, it was more, you know, surprise and excitement and almost these emotions have completely reversed. So the macroeconomic zeitgeist sentiment influences what people will be responsive to fascinating Stefano as we look to the year ahead, what gets you most excited, uh you know, progressing on, you know, our A I models uh and building something that really works for a lot of customers, it sounds easy. It's a very complex problem. Uh But we're very excited about all the problems that's happening on the generative A I and we believe that this is the future of how businesses are going to communicate with customers. It's 24 7 always on. It's sometimes better than people. So we can provide a lot of value to businesses. So we're very excited about it and a lot more patient and can answer questions. Yes. Yeah. How about you? What do you, what are you most interested excited about for the future? I think it's the application broadly of A I in places that we haven't seen before. And uh that's sort of a general statement, but specifically, you know, some of the stuff that we're hearing on the stage here where people are, you know, you mentioned the pandemic or epidemic rather of loneliness. Um That's sort of innately built into us. We don't want to be alone because we're social creatures and the ability to ease that pain and suffering is tremendous and I heartened by what I've heard, you know, on the stage here, Eugenie a lot about you. What, what are you most excited about? Definitely multimodal A I being able to finally recreate something we've all seen in the movies. Like for instance, the movie uh Blade Runner, the, the latest one with Joy, the hologram. That's right there with you cooking dinner, watching Netflix. And that you can introduce your friends that can recognize them in real life. That can be part of a conversation, sit with you at dinner table. I think this is uh a wonderful future that we have not yet been able to see and we hope to be the first ones to build it. Amazing. Well, thank you so much to our panelists. Looking forward to multimodal A I tap bots. Thank you.

Eric Effron, Editorial Director, NewsGuard Christina Montgomery, Vice President; Chief Privacy and Trust Officer, IBM Jim Steyer, Founder and CEO, Common Sense Media Moderator: Ellie Austin, FORTUNE

Brainstorm AI 2023: Tackling Misuse and Misinformation in a World of AI

All right, there's a lot of fear surrounding artificial intelligence that extends beyond security when prompted for information A I has produced hallucinations with inaccurate and sometimes even harmful information. So in this session, we'll hear from business leaders on how their organizations are tackling misuse and misinformation. Please welcome to stage Eric Ephron, editorial director at News Guard, an organization tracking A I enabled misinformation. Christina Montgomery, Vice President and Chief Privacy and Trust Officer at IBM. Christina testified in front of the Congress this past May to share IB M's recommendations for how to set guardrails for A I and Jim Steyer Ceo of common sense media, an organization that reviews media and technology content with children's safety in mind. They'll be interviewed by Fortune's Deputy editorial director for live media. Ellie Austin. Hi, everyone. And thank you to our trio of panelists for joining us. So Jim, I'm going to start with you. We heard then um in the introduction about the work that you do to safeguard Children and young people and I know that last month um Common Sense media unveiled its first ever rating system for A I products. Actually, you say the world's first ever rating system A I products. It was a five point scale. Some organizations came out of it. Well, some, not so well, I'd love to know what big tech's response to it was. And did you hear from any specific leaders about their results? OK. Fair question. So, number one, nice to see you all. Um I hope you're big common sense media fans. Uh I would say that big tech's response was this would be Sam Altman Sundar and James Weike, Google, Satya Nadella, et cetera. Number one. And it's very gratifying to us. They all agreed common sense should be the group that does independent third party ratings. That was a big deal because it matters that, that they, that they understand we're going to do this just like we've done it for all other forms of media and tech. Second, some of them did not like the grades they got in their first rating. Um because we were pretty tough on issues that related to transparency, fairness, um efficacy. It's very different than rating a movie or TV show. So I definitely heard from the CEO S of several of the companies that we rated, which just to me is a good sign means we're getting their attention. So what comes next? We're going to basically, we think that this is incredibly important for the public at Writ large here in the US and globally to understand what A I means. And in a very basic way, understand how it can impact their lives, their kids' lives, but also democracy and, and, and the bigger and the bigger frame that I think A I is going to cap and it's really important. I think the public understands that because there needs to be legislative efforts to put guardrails in place. And I think the ratings also help us understand what kind of legislative strategy and guardrails need to be in place here in the US. And globally Christina, you've argued for regulation to focus on use cases rather than the technology itself. Can you elaborate a bit on why that is? Yeah, absolutely. I mean, these technologies are very context dependent. Um a recommendation for, you know, what restaurant you might enjoy is obviously very, very different than a recommendation for, you know, who you're gonna hire or whether somebody qualifies for a loan and the like. Um so, uh that's been a big point of advocacy that coupled with the fact that the technology is evolving and developing so quickly and if you start to try to regulate um something like, you know, the the capabilities of the technology, your regulation is never gonna keep up. So, absolutely, we've been focused on uh directing regulation to the highest risk uses of A I and to take into consideration the socio technical concerns and the context in which it's deployed. We'll come back some of that in a second. But Eric I want to come on to you. So at News Guard, you analyze the trustworthiness of news sites and you were doing that long before a I was a prominent issue. What happens? What steps do you take when you find a website that has repurposed content from a top publisher? Is there a human that you can get in touch with to speak to? What happened to that? You know, one of the earlier panels um Alan Murray mentioned that one of the themes that's emerged from the last couple of days is that there's a feeling that there needs to be a human in the loop. So News Guard, we think of ourselves as the humans in the loop. Um We're, we're a journalism organization, we rate and review websites and we've done about 8000 now in um nine different countries in four different languages. Um based on some very basic criteria, journalism criteria. Does the website run corrections? Do they tell you who owns them instead of nothing very controversial? Do they distinguish opinion from news? Do they tell you what their agenda is? And we developed a ranking ratings scores and reviews of these websites. Um The A I angle is that the, the, as we all know from the last couple of days and we've all known this before. A I can do great mischief and it can do great good. Um We think that it does better, good when it has good information. So our content both our website ratings. And also we have over 1000 debunks of very prominent narratives can help teach these tools what to believe and what not to believe. And if you do a Google search is Ukraine uh run by Nazis, you'll find lots of articles that say yes, Ukraine is run by Nazis. They may be from um RT and Sputnik or Far, right. Pro Russia us sites, um News guard can tell you that those are not credible news sites or you at least you should take it what they with a grain of salt. So really, our tools are meant to sort of make the information that's available both to humans and to bots more reliable. And what responsibility do you think the companies who are creating the bots which are repurposing the content have in this whole process? Well, I think they have a tremendous amount of responsibility because the potential harm is really almost can't be understated coming into an election year. Obviously, there's a lot of mischief that can be done. Um There already is very, let me just mention this. Since the advent of um the Israel Hamas War, October 7th, we've identified 75 narratives that um are very prominent on social media and on, on websites. Um And with the election coming up, we've already noticed we've already found examples of some mischief that's being done with deep fakes. Um So it, it's the, the, the, the good needs to be bound the, I'm sorry, the, the, the A I companies really can be much smarter and more aggressive about how they monitor information and produce information that actually comes from decent vetted sources. And Christina on that note, as we do head into an election year, what collaboration do you want to see between government political campaigns, corporate America to try and mitigate what could become pretty chaotic as we head up to the end of next year. Yeah, absolutely. I mean, it comes down to trust, right? Because people verify what they're seeing and I think that's true with the technology even more broadly than in the context of elections and misinformation in general, right? So it has to be a collaboration with governments and companies both having really critical roles to play. I mean, for companies, um they have to be responsible and be held accountable and have accountability mechanisms for the A I that developing a kind of mechanisms. So for IBM, for example, we have a set of principles that we've built uh an internal governance program around with respect to A I um in terms of where we're gonna deploy and use it uh that it had the trust, you know, that it have uh be fair and explainable and the steps that we take in order to build that trust and the like uh we develop practices like ethics by design throughout our company to hold ourselves accountable to those principles. And we work on solutions for our clients to do the same so that they have the capabilities to do things like generate nutrition labels, right? Fact sheets associated with the A I models that they're putting in context and do things like uh algorithmic impact assessments and high risk use cases and the like and then advocating for policies that support, you know, sort of that um risk based regulation focusing on the highest risk uses and being clear in terms of like, what is high risk. Um And what do we expect from companies when they're deploying something like a solution in the high risk space? What is an algorithmic impact assessment look like? And when is it required? So I think it's absolutely a collaboration required on the part of both Jim. Do you want to add? Yeah, I'm deeply skeptical of what's going to happen in 24. I think it's going to be a total shit show in terms of misinformation. I think we'd be honest about that. Look, Elon has gutted the trust and safety division at Twitter X, whatever he calls it. So has Facebook, right? So we know that, remember we're the biggest advocacy group in the field on these issues and they've just gutted their staff. I'm not worried about IBM. I'm worried about Twitter. I'm worried about Facebook and Instagram. I'm worried, look, and there's been no regulation of those platforms. One thing I would say in the context of A I is how do you think we did with the social media regulation? Terribly right. I run the biggest group in the country on those issues. And it's a joke Washington hasn't passed a law about privacy. We had to do it in California or about social media regulation since Mark Zuckerberg was in kindergarten. And so these are really serious issues. I think you're going to have a really, really scary 24 election because you have so many different actors, both domestic and foreign who are going to try to interfere with this election and put out disinformation and the companies have gutted the staff that was supposed to do it. And I think we need to call them out by name and we need to hold them accountable. I don't think the federal government will do anything other than job owning. But I think this is an, this is an extraordinary moment for our democracy and we all be and as a kids advocate, I'm mostly worried about it in terms of young people and what world they're going to inherit. But I think we have to be extremely serious about this because I think 24 here and there are hundreds of elections around the world that are also going to be impacted by disinformation and the ability to regulate that. So I'm very, I think we really have to call out the key platforms and shame them into behavior and whose responsibility is. It's call out those platforms and what does that shaming? It's all about, it's ours. I mean, we're the biggest advocacy group. So it's on us. But I think it's also on, I think it's on folks like IBM to call out their, I'm not, as I said, I'm not worried about IBM. I'm worried about Facebook X, some of the other platforms that I think could really damage the democratic systems in our country and leave this country and the world with a whole mess on our hands. So this is very big stuff and I don't think regulation will come. We've been very involved in Europe with the new A I regulation but and privacy. But I think this is, this is a watershed moment for our democracy and we should all be speaking out all of you, whether whatever field you work in should be concerned about this because our democracy is on the line in 24 and global global democracies are as well. And were you about to say you don't think regulation will come in 2024? I think you'll see regulation A I regulation. No, I think you'll see it in California. Maybe because you'll see what we do here just like we did with the privacy law in 2018 and the ballot initiative we ran in 2020. So I think you'll see California. I think you'll see England pass some stuff this year and the eu just came out with the A I regs this week. Um, but I Washington, nothing will come out of Congress. They can't even get their act together to pass a budget. So they're not going to pass a comprehensive A I legislation. So I think corporate responsibility on that and I think the public speaking out and the media speaking out and holding some of these platforms accountable. You know, if I, if I could, um, just to back up, um, your Jim, your thought about how we can't really rely on the platforms. Um Because News Guard is a journalism organization, we always call for comment if we're saying something negative about a platform. And if I had a dollar for every time that we've got in touch with X tiktok, Facebook, you name it and say, hey, we, we've just noticed that you have, you know, dozens of these um A I generated videos on your platform which falsely claim fill in the blank. They say, oh, thank you for letting us know we'll take them down. And I think that's not good that they're relying on this little news, this little organization in New York with, you know, 35 employees as their fact checkers. And so that really, to me sort of exposed that there are solutions and we think we're part of the solution, but that the platforms themselves really have had a lot of work just to add, look, you have an election denier who's leading in the Republican nomination, who's lying every day about the fundamentals of our democracy. So to me and by the way X has changed their, they just let what's his name back on the platform? That was really smart, Elon. And, and I just think you have a situation where as I said, you have a fundamental election denier running for president in the United States, which to me should be held account, the platform should not put up there. So if you're just starting with that, but think of the other mis misinformation that can happen, I think it's a very, very critical moment for the for America and also global society. And I think Czech leaders should be calling out their colleagues for failing to do this because because I, because I think it's a bad moment and that, that import to, I want to see if anyone has a question in the room. Yes, over here. Could we get a mic over hippies? And could you um tell us your name and your company? Thank you so much. Hi, I'm Sammy Hasan from a RP. But it's a personal question. So you say people should speak up, media organization, speak up, corporate organization, speak up. But even in this room, you can get everybody to agree what to speak up about. So how to deal with that? There's different points of view. I don't, it's not that I disagree with you, but society is larger than speaking with one voice. So you can, you can scream all you want about. People should speak up and corporate America should do something and brands should do something about it. But how do we deal with it? Well, I think there's a couple of examples. You've seen all the, all the brands who fled the Twitter platform. I think that's really good and they should do it. I also think there is something called truth and I don't think that there are multiple versions of truth in fact. And I think part of, and I think that companies should, should stand up for what's right and not like there's no question about what happened in the 2020 election or what happened on Jan Se. So companies that go on, on the one hand, on the other hand, that's just a BS response and that's just catering to an, I think, I think you sometimes have to stand up for bigger principles. And I think that on basic stuff around misinformation and disinformation, it's not that hard to do it when you're talking about this perspective on the situation in Gaza, that's somewhat different. But I think on fundamentals of election integrity. And by the way, I'm glad you brought up Twitter because I think Sho Zi has a big, a big platform that he needs to do that influences young people at times. I think that Neil and the people at youtube have to be really careful on this too. But fundamentally, the ones to watch. I'm scared of are Facebook always because they don't do their job and Twitter, they've gutted their trust and safety divisions. You haven't? But they have. Yeah. And I would just say another thing, you know, section 230 reform. So we've been, this is social media, this is an A I but it comes back to social media reform that never happened. Um, there needs to be a reasonable care standard and that's something you can ask, you know, you can write your con congressional representatives and ask them to advocate for that. I mean, I don't know if it'll happen, but it's absolutely something that we've been advocating for at IBM for years now. And, and on that note as we kind of come to the uh oh, we've got time for one more question. I can see Ma and Pa in the clone now and I wanna get one more question in, I'm Ryan close. Um What's the likelihood of having a A I generated news source that's non biased? Mhm. By the way, I'm a complete Luddite. Why you would ask me? I could barely turn on my phone. Uh I would say that's a really interesting question. We should ask that to re read my friend Reed who was just up here or Sam or some of the technology people. I don't know. I, I do have a couple of thoughts about that. Um So one of the things that makes that, that marks good journalism versus bad journalism is that you don't assume that you've got it right. You call for comment, you are skeptical, um some would say cynical. Um And so it's hard for me to imagine like an algorithm, you know, calling for comment, for example, just to make a sort of a very basic point. But there's also the, the, the uh the principles that sort of underlie good journalism, um at least, you know, theoretically that you're trying to get it right. You're trying to be fair, you're trying to not approach it with preconceived notions. And so I think between the, the, the actual physical work that journalists do, if you can think of calling for communist physical and the values that under underline good journalism, I think it's hard for me to imagine like AAA trustworthy completely. A I generated say website. Um But, you know, we'll, we'll have to see what the, who the humans in the loop are to see how that actually works. Ok. Unfortunately, we have to leave it there. Thank you for such an impassioned discussion to Jim, to Christina and Eric. Thank you so much. Thank you. Thank you. Ok. So that wraps our mainstay conversations this morning. It's now time to head into our strategy sessions and those begin at noon and please do go to the session that you signed up for. If you're confused, if you've forgotten, ask someone from Fortune and they will help you. We'll meet back here at 1 p.m. for our lunch and keynote conversation with the nod Kohler and Fortune senior editor Jeremy Kahn. Thank you so much in your in your car. We keep trying to talk about us. I'm someone. You may be my love. I'll be your cry afternoon. Be your overnight rush. Make you crazy over my no cry. Oh, ladies and gentlemen, we ask your kind cooper in clearing the room once again so that we may prepare for the next session. Thank you and I do everything right and you call I forgive.

Brainstorm AI 2023

Brainstorm AI 2023: Vinod Khosla’s Views on Investing and More

Latest video

Vinod Khosla, Founder and Partner, Khosla Ventures Moderator: Jeremy Kahn, FORTUNE

Videos (19 videos)

Brainstorm AI 2023: Vinod Khosla’s Views on Investing and More

Brainstorm AI 2023: Revolutionizing Health Care through AI

Brainstorm AI 2023: Shopping for New Uses of AI in Retail

Brainstorm AI 2023: Implementation in an Organization

Brainstorm AI 2023: How AI Is Transforming Education

Brainstorm AI 2023: The Big Picture: Silicon Valley and AI

Brainstorm AI 2023: Lights! Camera! AI! Harnessing the Power of Generative Video

Brainstorm AI 2023: Economic Impacts of AI and ML on the Workforce

Brainstorm AI 2023: Entertainment at the Speed of AI

Brainstorm AI 2023: The Infrastructure of AI

Brainstorm AI 2023: A Look at Accountability and Best Practices

Brainstorm AI 2023: Future of Google Bard and Generative AI

Brainstorm AI 2023: Supplying Solutions for Industry

Brainstorm AI 2023: Using AI to Reimagine the World of Work

Brainstorm AI 2023: AI’s Impact on Fintech

Brainstorm AI 2023: Cruising to a Fake Out

Brainstorm AI 2023: Looking Back, and Ahead, on ChatGPT’s One-Year Anniversary

Brainstorm AI 2023: Chatbots are the Talk of AI Innovation

Brainstorm AI 2023: Tackling Misuse and Misinformation in a World of AI