Name: Brainstorm AI London 2024: Building The AI We Want
Uploaded: 2024-04-15
Duration: 13 min 22 s
Description: Paula Goldman, Chief Ethical and Humane Use Officer, Salesforce Moderator: Nick Lichtenberg, Executive News Editor, FORTUNE

Paula Goldman, Chief Ethical and Humane Use Officer, Salesforce Moderator: Nick Lichtenberg, Executive News Editor, FORTUNE

Brainstorm AI London 2024: Building The AI We Want

Photography by Joe Maher Fortune Brainstorm AI

Conferences

Hey, everyone. Thanks for joining us today. We are wanting questions from the crowd. So if anything we say resonates with you, if you've got a question, please raise your hand. Uh say your name and your title. We would love to have your questions for Paula. I'm super excited to talk to Paula today because she's sales forces first ever. Here's the title Chief Ethical and Humane Use Officer. And you've been doing that for five years, but you've been in ethical Technology for a decade. So you've been thinking about these things that the the wider world is now very aware of for about 12 months now. Um So describe to us your journey with Ethical Tech. Um Well, I should say I'm a deep believer in the ability of technology to open up incredible things for people in society. And I spent a lot of time working on tech for good. Um And about a decade ago at the time, I was working for Pierre, a Miar, the founder of ebay, we kind of looked up and said, oh, wow, tech is no longer the underdog tech needs guardrails. Um And, and it's been an amazing set of uh set of uh developments. But as you were just saying, it's also really gratifying to be in a moment where the field of tech ethics is in the news all the time. And that it's, it's actually, you know, one of the top questions we get from our customers when we're talking about A I is around, can we trust it? So it becomes a set of innovations and features on the product, right? And sales force has this concept of the human at the helm. And that to me that conjures up uh someone on a ship, the human at the helm of the ship, I think our art is very uh apt today. We've got this human, but they're driving all these gears. Um Does that define your philosophy at sales force? Yeah. So I mean, if you think about, so think about where we are in generative A I, right? So you just mentioned people have been captivated for about a year, a year and a half. Um And at phase one, a lot of people were talking about human in the loop. Actually, I feel that that term comes from the military, right? It was the notion of like having a person check before a really consequential decision was made by a machine. Um I think that's no longer good enough, right? And I think what is really important right now is we're moving from A I just generating content, I say just because it was blowing our minds, you know, to taking action on our behalf, the notion of co pilots, right? Um We need next level controls. Um We need people to be able to understand what's going on across the A I system. And most importantly, we need to be designing A I products that are taken into account what A I is good at and bad at, but also what people are good at and bad at in their own decision making and judgment. Yes. So you said co pilot there, I think that's an important word because uh we hear a lot about autopilot um and various discussions around A I but you insist it's a co pilot situation where we've got the human who's got the A I co pilot next to them. Right. That's right. Right. II I flew here. Uh I didn't go into a plane that was flying on its own. Right? And I think the idea of A I is that it's gonna augment human ability, it's gonna augment human judgment, it's gonna augment our ability to do our job better and more productively and to do that, we need to make sure that we've got the two strands working side by side. Right. So what have you found about what people are good at and what A I is good at which, which role should, which copilot have? Well, let me give you a few examples that I think are really incredible as we think about human at the home. Right. I think we do a lot of work. Uh, A I for service workers. Right. A I for service. Um, we, uh, one of our earliest generative A I products, um, we used with a major luxury consumer brand and their service workers. Right. And, uh, what they found was that, um, because the, the service agent didn't have to spend as much time looking up the answer to a routine question. They actually were able to do what really only people could do, which is connect with their customers. Right? And they found that all of the sudden service agents were 20% more doing product conversions. They were sales people now, right? Or think about the doctor that uses A I for note taking and is able to actually pay attention to what their patient is saying and what they're not saying and ask better questions and get to a better judgment or diagnosis at the end of that call. Really A I is already transforming jobs, people with their A I co pilots, uh people are becoming sales people overnight or doctors are becoming a new type of doctor. Yeah, it's changing. Well, it's definitely changing the nature of jobs and it's freeing up people to do things that are going to become higher value for the companies they're working in. Do you have any questions from the room so far? Uh One over here in the back there? And Tom Whitaker from law firm Burgess Salmon. Really interested to hear about A I ethics. How do you ensure that those who talk about, um A I ethics not only talk the talk but walk the walk. How do you ensure compliance? That's a very interesting question. So, I, I, what I would say is, you know, it's really important actually, I think a ethics is becoming a skill in all of our jobs, But the way that we approach it as sales force is actually like building it into the product itself, right? So you and I were talking about this a little earlier. The question of um how do you build friction into the product that people actually know how to use things? Mindful, mindful friction is a term that we use, right? So I'll give you an example, we have a marketing segmentation product. So it helps you, you want to send an email campaign, you want to generate an audience for that email a segment. So sometimes using demographics for that purpose is fine. Sometimes actually, it's not fine. It introduces bias. You might be overlooking audiences that you might, you know, might be better to target people who buy dresses recently than just women of a certain age group, right? And so, um the question then is how do you design a product? So that those types of questions which are matters of judgment are front and center. Well, we did a small thing in that product interface, where if the A I generates a segment that has demographics, those demographics are unchecked and you simply have to check them to bring them in. We also have another example, a model builder product um that enables people to build prediction of models or bring in their own generative models into our Einstein one platform. And there again, if you're building a model and you say, let's say you bring in zip code or postal code into your model that can be correlated with race. Again, sometimes that is OK. Sometimes that introduces unwanted bias, we have a little warning toggle that pops up, right? So there are many layers to the question compliance, you know, starting with regulation and a lot of other topics. But the piece that I'm actually most excited about is how you design products that it's actually, you know, what to trust and where you should take a second look and apply human judgments the A I outcome, right? Have you been surprised while designing mindful friction there in terms of things you think you can trust versus things, you know, you can trust. I, I think that the part that's been really interesting is like how do you design um controls that allow the human to look at the totality of what's going on in A I to the point about human and loop versus human at the home. Um One of the things that we built into our N 21 products was an audit trail. Um, so that you can say let's, you, you had a whole email campaign, you generated a million emails. Um, but there were the use 50 over here that got edited consistently before they were sent on to the customer. Maybe you have a problem with the knowledge article that you use. So that's like a signal to you. Exactly. 50 emails out of a million. Yeah, I think increasingly increasingly, you know, that's so we're at stage one, here's an audit trail. Increasingly, I think we're heading towards systems that can detect anomalies like that and encourage and prompt the human to take a second look at it. Fascinating. Do we have another question from the room over here? Thank you. Uh Bartik Organski from lever at lever, we coach human skills. Absolutely love what you're doing. He used the term human intelligence, which is, which is amazing and of course, much more important than in many aspects artificial intelligence for humans. How do you define or measure human intelligence? How long do you have? Obviously this is uh yeah, this has been debated the ages. I mean, I think there's a, there's a fierce ongoing debate about what is the nature of artificial intelligence, what is the nature of human intelligence. But I think the way that we measure it that measured, at least it kind of in the context of what sales force is building is their business outcomes that our products are um intended to achieve. And again, to be able to free up that sort of the service agent working to develop a deeper human connection or for example, we were working with an insurance company, a health insurance company in the UK. And again by freeing up service workers to deal with more routine, to not have to deal with the more routine insurance cases, but then pay more attention to their vulnerable populations and engage with them by phone. It's that type of interaction, those types of human connections that I think are deeply, deeply important. And on a part of the part of the landscape of human A I interaction and human intelligence that I think I would argue it cannot be automated, interesting. So human intelligence is about person to person interaction. Would you say, I mean, I would say that that is one factor of one factor, but the human intelligence is much broader than that. Of course, yes, I want to go back to your academic background. You have a phd from Harvard and this was just uncanny to me when I met you and how unorthodox ideas go mainstream. And you've been working in tech ethics for a decade now and now it's the most mainstream topic that there is. Uh so isn't that uncanny to you? Can you believe that that's what you studied? And here you are at Fortune brainstorm A I in London talking about this and it, it's the top issue in the world right now. I love hearing you say that. Um No, I think, you know, and when I started getting interested in tech ethics is because I saw that there was an, an unfilled need and that this needed to become, you know, super important. Um And so it is very gratifying that it has arisen in importance. And I hope that um I hope that the attention that's being paid to these issues of trust continues and is not kind of a momentary thing because you know, we talk about we, I'm sure all of you have watched these different waves of A IA I summers and A I winters. Um I think there's no doubt right now about the capabilities of A I, right, that you know, that it is the the the capabilities that are proceeding very, very quickly developing very quickly. My concern would be that it's possible that the next A I winter is caused by trust issues with A I or people adoption issues with A I, right. And so it is, it is very important to me that these types of trust innovations in products and the types of uh the types of training and on the job work and the sort of people adoption within companies. I think that's what's gonna continue to unlock A I productivity and gains for companies. Yes, we have just a moment for one final thought from you. One prediction. Of what you see next? You were just saying trust, do you think will be at the center of it? I also think since meeting you that you're kind of a psychic that you, your doctorate, you predicted your future career, you predicted the tech ethics would be the big question. So what is your clairvoyant view of what's, what's coming next? I think. So, two things. It is the sort of the company is paying attention to the people side of A I, I think we're gonna see a lot more attention not only to the interface and the sort of the, how, how you design A I for trust. Um But also how do you bring people along on the journey? How do you allow that? How employees to experiment? So that's one big thing. The second big thing I think is data. We just have not talked enough about data. And I think this year we will start to data governance, data governance as part of the regulatory conversation, which has been largely missing. Um And I think that's also like, I think that that's the second topic. Data. OK. All right. Well, we, we're gonna leave it there, Paula. Thank you so much for joining us. It was a pleasure. Thank you, everyone.