Name: Brainstorm AI London 2024: Navigating Roadblocks And Catalysts To Enterprise-Wide Adoption
Uploaded: 2024-04-15
Duration: 17 min 46 s
Description: Amy Challen, General Manager, Artificial Intelligence Sachin Dev Duggal, Founder and Chief Wizard, Builder.ai Jenny Griffiths, Vice President, Data Science, Oracle Moderator: May Habib

 Amy Challen, General Manager, Artificial Intelligence Sachin Dev Duggal, Founder and Chief Wizard, Builder.ai Jenny Griffiths, Vice President, Data Science, Oracle Moderator: May Habib

Brainstorm AI London 2024: Navigating Roadblocks And Catalysts To Enterprise-Wide Adoption

Photography by Joe Maher / Fortune Brainstorm AI

Conferences

Hey, hi again, everybody. Uh really excited to talk about getting those po CS out of purgatory and into scale uh with a fantastic panel. Uh First to kick us off how many folks have got scaled use cases in production for generative A I in the audience hands up. OK. Just what we thought. Um How many folks if you know, are spending more than a million dollars a year on generative A I? More than 3 million, more than 5 million, more than 10. OK. Well, you guys know what my answer is. Um but uh that's great. Uh uh insight and data folks are early and the, the stat, this is accenture BC G. Lots of folks who pulled executives fewer than 10% of the generative A I explorations are in production. Uh There is a graveyard of unused chat applications when those use cases do go to production. So I do think um we are in a um a bit of a trough from a adoption uh and scaling uh perspective and let us dive into um uh this conversation with, with that in mind. Um You all have scaled use cases in production and I would love for us to actually just maybe rapid fire, share those before getting into what you had to overcome to get to scales. And then we definitely want to take questions in, in this session. Uh Sachin, I'd love to start with you at Builder. What are you guys doing? How are you using A I internally? Um We are proud partners, mutual customers. So um tell us a little bit about how you guys are using A I internally. Sure. So, you know, at builder, we use the entire length and breadth. So, right, the way from, you know what you call classical A I machine learning, both supervised, unsupervised. Um We actually, the core foundations of our platform are built on a knowledge graph. So graph theory is really important because it creates the reasoning engine to give generative A I guard rails for all intents and purposes. Um but also um is a really strong method for reinforced learning. Um And um and then we use computer vision. Um So when we produce applications, you know, the old way of testing was you had a human that looked at an application compared to, to design. Uh It takes a really long time, we can test an application across thousands of devices in 30 seconds using computer vision. Um And then obviously, which everyone's looking forward to is we use a lot of generous technology. Um Now, I would say uh we have our own work especially on the small um sort of small language model side. Uh But the, the, the use of commoditized LLM SI use commoditized Lucy because I know it's a new technology but it's commoditized, it's actually quite important because we work with just about everyone um both from an image video and sort of the the tech side. And that allows us to do everything from user story generation based on ragging with the knowledge graph through to what we just launched last week, code generation. Um where the the our error rate is is single digit percentage um in in code produce because it plugs into the rest of our plumbing. So, really across the spectrum? Awesome. Thanks for sharing, Amy. What about you at Shell? What are we doing? Um We're doing a lot because we have the remit of the whole of Shell, which is a very large integrated value chain energy company. Um but some of our best established use cases and when I say best established, I mean, 10 years, it's things like predictive maintenance which you can use, you could use it in traditional oil and gas space, but also um wind farms, also the Holland Hydrogen one where we're, which we're building uh to be operational a year or two, which will create um green hydrogen from the Holland Coast, North Wind Farm that is being designed to have these integration as well. So that's a use case where we've got um So tens of thousands of pieces of equipment um under predicted maintenance models. Um Other examples are, are, are eb charging network. Um We've got optimized charging across that to ensure that um it will charge as efficiently as possible from where you're at. And then you've got other applications um such as um our marketing and loyalty pieces. Again, really obvious use case people have been doing it for years, but these have been a cut for years. So we've got a long tradition of making all the mistakes putting into production, scaling it. And that has put us in a much better place for the newer use cases and things like TV. I come out, everyone gets really excited. It's the same problem. It's, it's not the technology, which is the problem, it's the scaling, it's the implementation to processes, it's the change management. And so we have got that framework to do this. We have got these practices and we're doing the same again. So it's, it's been a really good couple of years in terms of the improvements in the models and improvements in the capabilities. But we are just putting it into our um, existing frameworks and existing ways of working in order to scale it and get the value. Awesome. We're gonna come back to that. That's great. And Jenny, what about you? You are both building A I into your products and using A I internally. So maybe tell us about both those sides of skilled use cases. Now, that's a really good point. We do have a kind of two pronged approach. Um So my group um I'm in Fusion A I and basically we develop embedded A I applications for Oracle Fusion Suite. Um and that covers pillars like uh human capital management, uh supply chain management, customer experience, uh finance. So loads of kind of different applications, but we try and use the same core underlying technology for loads of different customers and loads of different applications. So it makes us think about how you can design A I at scale. Um And yeah, when it comes to what we've been doing with generative A I, we have 25 use cases go live just last quarter alone, um A lot of others as well. So we're beginning to see this kind of customer adoption. And from my point of view, what's been really lovely to see is kind of people lean in at scale for kind of business knowledge and business application, but also on a more academic level of we now have customers really engaging with us about how A I models are put together what kind of data we're feeding systems with. So we're seeing this kind of business awareness and kind of academic awareness rise hand in hand, which has been really great to see awesome the end user adoption that you are seeing. Let's talk now, just your and your internal use cases for for generative A I, we have heard over a couple of panels around the reskilling challenge and the training challenge, training almost feels like an understatement or the kind of skill that you need uh in a workforce to be able to really use A I as leverage. Right? It's such a different type of technology. Can anybody speak to the types of things they did to really get scaled end user adoption inside of their companies? That would be good advice. So I think, um you know, I I'll sort of start off, we've looked at the implementation in two different lenses. One is where are you using this to remove human variants? And where are you using this to remove entire human workload and they're very different or, you know, tasks that someone was doing? Um And so where you're using it to remove variants, there is a, there's a co piloting or a companion experience, right? Because you're usually arming the human being. So for example, Natasha is ra I and we'll speak to customers, but at the same time, we'll speak to one of our customer success team and tell them when a customer this is, you actually need to answer with this. So there's a, there's a benefit collective insight, but that is very jarring for someone who's not used to having someone listening in their ear and then telling them what to ask. Um and, and so I think there is a, there is an upward training process in that and you need to get people comfortable with the fact that there's an on the call. Uh I think the removing work is a little bit easier because you're basically saying, hey, listen, uh as a good example, you used to manage your day. We have customer success team. They used to manage their day. You're not gonna manage your day anymore. Someone you're gonna have this assistant that's gonna move your calendar around and sort things out so that we can use the right time slots through the right things because they're all connected. Um There, it's more of an education to get people emotionally comfortable with the idea that, you know, they're sort of one step removed from how they were running a particular task. Very helpful. Yeah. And the way that we've been approaching it, at least to my team is that we've kind of segmented it into the different user base and depending on who it is, who will be consuming the technology. We apply different ways of educating the end customer. So for data scientists with my team, this time last year, we introduced, we called it LLM Friday, which was terribly branded from my point of view. Um But it was basically just my team, every Friday just took the time off tasks just to learn about LL MS because it's this huge new technology that's come in and we needed to be experts quickly. So we've gone to kind of that extreme around education, to the kind of day to day education around how our customers are actually engaging with this. So it could be hints and tips around prompt engineering. And we have a center of excellence to be formed to kind of lead people on that journey, help them engineer those prompts. But then, you know, we're leaving that knowledge with them as time goes on and then you've got the actual outreach to customers um doing events, videos and podcasts, basically, any data that we can feed people around how to get the most out of these systems. And that's what we're seeing is really important. Yeah, absolutely. Um I was in a conference recently about 50 A I CEO S and um there was a one of those like Slido, you know, respond to um the survey question and the number of um CEO S that had built uh A I features into their products versus the number of CEO S where there was meaningful PNL impact from using A I internally like 10 to 1, right? And so the, the onus for actual adoption is being pushed on to the customer's customer. Um And but by the A I vendors, um and I, I do think um the, the scale of change is really one of those blockers to, to wide enterprise uh adoption. So I love that you're doing that. Um I do think that um uh inside of even the A I companies, the um the the steepness of that learning curve, right is really being felt, it's only getting steeper back to you Amy at Shell. So you all have got a uh a deep hit history of integrating A I into your processes and getting a workforce comfortable um with relying on A I to make super critical like life and death types of decisions. Tell us a little bit about the framework that you're now adopting in the generative A I world to scale these use cases. What can you share with the audience? I mean, boring answer is not that different. It's like because again, it's like any change program, you just have to go through the steps and it will really depend on the individual use cases as the other panelists have mentioned about exactly, you know, what will be the blocker to change. So for an example, like with the particularly with safety use cases, you might call them parallel running is a really good one. So we have this technology called operator round by exception where you basically complement and operators around on an asset with um computer vision technology to sort of detecting what's going on. Now, you run it everything in parallel the old system, the new system for a long time and identify the variances before you take anything out. Um That can just once you've seen something working for how long you need days, weeks, months people start building trust they start realizing what's going on. They also, it's a big part of like how, you know, you've got the right model, right? This is the feedback loop. Um And I think that's, that's really the key for me. This is agile scaling where you always have a joint team. It's always a technical team, but also a business team. Never do data science without the business. You're always integrating the feed lap loop at every stage and you're always listening to what people are saying and understanding or if it's not being used, why it's your problem, it's not the user problem and then only scaling up and only taking to the next level when you're actually comfortable with that. And I think there's just no way out of that. It's never the case that you get something and it's perfect because it just doesn't work like that. So it thank you so much. OK. Questions. Um We've got time for a couple that is a fast hand. You in the middle. Yes, please uh tell everybody where you're from, who you are before you ask your question, Sam. Yes. Business school Barcelona. So we've talked a lot about employee training, user adoption and so on. What about the executives? How do we get the executives to a point that they can redesign organizational processes, structures and even business models? Oh, I love that question. There are so many execs who have to fill in airtime at board meetings. Right now and those 10 minutes last for a very long time. So, um, he would like to take executive training. So, you know, one of the things that we've said to our executive team and, and that we have a weekly war meeting, which is sort of weekly action review last week, next week. Uh, and I sort of put a man to everyone's like in every, every second meeting I wanna know a problem you wanna solve because I think less about education, less about process is actually first agreeing, what is the problem we're trying to solve? And is it really a problem that needs to be solved? And then, and then the question is how will you use the entire landscape of it? I not necessarily just generative. How will you use the landscape of how to solve that problem? Right. And where do you need it? That's just our experience. Yeah, I love that. Yeah, I'd be inclined to agree actually, if an executive comes to me with a problem, I always try and get to where they're trying to be and what KP it is that they want to impact rather than how they think they want to get there. And that's not because they might, they might know the answer, but it's kind of good for the data scientists behind the scenes to verify the approach that you're taking and that's really come into fruition kind of this year with generative A I because a lot of people think they want generative A I's case. But actually it turns out they want a deterministic output. They want the same thing again and again and then it's up to me to go, well, in that case, you should be looking at plastic A I, and it's not worse or it's not better. It's about that kind of education around which technologies are appropriate or which, which stages and the only way you can get there is actually talk about where you want to get to right at the very end. Yeah, I love that. I think it's really easy for people to come to you and say they want Blockchain and JV, I in an app and you're like, I think you need to business process automation and it's not because people are stupid. It's because they're curious, which is great and then you can have a conversation and we've had no problem at all having interested executives in all of this and the curiosity is great, but ultimately, there's no substitute for expertise. So it's always a joint conversation with people who really know their business about what can be done and what's sensible, helpful. Thank you. Other questions. Yes, back there. Hi there, Mark Riley. Mark Riley from Matheson A I. Um my clients have got clear vision and road maps and plans, but the stumbling block is always identifying ro I getting it past the CFO. How do we, how do we make roi a meaningful prediction. Yeah, I love this question. I can take that too. Um So it is so imperative to get the CFO on the same page and I actually think not getting them on the same page as executives uh really can result in some twisted incentives like execs, not sharing the actual ro I because they don't want the budget taken away from them. And so especially in the early days, um where you want to be able to reinvest savings into more explorations and not disincentivize execs from really taking costs out of the business. Um The Roy conversation, you know, should not be politicized. Um We track Roy with our customers weekly, not everybody is doing that, but it certainly requires being able to partner with procurement and finance and have this be kind of board level um priority uh to make that happen. But it's absolutely essential. Uh I think a big part of why Po CS that seem to be successful at a small scale aren't scaling is the CFO doesn't actually know how much this is going to cost at scale. Everything is priced by tokens and you have no idea what it's gonna cost the organization to run this at scale. So including them is just incredibly important. Um making sure that Ry is part of um the actual POC because that builds the scaffolding for what you're gonna take to production from people to process. Is it so important. Yeah, and, and look, and I, I just said whenever we've done an implementation and we, we do this entire thesis beforehand, which is we think it's gonna cut cost by this. Here is the cohort of things we need to test it across, here's the period we need it for. Um And depending on how close we get to the result that becomes the gatekeeper for whether we move to production or not, you know, if, if the cost to have that ro I obviously exceeds the RO I, then it's not worth doing. Um And I think that's, that's built in a really good discipline to test very early. Uh versus where we've seen in the past is it's tested too late. Uh And at this point, it's like, well, has it not shipped? Versus you might actually make the call and we've done this a few times. It's just not worth shipping. Or the approach we've taken is not the right approach because the more you generate, the more cost you have, but you could use classical A I or knowledge graphs, for example, to reuse something you previously generated and then actually makes it much more economical. Wonderful. Thank you folks so much. Thanks to the audience. Well, that's it.