Name: Brainstorm AI London 2024: Striking The Right Balance In AI Regulation
Uploaded: 2024-04-15
Duration: 18 min 48 s
Description: Balbir Bakhshi, Chief Risk Officer, London Stock Exchange Group Ian Hogarth, Chair, Phasecraft, AI Safety Institute; Partner, Plural Connor Leahy, Founder and CEO, Conjecture; Co-founder, EleutherAI Moderator: May Habib, Co-founder and CEO, Writer; Co-chair, Fortune Brainstorm AI London

Balbir Bakhshi, Chief Risk Officer, London Stock Exchange Group Ian Hogarth, Chair, Phasecraft, AI Safety Institute; Partner, Plural Connor Leahy, Founder and CEO, Conjecture; Co-founder, EleutherAI Moderator: May Habib, Co-founder and  CEO, Writer; Co-chair, Fortune Brainstorm AI London  

Brainstorm AI London 2024: Striking The Right Balance In AI Regulation

Photography by Joe Maher Fortune Brainstorm AI

Conferences

Balbir Bakhshi, London Stock Exchange Group, Ian Hogarth, Phasecraft, AI Safety Institute; Partner, Plural, Connor Leahy, Conjecture; EleutherAI, May Habib, Fortune Brainstorm AI London  

Really excited to get into the conversation. Uh National approaches may differ is an understatement. Thank you, Jeremy. I'm excited to get into it with everybody. Ian, I'd like to start with you. You have been writing about not just the opportunity for A I to transform society but the risks as well for two decades. And you now chair the A I Safety Institute in the UK, $100 million investment pound investment by the government to make sure that the ecosystem for A I in the UK is the right balance of grabbing the opportunity as well as mitigating the risk. So walk us through what you think is kind of state of the art. You guys have uh published the A I progress report, safety progress report. Uh not too long ago. What is the U K's pro innovation stance on managing A I safety? And, and how do you compare it to what the EU is doing and, and maybe what's happening in the US as well? I think um I think in the UK, I mean, the Prime Minister is, I think he would sort of describe himself as a, as a techno optimist. You're very excited about the potential of technology and start ups to transform the economy. And so I think the UK sort of tried to charter a course which really is sort of fundamentally built around empiricism. So um he created this uh A I safety institute um with the intention of basically creating uh research capacity inside government. So we've recruited some great researchers from uh from industry, from academia to come into government. So people like um Yarn Gal who's a professor at Oxford, um Jeffrey Irving, who was previously in charge of um alignment and safety at deep mind, Google, deep mind, um Jade Leung, who was previously at open A I and, and these people have come into government and we've built a research team and the research team is essentially looking at a variety of different risks at the frontier and trying to assess kind of how meaningful they are and then feeding that up to people making decisions about regulation, people making decisions about policy. So the idea is to sort of put, put risk on an empirical basis so that the government can make appropriate decisions about how and when it regulates. Um And so the, the the approach really is building state capacity, building, the capacity to understand what's going on. Um I think in, in Europe and in the US, there's been maybe a bit more of a kind of earlier move towards regulation with the EU A I Act and the Biden executive order, but the UK hasn't yet regulated. Um, we've just built this kind of empirical capacity. Um And in terms of the, you know, the, the innovation risk trade off. I mean, I think, I think that, um, maybe a good place to frame it would be around uh the ability of A I systems to write software. So I think it's something that potentially is very, very exciting because there's so many things that we could build if we, we we had more capable kind of software tooling. So, you know, I remember when I first kind of encountered more powerful programming languages, it was like, it was kind of like having a superpower, right, the ability to do, you know, right, code faster and, and, and and ship features faster. And so I think that for example, A I that makes coding more productive is, is potentially really, really positive in that we can generate loads of new applications and value for society. Um Obviously, there's always a risk with software which is, you know, um cyber crime, cyber attacks. And so, you know, one of the work streams that we have at the Safety Institute is just understanding how much these kind of frontier models would give a sort of malicious actor uplift in trying to perpetrate some kind of cyber attack. And we work with uh you know, we work in tandem with organizations like GCHQ and N CS C National Cybersecurity Center on that. So that's a good example of an area where there's lots and lots of opportunity and lots of opportunities for value creation for society. But then there's also this risk to mitigate. Yeah, thank you, Ian Connor. You have really, I think um kicked off the initial open source work with GP TJ and we build models at writer and I remember looking at that project um with, with a keen interest to see how it develops. Um You've also expressed grave concerns. I believe the exact wording was God like a I could be super. Um So there's definitely um a AAA large plurality of um A I researchers who, who agree with you, you've since started lecture um which is a, a effort to build models with bounded. Can you walk everybody through what you think that means? Um Do you still believe you guys are two years in? And it's, this is a, a for profit company. Um Tell us about what you have learned um on the frontier side to Ian's point around the potential to build in benevolence to humanity inside of these, these uh non deterministic model. So fundamentally, the way I got into a I, originally when I was a teenager was I was thinking about how can I do the most good for the world? How can I solve the most problems eventually? I figured out, you know, I'm just one guy with one brain. But if I figure out intelligence. Well, then I can make a lot of brains and they could work on all the problems. I could solve all the diseases and all the problems. Great. So I'll go do that. How hard can it be. Um, and so turns out relatively hard, but maybe not as hard as you might think. Um, so when I eventually though I figured out that I was thinking about that, the hard problem of A I of A G I is not how to build it. It's how to control it. It's much easier to build something that is intelligent, that does something than something that does what you want. Because think of yourself like what is what we want? That's a really hard question. Even someone we love very dearly, we could sometimes hurt them or we can misunderstand what they want even if we're trying our very, very best. Now, imagine a system built by for-profit companies to maximize profits. But presumably, uh what do you think that thing is going to do? Who knows if we build systems that are smarter than humans, which is the intent stated goal of all the major players that's better at business, science and technology, politics and everything else. And we don't know how to control it, which we don't currently know how to do. Please think for a moment, what do you think happens? Seriously? What happens? And so this is the question I, I've always been working on. Well, then what so well, even having the luxury of it, then what is a luxury? So conjecture, we work on this problem on the tech, what I call the technical problem or even hypothetically how can you make an artificial intelligence system do what you want? What does that even mean? So we focus on a sub problem of this which we call bounded where the question of how can you know what a system won't do? So this isn't even asking what it can do or guaranteeing that it will do the right thing. It's just can you guarantee that it will never do specific things? This is currently extremely difficult as I'm sure anyone here who has tried to use a GP T model in production can tell you. So we work on um this is a paradigm that we personally call cognitive emulation where we try to work on very, very precisely emulating the exact cognitive processes that humans use using LMS trained very, very, precisely on very, very precise tasks. So we know what these systems can and cannot do. This is not a full solution to the problem. It is a an important step forward to how can we get systems that in your business and your applications to bring you real value, do what you want them to do, do it in ways that is legible and comprehensible to humans and they don't go off the rails not enough for a good future in totality. Of course, since you know, as we are on a panel here about regulation, the problem of A I in general of full job automation, which is something that we will see in the near future is a question that is not just technical, it is also a social and a political problem. It is a humanist problem. And you know, techno optimism is nice. I love technology. I have worked with technology my entire life, I've worked with open source. Um I've been deeply involved in movements. I love technology, but uh techno optimism is not a replacement for humanism. If we just build better and better technology, it does not mean we may build a better future that would make a better place for people. We will make a better place for capital. Let me tell you those stock prices are going to go up. I guarantee you that. Now that's the prediction I'm happy to make. But will it be a better world for us to live in? I think that is a very difficult question. Yeah, I appreciate that. Um that perspective, the the gap between the the haves and have nots in a I enablement. Um When we think about like people at work who can be as productive as the next 20 or 30 colleagues and, and folks who, you know, haven't learned those skis. It's definitely not a recipe for, for social cohesion. And you think about um the government's role in regulating the social ills and harms of social media. And it is not a pretty picture of, you know, their role to be able to really bring the best of this next um platform shift in that um uh Harry World Bill Beer, uh as chair of the Responsible A I Institute of the group at um London Stock Exchange. Um you are really in the middle of trying to figure out how to incorporate A I internally in uh in, in, in your business while keeping, you know, one eye out for what the government is doing and is likely to um to to regulate right in the UK, nothing is law yet around um where and where not to invest and then also where the models and the technology is going. So um take us through, you know, a day in the life of beer trying to really help hopefully like push the envelope internally on the innovation side while balancing um all of the the risks and potential harms. Yeah. Thank you. I mean, se is an exchange, a clearing house, but most of our business is data and analytics. So, um when we started looking at the control side of things at the beginning of last year, um we established a set of principles aligned to our values responsible A I principles. Um and then we set up a steering committee uh to really govern what, what was going on because there was a huge, huge amount of hype in Q one last year and we wanted to get a handle on. OK, what's going on in a 25,000 person organization? How can we control it? So we started, so I I established that as an oversight group, not be because, because I'm the risk officer because this is a truly multidisciplinary activity and we see it fundamental to business transformation. So we had a, an overall um steering committee that really collected use cases, evaluated them, looked at feasibility commercially as well, but also looked at the risk. Um and then a product level working group, an efficiency, working group and a responsible A I working group. And that's really the construct that we've used ever since. And we keep the principles updated as new regulations come out. So they're in line with the EU A I Act as N brought out standards, we've aligned them to that. So all of these are very helpful, but we largely have taken a self regulation route. And in terms of managing the risk, I think our approach, there has very much been upgrading, constant upgrading of our capabilities, so particularly data management, security, but also model risk management and thinking about how, how A I systems can regulate other A I systems. And we've recently tested a couple of our LM models. So we're working very closely with Microsoft in developing A I tools that really expose our data and analytics and make clients lives easier. And then we're also working on tools to make internal efficiency improvements like a Q A tool to improve customer inquiries. And as we looked at our model validation approach and designed that in conjunction with some professors, we used LM as a judge as part of that as well as human oversight. And that showed interesting results particularly for out of domain questions that we might throw at a model. So that actually didn't surpass our expectations, but actually was very helpful in feeding back into the model development life cycle. So very much evolving uh the risk management approach at the same time. But, but the self regulation approach I think is really helpful, keeping it updated as new regulations sort of and principles arise. I love that 11 framework I've really found helpful in, in talking to execs about um A I safety is the concept of guardrails. And I think what I'm hearing here is there is a level of guard rails that you know, the the the tech vendors right are going to be competing to build into their systems. There is a level of guard rails that responsible executives like beer are going to be building into their own responsible A I and governance frameworks at a company level. And then there are A I guard rails that the government should be uh insisting on. And so with that framing in mind, um could we do rapid fire maybe starting with with Connor, you know, what are the, the, the top one or two guard rails, right? Uh If you do the, the vendor level, Connor, uh well, go then the bal bear government level and then, um our company level and then, uh Ian with, uh with government level, top one or two guard rails that folks should, should be building into products. Oh, God. Um uh I mean, for your own business, this highly recommend that you do not trust any outputs of LMS in general. I don't think there's any clear technical solutions there other than, you know, filters and the like, but even though it don't really cut it, if you're asking about the general societal level questions, these are the kinds of questions that I don't think companies should even have to answer. It's not, this might be controversial but it's not, shouldn't be the responsibility of oil companies to solve climate change. It should be the responsibility of government to stop oil companies from calling in climate change or at least make them pay to clean it up after they cause the mess. So really guardrails should be something that is coming down from the governmental level in terms of societal level risks. As for risks that apply to like a given business application or given thing. I can just say like, uh you know, careful what your consultants tell you A I is super unreliable at the moment. It is not a human level of reliability at the moment for basically any application that isn't fun fiction writing. So, just be very careful there. All right. Well, Connor is throwing the responsibility over to you guys and what are the top one or two guard rails that you would you think the UK? Let's start with the UK. Um, or keep it to the UK. Should, should insist on, I think that, um, I would say in most of our efforts right now are really trying to understand whether large models um enhance any existing national security threats. And so that would, that would seem to be a really clear area where, you know, we've already, we already are doing work to mitigate against, you know, for example, cyber attacks. And so understanding how a model can and can't contribute to some sort of malicious, um you know, code based attack would be the kind of thing. But again, like, I think we're still quite early in building the empirical base for talking about this stuff. Um But, you know, the great thing about what we're doing at the Safety Institute is we are working very closely with the, the experts in the UK national security community. People who are experts in biosecurity, experts in cybersecurity and nuclear security, who can help us kind of understand the level of uplift these systems truly provide because they think about these risk day in day out. Um I think like that, that's the sort of I guess the easy answer, I think the sort of thing I think we would want to be really on the lookout for, with these models is something like deceptiveness. You know, you wouldn't want a model that is deceptive. Um, and that's the, that's the sort of thing that's a bit broader than national security that I think it would be very important to, to, to, to sort of put guardrails in around. Yeah. Thank you. Yeah, I think um so we would worry a lot at a corporate level around being responsible for any misinformation, disinformation. Um So in, in terms of ensuring that you, you're putting things out there for people to make decisions on that are robust, I think having watermarked content and auditability. So, so that's what we're trying to build into our A I tools so that clients can see what was used to produce the output. And I think um just in terms of copyright type issues, I think having really strong data management and tagging your data as to what's the licensing agreement you have and where, where can you use it and who can have what, what are the permissions associated with that data? Give you just a bit more robust assurance that what you're producing at the end doesn't infringe anyone else's copyrights. And, but both of these areas you're already seeing some litigation that is concerning for organizations. So I think data management is, is probably quite fundamental. It's the fuel for, for A I. So getting that right. I think puts you in a much better place uh in your usage inside of London Stock Exchange. Have you created a guardrail around using models that are not trained on copyrighted data? We're seeing that now in some artists that are not trained. Yeah. So, so I think there is um have you gone that far? We are the main thing that we are focused on is actually um training models on our own data and a lot of our data, we actually contract some, some we create ourselves and some we actually have license agreements with third parties. So in doing so, we've been really careful. It's very easy for our own data. But where we actually contract data from somebody else, we're very careful about how and when we would use that within the training of, of an LLM that we would use with a client so trained and not fine tuned. Yes. Yeah. Very interesting. Well, I can't believe it, but that is all we have time for. Uh would love, I would have loved to do a Q and A session with everybody, but we'll have to save that for the break. So, thanks everybody.