Name: Brainstorm AI 2023: The Infrastructure of AI
Uploaded: 2023-12-12
Duration: 19 min 6 s
Description: Matthew Prince, Co-founder and CEO, Cloudflare Jennifer Tejada, Chairperson and CEO, PagerDuty Moderator: Michal Lev-Ram, FORTUNE

Matthew Prince, Co-founder and CEO, Cloudflare Jennifer Tejada, Chairperson and CEO, PagerDuty Moderator: Michal Lev-Ram, FORTUNE

Brainstorm AI 2023: The Infrastructure of AI

Conferences

OK. So on to the next session, when your organization's role is to keep your customers safe and protected. Implementing new technologies requires a secure foundation. It's incredibly difficult to unwind data once it's been integrated. And it's therefore very important for companies to protect their IP and trade secrets while putting guard rails in place to protect against bad actors as we were just talking about. So how do leading players deploy and harness the power of A I safely joining us now for a conversation on the challenges, strategies and solutions needed to build a strong A I infrastructure, please welcome Matthew Prince, co founder and CEO of Cloud flare which helps protect its clients by serving as a reverse proxy against web traffic. And Jennifer had chairperson and CEO for the incident response platform pager duty and they'll be interviewed by fortunes editor at large Nihal Lavra. Welcome. Um OK, so lots of talk about, um you know, concerns about ethics, philosophical kind of disagreements, um uh about A I but we're going to talk about nuts and bolts, which I know sounds very, very sexy. Uh but it is very important. So I, I wanna hear from both of you. You have so much, um you know, different perspectives, both internally your organizations, but also customers because the customer ba base that you support is, is very, very broad. So when it comes to infrastructure and deployment, what is, what are the top concerns that you're hearing and that you have today? And Matthew, maybe we'll start with you. No, I think the first is just access to it. Um We are still in a place where access to GP US is limited, although that limit is somewhat artificial. Um If you look at the big cloud providers, one of the things that the decisions they made early on was to make it expensive for you to move your data from one region to another or from one cloud provider uh to another AWS for example, marks up their transfer costs of data by almost 4000 times what their underlying costs are and what that is doing in the A I space is actually further artificially constraining access to GP US because there's GP U capacity across the different clouds at any given time. But getting your training set to those places is often very difficult. I think that's the first thing that we're hearing. I think the second thing which is starting to come up more and more and especially in the last 72 hours with the starting to internalize what the EU is doing around regulation is A I today is very much the infrastructure, 95% of GP U capacity for A I is being deployed in the United States. And I think that the rest of the world is waking up and saying, we don't want to make what they see as the mistake of the first generation internet where basically all the data cap back to the US, they want to not repeat that mistake going forward. And so whether you agree with that regulation or disagree with that regulation, I think more and more you're going to see that regulators are going to say, how do we make A I local and how do we make sure that we have some peace and some control over all of this innovation that's going forward? Sure, I think a lot of our customers are working through trying to find a path between research and operationally delivering products and services and changing the way their people work, looking for ways to keep strategies simple. But having to assess the capabilities you have within the organization, understand the uh data and security parameters with within which you need to work, training uh both your people uh and making sure that they're executing within guardrails. Uh but also not losing sight of the opportunity to really change the way they work. And even uh preparing your board members for dealing with some of the new issues that arise as a result of using generative A I in your work today. So there's the technology infrastructure. And then there's also how do we, how do we learn, how do we operate, how do we iterate whether you're a large company that is super conservative or a company that sees this as a game changer and is going to move really fast. So good segue to something that you've brought up before Jen, which is kind of this, you know, FOMO fear of missing out versus fear of jumping in or getting in. I'm not gonna say the acronym because it sounds too stupid but, but, but, but talk a little bit about that um that balance and sort of the pressures for, for both sides. How are organizations looking at it? Yeah, and we, we hear different things from different customers. Some customers are taking a path of um trying to enable every employee in their organization to get some practice and start finding ways to create efficiency and productivity within their teams to really embrace a culture of automation. Uh And then we see other customers who are saying uh you can't use generative A I within your team, you can't use it within the, we, we won't allow our employees to use your platform if you have it. And it's that um fear of the risk that is keeping them from some of the benefits of generative A I. And what's interesting in that case, when you take that risk, adverse stance, your employees can use it as an excuse it gets in, it becomes a convenient excuse for not moving as fast as maybe your competitors will move. And so, you know, what we advise our customers to do is find pockets where you can start test and learn and, um, understand some of the things that are going to work for you and some of the things that will not and then apply those against different use cases across your business. Matthew, you, uh, always have a, an interesting take on things. Um A and, and you've, you've talked about um sort of the, the share, the concern that a lot of the experimentation may not work out. So what's your approach to experimenting and, and what advice do you have for people in the room? You know, I, I um I think if we're all totally honest with ourselves, most of the people in this room, if they're doing anything in A I are effectively just lighting money on fire. Um And, and that's, are you guys happy to hear that? Um Sorry, uh Some of it will work, but the real challenge of A I is that the demo is easy, but production is incredibly hard. And so you're seeing all these incredibly flashy demos that people are, you know, selling their boards on selling V CS, on raising money on. But actually turning that into something that generates real value. That's where the gap is, what's here. And what I think is really tough about this space is we are so used to as product managers, as founders, as engineers working in incredibly deterministic systems, machines. When you tell them to do something are supposed to do the same thing every time you tell them to do it. And what I don't think we've gotten used to is that these are incredibly non determinist. In fact, they're by design non deterministic systems. And as a result, you know, in the security space, I try to understand all four corners of every application that we're securing. I want to understand what every input and every branch on the tree of the output that is impossible in A I. And so I actually think that as we think about this space, we're going to have to have different product managers that think about the space in a different way. We have to think about regulation differently. And how I've started to think about it is actually how we put the guardrails on A I look a lot more about how we put the guardrails on our people within organizations rather than the machines within organizations because A I systems actually behave unpredictably in the same ways that most of your employees behave somewhat unpredictably. Isn't that one of the most exciting aspects of this though? At the same time, the opportunities that that can present again, it depends on what you want. If you want, uh you know, a system where when you turn it on, it does the same thing every time and it's very predictable and it's solving a particular problem. A I is not very good at that. If on the other hand, what you want is a creative agent, then I can do that. Now, I think that then starts to get us down the path of what is a, I really good at A I is really good at replacing people and that's a very uncomfortable thing for us as a society to get used to. But that's what it's good at in a way that traditional deterministic machines haven't been as good at. They're much more tools whereas A I is in a lot of ways replacing it, potentially a lot of the creativity that we today rely on human beings to do. And as a result, it has different guard rails that you can put in place because it can say completely crazy things. Would you like to? I, I would, I would, I mean, I, we, we hear a lot of concerns about, oh my gosh is A I is A I gonna replace all the developers in the world is A I going to um take control over the world uh in, in some surprising and, and unknowing way. I mean, I'm generally very positive about the opportunity that, that generative A I and a more broadly um creates because a lot of what we see today and you heard Aaron Levy talk about it earlier that you still need a human in the loop to apply judgment and empathy and deep layers of context that a large language model can't today produce for you. And so um our belief is that as, as generative A I advances, yes, it's going to replace a lot of the toil and a lot of the meaningless less fulfilling tasks and jobs that we don't really like doing. But at the same time, it's going to create capacity for us to do more interesting things with human effort and human capacity and human context that we might not have been able to get to for many, many years. And so there's an acceleration uh that I think is really, really exciting, it needs to be monitored. So it's, it's another one of these technologies that you've got to have uh systems and automated instrument, instrumentation, tracking and managing. And you've also got to put your businesses in a position to respond to issues as they arise in real time. And so I think we also have to change the way we operate in order to support our teams and our people and our businesses and having success with A I. Um please raise your hand if you have a question for Jen and Matthew, um which we'll get to in just a second. But on the, on the, you mentioned toil and on the kind of experimentation side and trying to move forward with some, some innovative approaches, um At least when it comes to productivity maybe internally. Um you've talked about a a toil bucket and encouraging employees not to be confused with a toilet bucket. Very, very different. What is toil? So what's the bucket list of all the crap you hate doing as an employee? And can you apply some form of automation or a I to move that off your plate? Right? Like that's one interesting place to start. We started having this conversation because we found that employees that are closest to the business problem that understand the most about the business problem often come up with the most practical use case automation or for generative A I or A I more broadly, it's not necessarily the phd that, that does that it's somebody who actually is feeling the pain of the problem. Most founders start their company by having exposure to pain and agony that they want to make go away. So it's, you know, what is your bucket list for the toil and you know, the unfulfilling work that you'd like to never have to do again and start there? And there's remarkably a lot of it still around. Are you moving forward with? I mean, are any of these generating, like actual new processes, new ways? For sure. I mean, like some of the things that even we're doing within our own um products, one of the things that we're using generative A I to do is to draft the first version of the post mortem after a major incident which today takes multiple teams across multiple systems many, many hours times days to just capture what actually happened. What did we do? What did we learn? What remains to be fixed or changed? So this won't happen again. It's a very painful manual process that requires a lot of um you know, integration, coordinations between teams and now gender V I captures everything that went on in our platform and gives you the first draft within a few minutes. Questions for Jan and Matthew raise your hand if you have one. OK. I'm gonna ask you about uh another concern that's come up multiple times, which is vendor lock in. Can you talk a little bit about Matthew uh Matthew about your approach as a company? I know you've struck partnerships with so many different companies in the space. Well, you know, I think that um I think the good news is, this is a new space and that there's a lot of innovation. I think that right now um access to GP us is, is a lot of what's driving this, especially if you're trying to do things in the training space. Uh And the cost of that are just so, so, so enormous that it's very hard to compete with that if you don't have the capital uh to do it. I think over time, um You know, the great news is GPS are just silicon and over time silicon, we just watch time and time and time again comes down, down and down in price. I think the thing that we're trying to do is across the space, make it easy for data to move to wherever the resources are that need it. What I think is the biggest thing holding A I back is somewhat counter intuitive. I mentioned it already but the fact that you have to pay to take your data out of the big cloud providers, but you don't, they don't charge you anything to put it in even though that's a 4000 times mark up on what their underlying costs are costing the same amount to take the data out is put it in. And yet you have to pay that. That's actually restricting and locking people in. If I were a regulator, I would have one very simple regulation, which is the cost of taking data out should be the same as the same as the cost of putting data in. If you have that, we have a lot more competition in the space on different companies being able to compete with their individual models with their individual systems. And we wouldn't sort of have the sort of mediocrity of every different Amazon product which is out there. We'd have competition on every model. We'd have competition on every data storage layer. We'd have competition on all that space and you'd be able to mix and match and create better products. That's the best thing that we could do for the industry. That's what we're trying to drive forward. And I'm hopeful that more cloud providers will follow in our footsteps. And from your perspective, Jen, I think it's important to take a platform approach to this and, and create optionality for yourself in the future. Because to Matthew's point, we're very early and you can already see how one version of an LLM you know, surpasses another and then the next one leap frogs and the next one after that leaves, we're gonna learn a lot in the coming days, weeks, months and years. And we also learned a lot from cloud locking. And so creating optionality for yourself and taking more of a platform approach to this, I think gives you the most flexibility to apply the right technologies to the different types of problems you're going to try and solve because they don't have equal cost cases, they have differing degrees of difficulty in usage. Your people are gonna need different things to uh solve different problems. And so I think taking that platform approach and, and giving yourself the opportunity to sort of future proof uh where your business is going by working with a number of different partners. OK. And we have a question over here. Hi, I'm Anne Michael American Institute of Physics Publishing. And I really want to thank you both Matthew something that you said though really hit me and that was the idea that Oh, you're just setting your money on fire. And it was about scalability and I totally get that. Especially if you're seriously along the path, maybe where you are, where you're developing products, we're 100 and 50 people. And I think our primary use case right now is actually learning. So we are spending small amounts of money on, not what I think is on fire because we're going to ultimately have to pick between different providers of different services. And we can't be stupid when we do that. And so I'm just curious about where you see learning and education and being a smarter consumer because we're not going to build everything ourselves. Like where does that fit in? Yeah, I think again, I think that um if that's what you acknowledge you're doing, there's incredible value in figuring out how these systems work, figuring out where the edge cases are building those, those systems. But when I talk to some, you know, agricultural company and they say we're building our own LLM model. I'm like, mm probably not the best use of time. I, I do think that there is real value in education, getting people trained up on this. And I think especially making sure that we understand that the engineers, the product managers, that the people who are going to be using these systems in the future, understand that the difference between deterministic machines and non deterministic machines and both of them have a place and both of them have a use but they have very different parameters and one is actually, I think much more close to being like a huge manager of humans. I mean, I think A I systems in their non determinism almost act like human beings in a weird way. Whereas deterministic sort of the things we're much more familiar with as engineers have a different set of skills and a different set of constraints. My guess is that the education system is going to play a big role in making sure that we understand how to work with both of these tools in order to get the most out of them. Yeah. And I would just add to that, don't limit your ability to learn and the pace of your learning to just what you can do inside your own organization, apply a growth mindset. I mean, one of the things I've loved about being in Silicon Valley and being able to collaborate with people like Matthew, we compare notes on things all the time and we learn faster together, then we will learn individually on our own. And yes, you've got to be aware of competitive pressures and challenges, but at the same time, like, do you have to make every single mistake yourself? Do you have to learn everything the hard way? Can you apply that growth mindset and try and learn faster by learning together as a community? Um I, I'm guessing most of the people in the room. The, the investors here are not investing in agriculture companies that are building their own. Llm si, I think, but any other words of, of advice real quick, you know, I think it's, again, it's, we're in such early innings of this space. It's going to be interesting to see how it develops. I do think that fundamentally, we've got to think about whether it's the policy questions, whether it's the internal questions, thinking about how these systems work. They're not the solution to every problem, but they are a solution to a different set of problems that we haven't had before that we've often had to throw people at. And so I think that's going to open a bunch of possibilities, but it's going to require very different skill sets in order to make them 100% going, going to require very different skill sets and going to require a lot of practice. So you have to keep experimenting over and over again. You've got to give people the space and time to practice with different technologies and learn what they do and figure out the best path to take. And I think it's really easy to get somebody to try something once. But are they in it every day? Are they in it every week? Are they, are they really building a skill set around it? So do that without setting money on fire. Thank you so much, Jen and Matthew. Thank you for having us.