Name: Brainstorm AI 2024: The Essentials of Building AI
Uploaded: 2024-12-10
Duration: 16 min 35 s
Description: Daniela Braga, Founder and CEO, Defined.ai, Benjamin Plummer, Chief Executive Officer, Invisible Technologies , Jonathan Ross, Founder and CEO, Groq, Rana el Kaliouby, Managing Partner, Blue Tulip Ventures; Co-chair, Fortune Brainstorm AI

Daniela Braga, Founder and CEO, Defined.ai, Benjamin Plummer, Chief Executive Officer, Invisible Technologies , Jonathan Ross, Founder and CEO, Groq, Rana el Kaliouby, Managing Partner, Blue Tulip Ventures; Co-chair, Fortune Brainstorm AI

Brainstorm AI 2024: The Essentials of Building AI

Conferences

Daniela Braga, Founder and CEO, Defined.ai, Benjamin Plummer, Chief Executive Officer, Invisible Technologies, Jonathan Ross, Founder and CEO, Gro, Rana el Kaliouby, Managing Partner, Blue Tulip Ventures; Co-chair, Fortune Brainstorm AI

Daniela Braga, Defined.ai, Benjamin Plummer, Invisible Technologies, Jonathan Ross, Gro, Rana el Kaliouby, Blue Tulip Ventures

I am so excited for this conversation. Thank you for being here. So A I is making us all more productive, but man, it's so expensive to make. So I thought this in this conversation, we would go behind the scenes and talk about what it actually takes to make A I. So the data, the humans and the compute. And Jonathan, I wanted to start with you. Um People often think about Grok as a hardware company, but you actually insisted on not being on the tip panel at this event. Um You're more like a full stack solution and I more like Invidia actually, and and I wanted to point to one particular data point. So Grok cloud platform is used by more than 650,000 developers to build a wide range of A I applications. So can you explain why this software hardware combo is so important, especially in A I inference, which is what croc focuses on? Well, um most people don't know this but am DS GP us are actually faster than Nvidia's GP U SI actually had a VP from NVIDIA brag to me about this fact and then said it's actually their software that gives them the advantage. So when we started GROK, we actually spent the first six months working on the compiler and only after we were able to take programs and, you know, lower them into something that we thought we could build and run a chip with. Um did we start designing the chip? And so it was a massive advantage, follow on question. So you're often thought of as invidious challenger and there's also a lot of new entrants in the space like cerebra and etched and Amazon. What is it gonna take to win? Ok. So, um you never wanna directly compete. So we like to say that we do inference and they do training, they would love to do inference too, but we do fast inference. And why is that important? Why is fast inference important? Um Does anyone here want slow A I please raise your hand, you know, no hands. So just imagine if you're doing a Google search and it took 8, 1040 seconds to get an answer, right? That would be intolerable. And that's where we're at with A I today. So with us, we're about 2040 times faster than A GP U for these answers. It allows you to do a lot of the system two thinking and, and other stuff, improve the quality of results. Um But speed also results in increased engagement. So roughly every 100 milliseconds of speed up is about an 8% conversion rate increase on desktop and 30% on mobile. So it matters a lot amazing. Danielle, I want to come to you next. So we've known each other for many, many years. It's great that you're here. Um Earlier this year you presented at the United Nations. And I want to share a quote from that um um conversation. The race for data without permission is only broadening cultural language and gender imbalances because internet content is just a reflection of our society full of biases and misinformation. Your company defined A I is the largest uh marketplace of ethically sourced data and models. What is ethically sourced data mean? And why is it so important? Thank you. I'm very proud of that and actually very excited to hear the presentation before on pro pro rata uh A I which is in line of our belief that uh just because data is public, it doesn't mean it's free. This is always what I've been saying. And now it comes to fruition. Uh We uh created an ecosystem of a marketplace of training data that allows everyone to monetize their data. Uh And we as the brokers, in that sense, we vet legally the data, we make sure we can trace it back to the consent of the participant level. We apply a price, we add all the machine learning readiness and we sell it to a willing buyer that cares about brand reputation and not being sued by copyright infringement. Uh But it's beyond that, we have to take care of. There's another area which is the, as you, as you also mentioned, the uh if everything that is in the internet is biased by definition, it's mostly white male generated, uh it's English language generated mostly uh we can never have. Uh it's, it's just always augments the divide of our society. If we don't bring biased data, purposely built uh and ethically sourced with everybody paid on the chain to the models. And finally, the part that is hidden and nobody likes to talk about is is uh the humans in the loop component, which I guess is that about you uh where a lot of this work is being done under what is called digital. Uh Well, digital sweatshops uh in exploitation of people in developing and uh third world countries uh exposing them to very, very low paid jobs and very harsh conditions, especially uh including in the content moderation world, including uh psychological harm. So those three pillars is how we build our world and our marketplace and how we've been definitely want to come back to the digital sweatshop and the humans behind that in a second. But, you know, I'm an investor now in A I companies and you've actually been quite critical of the A I investment landscape. You've um you've talked a lot about how billions and billions of dollars of funding are going in to fund the same people, the same ideas over and over again. And I, I mean, how do we get to this world where we are actually funding underrepresented humans, applications problems? I, I did mention that too in that United Nations talk. Uh, pretty cool. By the way, I highly recommend it, especially, uh, there, if there, of course, this is a capitalist world and venture capital is focused on making, uh, money which, by the way, so far, very few A I companies make money. That's the other ironic part here. Uh The reality is uh if there must be an agreement and this is why it was important to speak it at the United Nations level where investors have tax breaks to, to incentivize them to look into other fields. The, the agriculture field itself has 1% of investment uh in relation to A I and it's what sustains humanity. So everything is very uh unequal in our world. Uh uh I think that's it. It's at the government level, it needs to be uh incentivized with money, which is what people understand. Then let's talk about the human is the loop. So invisible, you're the CEO of Invisible Tech and it's probably one of the best kept secrets, I think in the A I world, uh you're already profitable, you started off as a work for workflow automation company, but you actually also do a lot of work behind the scenes with open A I and you know, cohere, what do you actually do with an open A I give us an example. Yeah. So I categorize what we do in two broad areas, one loosely defined as sort of evaluations and understanding how these models performing, where are they strong, where are they weak, where are the gaps in their capabilities? So that they have a much better understanding of how these models are performing in the real world. And then the second part of that is creating really high quality data that can be used to retrain those models and actually close those gaps. And you can understand that there's a really symbiotic relationship between those two things and the faster you can crank that fly wheel, the faster you can drive model performance and improvement. Now, you also recently and I don't know if this is public or not, but you recently signed on NVIDIA. Um What are you doing with them? And shouldn't you then work with Rock? Yeah. Look, we've um you know, over the last couple of years worked with the majority of the frontier foundational model providers and it's we've really specialized in the sort of leading edge of that frontier, the most complex work. Um the most sort of intricate training of those models. And so any company looking to build a world class foundational model is either a client or a potential client. Um And given this is so uh exploratory in terms of trying different techniques and still very much in the R and D phase. A lot of that is co creating new capabilities. We're working with researchers understanding their goals and, and building evaluation suites or building data that they can use to actually close the gaps in model performance. Um I'll come to audience questions uh soon. So tee up your questions, please. I want to come back to this digital sweatshop because you employ thousands of humans around the world to basically do all this work. How do you think about what Daniella said? Yeah, it's really interesting because for us as a business, we've never really felt conflicted about needing to sort of decide what's best for our business or what's best for the people that participate in that ecosystem because we know that quality is paramount that the highest quality data is absolutely necessary and to get that you need really happy, really engaged people. And so we spend a lot of time curating this community, investing in them, training them, making sure we understand we're paying well above living wages and really investing and building that out. Yes, it's a good thing to do, but it's also good for business in that these engaged people produce higher quality results. I think probably the most surprising was when I discovered that a few of them had gotten tattoos with the invisible logo, which was probably a little too far in the engagement side of things. But um yeah, really a fantastic community that's loyalty. That's next level loyalty. All right, we have a question here. Can you please say your name and your affiliation of investor in? My question is, you know, having been in the semis space for three decades, I cannot recall the last time a semis start up made it. It's a scale business um working with the foundry and the supply chain and so on and so forth. So while I love LP U and NVIDIA is showing the way Jonathan, how do you think about scaling vis A vis NVIDIA ne MD for that matter. But I don't know if you intended to t me up perfectly, but I think you just did. So I appreciate this. So um A I is a scale game and if you can't get the scale, there's no point. Um One of the unique things about what we did was we actually designed our chips to be using 14 nanometer, which is an old technology. It's under utilized. The fab that manufactures our chips is only about 50% used, which means that next year, if you look at that, if you look at our contract manufacturers, we actually have the ability to scale up to over 2 million of our LP US physically, we can manufacture that. So what we've been doing to get the scale is we've been working with partners. So actually, I've been playing around with this token. So uh if some of you look at me on linkedin. Uh you've probably seen this. So this is my 25 million tokens per second target. Everyone at GROK carries one of these that scale. Uh So when you produce to uh output from these models, they produce tokens and a token is about 1.3 tokens is a word. And so um 25 million is about where open A I and Microsoft combined were at the beginning of this year. And so our goal is to get there, we'll get there by the end of next quarter. So that'll make us hyper scale or scale. But I also happen to have this other one which is um 1 billion tokens. I'll even do this 1 billion tokens. And so this is with a ROM code digital and we're working together to do this. If we do that, that'll be more than all of the other cloud providers combined. And we're, we partnered with them, they're covering our cost to deploy and then we, we split the profits of that and it's a very um beneficial arrangement together because we're really the only ones who can get to that scale. Um NVIDIA has a lot of obligations with existing customers and we can actually build as much compute as all of NVIDIA combined. Wow. You know, I, I have to ask a follow up question. You're very passionate about, about democratizing access to A I. Can you talk about how the scale will be a path to doing that. Yeah, because right now if you want to get access to a GP U, you have to wait in line and it's a long line and it's not a very transparent line. You don't know how long you're gonna wait. Um So if we, so right now Rock gives away about four times as much uh many tokens for free every day as GCP does and we're a start up. And so our intention is to get to a point where we make enough money where we can give access away for free to everyone in the world. Just like you go, you know, into any room here, you plug into an outlet and no one's going to be upset, right? They're not going to say you're stealing our electricity now, for those who are using a large amount of tokens they will pay, but that will cover the cost for everyone else. Amazing. Do we have more questions in the audience? Uh I'm not seeing any right now. OK. Let's talk about jobs. Um What do, do you both see kind of an evolution of A I jobs that are created um in A I that that's needed kind of behind the scenes to train all these A I models and create all this ethically sourced data. Yeah, I mean, we've already started to see shifts and the demand for certain jobs is, is gone up and being created and the demand for other types of job translation and some of these things is, is clearly gone down. And that's not that different from, I think a lot of sort of technology shifts. Um what is different about this one is a, the speed I think these shifts have sort of happened um much more quickly than typically. And this, the second is this really unique attribute about A I and the technologies and their ability to democratize knowledge and skills and capabilities. And so you have people who might not have any technical knowledge can create a website and create these technologies. And so I think it would be really interesting to give these powerful tools to people and free them up to go back to inventing and creating new things and, and removing them from the sort of boring mundane work that holds most people back. I was gonna add two more A I life cycle, more constrained and not uh more philosophically what A I is gonna change. There's uh I like the Xa I calls them A I Twitter, which is a glorified way but really needed way of, of uh calling the humans in the loop uh in the fine tuning process of the model. Uh And uh and the legal, the lawyers, the amount of legal uh with A I, it's a, it's a huge, every company now has to either hire them on in house or, or, or, or contract the consultants. It's really huge. OK. One word answers what is it gonna take for A I to be faster, smarter, impactful and equitable? Just one word that comes to mind. If you have the answer, just say it. Humans, humans. OK. Commitment, commitment, love it. Um I, I think more, more e equitable investment, equitable investment. There we go. Thank you so much to our panelists. Great conversation.