Name: Brainstorm AI 2024: OpenAI Demo
Uploaded: 2024-12-10
Duration: 10 min 42 s
Description: Presenter: Romain Huet, Head, Developer Experience, OpenAI

Presenter: Romain Huet, Head, Developer Experience, OpenAI

Brainstorm AI 2024: OpenAI Demo

Conferences

Thank you. Good afternoon, everyone. Super thrilled to be here today. It's an interesting time in A I, the pace of change is really accelerating and there are more A I applications going from prototype to production every single day. I'm Roma and I lead developer experience at open and you know, a lot of people know us for Chad GPT. But in fact, our first product was the open API the ability for developers and for companies to take our A I models and bring them into their own products and services. And today, we are very fortunate to work with more than 3 million developers on the platform and 92% of Fortune Five Android actually use our products. So it's an amazing time to be all of us together and building in A I, you know, one of the favorite things in my role day to day is to work with amazing founders and companies who are pushing the envelope of these models every day and also showing them the art of the possible with what we, what we put out there in the world. So in that spirit, instead of having any presentation of slides today, let's jump to my laptop and take a look at some live demos. So first off at the remiss not to mention Sora, we just launched Sora this morning, our video generation model and it's now available on sora.com. Uh So what you're looking at here is the home page with features video already created in the last few hours by the community. Uh It's pretty amazing. In fact, we can fire off maybe a first uh video generation here. I'll say a golden retriever dressed as Santa on Christmas Day, you know, sounds pretty uh pretty cool. Um There is an enormous amount of demand at the moment for Sora. So sounds like uh you know, the service is already at capacity. That's OK, you can check it out later. Uh uh You know, and these are some of the beautiful videos that are already creating. Yeah. So now in fact, to show you something more concrete but with the wind to Sora, I'm going to bring here something about multi modality, multimodality is what we refer to as the ability for A I models to not just not just work with text, but also work with speech, with audio, with vision and also with video as you've seen with Sora. So I wanted to bring you here with a demo of how we can combine all of these modalities together to create something very unique. So this is the only part of the demo that is not live. This is describing a prompt as you can read on the screen in great detail about kind of the history of the German automotive automotive industry. And so if I click here, this is the video that came out of Sora. So what's pretty amazing here is that uh it's only about us switching together some themes out of Sora and adding a little bit of a soundtrack on top, but all of the images are untouched. This is straight out of the model. It's pretty magical. So now let's say we want to use such a video as a basis for a presentation. And what if we want to tell a story that this image, these images in this video can tell us. So what I'll do next is I'll click this button extract frames and I'll slice a few frames out of this video with A Z are generated. And now what I'll do next, I'll bring another modality of GPT photo, the ability to understand and extract insights from images or pictures and that's what we call vision. And so what I'll do next here is I'll keep this button and I'll take all of these six frames. I sliced out of the so video and I'll send them to GPT for all with vision to come up with a story. So don't blank, it goes fast. There we go. So every single time this is unique again, this is all happening in real time as we speak. So now out of these few frames, we were able to come up with that story. So this could be, for instance, the narration for a, for a documentary or this could be something that you can adapt to your own use cases and industries. But next, what if we were to bring this to life? What if we were to be able to bring a voice onto that uh that narration? Now, I'm sure some of you are already thinking ahead here. So what I'll do next is I'll click here to create a voice over and we now have amazing voices in our api like preset voices and they are much more expressive. We'll see that in a minute with the, with the next demo. But here, I also wanted to show you a preview of our voice engine model. The voice engine model is the ability to take a short sample of a voice and turn it into a voice you can use in your apps. So this is only available to trusted partners for now. And we are actively testing this West Dem. We obviously take safety very seriously. We wanted to show you a quick preview of what that would look like. So I'll go ahead and here, I'm going to record a few seconds of my voice. I'm extremely excited to be on stage at the Fortune brainstorm A I conference today and all I need is a few more seconds and I think we should be good to go for me to actually narrate this beautiful video that Sora created. All right. So now that this is so I'll go ahead record it right there. I can just send the script that we created before. Now send this little sample to our voice engine and we'll see how all of these can be combined together to create something a little unique here. So in a second, we should see that exact same video in the heart of the German automotive industry. But now narrated, Rily assemble the machine that would do one more modality that you can play with, right? And every bolt secures the legacy of engineering precision and innovation. Next, I could also do translation. So as you can hear from my French accent, I can maybe also narrated in French. So I'll go ahead and click French here, commitment to quality and efficiency setting. And now this is me narrating this video in French. And last but not least another modality that you can now tap into from our multimodal A I model is the ability to bring the speech to text as well. So here for since most of you don't speak French, but I can actually turn on the caps and all of a sudden you have this other modality of speaking text available. So this is just one example of how you can combine these multi modal models like GPT photo to create something very unique that you can adapt to your own businesses and products and services. But next, I want to show you something else about real time and our real time api so, first of all, here, how many of you in the room have tried Chad GPT with the advanced voice mode before? Wow. Quite a few of you. That's awesome. Well, you know, one question that we've heard so many times over the past few months has been, how can I bring these exact same experiences into my own products into my own services, whether it's to re invent customer service at scale or if it's about like a new way to interface with products? Well, here what I'm going to show you is a loop, sneak peek behind the scenes. So this is the open eye playground. So this is where developers and builders start their journey with our models. And so here I can start a session at any time and actually engage to start to have a feel for how the real time api in the speech to speech capabilities of GPT F come to life. So let's try that. Hi there. Hello there. How can I assist you today? Yeah, I'm on stage at the Fortune Prince, Tommy I conference. Could you welcome the attendees for me? Welcome everyone to the fortune brainstorm A I, so I like how you started, but I think I want you to bring more energy, more enthusiasm and don't forget people watching on the live stream. Welcome everyone to the Fortune Brainstorm A I conference at the Fabulous Saint Regis Hotel in San Francisco. To our incredible attendees and viewers online prepare for an inspiring and groundbreaking event as we dive into the future of A I. Um Now, could you whisper with a very low voice? A secret about San Francisco that maybe only the true locals could know like very, very low like subscription. Did you know that underneath San Francisco's bustling streets? There's a hidden network of tunnels. I did not know that these tunnels were used during prohibition to transport liquor. Discreetly, it's a little known secret. So as you can see the ability to interrupt the model at any time, the ability to jump back and generate intonations, all of that is at your disposal to create amazing ways to interface between your brand, your product and your own customers. Maybe one last example here, I have a test for you here. So here is the deal. I'm going to speak to you in three different languages and I'd like you to speak back to me with those same three languages mixed up. Does that sound good? Absolutely. I'm ready for the challenge. Let's hear it. All right. Chow, Pala Italiano, Francais de Christian stage and I can switch back to English at any time. Your turn. Chow Franci and I'm switching to English now. It's a pleasure to mix these languages on stage with you. So this is the kind of power now you have with these multi modal abilities for speech to speech. And here one example, for instance is that if you want to imagine what that could look like briefly, well, this is what you can do. You can now have voice as a modality to interface with new keyboards. What if I were to say, hey, can you show me on the map where we are? Sure here's Paris, a beautiful city with rich history. I'd like to see amazing Christmas decorations when I'm there. Can you show us a few examples? To me, here are some spots in Paris with amazing Christmas decorations. Galerie Lafayette, the Champs Elysees. This is one way you can imagine how we'll be able to re invent computer to human interfaces in the future with multi modality. And of course, this is not the only paradigm shift that we are excited about because we have also launched the one series of models that can now attack very complex task with reasoning. And we strongly believe that reasoning plus multi modality combined will unleash A I agents next year. And so we're extremely excited about these two paradigm shifts. Thank you so much for your time and I can't wait to see what you build. Thank you.