Name: Brainstorm AI Singapore 2024: How to write effective AI prompts
Uploaded: 2024-07-31
Duration: 10 min 11 s
Description: Teodora DANILOVIC, Prompt Engineer, Autogenai

Teodora DANILOVIC, Prompt Engineer, Autogenai 

Brainstorm AI Singapore 2024: How to write effective AI prompts

Conferences

So in the short time I have with you all today, I want to demonstrate how smart prompting leads to smart outputs and hopefully give you a few practical tips and techniques that you can take away and use for your own prompt creation process. So I work at auto A where we help organizations around the world write more winning bids, tenders and proposals by leveraging large language models and linguistic engineering. I get a lot of questions around what prompt engineering actually is and what I do in my role. So I thought it would be useful to clarify that. Can I get a quick show of hands who here has used chat GP T or any other similar platform for their prompting? Wonderful everyone. Um Perfect. So what most of you have been doing in those instances is called prompt, crafting. Prompt, crafting is when you interact real time with a model and give it a prompt. For that individual instance, you receive useful and relevant responses, but you wouldn't necessarily expect that prompt to work on any other piece of text that anyone else would use. So, prompt engineering is curating prompts which produce replicable, reliable outputs to fulfill a specific function whilst continuously objectively measuring and improving them. It's about setting up frameworks that scale well in the future, with any unknown input all the time, there are many prompting techniques out there. But these are some of the most popular today. I will demonstrate a few of these as we consider their benefits and their drawbacks. And we'll do this by focusing on one task which is extracting and classifying things from a data set. I'll be using autogenerated platform today to demonstrate these prompts. So the feature that I'll use, it offers multiple options of output for each prompt. So you'll see three options on the right hand side up here, we have a zero shot prompt. It's an instruction with no examples. It's what everyone does the first time they interact with the large language model. It works well most of the time, but it does have a few drawbacks. Sometimes it can lack a nuanced understanding of the task we're trying to achieve. And we'll see that in practice today. So in this example, I bought it to classify a piece of text and that piece of text has a debatable output. So as a human, I would expect the product arrives late, but the quality is excellent to generally be a positive statement. Although it has some positive and some negatives in it, it's weighted heavier on the positive statement. We can see that the model has given neutral for all three outputs. So for this case, zero, prompting doesn't encourage that nuanced understanding and I probably wouldn't trust it for even larger sets of data here. The model lacks understanding of what I mean by each of those sentiments. What does positive mean for me? What does neutral mean for me? In the previous example, I didn't have enough context. So one way of providing it with more context is offering examples of what you want. This is called multi shot prompting a shot being an example, chain of thought, prompting means asking the model to think step by step and show its reasoning. So I've given it pretty much the same prompt, but I've given it three examples of what I think a positive, negative and neutral statement is you can see as we transform the text, the outputs are far more new. The first one says positive. The second one after explaining its chain of thought where I've instructed it to think step by step also concludes to be positive. In the end, it's far closer to what we're looking for. One thing I will say is that you should be wary of bias with multi shop prompting. In this example, the model might take it to mean that positive statement must always talk about product quality or it always must mention that the site was confusing or that it always has to have one positive and one negative side to it. If you're using multi shot prompting for larger sets of data, you have to make sure your examples cover all bases. This can be quite difficult to do as you have to think of every way that something can be interpreted chain of thought also helps in this scenario. Because if I can see the models thought process, I can see where it went wrong and it can help with my model debugging multi short hair was really great for improving this task and giving the model some more understanding of my intention. But sometimes I want to do something a bit more complicated and I don't want it to give me just a one word classification for something. Let's look at how we can introduce even more complexity with some prompting techniques, prompt training or multi step prompting is best for complex reasoning tasks that cannot be instructed in one go. It ensures you're working on the best piece of text at each stage and it doesn't leave room for model inconsistency. It makes sure that the potentially conflicting instructions don't interfere with one another. Let's say I want a more complex analysis of some sentiment on a larger body of text. As a human, I might break this process down into a few different steps. First classifying the statements as a whole, then extracting themes from the statement and then grouping those themes. This type of breakdown also works great with prompting. So for the sake of brevity, I'm just showing the output of the first prompt that I gave the model. The prompt instructed it to classify a list of customer feedback into sentiments using the same multi shot prompt that I showed you guys before. Except on a larger set of data. There's around 2526 pieces of customer feedback. Here, this is the first prompt in this chain. The second prompt in the chain is a really good zero shot prompt up at the top here, you can see we've asked it to identify all the themes in the following list of customer feedback. Your response should be an exhaustive list of accurate and relevant themes with a number beside them indicating how many times it appeared, do not write the same theme out twice, feedback prolong. So I think it's important to note some key parts of that prompt. So I've asked it to identify all the themes and then I repeated myself and said it should be an exhaustive list. Repetition is always good. I've said that it's a list of customer feedback which provides the model with context of what I'm talking about. I've asked it to be accurate and relevant. So it's clear that I want the themes only to do with customer feedback. And then I've said how I want the structure to look, you can see with all of the options. The structure is exactly how I want it to look. The answers are exactly what I'm looking for. So I'm happy with this and we bring it over into the editor. So as we move on to the third prompt in the chain, you can see how with each step, the output is a combination of everything that came before it context which is implicitly woven into the answer gets carried on to the next prompt. The prompt that I've given it here. The third prompt in the chain is classify the following themes into positive, negative, neutral or other categories. Your response should have the sentiment as a heading followed by the themes which fall under that sentiment under each theme, write a brief justification of why it falls under that sentiment. And then I've given the list of themes, as you can see on the right hand side, this is far more nuanced, accurate and useful than the first thing we looked at. This is this output is actually a combination of all the techniques we've looked at so far and we could not have gotten this type of information, which is this thorough and this accurate through only using one of the techniques. Once you've got an output that you're happy with, the possibilities really are endless. You can translate it into Jason, you can play around with a toe, you can turn it into a powerpoint presentation and much much more so to conclude this short segment, simplicity is always best. Although a single shot prompt wasn't nuanced or accurate enough. For example, today, it is most often the best choice. The last three examples I showed you of turning it into Jason or translating the tone. All of those used a really good successful zero shot prompt. A prompt should be direct, unambiguous and relevant. And each of the techniques we've gone through today are just different ways of ensuring that the prompt meets those requirements. I really hope this offers some insight into what prompt engineering actually is and gave you some tools that you can take away and implement into your prompt. Crafting. I think we might have time for one quick question. Is there any questions in the audience? Yeah, we have the one here. How do you think about it? And do you use any tools to refine your prompts and how? Yeah. Yeah. So that's, that's actually a really interesting question. Um You can prompt models with how you want them to refine your prompts interestingly. So you have to have the techniques in order to instruct the models and how you want your prompts to look. Anyway, sometimes if I'm lacking inspiration on how to begin writing a prompt, I will definitely use a model. I'll be direct, unambiguous relevant, clear about what my parameters are clear about what my instructions are. And often it gives me a really good framework. The only problem with models are, it's sometimes a bit more nuanced than that. And you're looking to write a prompt for a specific use case. So I know my target audience. I know my, my customers and I know what they're looking for and I can try put that into prompting. But ultimately, it's your subjective opinion on whether you think it meets these metrics that you set out. So, yes, absolutely. You can use other models, not only Claude but any of the providers to give you a first draft of a good prompt and to better your own prompt. Awesome. I think that's all the time we have today. But if you have any more questions, I'll be around afterwards and please feel free to add me on linkedin and ask me any questions there. Thank you so much.