The quest for seamless app user experience begins with crafting the perfect prompt. But what if you could optimize your prompts using cutting-edge technology? Enter the prompt optimizer, a game-changing tool that revolutionizes the way you interact with users.

Preparing Your Data

To unlock the full potential of the prompt optimizer, you'll need to prepare a dataset containing the prompt you want to optimize and an evaluation dataset. This dataset should include at least three rows of data with responses and corresponding annotations or graders results. The more detailed and specific your annotations are, the better the prompt optimizer will be able to improve your prompts.

The prompt optimizer can utilize various elements from your dataset to enhance your prompts, including:

  • Annotations containing Good/Bad ratings and detailed critiques
  • Text critiques written in output_feedback
  • Results from graders

Optimizing Your Prompt

Once you've prepared your dataset, it's time to optimize your prompt. To do this, click the Optimize button at the bottom of the prompt pane, which will create a new tab for the optimized result and initiate an optimization process that runs in the background.

When the optimized prompt is ready, view and test the new prompt. Don't be afraid to repeat the optimization process on the new prompt, generating outputs, annotating outputs, running graders, and optimizing until you achieve the desired results.

The Power of Gradable Properties

The effectiveness of prompt optimization relies heavily on the quality of your graders. To build effective graders, focus on creating narrowly-defined graders for each of the desired output properties where you see your prompt failing to meet expectations. By doing so, you'll be able to identify and address specific areas for improvement.

Evaluating Your Results

While the prompt optimizer is designed to provide a strict improvement in your prompt's effectiveness, it's crucial to manually review optimized prompts before using them in production. This ensures that the optimized prompt performs as expected on various inputs.

Next Steps: Elevating Your App User Experience

Ready to take your app user experience to the next level? Explore the OpenAI Cookbook for example code and links to third-party resources, or learn more about our tools for evaluations:

  • Operate a flywheel of continuous improvement using evaluations
  • Evaluate against external models, interact with evals via API, and more
  • Build sophisticated graders to improve the effectiveness of your evals
  • Improve a model's ability to generate responses tailored to your use case

By incorporating these best practices into your workflow, you'll be well on your way to creating an exceptional app user experience that sets you apart from the competition.