Prompt Playground

When working with LLMs, there are time when you want to quickly try out different prompts and see how they perform. Opik’s prompt playground is a great way to do just that.

Using the prompt playground

The prompt playground is a simple interface that allows you to enter prompts and see the output of the LLM. It allows you to enter system, user and assistant messages and see the output of the LLM in real time.

You can also easily evaluate how different models impact the prompt by duplicating a prompt and changing either the model or the model parameters.

All of the conversations from the playground are logged to the playground project so that you can easily refer back to them later.

Configuring the prompt playground

The playground supports the following LLM providers:

• OpenAI
• Anthropic
• Ollama
• Gemini
• Vertex AI
• LM Studio (coming soon)

Go to configuring AI Providers to learn how to configure the prompt playground.

Running experiments in the playground

You can evaluate prompts in the playground by using variables in the prompts using the {{variable}} syntax. You can then connect a dataset and run the prompts on each dataset item. This allows both technical and non-technical users to evaluate prompts quickly and easily.

When using datasets in the playground, you need to ensure the prompt contains variables in the mustache syntax ({{variable}}) that align with the columns in the dataset. For example if the dataset contains a column named user_question you need to ensure the prompt contains {{user_question}}.

Once you are ready to run the experiment, simply select a dataset next to the run button and click on the Run button. You will then be able to see the LLM outputs for each sample in the dataset.