Create TestSet & Upload TestCases

Overview

Uploading test cases is the first step in building your evaluation pipeline. You can import test cases in bulk through CSV or JSON files, or create them manually through the Studio interface.

Test cases uploaded to your project are stored in test sets, which can be accessed via SDK or Studio for running evaluations. Once uploaded, test cases are immediately available for evaluation.

Field Specifications

Required Fields

Input Type: text The prompt or query sent to your LLM.

Optional Fields

Expected Output Type: text The ideal response for comparison-based metrics.

Retrieval Context Type: list The re-ranked context used to generate the response in RAG applications. Provide as an array of text strings.

Context Type: list Additional contextual information or source documents. Provide as an array of text strings.

File Format Requirements

JSON Format

json

[
  {
    "input_prompt": "What is your refund policy?",
    "expected_output": "We offer 30-day refunds for all products.",
    "retrieval_context": ["Policy document excerpt..."],
    "context": ["Additional context..."]
  }
]

CSV Format

csv

input,expected_output,retrieval_context,context
"What is your refund policy?","We offer 30-day refunds for all products.","['Policy document e

Uploading via Studio

1

TestSets Page

Go to TestSets in the sidebar under the TESTING section. If this is your first time, you'll see an empty state with "No testsets yet".

Click the Create TestSet button to begin uploading your test cases.

2

Create a TestSet

A modal will appear prompting you to create a new dataset. Enter a unique TestSet Alias for your project. This alias will be used to reference the test set in SDK calls.

The alias must be unique to your project. You're creating a single-turn dataset.

Example: customer-support, edge-cases, prod-sample-jan-2025

Click Create to proceed.

3

TestSet Created

Your testset is successfully created and appears in the list. The status shows EMPTY as no test cases have been uploaded yet.

4

Open TestSet Editor

Click on your testset row to open the editor. You'll see an empty state prompting you to import test cases.

Click Import from File to begin uploading your CSV or JSON file.

5

Upload TestSet File

The Upload TestSets modal appears with a 3-step process:

1. Import File

2. Select Fields

3. Final Review

Drag and drop your CSV or JSON file into the upload area, or click Select a CSV or JSON file to import to browse your files.

6

Map Fields

Now you're on the Select Fields step. Map your file's columns to the Golden Fields.

Your uploaded file is shown with its size and row count. The system automatically detects all columns from your CSV or JSON file.

Use the dropdown menus on the right to select which column from your file corresponds to each Golden Field on the left.

Map at least the required Input field. Once mapping is complete, click Next to proceed to the final review.

7

Processing and Completion

After clicking Next, the system processes your uploaded file. You'll see a notification banner stating "Processing uploaded file... Please wait while we process your data and update the test cases."

Once processing is complete, you'll be redirected to the TestSet list page, where you can see your test cases successfully uploaded. The status will update from EMPTY to show the number of test cases imported.

Your test cases are now ready to use for evaluations via SDK or Studio.