Testing & Fine-tuning Templates

Learn how to test and optimize your templates for better performance and accuracy.

# Step 1: Prepare Test Inputs

For the Geo Guesser example, you might test with inputs like:

• Famous city nicknames ("The Big Apple")
• Descriptive phrases ("It's a place that never sleeps")
• Landmark references ("the city holding the status of liberty")
• Images of landmarks (when using image input)

# Step 2: Run Tests in the Playground

Use the template playground to process each input and verify the results.

{
  "city": "New York",
  "country": "USA",
  "currency": "USD",
  "isCapital": false,
  "population": "8.4 million"
}

# Step 4: Analyze Results

When reviewing test results, check for these key aspects:

• Schema compliance: Does the output match your defined structure?
• Data accuracy: Is the extracted information correct?
• Completeness: Is all relevant information captured?
• Edge case handling: How does it handle unusual inputs?
• Processing time: How quickly does it complete?

# Fine-tuning Your Template

If your tests reveal issues or opportunities for improvement, you can fine-tune your template using the "Tune" tab.

# 1. Add or Refine Custom Prompts

Custom prompts can significantly improve template performance by giving the AI more specific instructions.

# 2. Select the Optimal AI Model

Different models have different strengths. Try alternative models to see which performs best for your use case.

Model selection considerations:

• For complex tasks requiring nuanced understanding, try GPT-4o
• For faster response times and lower costs, consider GPT-3.5 Turbo
• For image analysis tasks, ensure you select a model with vision capabilities

# 3. Refine Your Schema

Sometimes issues stem from ambiguities in your output schema. Consider these improvements:

• Make field names clear and descriptive
• Choose appropriate data types for each field
• Use boolean fields (like isCapital) for yes/no data
• Consider adding more specific fields if needed

# Testing Workflow Best Practices

To ensure your template performs reliably in production:

1. Test with a variety of inputs that represent real-world usage
2. Include edge cases and potential ambiguities
3. Test performance with different AI models
4. Save successful test cases for regression testing
5. Regularly re-test your template as you make changes