Test LLM Prompts
At Scale
Create test cases, define assertions, and evaluate prompt performance
across a wide range of inputs with Langtail's intuitive testing suite.
Comprehensive Testing Suite
Create and manage tests
- Intuitive Test Creation
- Build tests using a user-friendly, Excel-like table interface.
- Flexible Data Input
- Import test cases from CSV files or paste directly from Excel.
- Efficient Test Management
- Organize tests into collections and run them against different prompt versions.
- Test Run History
- Track and review past test runs to monitor performance over time.
Versatile Assertions
Evaluate prompt outputs with precision
- String Manipulation Assertions
- Validate outputs using contains, exact, and other string-based assertions.
- Custom JavaScript Assertions
- Write custom assertion functions in JavaScript for advanced evaluation.
- External API Integration
- Verify outputs by calling external APIs within assertion functions.
- LLM-powered Assertions
- Assess output quality, relevance, and truthfulness using LLM-based assertions.
Future-proof Testing
Upcoming features to enhance testing experience
- Human-like Assertions
- Evaluate outputs using assertions that mimic human judgment.
- API Data Retrieval
- Fetch data from APIs to use as inputs for tests, ideal for RAG apps.
- Function Resolvers
- Call external APIs during function resolution for enhanced testing capabilities.
- Assistant API Testing
- Expand testing capabilities to cover Langtail's Assistants API.
Engineering and AI teams Langtail
“Langtail simplifies the development and testing of Deepnote AI, enabling our team to focus on further integrating AI features into our product”
“This is already a killer tool for many use-cases we are already using it for. Super excited for the upcoming features and good luck with the launch and further development! 💜”
“Been using LangTail for a few months now, highly recommend. It has kept me sane. If you want your LLM apps to behave uncontrollably all the time, don't use LangTail. On the other hand, if you are serious about the product you are building, you know what to do :P Love the product and the team's hard work. Keep up the great work!”
“I have used Langtail for prompt refinement, and it was a real timesaver for me. Debugging and refining prompts is sometimes a tedious task, and Langtail makes it so much easier. Good work!”
“LLM products are creating a flurry of bad experiences in their rush to hit the market quickly. But Petr and his team have been demonstrating since day one just how serious they are about doing this job with outstanding designs. I've been following them for over a year now and I highly recommend them to everyone. I'm certain they're going to reach fantastic places.”
“Been using Langtail for a while, and it has made working with our clients a breeze”
“Unpredictable behavior of LLMs, team collaboration on prompts and robust evaluation were the biggest pains for me when I was building my app. But now it's solved thanks to LangTail. It's a great product.”
- Intuitive Test Creation
- Create tests effortlessly using a user-friendly, Excel-like table interface.
- CSV & Excel Import
- Import test cases from CSV files or paste directly from Excel.
- Test Collections
- Organize tests into collections for better management and organization.
- Multiple Prompt Versions
- Run tests against different prompt versions to compare performance.
- String-based Assertions
- Validate outputs using contains, exact, and other string-based assertions.
- JavaScript Assertions
- Write custom assertion functions in JavaScript for advanced evaluation.
- External API Assertions
- Verify outputs by calling external APIs within assertion functions.
- LLM-powered Assertions
- Assess output quality, relevance, and truthfulness using LLM-based assertions.
- Test Run History
- Track and review past test runs to monitor performance over time.
Frequently asked questions
Ready to test your prompts at scale?
Start testing and evaluating your prompts for free today.