Changelog
๐ง Improved Test UX and Bug Fixes
- ๐ Itโs much easier to switch between prompt editing (Playground) and Tests now using tab navigation
- ๐ New users now get limited free usage for Anthropic & Google without providing their own API keys
- ๐งช Added Gemini experimental model 1206
- ๐ Fixed index & id in response for multiple tool calls from Anthropic
- ๐ Fixed variables list rendering in prompt detail view
- ๐ Fixed missing user message in assistant playground
- ๐ Fixed copy-pasting prompts from playground to include new lines correctly
- ๐ Fixed dev console error when viewing non-JSON test results
๐จ Test design updates and new onboarding experience
- ๐จ Redesigned test interface to match playground styling and UX
- ๐ Introduced new user onboarding flow for better getting started experience
- โจ Magic prompt enhancer with diff view and accept functionality
- ๐ Fixed assistant deployment snippet generation
- ๐ Added support for video URLs without file extensions
๐ Improved test response UX and other fixes
- ๐ You can now enable test response text wrapping to easily see longer responses
- ๐ Added support for markdown in the test response drawer
- ๐ง Fixed selection of content in the test response drawer
- ๐จ Fixed test variable icon color on theme switch without reload
- ๐ง Adjusted LLM assertion to allow shorter Yes/No descriptions
- ๐ Fixed thread messages not saving with assistant API in Anthropic/Google
- ๐ Fixed a rare bug where a second main test config could be created
๐ Shareable Prompts and Assistants, and UX Enhancements
- ๐ Added OG tags to shareable prompt assistants for better visibility and discoverability.
- ๐ When copy-pasting multiple rows into Tests, the order is now fixed.
- โจ๏ธ Enhanced tab navigation in the messages drawer for better accessibility.
- ๐ Implemented proper handling of multiple test runs.
- ๐ Formatted the test output preview and added a copy button for improved readability.
- ๐ Resolved issues with Gemini tool calls in tests and Anthropic model behavior.
- ๐ Added undo/redo functionality to the test table for easier editing.
- ๐ค Introduced a prototype for LLM assertion generation.
- ๐ค Added support for the xAI grok-beta model.
- โจ Implemented magic to generate descriptions and titles for assistants.
๐ค New Model Support, Magic buttons, Testing Improvements, and Onboarding
- ๐ค Added support for the Claude 3.5 Haiku model from Anthropic.
- โจ Introduced several โmagicโ buttons throughout the app to improve your experience with LLMs. These include prompts for enhancing prompts, generating test cases, and more! Let us know which ones you find and how useful they are - itโs like an Easter egg hunt!
- ๐ Fixed an issue where the test data grid couldnโt delete repeated test cases.
- โ๏ธ Introduced a โnoneโ option for tool choice.
- ๐ Added a history viewer for better visibility into past events.
- ๐ฑ Improved onboarding experience for mobile devices.
๐ Post 1.0 Fixes and UI Improvements
- ๐จ Implemented various small UI fixes and improvements.
- ๐ข Fixed an issue with ordering items in the testing table when they have the same creation date.
- ๐ Improved test generation by passing test_inputs to the magic function for better test data.
- ๐ Added templates and documentation links to the sidebar for easier access.
๐ Langtail 1.0: Brand New Testing Interface, Assistants, and More!
Weโre thrilled to announce the release of Langtail 1.0, packed with exciting new features and improvements!
๐ Brand New Testing Interface
- ๐ At the core of Langtail 1.0 is our new spreadsheet-like interface for testing LLM applications, designed to feel natural for anyone used to working with Excel or Google Sheets.
๐ Test Configurations
- ๐๏ธ Introducing test configurations, allowing you to easily create and compare tests across different models with just a few clicks.
๐ ๏ธ Tools with Hosted Code
- ๐ป Your prompts can now call tools and have the code run directly within Langtail, making prototyping and testing much easier without the need for mocking responses.
- ๐ Enjoy secure and fast code execution in a sandboxed environment powered by QuickJS.
๐ง Assistants: Memory Management for Prompts
- ๐ง Langtail 1.0 introduces Assistants โ stateful entities that automatically manage memory and conversation history, reducing the need for additional code.
- ๐ Assistants can be used across models, in tests, deployed as APIs, or integrated with tools.
๐ Additional Features
- ๐ Light Mode: Introduced a new Light mode for the Langtail app, providing a refreshing alternative to the default Dark mode.
- ๐ก๏ธ Protect your AI apps from prompt injection attacks and other security vulnerabilities with our built-in AI Firewall.
- ๐ Execute LLM-generated code securely in a sandboxed container, thanks to our partnership with E2B.
- ๐ Explore a gallery of templates showcasing different ways Langtail can help your team.
- ๐ค Added support for Google Gemini models, including the ability to attach videos to prompts.
- ๐ข Self-hosting: Langtail can now be self-hosted entirely on your infrastructure (contact us for more information).
๐ ๏ธ Tool Calls in Assistant Playground and Gemini Support
- ๐ ๏ธ Added the ability to handle tool calls in the Assistant Playground, allowing you to provide responses for tool calls not handled by Langtail function handlers.
- ๐ค Introduced native support for Gemini models, leveraging the @google/generative-ai library under the hood, with support for tools, streaming, and various media formats as input.
๐ Other Improvements
- ๐ Fixed an issue with opening logs with prompts in the playground.
- ๐ฐ Set Gemini model prices.
๐ Assistant Sharing Redesign, LLM Leaderboard, and Hosted Functions
๐ค Assistant Sharing Redesign
- ๐ Redesigned the assistant sharing feature, allowing you to make your AI assistants publicly accessible and share them as standalone apps.
๐ LLM Leaderboard
- ๐ Introduced a new LLM leaderboard page to compare the performance of different language models.
๐ ๏ธ Hosted Code Support
- ๐ Added support for Hosted Code, enabling you to test your tools right inside Langtail by directly calling external APIs.
๐ Security and Stability Improvements
- ๐ Ensured that only team members can edit shared assistants.
- ๐ค Added support for new OpenAI models: o1-mini and o1-preview.
- ๐ง Fixed issues with tool usage for Azure models and display names.
๐ Shared Assistant Indexes
- ๐ Implemented shared assistant indexes for improved discoverability and searchability.
๐ Text Assertion Enhancements and LLaMa 3.1 Support
- ๐ Grouped all text operation assertions under โTextโ for better organization.
- ๐ Added new text assertion options: not equals, does not contain, and does not icontain.
- ๐ Introduced a text editor for the โTextโ assertion modal form, with support for variables.
- ๐ค Added support for new LLaMa 3.1 models from groq.
๐ฐ Usage Limits, Test Improvements, and GPT-4o Mini Support
- ๐ค Added support for GPT-4o Mini, OpenAIโs cost-efficient model offering advanced text and multimodal performance
- ๐ฐ Implemented a banner to notify free users when they exceed usage limits.
- ๐ Fixed an issue with the test ID for the assertion popup in test results.
- ๐๏ธ Improved the layout styles for project logs.
๐ Expanded Message Types and Provider Visibility
- ๐จ Added support for other message types, tools, and images in the โmessagesโ column, allowing you to simulate user and assistant messages, as well as mock tools and upload images for Vision testing.
- ๐ Unlocked providers are now displayed at the top for better visibility.
๐จ Layout Redesign and Test Table Improvements
๐ Enhanced Project-Level Visibility
Weโve redesigned the layout to provide better visibility and access to key project-level components:
- Playground
- Tests
- Deployments
- Assistants
๐ Test Table Fixes and Enhancements
- ๐ฅ Fixed an issue with the test table width when the drawer is open.
- ๐ Added the ability to close the cell edit drawer by pressing the ESC key.
- ๐ Resolved issues with HTML pasting in the data table.
- ๐ Improved the display of relevant information in the test history.
๐ Security Enhancements
- ๐ Fixed issues related to system_fingerprint and model value for Anthropic models.
- ๐ผ Updated to the latest SDK, resolving an OpenAI issue with images.
๐ค Introducing Anthropicโs Claude 3.5 Sonnet Model
- ๐ Added support for Anthropicโs Claude 3.5 Sonnet model, offering faster and cheaper performance compared to Claude 3 Opus, with enhanced reasoning, knowledge, and coding abilities.
๐ Test Table UX Enhancements and Bug Fixes
Weโve made several improvements to the test table user experience and addressed some pesky bugs:
- ๐ผ๏ธ Text table UX improvements
- ๐ฏ Streamlined the handling of messages in the test table for a smoother workflow.
- ๐ Enhanced the behavior of the edit drawer, ensuring a more intuitive experience.
- โจ๏ธ Improved keyboard navigation and selection within the test table.
- ๐ Fixed issues related to arrow key navigation and drawer behavior.
๐ฏ Testing Enhancements: Keyboard-Friendly Table, Helper Variables, and Improved LLM Assertions
โจ๏ธ Keyboard-Friendly Testing Table โจ๏ธ
Weโve made our testing table more keyboard-friendly, allowing you to bulk delete rows and copy multiple values at once.
๐งฎ Helper Variables for Assertions ๐งฎ
You can now add additional columns that act as helper variables for assertions (JavaScript, LLM assertions). Place the correct answer in these helper variables and use them in assertions for more efficient and accurate testing.
๐ Improved LLM Assertions ๐
Based on user feedback, weโve improved LLM assertions. Now, the entire conversation history is included in the context, providing a more comprehensive view for better testing.
๐จ Multiple Messages in the โMessagesโ Column ๐จ
You can now use multiple messages in the โmessagesโ column. Currently, itโs possible to simulate user and assistant messages. Weโre working on the ability to mock Tools and upload images for Vision testing.
Tests are a big focus for us, and weโre committed to continuously improving them. Stay tuned for more updates!
โก Major Upgrade to Langtailโs Testing Table Component and Upcoming Features
โก Langtailโs Testing Table Component Upgrade โก
Weโre excited to announce significant updates to Langtailโs table component aimed at improving your test-writing experience:
- ๐ Lightning-fast row addition and cell content deletion for efficient test setup
- โก๏ธ Tab navigation between cells for seamless navigation
- ๐๏ธ Bulk row deletion for quick test cleanup
- ๐ฏ Selective test case execution for focused testing
- ๐ Test result display with LLM and assertion outcomes for comprehensive insights
๐ Upcoming Feature: LLM as a Judge Assertion ๐
Stay tuned! Weโre in the process of developing an โLLM as a judge assertionโ feature, set to take our testing capabilities to the next level. More updates coming your way soon!
New Dashboard View and Launch of Langtail SDK ๐
๐ New Dashboard View ๐
Weโre thrilled to introduce our new dashboard view! Now serving as the main hub for each project, the dashboard provides a snapshot of your projectโs activity and performance at a glance. With recent logs and quick statistics, you can stay updated the moment you log in.
๐ฏ Langtail SDK Release ๐ฏ
The first version of the Langtail SDK is now live! ๐ฅณ The SDK can function as an OpenAI proxy and is also compatible with prompts directly deployed within Langtail. Dive into the details over at our GitHub docs.
Currently, weโve rolled out the JS SDK. If youโre interested in a Python SDK, weโd love to hear from you.
New Homepage Video, About Us Page, and Embeddable Prompts
๐๏ธ New Homepage Video ๐๏ธ
Weโve released a new video thatโs now featured on our homepage. Get a quick overview of what Langtail offers by watching our latest video.
๐งโ๐ป About Us Page ๐งโ๐ป
Curious about the team behind Langtail? Visit our newly updated About Us page to learn more about our mission and the people who make Langtail possible. Learn more
๐ฃ๏ธ Roadmap, Changelog, and Feedback ๐ฃ๏ธ
Your feedback is crucial in helping us improve. Visit our feedback portal to see our roadmap, changelog, and to share your thoughts. Give feedback
๐ผ๏ธ Embeddable Prompts ๐ผ๏ธ
Weโre happy to announce that you can now embed publicly shared prompts into iframes! Hereโs a quick example:
๐ Docs Expansion, ๐ผ๏ธ Dynamic OG Images, and ๐ Project-Level Env Variables
๐ Expanded Documentation for Seamless Integration ๐
Weโve significantly expanded our documentation to help you hit the ground running:
- ๐ Deploying Prompts: Learn how to deploy your prompts effortlessly. Read more
- ๐ REST API Usage: Discover how to leverage our powerful REST API. Learn how
๐ผ๏ธ Dynamic OG Images for Shared Prompts ๐ผ๏ธ
Sharing is caring, and weโve made it even better! When you share a prompt publicly, weโll automatically generate an OG image for you.
๐ Project-Level Environment Variables ๐
Tired of hardcoding API endpoints and keys directly in code? Weโve got you covered! Introducing project-level environment variables:
- ๐ Secure: Keep your sensitive information safe and separate from the code
- ๐๏ธ Centralized: Set your variables once at the project level
- ๐ Accessible: Use your env variables in the JavaScript editor for Test Assertions and soon in the Playground too
Langtail.com updates, Proxy Integration, and Quality of Life Improvements ๐
โจ Landing page updates โจ
Our brand new website is live, showcasing the four key features that make Langtail the ultimate tool for prompt engineering:
- ๐ Debug prompts effortlessly in the playground
- โ Run tests to ensure your prompts are top-notch
- ๐ Deploy prompts in a flash
- ๐ Monitor production with comprehensive logs and metrics
๐ธ Pricing Preview ๐ธ
Weโve got a pricing plan for everyone! Choose from three tiers:
- ๐ Free Tier: Perfect for getting started
- ๐ฐ Mid Tier: Unleash more power at just $99 per month
- ๐ข Enterprise Tier: Tailored solutions for your business needs
๐ Docs Page Expansion ๐
Weโve been busy expanding our documentation to help you get the most out of Langtail. Head over to our docs page to explore the latest information and guides.
๐ Langtail Proxy: Seamless Integration ๐
Already building your application using the OpenAI SDK? No problem! With Langtail Proxy, you can experiment with Langtail without migrating your prompts. Hereโs how:
- Create your Langtail account
- Generate your project API key
- Modify your code by updating the base URL to
"https://proxy.langtail.com/v1
and passing the Langtail API key - Test it out and witness the magic!
๐จ Quality of Life Improvements ๐จ
Weโve also released some handy enhancements to make your life easier:
- ๐ Log Filtering: Easily filter logs by categories like proxy
- ๐ฏ Single Test Run: Run individual tests instead of the entire suite
- โ Test Cancellation: Cancel test runs whenever you need
- ๐ Current Date in System Prompt: Add the current date using handlebars
๐ $Date Helper, ๐ Enhanced Log Filters, and ๐ Webpage Updates
๐ $Date Helper for Chatbots ๐
Chatbots can now stay up-to-date with the current date! Introducing the $date
helper, a new feature designed to keep your chatbots in sync with real-time.
๐ Log Filtering ๐
Say hello to granular control over your logs. You can now filter logs by any column, even custom-send metadata. Find what you need, when you need it.
๐ฐ Pricing Page is Live ๐ฐ
Ever wondered about our pricing? Weโve just launched our pricing page, making it easier for you to find the perfect plan for your needs. Check it out!
๐ Introducing Public Sharing of Prompts and Read-Only Playground!
๐ก Public Sharing of Prompts ๐ก
Good news, everyone! Weโve just launched a new feature that lets you publicly share your prompts. Previously, users needed an account to view a shared prompt, but now, anyone can view it thanks to our new read-only Playground view. ๐
๐๏ธ Read-Only Playground ๐๏ธ
In the read-only Playground, you can see all messages, variables, and Tools (functions). Itโs an open window to the world of your prompts. Currently, only a fixed version of the prompt can be shared, but weโre already working on updates that will let you share and always view the most recent version of a prompt.
๐ Major DX Improvements for JavaScript Assertions in Tests and Upcoming Playground Enhancements ๐ ๏ธ
๐งช Enhanced JavaScript Assertions in Tests ๐งช
Weโve made some significant improvements to our developer experience for JavaScript assertions in Tests:
- ๐ Run functions with example input: Simply fill in your example input in the left column and run the function.
- โ Auto Input Addition: If any input already exists, weโll automatically add it for you.
- ๐ Debugging Support: Use
console.log
to debug your tests effectively. - ๐ API Calls: For advanced use cases, you can now call external APIs using the Fetch API.
- โก๏ธ Ultra-fast API requests: Thanks to our robust infrastructure, your API requests will execute swiftly.
๐ New Templating Engine Based on Handlebars for Enhanced Prompt Handling
๐จ Templating Engine Support ๐จ
Weโre excited to announce that weโve released support for a templating engine, based on Handlebars! With this new feature, you can now tackle more complex use cases, including:
- Conditional parts of prompts
- Variations of prompts
Simply deploy a prompt with this template, send the correct variable, and watch as the final version of the prompt resolves. You can then view this in the Logs. This added flexibility opens up a world of possibilities for your prompt handling.
๐ Project Settings, Workspace Refinement, Redesigned Logs, and OpenAI Tools Support!
๐ง New Project Settings ๐ง
Managing your projects just got easier! Weโve added a new item in the project-level sidebar that lets you perform common actions like renaming or deleting a project.
๐ ๏ธ Enhanced Workspace Settings ๐ ๏ธ
Your workspace is now even more flexible! Weโve refined the settings to allow you to rename your workspace and made the UI more consistent across the different sections for a smoother experience.
๐ Redesigned Logs ๐
Weโve completely redesigned our Logs for a cleaner, more user-friendly interface. Plus, Logs now also display the JSON representation of whatโs being sent from Langtail to end providers like OpenAI. We think youโll love this upgrade as much as we do!
๐ OpenAI Tools Support ๐
Weโre thrilled to announce that support for the new OpenAI Tools is here! Functions now visually resemble TypeScript functions and are even typesafe. The types are generated based on the JSON schema. Weโre confident youโll find them incredibly useful.
๐ GPT-4 Vision is Live, Plus New Workspace Settings and Refined UX ๐
๐ GPT-4 Vision Now Live ๐
Weโre thrilled to announce that GPT-4 Vision is now live in production! You can now input images along with text, opening up a whole new realm of creative use cases. For instance, take a photo of an invoice and ask for the cost of each item โ Vision has you covered.
๐จ Experimenting with Vision in Playground ๐จ
Want to give Vision a spin? Head over to our Playground and upload an image or insert an image URL. Please note that Vision takes a few seconds to process, so a bit of patience is required.
๐ Deploy Prompts with Vision ๐
Great news for developers โ deploying prompts with Vision in Langtail is now supported!
๐ ๏ธ Workspace Settings and UX Updates ๐ ๏ธ
Weโve also rolled out a couple of key updates for improved user experience:
-
New Workspace Settings: Now, you can manage your provider API keys and team members all in one place.
-
Refined UX for Per API Budget: Based on your feedback, weโve refined the UX and terminology for the per API budget. You must now enable the budget before setting the total for your API key and specific user allowances.
Was this page helpful?