​

🔄 Test Improvements and Infrastructure Updates

• 📊 Added display of pass rate by each configuration in test results
• 🛠️ Fixed tool result focus behavior when there are multiple tool calls in playground
• 🐛 Fixed test run detail sidebar not updating when switching between cells in test table
• 🐛 Fixed o1-preview and o1-mini which dont support reasoning effort level - Langtail is no longer showing it
• 🔀 Added support for changing thread message list order direction in the API (order=desc)

​

🤖 New Gemini Models and UI Improvements

• ✨ Added new Google models:
  • Gemini Flash 2
  • Gemini Flash 2 Lite
  • Gemini Pro 2 (Experimental)
• 🔄 Added New Flash 2 model and Sonnet 3.5 to OpenRouter models
• 🎛️ Replaced temperature setting with reasoning effort level for OpenAI reasoning models (o1/o3)
• 💾 Test run details now persist when switching between playground, tests and test run tabs

​

🎉 OpenAI o3-mini Launch and Performance Updates

• 🚀 Added support for OpenAI’s new o3-mini model
  • More powerful and cost-effective compared to o1-mini
  • Lower latency responses
  • Full support for function calling, structured outputs, and developer messages
  • Note: No vision capabilities, requires temperature 1.0
• 💲 Fixed request cost calculations in logs & metrics to account for cached prompt tokens
• 🗃️ Added caching support for Anthropic models

​

🔄 Anthropic Caching and Usage Metrics Improvements

• 🔄 Added caching support for Anthropic models
  • Enable via UI on template messages
• 🐛 Improved token count and pricing calculations for Anthropic models and Gemini models
• 💰 Implemented cached token price calculations for OpenAI and Anthropic providers

​

🤖 Fixed bugs and added support for Deepseek-R1 model

• 🤖 Added OpenRouter Deepseek-R1 model
• 🔄 Fixed bug: OpenAI compatible providers (including OpenAI) didn’t return a proper error response message - only “Whoops! We couldn’t send your message. Let us know if this keeps happening.”
• 🔄 Fixed bug: Test inputs (rows) could not be deleted if some columns were hidden

​

🧪 Test Duplication and UI Improvements

• ✨ Added test duplication dialog to copy tests between prompts
• 🗑️ Removed model column from test runs list for cleaner UI
• 🖱️ Fixed scroll behavior on message removal in playground
• 📊 Improved test column actions and styling
• 🔧 Fixed various UI issues with prompt and test form lists
• 🛠️ Fixed tool calls styling in tests - support when LLM responds with both text and tool call
• 🐛 Fixed test runs failing with exceptions in rare cases
• 🔒 Fixed 404 issue after logging out and in as a different user

​

🤖 DeepSeek-V3 Support Added

• 🧠 Added support for DeepSeek-V3, the latest model from the DeepSeek team
• 📚 Pre-trained on nearly 15 trillion tokens
• ⌨️ Excels at instruction following and coding tasks
• 🏆 Outperforms other open-source models and rivals leading closed-source models

​

🧠 OpenAI o1 Models Now Available

• 🤖 Added support for OpenAI’s new o1 models
• 🎯 Features internal reasoning chains for complex problem-solving
• 🤔 Models demonstrate improved reasoning capabilities by thinking before answering

​

🔍 Gemini 2.0 Flash (Experimental) Integration

• 🆕 Added Gemini 2.0 Flash (Experimental) model
• 🎯 First model with precise object bounds detection in images
• 🖼️ Eliminates guesswork in image analysis tasks
• ✅ Extensively tested with impressive results

​

🔧 Improved Test UX and Bug Fixes

• 🚀 It’s much easier to switch between prompt editing (Playground) and Tests now using tab navigation
• 📊 New users now get limited free usage for Anthropic & Google without providing their own API keys
• 🧪 Added Gemini experimental model 1206
• 🐛 Fixed index & id in response for multiple tool calls from Anthropic
• 🐛 Fixed variables list rendering in prompt detail view
• 🐛 Fixed missing user message in assistant playground
• 🐛 Fixed copy-pasting prompts from playground to include new lines correctly
• 🐛 Fixed dev console error when viewing non-JSON test results

​

🎨 Test design updates and new onboarding experience

• 🎨 Redesigned test interface to match playground styling and UX
• 📚 Introduced new user onboarding flow for better getting started experience
• ✨ Magic prompt enhancer with diff view and accept functionality
• 🐞 Fixed assistant deployment snippet generation
• 🐞 Added support for video URLs without file extensions

​

🔗 Improved test response UX and other fixes

• 📏 You can now enable test response text wrapping to easily see longer responses
• 📝 Added support for markdown in the test response drawer
• 🔧 Fixed selection of content in the test response drawer
• 🎨 Fixed test variable icon color on theme switch without reload
• 🔧 Adjusted LLM assertion to allow shorter Yes/No descriptions
• 🔄 Fixed thread messages not saving with assistant API in Anthropic/Google
• 🐛 Fixed a rare bug where a second main test config could be created

​

🔗 Shareable Prompts and Assistants, and UX Enhancements

• 🔗 Added OG tags to shareable prompt assistants for better visibility and discoverability.
• 🐞 When copy-pasting multiple rows into Tests, the order is now fixed.
• ⌨️ Enhanced tab navigation in the messages drawer for better accessibility.
• 🔄 Implemented proper handling of multiple test runs.
• 📋 Formatted the test output preview and added a copy button for improved readability.
• 🐞 Resolved issues with Gemini tool calls in tests and Anthropic model behavior.
• 📝 Added undo/redo functionality to the test table for easier editing.
• 🤖 Introduced a prototype for LLM assertion generation.
• 🤖 Added support for the xAI grok-beta model.
• ✨ Implemented magic to generate descriptions and titles for assistants.

​

🤖 New Model Support, Magic buttons, Testing Improvements, and Onboarding

• 🤖 Added support for the Claude 3.5 Haiku model from Anthropic.
• ✨ Introduced several “magic” buttons throughout the app to improve your experience with LLMs. These include prompts for enhancing prompts, generating test cases, and more! Let us know which ones you find and how useful they are - it’s like an Easter egg hunt!
• 🐞 Fixed an issue where the test data grid couldn’t delete repeated test cases.
• ⚙️ Introduced a “none” option for tool choice.
• 🔍 Added a history viewer for better visibility into past events.
• 📱 Improved onboarding experience for mobile devices.

​

🐞 Post 1.0 Fixes and UI Improvements

• 🎨 Implemented various small UI fixes and improvements.
• 🔢 Fixed an issue with ordering items in the testing table when they have the same creation date.
• 📝 Improved test generation by passing test_inputs to the magic function for better test data.
• 📚 Added templates and documentation links to the sidebar for easier access.

​

🚀 Langtail 1.0: Brand New Testing Interface, Assistants, and More!

​

📊 Brand New Testing Interface

• 📝 At the core of Langtail 1.0 is our new spreadsheet-like interface for testing LLM applications, designed to feel natural for anyone used to working with Excel or Google Sheets.

​

🔄 Test Configurations

• 🎚️ Introducing test configurations, allowing you to easily create and compare tests across different models with just a few clicks.

​

🛠️ Tools with Hosted Code

• 💻 Your prompts can now call tools and have the code run directly within Langtail, making prototyping and testing much easier without the need for mocking responses.
• 🔒 Enjoy secure and fast code execution in a sandboxed environment powered by QuickJS.

​

🧠 Assistants: Memory Management for Prompts

• 🧠 Langtail 1.0 introduces Assistants – stateful entities that automatically manage memory and conversation history, reducing the need for additional code.
• 🔄 Assistants can be used across models, in tests, deployed as APIs, or integrated with tools.

​

🔒 Additional Features

• 🌞 Light Mode: Introduced a new Light mode for the Langtail app, providing a refreshing alternative to the default Dark mode.
• 🛡️ Protect your AI apps from prompt injection attacks and other security vulnerabilities with our built-in AI Firewall.
• 🔐 Execute LLM-generated code securely in a sandboxed container, thanks to our partnership with E2B.
• 📚 Explore a gallery of templates showcasing different ways Langtail can help your team.
• 🤖 Added support for Google Gemini models, including the ability to attach videos to prompts.
• 🏢 Self-hosting: Langtail can now be self-hosted entirely on your infrastructure (contact us for more information).

​

🛠️ Tool Calls in Assistant Playground and Gemini Support

• 🛠️ Added the ability to handle tool calls in the Assistant Playground, allowing you to provide responses for tool calls not handled by Langtail function handlers.
• 🤖 Introduced native support for Gemini models, leveraging the @google/generative-ai library under the hood, with support for tools, streaming, and various media formats as input.

​

📋 Other Improvements

• 🔍 Fixed an issue with opening logs with prompts in the playground.
• 💰 Set Gemini model prices.

​

🌐 Assistant Sharing Redesign, LLM Leaderboard, and Hosted Functions

​

🤖 Assistant Sharing Redesign

• 🌐 Redesigned the assistant sharing feature, allowing you to make your AI assistants publicly accessible and share them as standalone apps.

​

📊 LLM Leaderboard

• 📈 Introduced a new LLM leaderboard page to compare the performance of different language models.

​

🛠️ Hosted Code Support

• 🔌 Added support for Hosted Code, enabling you to test your tools right inside Langtail by directly calling external APIs.

​

🔒 Security and Stability Improvements

• 🔒 Ensured that only team members can edit shared assistants.
• 🤖 Added support for new OpenAI models: o1-mini and o1-preview.
• 🔧 Fixed issues with tool usage for Azure models and display names.

​

🔍 Shared Assistant Indexes

• 🔍 Implemented shared assistant indexes for improved discoverability and searchability.

​

📝 Text Assertion Enhancements and LLaMa 3.1 Support

• 📝 Grouped all text operation assertions under “Text” for better organization.
• 🔖 Added new text assertion options: not equals, does not contain, and does not icontain.
• 📝 Introduced a text editor for the “Text” assertion modal form, with support for variables.
• 🤖 Added support for new LLaMa 3.1 models from groq.

​

💰 Usage Limits, Test Improvements, and GPT-4o Mini Support

• 🤖 Added support for GPT-4o Mini, OpenAI’s cost-efficient model offering advanced text and multimodal performance
• 💰 Implemented a banner to notify free users when they exceed usage limits.
• 🐞 Fixed an issue with the test ID for the assertion popup in test results.
• 🎚️ Improved the layout styles for project logs.

​

🌐 Expanded Message Types and Provider Visibility

• 📨 Added support for other message types, tools, and images in the “messages” column, allowing you to simulate user and assistant messages, as well as mock tools and upload images for Vision testing.
• 🔓 Unlocked providers are now displayed at the top for better visibility.

​

🎨 Layout Redesign and Test Table Improvements

​

🔍 Enhanced Project-Level Visibility

• Playground
• Tests
• Deployments
• Assistants

​

🐞 Test Table Fixes and Enhancements

• 🎥 Fixed an issue with the test table width when the drawer is open.
• 🔘 Added the ability to close the cell edit drawer by pressing the ESC key.
• 🐛 Resolved issues with HTML pasting in the data table.
• 📋 Improved the display of relevant information in the test history.

​

🔒 Security Enhancements

• 🔒 Fixed issues related to system_fingerprint and model value for Anthropic models.
• 🔼 Updated to the latest SDK, resolving an OpenAI issue with images.

​

🤖 Introducing Anthropic’s Claude 3.5 Sonnet Model

• 🆕 Added support for Anthropic’s Claude 3.5 Sonnet model, offering faster and cheaper performance compared to Claude 3 Opus, with enhanced reasoning, knowledge, and coding abilities.

​

🐞 Test Table UX Enhancements and Bug Fixes

• 🖼️ Text table UX improvements
• 🎯 Streamlined the handling of messages in the test table for a smoother workflow.
• 🔍 Enhanced the behavior of the edit drawer, ensuring a more intuitive experience.
• ⌨️ Improved keyboard navigation and selection within the test table.
• 🐛 Fixed issues related to arrow key navigation and drawer behavior.

​

🎯 Testing Enhancements: Keyboard-Friendly Table, Helper Variables, and Improved LLM Assertions

​

⌨️ Keyboard-Friendly Testing Table ⌨️

​

🧮 Helper Variables for Assertions 🧮

​

📚 Improved LLM Assertions 📚

​

📨 Multiple Messages in the “Messages” Column 📨

​

⚡ Major Upgrade to Langtail’s Testing Table Component and Upcoming Features

​

⚡ Langtail’s Testing Table Component Upgrade ⚡

• 🚀 Lightning-fast row addition and cell content deletion for efficient test setup
• ➡️ Tab navigation between cells for seamless navigation
• 🗑️ Bulk row deletion for quick test cleanup
• 🎯 Selective test case execution for focused testing
• 🎁 Test result display with LLM and assertion outcomes for comprehensive insights

​

🔜 Upcoming Feature: LLM as a Judge Assertion 🔜

​

New Dashboard View and Launch of Langtail SDK 🚀

​

📊 New Dashboard View 📊

​

🎯 Langtail SDK Release 🎯

​

New Homepage Video, About Us Page, and Embeddable Prompts

​

🎞️ New Homepage Video 🎞️

​

🧑‍💻 About Us Page 🧑‍💻

​

🛣️ Roadmap, Changelog, and Feedback 🛣️

​

🖼️ Embeddable Prompts 🖼️

<iframe src="https://app.langtail.com/playground?shareid=cbGLHKSIff" width="650" height="400" allow="clipboard-write"> </iframe>

​

📚 Docs Expansion, 🖼️ Dynamic OG Images, and 🌍 Project-Level Env Variables

​

📚 Expanded Documentation for Seamless Integration 📚

• 🚀 Deploying Prompts: Learn how to deploy your prompts effortlessly. Read more
• 🌐 REST API Usage: Discover how to leverage our powerful REST API. Learn how

​

🖼️ Dynamic OG Images for Shared Prompts 🖼️

​

🌍 Project-Level Environment Variables 🌍

• 🔒 Secure: Keep your sensitive information safe and separate from the code
• 🎛️ Centralized: Set your variables once at the project level
• 🌐 Accessible: Use your env variables in the JavaScript editor for Test Assertions and soon in the Playground too

​

Langtail.com updates, Proxy Integration, and Quality of Life Improvements 🚀

​

✨ Landing page updates ✨

1. 🐛 Debug prompts effortlessly in the playground
2. ✅ Run tests to ensure your prompts are top-notch
3. 🚀 Deploy prompts in a flash
4. 📊 Monitor production with comprehensive logs and metrics

​

💸 Pricing Preview 💸

• 🆓 Free Tier: Perfect for getting started
• 💰 Mid Tier: Unleash more power at just $99 per month
• 🏢 Enterprise Tier: Tailored solutions for your business needs

​

📚 Docs Page Expansion 📚

​

🌉 Langtail Proxy: Seamless Integration 🌉

1. Create your Langtail account
2. Generate your project API key
3. Modify your code by updating the base URL to "https://proxy.langtail.com/v1 and passing the Langtail API key
4. Test it out and witness the magic!

​

🎨 Quality of Life Improvements 🎨

• 🔍 Log Filtering: Easily filter logs by categories like proxy
• 🎯 Single Test Run: Run individual tests instead of the entire suite
• ❌ Test Cancellation: Cancel test runs whenever you need
• 📅 Current Date in System Prompt: Add the current date using handlebars

​

📆 $Date Helper, 🔍 Enhanced Log Filters, and 🏠 Webpage Updates

​

📆 $Date Helper for Chatbots 📆

​

🔍 Log Filtering 🔍

​

💰 Pricing Page is Live 💰

​

🚀 Introducing Public Sharing of Prompts and Read-Only Playground!

​

🎡 Public Sharing of Prompts 🎡

​

🏞️ Read-Only Playground 🏞️

​

🚀 Major DX Improvements for JavaScript Assertions in Tests and Upcoming Playground Enhancements 🛠️

​

🧪 Enhanced JavaScript Assertions in Tests 🧪

• 📝 Run functions with example input: Simply fill in your example input in the left column and run the function.
• ✅ Auto Input Addition: If any input already exists, we’ll automatically add it for you.
• 🐞 Debugging Support: Use console.log to debug your tests effectively.
• 🌐 API Calls: For advanced use cases, you can now call external APIs using the Fetch API.
• ⚡️ Ultra-fast API requests: Thanks to our robust infrastructure, your API requests will execute swiftly.

​

🚀 New Templating Engine Based on Handlebars for Enhanced Prompt Handling

​

🎨 Templating Engine Support 🎨

• Conditional parts of prompts
• Variations of prompts

​

🚀 Project Settings, Workspace Refinement, Redesigned Logs, and OpenAI Tools Support!

​

🔧 New Project Settings 🔧

​

🛠️ Enhanced Workspace Settings 🛠️

​

📜 Redesigned Logs 📜

​

🔍 OpenAI Tools Support 🔍

​

🌟 GPT-4 Vision is Live, Plus New Workspace Settings and Refined UX 🔍

​

🌟 GPT-4 Vision Now Live 🌟

​

🎨 Experimenting with Vision in Playground 🎨

​

🚀 Deploy Prompts with Vision 🚀

​

🛠️ Workspace Settings and UX Updates 🛠️

• New Workspace Settings: Now, you can manage your provider API keys and team members all in one place.
• Refined UX for Per API Budget: Based on your feedback, we’ve refined the UX and terminology for the per API budget. You must now enable the budget before setting the total for your API key and specific user allowances.