Deepnote Accelerates AI Feature Development From Days to Hours Using Langtail
Snapshot:
Deepnote, the AI-powered data workspace, launched its AI assistant to help users write notebook blocks faster, but it needed a way to ensure that AI outputs are contextually relevant and solve user problems. Initially, Deepnote’s team spent days fine-tuning prompts for its complex use cases. Then it started using Langtail to make its large language model (LLM) more accurate and consistent. With Langtail, Deepnote follows a data-driven, systematic approach to AI testing and development, saving its team hundreds of hours and giving developers confidence that their work will pay off.
Key Metrics:
- Saved Deepnote 100s of hours on AI feature development
- Boosted user adoption of AI by 31%
- Achieved a 20% as-is acceptance rate for AI suggestions
- Increased productivity for 75% of users
About Deepnote
Deepnote, the AI-powered data workspace, launched an AI assistant in July 2023 to help its users write data blocks faster. Since then, Deepnote AI has grown into one of the company’s most-used features, putting advanced data science capabilities in the hands of business professionals.
To learn more about Deepnote’s AI journey, check out the links at the end of this page.
Challenge | Saving Hundreds of Hours by Achieving LLM Consistency Using Langtail
While LLMs excel at natural language processing, data science is a different beast. For instance, Deepnote found that it was easy to achieve a 75 percent success rate with AI features and get a proof of concept approved — but that’s when the trouble started.
Deepnote users often work with a number of databases, many of which are written in distinct SQL dialects. Early AI features frequently ignored these nuances and delivered inconsistent and unusable results. Engineers then had to spend days refining prompts to achieve quality targets. Often, they simply ran into dead ends.
“Before discovering Langtail, developers would hit dead ends for hours — sometimes days. We simply didn’t know how to tame the LLM to make it consistent. Using Langtail, we’ve saved our team hundreds of hours and plenty of headaches.”
Solution | Came to Langtail for the User-Friendly Playground, Stayed for the Data-Driven Testing and Evaluation
Deepnote developers knew they needed a data-driven approach to AI feature development. After evaluating several options, they started using Langtail to manage LLM prompts in October 2023. Then they discovered they could use Langtail to systematically test prompts.
Deepnote started generating test sets in Langtail and verifying that its AI produces consistent results with a wide spectrum of user inputs. Now developers are confident in their systematic, data-driven process, and customers are getting new features faster than ever.
“Using Langtail’s testing and evaluation environment, we reduced our AI production timeline from days to hours.”
Results | Accelerating AI Development with Confidence Using Langtail
Data professionals are working smarter and faster with Deepnote AI. In year one, AI user adoption increased by 31 percent, and three-quarters of Deepnote users say that AI increases their productivity. Moreover, about 20 percent of Deepnote AI suggestions are accepted as-is — a remarkable rate considering the complexity of data science use cases. As Romancov puts it, “For AI, data science is a trial by fire, and Langtail passed.”
Deepnote shaved days off AI feature development by using Langtail. With quality improvements evident in vibe checks, developers are excited to be shipping new features and increasing the value they offer Deepnote users.
Testimonial
“During vibe checks, it’s obvious that using Langtail has transformed our AI features. Not only is our team enjoying significant time savings, but our developers now have a systematic, data-driven approach to AI development. Before using Langtail, we didn’t have a systematic way to verify that our work was heading in the right direction, so we often took one step forward and two steps back. With Langtail, our AI feature development is based on data insights, and we’re shipping to customers faster than ever.”
See For Yourself
To learn more about how Langtail can help your team develop and ship AI applications faster, we would love to talk with you.
To read about Deepnote’s AI journey, check out these blogs: