Integrate via OpenAI library

If you’ve already started building your application using the OpenAI SDK but want to start experimenting with Langtail, you can configure your existing code to start sending requests to the OpenAI Proxy. In this guide, we’ll generate a Langtail API key, modify your existing code to send LLM requests via the proxy, and then use Langtail’s observability tools to see the requests coming from your app. Let’s get started:

Prerequisites

Before moving on, make sure you have:

A Langtail account (create one here)
An OpenAI key already added to your Langtail workspace

Generate a project API key

In Langtail, open an existing project or create a new one.
In the left sidebar, click Secrets.
Click New API Key, optionally set a name and a budget limit, then click Create.
Copy the key.

Modify your code to use the proxy

Add your Langtail project API key to an environment variable. In the following snippets, we use LANGTAIL_API_KEY.
In your codebase, find where you initialize the OpenAI SDK and make these modifications:

OpenAI SDK v4

import OpenAI from "openai";

// Add baseURL to send requests to Langtail instead of directly to OpenAI
const openai = new OpenAI({
  apiKey: process.env.LANGTAIL_API_KEY,
  baseURL: "https://proxy.langtail.com/v1",
  defaultHeaders: {
    "X-Langtail-Metadata-key": "value", // optional; 'key' can be any string
  },
});

// Then submit a request as usual and it will be proxied through Langtail.
// Optionally, add headers for easier trace filtering.
const chatCompletion = await openai.chat.completions.create(
  {
    messages: [{ role: "user", content: "Hello world" }],
    model: "gpt-3.5-turbo",
  },
  {
    headers: {
      "X-Langtail-Prompt": "prompt-slug", // optional
      "X-Langtail-Metadata-key": "value", // optional; 'key' can be any string
    },
  }
);
console.log(chatCompletion.choices[0].message)

OpenAI SDK v3 or lower

import { Configuration, OpenAIApi } from "openai";

// Add basePath to send requests to Langtail instead of directly to OpenAI
const configuration = new Configuration({
  apiKey: process.env.LANGTAIL_API_KEY,
  basePath: "https://proxy.langtail.com/v1",
  baseOptions: {
    headers: {
      "X-Langtail-Metadata-key": "value", // optional; 'key' can be any string
    },
  },
});
const openai = new OpenAIApi(configuration);

// Then submit a request as usual and it will be proxied through Langtail.
// Optionally, add headers for easier trace filtering.
const chatCompletion = await openai.createChatCompletion(
  {
    model: "gpt-3.5-turbo",
    messages: [{ role: "user", content: "Hello world" }],
  },
  {
    headers: {
      "X-Langtail-Prompt": "prompt-slug", // optional
      "X-Langtail-Metadata-key": "value", // optional; 'key' can be any string
    },
  }
);
console.log(chatCompletion.data.choices[0].message);

Test it out

Run your app and make a request to verify that everything works properly.

Observe metrics and logs

Back in Langtail, navigate to the project and then click Logs in the left sidebar.
You should see log entries coming from your app (they have an environment value of “proxy”).
Click on a log to see detailed information about the request and response from OpenAI.
If you want to experiment with this prompt, click Open in Playground. Here you can change parameters and modify different parts of the prompt and then see the result.

Introduction

Concepts and Examples

References

Media & Presentations

Support & Community

Integrate via OpenAI library