If you’ve already started building your application using the OpenAI SDK but want to start experimenting with Langtail, you can configure your existing code to start sending requests to the OpenAI Proxy.
In this guide, we’ll generate a Langtail API key, modify your existing code to send LLM requests via the proxy, and then use Langtail’s observability tools to see the requests coming from your app.
An OpenAI key already added to your Langtail workspace
2
Generate a project API key
In Langtail, open an existing project or create a new one.
In the left sidebar, click Secrets.
Click New API Key, optionally set a name and a budget limit, then click Create.
Copy the key.
3
Modify your code to use the proxy
Add your Langtail project API key to an environment variable. In the following snippets, we use LANGTAIL_API_KEY.
In your codebase, find where you initialize the OpenAI SDK and make these modifications:
OpenAI SDK v4
Copy
Ask AI
import OpenAI from "openai";// Add baseURL to send requests to Langtail instead of directly to OpenAIconst openai = new OpenAI({ apiKey: process.env.LANGTAIL_API_KEY, baseURL: "https://proxy.langtail.com/v1", defaultHeaders: { "X-Langtail-Metadata-key": "value", // optional; 'key' can be any string },});// Then submit a request as usual and it will be proxied through Langtail.// Optionally, add headers for easier trace filtering.const chatCompletion = await openai.chat.completions.create( { messages: [{ role: "user", content: "Hello world" }], model: "gpt-3.5-turbo", }, { headers: { "X-Langtail-Prompt": "prompt-slug", // optional "X-Langtail-Metadata-key": "value", // optional; 'key' can be any string }, });console.log(chatCompletion.choices[0].message)
OpenAI SDK v3 or lower
Copy
Ask AI
import { Configuration, OpenAIApi } from "openai";// Add basePath to send requests to Langtail instead of directly to OpenAIconst configuration = new Configuration({ apiKey: process.env.LANGTAIL_API_KEY, basePath: "https://proxy.langtail.com/v1", baseOptions: { headers: { "X-Langtail-Metadata-key": "value", // optional; 'key' can be any string }, },});const openai = new OpenAIApi(configuration);// Then submit a request as usual and it will be proxied through Langtail.// Optionally, add headers for easier trace filtering.const chatCompletion = await openai.createChatCompletion( { model: "gpt-3.5-turbo", messages: [{ role: "user", content: "Hello world" }], }, { headers: { "X-Langtail-Prompt": "prompt-slug", // optional "X-Langtail-Metadata-key": "value", // optional; 'key' can be any string }, });console.log(chatCompletion.data.choices[0].message);
OpenAI SDK v4
Copy
Ask AI
import OpenAI from "openai";// Add baseURL to send requests to Langtail instead of directly to OpenAIconst openai = new OpenAI({ apiKey: process.env.LANGTAIL_API_KEY, baseURL: "https://proxy.langtail.com/v1", defaultHeaders: { "X-Langtail-Metadata-key": "value", // optional; 'key' can be any string },});// Then submit a request as usual and it will be proxied through Langtail.// Optionally, add headers for easier trace filtering.const chatCompletion = await openai.chat.completions.create( { messages: [{ role: "user", content: "Hello world" }], model: "gpt-3.5-turbo", }, { headers: { "X-Langtail-Prompt": "prompt-slug", // optional "X-Langtail-Metadata-key": "value", // optional; 'key' can be any string }, });console.log(chatCompletion.choices[0].message)
OpenAI SDK v3 or lower
Copy
Ask AI
import { Configuration, OpenAIApi } from "openai";// Add basePath to send requests to Langtail instead of directly to OpenAIconst configuration = new Configuration({ apiKey: process.env.LANGTAIL_API_KEY, basePath: "https://proxy.langtail.com/v1", baseOptions: { headers: { "X-Langtail-Metadata-key": "value", // optional; 'key' can be any string }, },});const openai = new OpenAIApi(configuration);// Then submit a request as usual and it will be proxied through Langtail.// Optionally, add headers for easier trace filtering.const chatCompletion = await openai.createChatCompletion( { model: "gpt-3.5-turbo", messages: [{ role: "user", content: "Hello world" }], }, { headers: { "X-Langtail-Prompt": "prompt-slug", // optional "X-Langtail-Metadata-key": "value", // optional; 'key' can be any string }, });console.log(chatCompletion.data.choices[0].message);
OpenAI SDK v1
Copy
Ask AI
import osfrom openai import OpenAI# Add base_url to send requests to Langtail instead of directly to OpenAIclient = OpenAI( api_key=os.environ["LANGTAIL_API_KEY"], base_url="https://proxy.langtail.com/v1", default_headers={ "X-Langtail-Metadata-key": "value", # optional; 'key' can be any string },)# Then submit a request as usual and it will be proxied through Langtail.# Optionally, add headers for easier trace filtering.chat_completion = client.chat.completions.create( model="gpt-4", messages=[{"role": "user", "content": "Hello world!"}], extra_headers={ "X-Langtail-Prompt": "prompt-slug", # optional "X-Langtail-Metadata-key": "value", # optional; 'key' can be any string },)print(chat_completion)
OpenAI SDK v0.x
Copy
Ask AI
import osimport openai# Add api_base to send requests to Langtail instead of directly to OpenAIopenai.api_key = os.environ["LANGTAIL_API_KEY"]openai.api_base = "https://proxy.langtail.com/v1"# Then submit a request as usual and it will be proxied through Langtail.# Optionally, add headers for easier trace filtering.completion = openai.ChatCompletion.create( model="gpt-4", messages=[{"role": "user", "content": "Hello world!"}], headers={ "X-Langtail-Prompt": "prompt-slug", # optional "X-Langtail-Metadata-key": "value", # optional; 'key' can be any string },)print(completion)
4
Test it out
Run your app and make a request to verify that everything works properly.
5
Observe metrics and logs
Back in Langtail, navigate to the project and then click Logs in the left sidebar.
You should see log entries coming from your app (they have an environment value of “proxy”).
Click on a log to see detailed information about the request and response from OpenAI.
If you want to experiment with this prompt, click Open in Playground. Here you can change parameters and modify different parts of the prompt and then see the result.
