If you’ve already started building your application by using OpenAI’s models directly via the REST API but want to start experimenting with Langtail, you can configure your existing code to start sending requests to the OpenAI Proxy. In this guide, we’ll generate a Langtail API key, modify your existing code to send LLM requests via the proxy, and then use Langtail’s observability tools to see the requests coming from your app. Let’s get started:
1

Prerequisites

Before moving on, make sure you have:
  • A Langtail account (create one here)
  • An OpenAI key already added to your Langtail workspace
2

Generate a project API key

  1. In Langtail, open an existing project or create a new one.
  2. In the left sidebar, click Secrets.
  3. Click New API Key, optionally set a name and a budget limit, then click Create.
  4. Copy the key.
3

Modify your code to use the proxy

  1. Add your Langtail project API key to an environment variable. In the following snippet, we use LANGTAIL_API_KEY.
  2. In your codebase, find where you call the OpenAI API. Then, make the following adjustments:
    • In the url, replace https://api.openai.com with https://proxy.langtail.com.
    • In the Authorization header, use your Langtail project API key instead of your OpenAI key.
  3. Here’s an example calling the proxy using cURL:
curl --request POST \
--url https://proxy.langtail.com/v1/chat/completions \
--header "Authorization: Bearer ${LANGTAIL_API_KEY}" \
--header 'Content-Type: application/json' \
--data '{
    "model": "gpt-3.5-turbo",
    "messages": [
        {
            "role": "system",
            "content": "Generate 5 random words."
        }
    ]
  }'
4

Test it out

Run your app and make a request to verify that everything works properly.
5

Observe metrics and logs

  1. Back in Langtail, navigate to the project and then click Logs in the left sidebar.
  2. You should see log entries coming from your app (they have an environment value of “proxy”).
  3. Click on a log to see detailed information about the request and response from OpenAI.
  4. If you want to experiment with this prompt, click Open in Playground. Here you can change parameters and modify different parts of the prompt and then see the result.