Basic Tests

Here is a prompt that that analyzes the sentiment of some text and returns a JSON object containing that sentiment. Here’s what a basic set of tests could look like:

1. Test 1 - Positive Sentiment

   • Case: “I am happy.”
   • Assertion: An equals assertion with the value {"sentiment": "positive"}

2. Test 2 - Positive Sentiment

   • Case: “I am sad.”
   • Assertion: An equals assertion with the value {"sentiment": "negative"}

3. Test 3 - Is Valid JSON

   • Case: “I saw a blue car.”

   • Assertion: A javascript assertion with the following code:

     test(async ({ message, variables }) => {
       try {
             JSON.parse(message.content);
             return true; // Parsing succeeded, so it's valid JSON
         } catch (e) {
             return false; // Parsing failed, so it's not valid JSON
         }
     });
     

There’s a lot more that could be tested, but having even a basic test suite for each of your prompts can be a great way to safeguard against typos (like misspelling “positive”) or parameter changes (like turning off JSON mode).

If you have any questions regarding tests or best practices for your project, get in touch with us.