OpenAI Functions and Commerce

It’s been almost two weeks since OpenAI launched function calling and we have been busy implementing this into the Response app.

With Functions we are able to define different functions using JSON; we pass in the name, description and parameters of the functions to the model. The model can understand this structure and attempt to do the intent and parameter detection.

Functions are essentially intents.

Parameters for an API call are basically entities.

Where is this very useful?

With response often we would need to pass in specific slash commands which allow us to route specific requests let’s say to add or update a vector embeddings. This can be done on the client side (in the web app UI code).

However, what we can do now is completely remove the “/add” or “/update” commands and now actually pass the intent into the Response Chat API.

The Idea

Here are the functions we can now use to make Response an intent driven autonomous agent which will be able to update it’s context dynamically (without us having to use slash commands to have it update it’s memory).

    let functionList = [{
      "name": "add",
      "description": "the add function is used when a user wants to adds data to the knowledge base that is powering the ReSponse app.",
      "parameters": {
        "type": "object",
        "properties": {
          "knowledge": {
            "type": "string",
            "description": "the knowledge property is the the data that needs to be added new new to the knowledge base"
          }
        },
        "required": ["knowledge"]
      },
    },
    {
      "name": "update",
      "description": "the update function is used when a user wants to update data that exist in the knowledge base that is powering the ReSponse app.",
      "parameters": {
        "type": "object",
        "properties": {
          "knowledge": {
            "type": "string",
            "description": "the knowledge property is the what the data needs to be updated to in the knowledge base"
          }
        },
        "required": ["knowledge"]
      }
    },
    {
      "name": "respond",
      "description": "the response function is used when a user is asking a question and wants to be responded to with the answer of the question based on data in the knowledge base that is powering the ReSponse app",
      "parameters": {
        "type": "object",
        "properties": {
          "question": {
            "type": "string",
            "description": "the question property is the question to be responded to from the knowledge base"
          }
        },
        "required": ["question"]
      },
    }];

The function calls the OpenAI API using fetch and waits for the response. If the response indicates that a function call is required, it extracts the function call name and arguments from the response and invokes the corresponding function. Otherwise, it returns the JSON response.

We can now use the chat function as a router, a parameter recognizer; a more deterministic output generator instead.

With functions, the output is the function name (add, update etc), and parameters in this case, knowledge. We can then call the API to update the vector embeddings and store the function response in the chat history.

{
  id: 'chatcmpl-7RhaulPv35ko1PhxbkjTPfp3xX5UL',
  object: 'chat.completion',
  created: 1686836788,
  model: 'gpt-4-0613',
  choices: [ { index: 0, message: [Object], finish_reason: 'function_call' } ],
  usage: { prompt_tokens: 395, completion_tokens: 103, total_tokens: 498 }
}
function: add
{
  knowledge: 'Stateset Inc. is an AI-powered software as a service (SaaS) company focused on empowering organizations in the commerce space. Our mission is to revolutionize commerce operations by providing innovative technology solutions that enable brands to achieve efficiency and scale. With our three SaaS products - Stateset ReSponse App, Stateset One View, and Stateset Cloud Platform - along with our Commerce Network, we aim to become the leading technology provider in the industry.'
}

knowledge: Stateset Inc. is an AI-powered software as a service (SaaS) company focused on empowering organizations in the commerce space. Our mission is to revolutionize commerce operations by providing innovative technology solutions that enable brands to achieve efficiency and scale. With our three SaaS products - Stateset ReSponse App, Stateset One View, and Stateset Cloud Platform - along with our Commerce Network, we aim to become the leading technology provider in the industry.

{ status: 'updated_pinecone_vector' }

Knowledge Base Updated!

Why is this impressive… all of the request that we were previously front routing on the client to API calls; but now they can actually now all go the same chat endpoint and based on the intent detected it will map to the specific function with the needed parameters! This opens up a ton of opportunities and make adding and updating data in response much more dynamic and intent based!

Entity Detection Use Case in Commerce

This brings up a few questions... if it doesn't have all of the required parameters is it able to ask for the parameter and pull both from chat history and make the call?

For an example use case related to Subscriptions Management.

If the customer asked ReSponse could you please pause my subscription for [email protected]? If it requires email and paused until date, detects that the data was provided, asks for it and then can execute the call after it has both?

Before (back in 2016) these sorts of entity detection engine like LUIS worked great… however were fairly rigid, it needed examples, they needed to be labeled etc. This new parameter detection actually makes it much more generic (again) and by using the chat history we can design different workflows across multiple functions as well.

If you have more control over the functions that are executed based on inputs / outputs from the model, and the history of the model output this could make sense.

For example if it is unable to find the subscription id by the email and returns an error back in the output and says something along the lines of; we were not able to find a subscription with that email, could you please provide another one... and then can re-execute the function call because it knows it didn't successfully the try before...?

Again, this makes things much more dynamic and generalized for chaining different API calls together and automating tasks related to CX & Ops.

Chaining Functions

The chaining idea of providing examples is a way to increase output accuracy however this actually introduces a chaining way to model whether or not external state has been successfully updated…

So let's take the same example: If the model knows that the subscription was updated in the history based on the response from the call to “pause the subscription until the end of July” then any subsequent call should also maintain this in perfect, though finite memory.

Depending on the context / prompt window this model could be a great way to log each interaction with the customer. It already seems to be gearing up to be that sort of feed of interactions between user, assistant, function, and system messages.

Authentication Function Flows

Function calling for automating specific tasks that currently require login / authenticate credentials still may require some additional system prompts. For example if you want to confirm that the user if actually indeed the one that wants to pause their subscription update until end of July, maybe there is another call that is made to an API that sends a confirmation code to the user.

This response could be something like, "happy to help you with that, I just sent a confirmation code to your email, could you please verify that with me"?

Then upon entering the code another call is made to verify the code and that this is indeed an authorized user that wants to pause their subscription.

Intent Confirmation seems to also be a very important requirement for function calling (and using the chat history state as whether or not it has confirmation).

Future Apps

The chat history as a stored array with functions making external calls has a number of applications and will continue to drive new innovative solutions for integrations.

It already saved us a couple hundred lines of code of doing intent detection using davinci-text-003 to make API calls dynamically based on the intent / route that was detected.

The combination of chat history for state, vector based context for knowledge / memory and now functions will continue to drive the innovation of generative AI and help us build the autonomous CX agents that we are very excited about.