Achieve Function Calling and Tool Use in Large Language Models

This article will tell you how to achieve a function calling ability in Large Language Models (LLM) and how to use the function in the LLM.

Implementation architecture: Client + Service

Client

User Input: Query + Function Description (system)
Example:
Query: Show 5 rows of data.
Function Description:

[{
   "name": "show_data_head",
   "description": "Show top n row of data.",
   "parameters": {
       "type": "object",
       "properties": {
       "row": {
           "type": "string",
           "description": "number of rows to show."
          }
       }
      }
   },
....
]

Service

Use React Template:

"""Answer the following questions as best you can. You have access to the following APIs:

{tools_text}

Use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tools_name_text}]
Action Input: the input to the action, if no parameters are provided, marking this as empty.
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can be repeated zero or more times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question

Begin!"""

The {tools_text} is the processed nature langue description of above API description.

There is an example for processing it to nature langue:

TOOL_DESC_WITH_PARAMETERS = (
    '{name_for_model}: Call this tool to interact with the {name_for_human} API.'
    ' What is the {name_for_human} API useful for? {description_for_model} Parameters: {parameters}'
)

So, for request above from client. The overall prompt shoud be:

"""Answer the following questions as best you can. You have access to the following APIs:

[
		'show_data_head: Call this tool to interact with the show_data_head API. '
    'What is the show_data_head API useful for? show top n row of data. '
    'Parameters: {"type": "object", "properties": {"row": {"type": "string", '
    '"description": "number of rows to show."}}}\n
    ....
]

Use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [***show_data_head***, ....]
Action Input: the input to the action, if no parameters are provided, marking this as empty.
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can be repeated zero or more times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question

Begin!"""

Stopword

If we let the LLM do text generation by the above react prompt directly, the model will predict the text of Observation: the result of the action directly. That isn’t we want. We want the text here is the result by our API. So it is essentially for make model stop generation in Observation.

We will add additional stop word to model for let it pauses in the stop words.

Here, we add “Observation” to the stop text.

Response

We will parse the “Action” above to Json formation to return. We refer the response data of Openai here

{
  "model": "Qwen",
  "object": "chat.completion",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Thought: I need to use the show_data_head API to display the first few rows of the data.",
        "function_call": {
          "name": "show_data_head",
          "arguments": {"row": "5"}
        }
      },
      "finish_reason": "function_call"
    }
  ],
  "created": 170228275
}

API Calling(Client)

We got the response Json of LLM. Now, we can call the API and passing the parameters according to Json. Then, we will get the result of our API.

def show_data_head(self, row: int = 5):
    try:
        #self.message_recorder.append(f"{} rows of data: {self.data.head(int(row))}")
        return self.data.head(int(row))
    except Exception as e:
        print(f"Error in show data: {e}")

Result:

Sepal.Length	Sepal.Width	Petal.Length	Petal.Width	Species
5.1	3.5	1.4	0.2	setosa
4.9	3.0	1.4	0.2	setosa
4.7	3.2	1.3	0.2	setosa
4.6	3.1	1.5	0.2	setosa
5.0	3.6	1.4	0.2	setosa

Importantly, we should add both of LLM’s response (Json) and the result of API to message of next request.

Then, we will call LLM again.

Second Response for API Results

Note that previously we use template of “chat” for LLM

But now, we want model do text continuation continuing the “Observation: {result of our API}”. So we should use “completion” template here.

<completion>
<|im_start|>system
You are a data scientist, your mission is help human to do data analysis, data mining and generate report.<|im_end|>
<|im_start|>user
hi<|im_end|>
<|im_start|>assistant
Hello! How can I assist you with your data analysis, data mining, or report generation? Please provide me with some details about the data you have and what insights you're hoping to gain, and we can get started.<|im_end|>
<|im_start|>user
show 5 rows of data<|im_end|>
<|im_start|>assistant
Thought: I need to use the show_data_head API to display the first few rows of the data.
Action: show_data_head
Action Input: {"row": "5"}
Observation:    Sepal.Length  Sepal.Width  Petal.Length  Petal.Width Species
0           5.1          3.5           1.4          0.2  setosa
1           4.9          3.0           1.4          0.2  setosa
2           4.7          3.2           1.3          0.2  setosa
3           4.6          3.1           1.5          0.2  setosa
4           5.0          3.6           1.4          0.2  setosa
Thought:
<!-- *** -->

The model will do text continuation from the last Thought:

Thought:
<!-- *** -->
The API has displayed the first 5 rows of the data as requested.
Response: Here are the first 5 rows of the data:

|Sepal.Length| Sepal.Width| Petal.Length| Petal.Width|   Species|
|------------|------------|-------------|------------|----------|
|       5.1|         3.5|          1.4|         0.2|    setosa|
|       4.9|         3.0|          1.4|         0.2|    setosa|
|       4.7|         3.2|          1.3|         0.2|    setosa|
|       4.6|         3.1|          1.5|         0.2|    setosa|
|       5.0|         3.6|          1.4|         0.2|    setosa|
</completion>

Finally, parse the response again and return to the client.