Function calling and tool use

Function calling is a feature available with some large language models (LLMs) that allows them to call external program functions (or tools). This allows the model to interact with external systems to retrieve new data for use as input or execute other tasks. This is a foundational building block for agentic AI applications, in which an LLM can chain together various functions to achieve complex objectives. Function calling is also called “tool use” because the manner in which you tell the LLM what functions are available is with a tools parameter in the request body. The inference API is OpenAI-compatible, so you can use the OpenAI SDK without changes.

Function calling is enabled by default, but its availability is model-dependent and will produce valid output only if the model is pretrained to return tool-use responses. Function calling is supported on the /v1/chat/completions endpoint only; it does not apply to /v1/responses, which is used for image and video generation models. See Supported models below.

When to use function calling

You should use function calling when you want your LLM to:

Fetch data: Such as fetch weather data, stock prices, or news updates from a database. The model will call a function to get information, and then incorporate that data into its final response.
Perform actions: Such as modify application states, invoke workflows, or call upon other AI systems. The model will call another tool to perform an action, effectively handing off the request after it determines what the user wants.

How function calling works

When you send an inference request to a model that supports function calling, you can specify which functions are available to the model using the tools body parameter. The tools parameter provides information that allows the LLM to understand:

What each function can do
How to call each function (the arguments it accepts/requires)

For example, here’s a request with the chat completions API that declares an available function named get_weather():

import os
from openai import OpenAI

def get_weather(city: str) -> str:
    print("Get weather:", city)

client = OpenAI(
    base_url="https://api.modular.com/v1",
    api_key=os.environ["MODULAR_API_KEY"],
)

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current temperature for a given location.",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City and country e.g. Bogotá, Colombia"
                }
            },
            "required": [
                "location"
            ],
            "additionalProperties": False
        },
        "strict": True
    }
}]

messages = [
  {
    "role": "user",
    "content": "What's the weather like in San Francisco today?"
  }
]

completion = client.chat.completions.create(
    model="google/gemma-4-31B-it",
    messages=messages,
    tools=tools
)

Let’s take a closer look at each parameter shown in the tools property:

type: Currently this is always function
function: Definition of the function
- name: The function name used by the LLM to call it. Must be at most 64 characters; the default character set is [a-zA-Z0-9_-], though some model parsers widen it to allow dotted or namespaced names (for example, my-tool.v2)
- description: A function description that helps the LLM understand when to use it
- parameters: Definition of the function parameters
  - type: Defines this as an object containing parameters
  - properties: Lists all possible function arguments and their types
  - required: Specifies which function arguments are required

This format follows the OpenAI function calling specification to specify functions as tools that a model can use. You can also control whether (and how) the model is required to call a tool with the tool_choice parameter:

none: The model won’t call any tool.
auto (default): The model decides whether to call a tool or respond with a message.
required: The model must call one or more tools.
A specific-function object: Forces the model to call that named function, for example:
{"type": "function", "function": {"name": "get_weather"}}

Using this information, the model will decide whether to call any functions specified in tools. In this case, we expect the model to call get_weather() and incorporate that information into its final response. So, the initial completion response from above includes a tool_calls parameter like this:

print(completion.choices[0].message.tool_calls)

[ChatCompletionMessageToolCall(
  id='call_a175692d9ff54554',
  function=Function(
    arguments='{
      "location": "San Francisco, USA"
    }',
    name='get_weather'
  ),
  type='function'
)]

From here, you must parse the tool_calls body and execute the function as appropriate. For example:

import json

tool_call = completion.choices[0].message.tool_calls[0]
args = json.loads(tool_call.function.arguments)

result = get_weather(args["location"])

If the function is designed to fetch data for the model, you should call the function and then call the model again with the function results appended as a message using the tool role. Append the assistant message that contained the tool_calls, then one message per result with the tool role and the matching tool_call_id:

"messages": [
  { "role": "user", "content": "What's the weather in San Francisco?" },
  {
    "role": "assistant",
    "tool_calls": [
      {
        "id": "call_abc123",
        "type": "function",
        "function": {
          "name": "get_weather",
          "arguments": "{\"location\": \"San Francisco\"}"
        }
      }
    ]
  },
  {
    "role": "tool",
    "tool_call_id": "call_abc123",
    "content": "{\"temperature\": 62, \"unit\": \"F\"}"
  }
]

Each tool message’s tool_call_id must match the id of a tool call in the preceding assistant message. A request with a non-JSON tool_calls argument, an unmatched tool_call_id, or only a partial set of tool replies is rejected with a 400 error. If the function is designed to perform an action, then you don’t need to call the model again. For detail about how to execute the function and feed the results back to the model, see the OpenAI docs about handling function calls. The OpenAI function calling spec is compatible with multiple agent frameworks, such as AutoGen, CrewAI, and more.

Supported models

Function calling is model-dependent and will produce valid output only if the model is pretrained to return tool-use responses. Streaming (stream: true) with function calling also varies by model. If you see incomplete or malformed tool-call output while streaming, set stream: false for that request. Function calling only applies to text-generation models called through /v1/chat/completions. It isn’t applicable to /v1/responses, since that endpoint is used exclusively for image and video generation models, which don’t accept a tools parameter. See the supported models page for the current list of models available on shared and dedicated endpoints, and which ones support function calling.

Quickstart

This walks through the same get_weather example from above as a single, runnable script. If you haven’t already, create an API key in the Modular Console and export it:

export MODULAR_API_KEY="your_api_key_here"

Then send a request with the tools parameter to a model that supports function calling:

Python
curl

Install the OpenAI SDK:

pip install openai

Create a program to send a request specifying the available get_weather() function:

function-calling.py

import os
import json
from openai import OpenAI

def get_weather(city: str) -> str:
    print("Get weather:", city)

client = OpenAI(
    base_url="https://api.modular.com/v1",
    api_key=os.environ["MODULAR_API_KEY"],
)

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current temperature for a given location.",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City and country e.g. Bogotá, Colombia"
                }
            },
            "required": [
                "location"
            ],
            "additionalProperties": False
        },
        "strict": True
    }
}]

messages = [
  {
    "role": "user",
    "content": "What's the weather like in San Francisco today?"
  }
]

completion = client.chat.completions.create(
    model="google/gemma-4-31B-it",
    messages=messages,
    tools=tools
)

tool_call = completion.choices[0].message.tool_calls[0]
args = json.loads(tool_call.function.arguments)

result = get_weather(args["location"])

Run it and the get_weather() function should print the argument received:

python function-calling.py

Get weather: San Francisco, USA

Use the following curl command to send a request specifying the available get_weather() function:

curl -X POST https://api.modular.com/v1/chat/completions \
    -H "Authorization: Bearer $MODULAR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
        "model": "google/gemma-4-31B-it",
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "What is the weather like in Boston today?"}
        ],
        "tools": [
            {
                "type": "function",
                "function": {
                    "name": "get_weather",
                    "description": "Get the current weather in a given location",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "location": {
                                "type": "string",
                                "description": "The city and state, e.g. Los Angeles, CA"
                            }
                        },
                        "required": ["location"]
                    }
                }
            }
        ],
        "tool_choice": "auto"
    }'

You should receive a response similar to this:

"tool_calls": [
  {
    "id": "call_ac73df14fe184349",
    "type": "function",
    "function": {
        "name": "get_weather",
        "arguments": "{\"location\": \"Boston, MA\"}"
    }
  }
]

For a more complete walkthrough of how to handle a tool_calls response and send the function results back to the LLM as input, see the OpenAI docs about handling function calls.

Inference

Deployments

Administration

Function calling and tool use

When to use function calling

How function calling works

Supported models

Quickstart

​When to use function calling

​How function calling works

​Supported models

​Quickstart

When to use function calling

How function calling works

Supported models

Quickstart