> ## Documentation Index > Fetch the complete documentation index at: https://docs-preview.modular.com/llms.txt > Use this file to discover all available pages before exploring further. # Create chat completion > Generate a response from a conversation. Accepts text, image, and video input for multimodal models, and supports function calling via `tools`/`tool_choice`. Request and response shapes match the OpenAI Chat Completions API, so existing OpenAI SDK clients work unchanged. ## OpenAPI ````yaml /reference/openapi-inference.json post /v1/chat/completions openapi: 3.1.0 info: title: Modular Cloud Inference API version: 0.1.0 description: >- REST API for Modular Cloud inference. All endpoints are served at `https://api.modular.com`. Run inference against hosted models. Different model types use different endpoints. See the [supported models](/models) page to check which endpoint each model uses. servers: - url: https://api.modular.com security: - BearerAuth: [] tags: - name: Inference description: >- Run inference against hosted models. Text and chat models use `POST /v1/chat/completions`; image and video generation models use `POST /v1/responses`. Check the [supported models](/models) page to see which endpoint each model uses. paths: /v1/chat/completions: post: tags: - Inference summary: Create chat completion description: >- Generate a response from a conversation. Accepts text, image, and video input for multimodal models, and supports function calling via `tools`/`tool_choice`. Request and response shapes match the OpenAI Chat Completions API, so existing OpenAI SDK clients work unchanged. operationId: createChatCompletion requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/ChatCompletionRequest' examples: text-to-text: summary: Text input value: model: google/gemma-4-26B-A4B-it messages: - role: user content: What is the capital of France? image-to-text: summary: Image input value: model: google/gemma-4-26B-A4B-it messages: - role: user content: - type: image_url image_url: url: https://example.com/image.jpg - type: text text: Describe this image. video-to-text: summary: Video input value: model: google/gemma-4-26B-A4B-it messages: - role: user content: - type: video_url video_url: url: https://example.com/video.mp4 - type: text text: Describe what happens in this video. function-calling: summary: Function calling value: model: google/gemma-4-31B-it messages: - role: user content: What's the weather like in San Francisco today? tools: - type: function function: name: get_weather description: Get current temperature for a given location. parameters: type: object properties: location: type: string description: City and country e.g. Bogotá, Colombia required: - location additionalProperties: false strict: true tool_choice: auto responses: '200': description: Chat completion response. content: application/json: schema: $ref: '#/components/schemas/ChatCompletionResponse' examples: text: summary: Text response value: id: chatcmpl-abc123 object: chat.completion model: google/gemma-4-26B-A4B-it choices: - index: 0 message: role: assistant content: The capital of France is Paris. finish_reason: stop usage: prompt_tokens: 12 completion_tokens: 9 total_tokens: 21 tool-calls: summary: Tool call response value: id: chatcmpl-abc123 object: chat.completion model: google/gemma-4-31B-it choices: - index: 0 message: role: assistant content: null tool_calls: - id: call_ac73df14fe184349 type: function function: name: get_weather arguments: '{"location": "San Francisco, USA"}' finish_reason: tool_calls usage: prompt_tokens: 45 completion_tokens: 18 total_tokens: 63 x-codeSamples: - lang: Python label: Text → text source: |- import os from openai import OpenAI client = OpenAI( base_url="https://api.modular.com/v1", api_key=os.environ["MODULAR_API_KEY"], ) response = client.chat.completions.create( model="google/gemma-4-26B-A4B-it", messages=[{"role": "user", "content": "What is the capital of France?"}], ) print(response.choices[0].message.content) - lang: Python label: Function calling source: |- import os from openai import OpenAI client = OpenAI( base_url="https://api.modular.com/v1", api_key=os.environ["MODULAR_API_KEY"], ) tools = [{ "type": "function", "function": { "name": "get_weather", "description": "Get current temperature for a given location.", "parameters": { "type": "object", "properties": { "location": {"type": "string", "description": "City and country e.g. Bogotá, Colombia"} }, "required": ["location"], "additionalProperties": False }, "strict": True } }] response = client.chat.completions.create( model="google/gemma-4-31B-it", messages=[{"role": "user", "content": "What's the weather like in San Francisco today?"}], tools=tools, ) print(response.choices[0].message.tool_calls) - lang: Python label: Image → text source: |- import os from openai import OpenAI client = OpenAI( base_url="https://api.modular.com/v1", api_key=os.environ["MODULAR_API_KEY"], ) response = client.chat.completions.create( model="google/gemma-4-26B-A4B-it", messages=[ { "role": "user", "content": [ {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg"}}, {"type": "text", "text": "Describe this image."}, ], } ], ) print(response.choices[0].message.content) - lang: Python label: Video → text source: |- import os from openai import OpenAI client = OpenAI( base_url="https://api.modular.com/v1", api_key=os.environ["MODULAR_API_KEY"], ) response = client.chat.completions.create( model="google/gemma-4-26B-A4B-it", messages=[ { "role": "user", "content": [ {"type": "video_url", "video_url": {"url": "https://example.com/video.mp4"}}, {"type": "text", "text": "Describe what happens in this video."}, ], } ], ) print(response.choices[0].message.content) - lang: Shell label: Text → text source: |- curl -X POST https://api.modular.com/v1/chat/completions \ -H "Authorization: Bearer $MODULAR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "google/gemma-4-26B-A4B-it", "messages": [{"role": "user", "content": "What is the capital of France?"}] }' - lang: Shell label: Image → text source: |- curl -X POST https://api.modular.com/v1/chat/completions \ -H "Authorization: Bearer $MODULAR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "google/gemma-4-26B-A4B-it", "messages": [ { "role": "user", "content": [ {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg"}}, {"type": "text", "text": "Describe this image."} ] } ] }' - lang: Shell label: Video → text source: |- curl -X POST https://api.modular.com/v1/chat/completions \ -H "Authorization: Bearer $MODULAR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "google/gemma-4-26B-A4B-it", "messages": [ { "role": "user", "content": [ {"type": "video_url", "video_url": {"url": "https://example.com/video.mp4"}}, {"type": "text", "text": "Describe what happens in this video."} ] } ] }' - lang: Shell label: Function calling source: |- curl -X POST https://api.modular.com/v1/chat/completions \ -H "Authorization: Bearer $MODULAR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "google/gemma-4-31B-it", "messages": [ {"role": "user", "content": "What is the weather like in Boston today?"} ], "tools": [ { "type": "function", "function": { "name": "get_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": {"type": "string", "description": "The city and state, e.g. Los Angeles, CA"} }, "required": ["location"] } } } ], "tool_choice": "auto" }' components: schemas: ChatCompletionRequest: type: object properties: model: type: string description: Model identifier. See the [supported models](/models) page. example: google/gemma-4-26B-A4B-it messages: type: array items: $ref: '#/components/schemas/ChatMessage' description: The conversation history. max_tokens: type: integer description: Maximum number of tokens to generate. temperature: type: number description: >- Sampling temperature (0–2). Higher values produce more varied output. minimum: 0 maximum: 2 stream: type: boolean description: >- If true, stream partial tokens as server-sent events. Support for `tools` while streaming is model-dependent; set this to false if you see incomplete or malformed tool-call output. tools: type: array description: >- List of functions the model may call. Only supported on models that are pretrained for tool use; see the [supported models](/models) page. items: $ref: '#/components/schemas/ChatCompletionTool' tool_choice: description: >- Controls whether and how the model calls a tool. `none` disables tool calls; `auto` (default) lets the model decide; `required` forces at least one tool call; or pass an object naming a specific function to force that call. oneOf: - type: string enum: - none - auto - required - type: object properties: type: type: string enum: - function function: type: object properties: name: type: string description: Name of the function the model must call. required: - name required: - type - function required: - model - messages ChatCompletionResponse: type: object properties: id: type: string object: type: string example: chat.completion model: type: string choices: type: array items: $ref: '#/components/schemas/ChatCompletionChoice' usage: $ref: '#/components/schemas/CompletionUsage' required: - id - object - model - choices ChatMessage: type: object properties: role: type: string enum: - system - user - assistant - tool description: >- The role of the message author. Use `tool` to return a function's result to the model, matched to the originating call via `tool_call_id`. content: oneOf: - type: string description: >- Plain text content for text-only messages, or a JSON-encoded result string for `tool` messages. - type: array description: >- Multimodal content array for messages that include images or video. items: $ref: '#/components/schemas/ChatMessageContentPart' - type: 'null' description: >- Null when an `assistant` message only contains `tool_calls` and no text content. description: >- Message content. Pass a string for text-only inputs, an array of content blocks for image or video inputs, or null for an assistant message that only contains tool calls. tool_calls: type: array description: >- Tool calls requested by the model. Present on `assistant` messages when `finish_reason` is `tool_calls`. items: $ref: '#/components/schemas/ChatCompletionMessageToolCall' tool_call_id: type: string description: >- ID of the tool call this message answers. Required on `tool` role messages and must match the `id` of a tool call in the preceding assistant message. required: - role - content ChatCompletionTool: type: object properties: type: type: string enum: - function description: Currently always `function`. function: $ref: '#/components/schemas/ChatCompletionToolFunction' required: - type - function ChatCompletionChoice: type: object properties: index: type: integer message: $ref: '#/components/schemas/ChatMessage' finish_reason: type: string enum: - stop - length - tool_calls required: - index - message - finish_reason CompletionUsage: type: object properties: prompt_tokens: type: integer completion_tokens: type: integer total_tokens: type: integer ChatMessageContentPart: type: object description: A single content block within a multimodal message. properties: type: type: string enum: - text - image_url - video_url description: Content block type. text: type: string description: Text content. Required when `type` is `text`. image_url: type: object description: Image source. Required when `type` is `image_url`. properties: url: type: string description: >- URL or base64 data URI of the image (`data:;base64,`). required: - url video_url: type: object description: Video source. Required when `type` is `video_url`. properties: url: type: string description: URL of the video file. required: - url required: - type ChatCompletionMessageToolCall: type: object description: A single tool call requested by the model. properties: id: type: string description: >- Unique identifier for this tool call. Echo it back in the matching `tool` message's `tool_call_id`. type: type: string enum: - function description: Currently always `function`. function: type: object properties: name: type: string description: >- Name of the function to call, matching a `tools[].function.name` from the request. arguments: type: string description: >- JSON-encoded string of arguments to pass to the function. Parse it before use; malformed JSON indicates the model failed to produce valid tool-call output. required: - name - arguments required: - id - type - function ChatCompletionToolFunction: type: object description: >- Definition of a callable function, following the OpenAI function calling specification. properties: name: type: string description: >- Function name the model uses to call it. At most 64 characters; default character set is `[a-zA-Z0-9_-]`. description: type: string description: >- Description of what the function does, used by the model to decide when and how to call it. parameters: type: object description: JSON Schema object describing the function's parameters. strict: type: boolean description: >- If true, the model's arguments are constrained to strictly match the `parameters` schema. required: - name securitySchemes: BearerAuth: type: http scheme: bearer description: >- Modular Cloud API token. Obtain from the [API Tokens page](https://console.modular.com/api_tokens). ````