> ## Documentation Index
> Fetch the complete documentation index at: https://docs-preview.modular.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Text generation
> Generate text and analyze images and video using an OpenAI-compatible chat completions endpoint
Generate text from a prompt or conversation with `/v1/chat/completions`, or
analyze images and video by passing them alongside text in the same request.
The endpoint is fully compatible with the OpenAI Chat Completions API, so
existing OpenAI SDK code works unchanged.
Check the [supported models](/models) page to see which models are available
via shared or dedicated endpoints, and which support text generation versus
image/video analysis.
Already using the OpenAI SDK? Just change `base_url`
to `https://api.modular.com/v1` and set `api_key` to your API
key. No other code changes required.
## The `v1/chat/completions` endpoint
The `v1/chat/completions` endpoint supports both single-turn and multi-turn
interactions. You provide a sequence of structured messages with roles
(`system`, `user`, `assistant`), and the model generates a response.
For example, within the `v1/chat/completions` request body, the `"messages"`
array might look similar to the following:
```json theme={null}
"messages": [
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "Who won the world series in 2020?"
}
]
```
Use a combination of roles to give the model the context it needs. A `system`
message can define overall model response behavior, `user` messages represent
instructions or prompts from the end-user, and `assistant` messages incorporate
past model responses into the context.
You can also include image or video inputs alongside text to work with
multimodal models.
## Generate text
Generate text from a prompt or conversation history using `v1/chat/completions`:
Send a chat completion request using the OpenAI Python SDK:
```python title="generate-text.py" theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.modular.com/v1",
api_key=os.environ["MODULAR_API_KEY"],
)
response = client.chat.completions.create(
model="google/gemma-4-31B-it",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Who won the world series in 2020?"},
{"role": "assistant", "content": "The LA Dodgers won in 2020."},
{"role": "user", "content": "Where was it played?"}
]
)
print(response.choices[0].message.content)
```
```text theme={null}
The 2020 World Series was played at Globe Life Field in Arlington, Texas. It was a neutral site due to the COVID-19 pandemic.
```
Send the same request using curl:
```bash theme={null}
curl -X POST https://api.modular.com/v1/chat/completions \
-H "Authorization: Bearer $MODULAR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "google/gemma-4-31B-it",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "Hello, how are you?"
}
],
"max_tokens": 100
}'
```
```json theme={null}
{
"choices": [
{
"finish_reason": "stop",
"index": 0,
"message": {
"content": "I'm doing well, thank you for asking. How can I assist you today?",
"role": "assistant"
}
}
],
"model": "google/gemma-4-31B-it",
"object": "chat.completion",
"usage": {
"completion_tokens": 17,
"prompt_tokens": null,
"total_tokens": 17
}
}
```
### Stream responses in real time
Set `stream=True` to receive tokens as they are generated instead of waiting
for the full response:
Set `stream=True` and iterate over the response chunks as they arrive:
```python title="stream-text.py" theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.modular.com/v1",
api_key=os.environ["MODULAR_API_KEY"],
)
stream = client.chat.completions.create(
model="google/gemma-4-31B-it",
messages=[{"role": "user", "content": "Write a short poem about the sea."}],
stream=True,
)
for chunk in stream:
if chunk.choices[0].delta.content is not None:
print(chunk.choices[0].delta.content, end="", flush=True)
print()
```
Pass `"stream": true` in the request body to receive server-sent events:
```bash theme={null}
curl -X POST https://api.modular.com/v1/chat/completions \
-H "Authorization: Bearer $MODULAR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "google/gemma-4-31B-it",
"messages": [{"role": "user", "content": "Write a short poem about the sea."}],
"stream": true
}'
```
Each streamed chunk is a server-sent event with a `data:` prefix:
```text theme={null}
data: {"choices":[{"delta":{"content":"The"},"index":0}],"object":"chat.completion.chunk"}
data: {"choices":[{"delta":{"content":" sea"},"index":0}],"object":"chat.completion.chunk"}
...
data: [DONE]
```
## Analyze images
Send images to the model alongside text prompts to get descriptions, answers to
questions, or extracted information from visual content.
### Pass image or video URLs
Within the `v1/chat/completions` request body, the `"messages"` array accepts
inline image or video URLs.
Use `image_url` to pass an image:
```json theme={null}
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "What is in this image?"
},
{
"type": "image_url",
"image_url": {
"url": "https://example.com/path/to/image.jpg"
}
}
]
}
]
```
Use `video_url` to pass a video:
```json theme={null}
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "What is happening in this video?"
},
{
"type": "video_url",
"video_url": {
"url": "https://example.com/path/to/video.mp4"
}
}
]
}
]
```
Both `image_url` and `video_url` also accept base64-encoded data URIs
(such as `data:image/jpeg;base64,...` or `data:video/mp4;base64,...`).
### Describe an image
Send an image URL alongside a text question to get a description or analysis of
the image:
Include an `image_url` content block alongside your text prompt:
```python title="generate-image-description.py" theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.modular.com/v1",
api_key=os.environ["MODULAR_API_KEY"],
)
response = client.chat.completions.create(
model="google/gemma-4-31B-it",
messages=[
{
"role": "user",
"content": [
{
"type": "text",
"text": "What is in this image?"
},
{
"type": "image_url",
"image_url": {
"url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/0052a70beed5bf71b92610a43a52df6d286cd5f3/diffusers/rabbit.jpg"
}
}
]
}
],
max_tokens=300
)
print(response.choices[0].message.content)
```
```text theme={null}
Here's a breakdown of what's in the image:
* **Peter Rabbit:** The main focus is a realistic-looking depiction of Peter
Rabbit, the character from Beatrix Potter's stories...
```
Pass the image URL and text prompt in the `content` array of the request body:
```bash theme={null}
curl -X POST https://api.modular.com/v1/chat/completions \
-H "Authorization: Bearer $MODULAR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "google/gemma-4-31B-it",
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "What is in this image?"
},
{
"type": "image_url",
"image_url": {
"url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/0052a70beed5bf71b92610a43a52df6d286cd5f3/diffusers/rabbit.jpg"
}
}
]
}
],
"max_tokens": 300
}' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '\n' | sed 's/\\n/\n/g'
```
```text theme={null}
Here's a breakdown of what's in the image:
* **Peter Rabbit:** The main focus is a realistic, anthropomorphic
(human-like) rabbit character...
```
## Describe video content
Send a video URL alongside a text question to get a description or analysis of
the video's visual content.
### Describe a video
Include a `video_url` content block in your request to analyze what happens in
a video:
Pass a video URL and a text question to get a natural-language description of
the video:
```python title="generate-video-description.py" theme={null}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.modular.com/v1",
api_key=os.environ["MODULAR_API_KEY"],
)
completion = client.chat.completions.create(
model="google/gemma-4-31B-it",
messages=[
{
"role": "user",
"content": [
{
"type": "text",
"text": "Describe what is happening in this video"
},
{
"type": "video_url",
"video_url": {
"url": "https://avtshare01.rz.tu-ilmenau.de/avt-vqdb-uhd-1/test_1/segments/bigbuck_bunny_8bit_15000kbps_1080p_60.0fps_h264.mp4"
}
}
]
}
],
max_tokens=300
)
print(completion.choices[0].message.content)
```
```text theme={null}
The video is an animated short film featuring a large, fluffy rabbit in a
colorful meadow. The rabbit wanders through the environment, encountering
butterflies and small birds. The animation has a warm, lighthearted tone with
vibrant natural scenery...
```
Pass the video URL and text prompt in the `content` array of the request body:
```bash theme={null}
curl -X POST https://api.modular.com/v1/chat/completions \
-H "Authorization: Bearer $MODULAR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "google/gemma-4-31B-it",
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Describe what is happening in this video"
},
{
"type": "video_url",
"video_url": {
"url": "https://avtshare01.rz.tu-ilmenau.de/avt-vqdb-uhd-1/test_1/segments/bigbuck_bunny_8bit_15000kbps_1080p_60.0fps_h264.mp4"
}
}
]
}
],
"max_tokens": 300
}' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '\n' | sed 's/\\n/\n/g'
```
```text theme={null}
The video is an animated short film featuring a large, fluffy rabbit in a
colorful meadow. The rabbit wanders through the environment, encountering
butterflies and small birds. The animation has a warm, lighthearted tone with
vibrant natural scenery...
```