Leaders Insights
Leaders Insights

Stay at the top of your field, a little every day.

DomainsMarketingDataFinanceAI
ResourcesLearnTestToolsBlogGlossary
© 2026 Leaders Insights — All rights reserved.
Tracks/AI Essentials/Gemini & Google AI/Building deeper: tools, RAG, and files/Function calling and structured outputs
1/3+200 XP

Building deeper: tools, RAG, and files

1Function calling and structured outputs+2002Files, embeddings, and RAG with Google+1903Vertex AI: taking Gemini to production+180

Function calling and structured outputs

# Function calling and structured outputs

Giving Gemini a function it can call turns it from a text generator into something that can read your calendar, hit your pricing APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition →, or query your database, and getting clean JSON back means your code can actually use the result without regex hacks. These two capabilities are the backbone of every serious Gemini integration, and they work very differently under the hood. Let's go deep on both.

Function calling is a negotiation, not an execution

The most important mental shift: Gemini never runs your function. It only decides *that* a function should be called and *with what arguments*. You run the code. You send the result back. Gemini continues.

That round trip is the tool-calling loop:

1. You send a prompt plus a list of function *declarations* (name, description, parameters).

2. Gemini responds either with normal text, or with a functionCall

part containing the chosen function and arguments.

3. Your code executes that function for real.

4. You send the result back as a functionResponse.

5. Gemini uses the result to produce its final answer (or call another function).

The model is doing structured intent recognition. Your job is to wire up the actual execution and keep passing state back into the conversation.

Declaring a function

A function declaration is a schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition →. Gemini reads the description fields to decide when and how to call it, so treat those as prompt engineeringprompt engineeringPrompt engineering is the practice of designing and refining text inputs to guide large language models toward accurate, relevant, and reliable outputs.View full definition →, not documentation. Vague descriptions produce wrong calls.

python
from google import genai
from google.genai import types

client = genai.Client()

get_weather = types.FunctionDeclaration(
    name="get_weather",
    description="Get the current temperature for a city. Use only when the user asks about weather.",
    parameters={
        "type": "object",
        "properties": {
            "city": {"type": "string", "description": "City name, e.g. 'Lisbon'"},
            "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
        },
        "required": ["city"],
    },
)

weather_tool = types.Tool(function_declarations=[get_weather])

This uses the unified Google Gen AI SDK, the current recommended Python library. The same SDK targets both the Gemini APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition → (via AI Studio keys) and Vertex AI by changing client config, so you write the loop once.

Running the loop

Here is the full round trip with a Flash model. Note that the *function result* goes back into the conversation as a new content part, not as plain text.

python
def get_weather_impl(city, unit="celsius"):
    return {"city": city, "temp": 19, "unit": unit}  # real API call goes here

config = types.GenerateContentConfig(tools=[weather_tool])
contents = ["What's the weather in Lisbon right now?"]

resp = client.models.generate_content(
    model="gemini-2.5-flash", contents=contents, config=config
)

part = resp.candidates[0].content.parts[0]
if part.function_call:
    call = part.function_call
    result = get_weather_impl(**call.args)

    contents.append(resp.candidates[0].content)  # the model's call
    contents.append(types.Content(role="user", parts=[
        types.Part.from_function_response(name=call.name, response=result)
    ]))

    final = client.models.generate_content(
        model="gemini-2.5-flash", contents=contents, config=config
    )
    print(final.text)  # "It's 19°C in Lisbon right now."

The key lines are the two contents.append calls. You replay the model's own function call back into the history, then append the result. Skip the first append and Gemini loses track of what it asked for.

Letting Gemini call several functions

Give it more than one declaration and Gemini picks the right one, or asks for several. Parallel function calling is when a single response contains multiple functionCall parts (e.g. "compare weather in Lisbon and Madrid" yields two calls at once). Loop over resp.candidates[0].content.parts, execute each, and append all the responses together before the next turn.

You can also constrain behavior with tool_config. Setting the function calling mode to ANY forces Gemini to call a function rather than reply in prose, which is useful when a tool call is the only acceptable outcome:

python
config = types.GenerateContentConfig(
    tools=[weather_tool],
    tool_config=types.ToolConfig(
        function_calling_config=types.FunctionCallingConfig(mode="ANY")
    ),
)

Modes are AUTO (default, model decides), ANY (must call something), and NONE (never call). Use ANY with an allowed_function_names list to corral the model into a specific subset.

Structured outputs: when you want data, not a tool

Function calling is for *acting*. Structured output is for *extracting*. If you just want Gemini to return clean JSON you can parse, don't fake it with a tool. Use the response schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition → feature, which constrains the model's decoding so the output is guaranteed to match your schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition →.

This is constrained decoding: Gemini is restricted at generation time to only produce tokenstokensA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition → that keep the output valid against your schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition →. You are not hoping for JSON. You are getting it.

python
from pydantic import BaseModel

class Invoice(BaseModel):
    vendor: str
    total: float
    currency: str
    line_items: list[str]

resp = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Extract the invoice details from this email: ...",
    config=types.GenerateContentConfig(
        response_mime_type="application/json",
        response_schema=Invoice,
    ),
)

invoice = Invoice.model_validate_json(resp.text)
print(invoice.total, invoice.currency)

Passing a Pydantic model as response_schema is the idiomatic path: the SDK converts it to a schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition →, Gemini honors it, and you validate straight back into typed objects. No prompt that begs "return only JSON." No stray markdown fences to strip.

When the schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition → is dynamic

If your shape is decided at runtime, pass a raw schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition → dict instead of a Pydantic class. Use propertyOrdering to lock field order, since order can affect quality on complex extractions:

python
schema = {
    "type": "object",
    "properties": {
        "sentiment": {"type": "string", "enum": ["positive", "neutral", "negative"]},
        "topics": {"type": "array", "items": {"type": "string"}},
    },
    "required": ["sentiment", "topics"],
    "propertyOrdering": ["sentiment", "topics"],
}

For classification specifically, there is also response_mime_type="text/x.enum" with an enum schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition →, which forces the answer to be exactly one of your labels and nothing else. That is the cleanest way to do single-label classification.

Choosing between the two

They overlap, so be deliberate:

  • Structured output when you control the whole interaction and just need parsed data back from one call. Faster, simpler, guaranteed shape.
  • Function calling when Gemini needs to reachreachThe number of unique people exposed to your message in a given period. Unlike impressions, reach counts each person once, no matter how often they see it.View full definition → *outside* itself: live data, your APIs, side effects, multi-step reasoning where each step depends on real results.

A common pattern combines them: a function call fetches real data, then a final generation step returns it through a response schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition → so your downstream code stays typed.

Don't hand-roll what the platform gives you

Some "functions" already exist as built-in tools. Grounding with Google Search, code execution, and URL context are native tools you enable in config rather than implement yourself. If you need fresh facts, turn on Search grounding instead of writing your own search function:

python
config = types.GenerateContentConfig(
    tools=[types.Tool(google_search=types.GoogleSearch())]
)

You cannot freely mix some built-in tools with your own function declarations in a single call, and the rules differ between the Gemini APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition → and Vertex AI, so check the tool you need before combining.

Function Calling with the Gemini API

Watch on YouTube

Knowledge check

1. What is the most important conceptual distinction about how Gemini handles function calling?

2. Why does the lesson say the function's `description` fields should be treated as prompt engineering rather than documentation?

3. In the tool-calling loop, what does your code send back to Gemini after actually executing the chosen function?

MULTIPLE CHOICE

4. Select ALL correct statements about the tool-calling loop and function declarations.

Select all the correct answers.

MULTIPLE CHOICE

5. Select ALL correct statements about the benefits and design of function calling and structured outputs described in the lesson.

Select all the correct answers.

Prototype in AI Studio, then scale the right way

Before writing any loop, build the function declaration visually in Google AI Studio. You can define tools, watch Gemini emit functionCall parts, and copy working code in your language. It is the fastest way to tune those all-important description strings.

When you move to production, the decision is *where* the loop runs:

  • Gemini API (ai.google.dev) for app backends and quick services. The SDK code above is all you need.
  • Vertex AI when you need enterprise controls: IAM, VPC, data residency, and managed deployment. Same SDK, different client init.
  • Agent Development Kit (ADK) when one model with a handful of tools grows into a real agent with planning, memory, and many tools. ADK gives you the loop, state, and tool orchestration as a framework so you stop writing append-the-response plumbing by hand. See the ADK docs for the agent-first approach.

For internal Workspace automation, you don't always need any of this. Apps Script can call Gemini and act on Docs, Sheets, or Gmail directly, which is often the shortest path to a working tool inside Google Workspace.

Hard-won details that save you hours

Descriptions are prompts. Gemini's accuracy at choosing and filling functions tracks almost entirely with the quality of your description fields and parameter docs. Spell out *when* to use a function and *when not to*.

Validate every argument. Gemini fills arguments from a schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition →, but it can still hallucinate a plausible-but-wrong value, especially for IDs or enums it half-remembers. Treat function arguments like untrusted user input. Validate before you execute anything with side effects.

Use `required` and `enum` aggressively. Every constraint you put in the schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition → is a constraint the model must satisfy. Enums eliminate whole classes of garbage values. Required fields stop half-filled calls.

Pick the tier per task. Flash handles the vast majority of tool routing and extraction with low latency and cost. ReachReachThe number of unique people exposed to your message in a given period. Unlike impressions, reach counts each person once, no matter how often they see it.View full definition → for Pro when the decision about *which* tool to call requires real reasoning over a long context, or when extraction depends on subtle multi-document logic.

Feed function results back, always. The single most common bug is forgetting to append the model's own function call before the response. Without it, the conversation history is inconsistent and the final answer drifts.

Key Takeaways

  • You execute, Gemini decides. The tool-calling loop is: declare functions, receive a functionCall, run it yourself, append both the call and the functionResponse, then generate the final answer.
  • Use `response_schema` with a Pydantic model to get guaranteed-valid JSON through constrained decoding. Stop promptingpromptingPrompt engineering is the practice of designing and refining text inputs to guide large language models toward accurate, relevant, and reliable outputs.View full definition → for JSON and stop stripping markdown fences.
  • Act vs. extract: function calling reaches outside the model for live data and side effects; structured output parses data from a single call. Combine them when you fetch real data, then return it typed.
  • Invest in `description` fields, enums, and `required`. Tool accuracy is mostly schema quality, and every constraint is one the model must satisfy.

Next

Files, embeddings, and RAG with Google

schema
A schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.
View full definition →
  • Prototype in AI Studio, then graduate to the Gemini APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition →, Vertex AI, or ADK based on whether you need scale, enterprise controls, or full agent orchestration, and always validate function arguments before acting on them.