+190 XP

The Gemini API: your first real calls

Here is a complete, multimodal Gemini call in Python: one image, one question, one response, with each piece doing something specific to Gemini.

python
from google import genai

client = genai.Client()  # reads GEMINI_API_KEY from the environment

with open("invoice.png", "rb") as f:
    image_bytes = f.read()

response = client.models.generate_content(
    model="gemini-3-flash-preview",
    contents=[
        {"text": "Extract the total amount and due date. Reply as JSON."},
        {"inline_data": {"mime_type": "image/png", "data": image_bytes}},
    ],
)

print(response.text)

That is the whole thing. You don't need a base64 dance, a separate vision endpoint or an OCR preprocessor. You hand the model bytes and text together, and it reads both. The rest of this lesson unpacks what is Gemini-specific here, because that is where the value is.

The SDK and the client

The package is google-genai, the unified Google GenAI SDK. Install it with pip install google-genai. This is the current SDK. If you find old tutorials importing google.generativeai, that is the legacy library. Use the new one.

bash
pip install google-genai
export GEMINI_API_KEY="your-key-from-aistudio"

Grab the key from aistudio.google.com under "Get API key". The `genai.Client()` constructor reads `GEMINI_API_KEY` automatically, so you rarely pass it in code. Full reference lives at ai.google.dev.

One SDK talks to two backends. The same client works with either the Gemini Developer API (the AI Studio key, fast to start) or Vertex AI (Google Cloud, with IAM, VPC controls and enterprise billing). You switch by setting vertexai=True and a project plus location. You don't rewrite your code. Prototype on the Developer API, then move to Vertex when you need governance. We cover that move later in this path.

Picking the model: flash vs pro

The model string is a real decision. The current generation is Gemini 3, and it ships in tiers:

  • Flash is the workhorse. It is fast and cheap, and it handles extraction, classification, chat and high-volume jobs well. The invoice task above is a Flash job.
  • Pro is the reasoner, for harder multi-step problems, dense code and long analytical chains. It is slower and pricier per token.
  • Flash-Lite variants cover the highest-volume, lowest-cost cases.

Model IDs change often. Some launch with a -preview suffix, graduate to a stable ID, and older generations (Gemini 2.5 and earlier) get deprecation dates. Check the live list on the models page before you ship, and pin the exact ID you tested instead of a floating "latest" alias. The pattern holds as version numbers tick up: reach for Flash first, escalate to Pro only when Flash visibly struggles.

Why this matters more on Gemini

Gemini is natively multimodal. Text, images, audio, video and PDFs go through the same model rather than a bolted-on vision module. So the cost and latency tradeoff between Flash and Pro applies to image and document work too. A Flash model reading a 40-page PDF is often all you need.

The contents structure

contents is a list of parts. Each part is a chunk of input: a text part, an inline_data part (raw bytes plus a MIME type), or a reference to an uploaded file. The model sees them in order, so placement matters. For "do X to this thing" tasks, putting the instruction before the image, as above, tends to work well.

For small images, inline_data is fine. For anything large, such as long video, big PDFs or files you reuse across calls, upload once with the Files API and pass a handle instead:

python
uploaded = client.files.upload(file="contract.pdf")
response = client.models.generate_content(
    model="gemini-3-pro-preview",
    contents=["Summarize the indemnification clauses.", uploaded],
)
print(response.text)

You can pass plain strings and file objects directly, and the SDK wraps them into parts for you. The explicit dict form in the first example is the same thing spelled out.

Long context, used deliberately

Gemini's context window is large enough to hold entire documents, codebases or transcripts, so for many tasks you can put the material straight into contents and skip retrieval. Sometimes the simplest "RAG" is no RAG at all, just the whole corpus in the prompt.

It has costs, though. More tokens mean more cost and more latency, and very long contexts can dilute attention on the one detail you care about. Pasting everything is a trap when a tight, relevant slice would do. Use long context when the task calls for it, and don't make it your default.

Configuration that changes behavior

Pass a config to control generation. Two settings are useful right away.

Structured output. You don't have to ask for JSON in the prompt and hope. You can constrain the model to a schema, and Gemini will return valid JSON that matches it.

python
from google import genai
from pydantic import BaseModel

class Invoice(BaseModel):
    total: float
    due_date: str

client = genai.Client()
response = client.models.generate_content(
    model="gemini-3-flash-preview",
    contents=["Extract total and due date.", uploaded],
    config={
        "response_mime_type": "application/json",
        "response_schema": Invoice,
    },
)

invoice = response.parsed  # a typed Invoice object
print(invoice.total, invoice.due_date)

response.parsed gives you a real Python object instead of a string you have to json.loads and pray over. This is how you make Gemini calls safe to wire into downstream code.

System instructions. Set persistent behavior with system_instruction in the config rather than repeating it in every prompt:

python
config={"system_instruction": "You are a terse financial analyst. Cite figures exactly as written."}

The temperature, max_output_tokens and thinking_config settings also live here. Gemini 3 models reason before answering, and you control how much with thinking_level inside thinking_config (for example {"thinking_config": {"thinking_level": "low"}}). Use low for speed on easy extraction and high for hard problems. Older 2.5 models used a numeric thinking_budget instead, so expect to see both in tutorials. For Gemini 3, Google recommends leaving temperature at its default of 1.0.

Gemini API in Python: Getting Started

Watch on YouTube

Grounding with Google search

Few other APIs offer this: you can let Gemini ground its answers in live Google Search results, with citations, through one tool entry in the config.

python
from google import genai
from google.genai import types

client = genai.Client()
response = client.models.generate_content(
    model="gemini-3-flash-preview",
    contents="What changed in the latest Gemini API pricing?",
    config=types.GenerateContentConfig(
        tools=[types.Tool(google_search=types.GoogleSearch())]
    ),
)

print(response.text)

The model decides when to search, runs the queries and writes an answer, with sources attached in the response metadata. For factual, time-sensitive questions this is the cleanest fix for stale knowledge, and it is built in. Details are on the grounding docs.

Knowledge check

1. In the multimodal Gemini call shown, how is the image provided to the model alongside the text prompt?

2. Why does the lesson recommend using a pinned, dated alias like 'gemini-2.5-flash' instead of always selecting the newest model?

3. For a high-volume invoice data extraction task, which Gemini tier does the lesson recommend and why?

MULTIPLE CHOICE

4. Select ALL correct statements about the Google GenAI SDK and client as described in the lesson.

Select all the correct answers.

MULTIPLE CHOICE

5. Select ALL correct statements about the distinction between the Gemini Developer API and Vertex AI.

Select all the correct answers.

Streaming, chat, and errors

A few practical things before you ship.

Streaming. For anything a human waits on, stream tokens as they arrive instead of blocking for the full response:

python
for chunk in client.models.generate_content_stream(
    model="gemini-3-flash-preview",
    contents="Explain native multimodality in two sentences.",
):
    print(chunk.text, end="", flush=True)

Chat sessions. For multi-turn conversations, client.chats.create(model=...) keeps history for you. You call chat.send_message(...) and it remembers prior turns, so you don't rebuild the transcript by hand.

Errors and limits. Free-tier keys have rate limits, and you will hit 429 responses under load, so build in retry with backoff. RESOURCE_EXHAUSTED means quota. INVALID_ARGUMENT means your request is malformed, often because of a bad MIME type or an oversized inline payload. When inline data gets large, switch to the Files API. That fixes a surprising share of early failures. Also watch for errors from retired model IDs when Google deprecates an older generation.

AI Studio: prototype, then export

Don't write Python to explore. Open AI Studio, paste your prompt, drop in an image, toggle structured output and grounding, tune settings and watch it work. When the prompt behaves, click "Get code" and AI Studio generates the exact google-genai call, with the model ID and config included. Your loop becomes: experiment in AI Studio, export, then refine in your editor.

AI Studio is also where you check token usage and try models side by side before committing one to production.

When to reach for Vertex AI instead

The Developer API key works well for prototypes and small apps. Move to Vertex AI when you need org-level IAM, data residency guarantees, VPC Service Controls, customer-managed encryption or consolidated Google Cloud billing. The same google-genai code carries over: you flip the client to Vertex mode and authenticate through Google Cloud instead of an API key. See cloud.google.com/vertex-ai. The decision is about governance and scale. The underlying models are the same family, so you don't gain capability by switching.

A note on where the API sits

The API is one of several ways to reach Gemini, and each targets a different job. The Gemini app and Gems are end-user surfaces. Gemini in Workspace lives inside Docs and Gmail. Gemini CLI and Code Assist serve developers in the terminal and IDE. The API is the layer underneath your own products. You call it when you are the one building the thing other people use, and everything in this lesson is about that layer.

Key Takeaways

  • Install google-genai, not the legacy google.generativeai. One genai.Client() runs against both the AI Studio Developer API and Vertex AI.
  • Default to Gemini 3 Flash and escalate to Pro for hard reasoning. Pin the exact model ID you tested and check the models page for deprecations.
  • Pass multimodal input as parts: inline_data for small images, the Files API for large PDFs, video or anything you reuse.
  • Constrain output with `response_schema` and read `response.parsed` to get typed objects you can safely use downstream.
  • On Gemini 3, tune reasoning with thinking_level. Prototype in AI Studio, then use "Get code" to export the exact call.

Frequently asked questions

Which Python package should I install to call the Gemini API?

Install `google-genai` with `pip install google-genai`. That is the current unified Google GenAI SDK. Tutorials that import `google.generativeai` use the legacy library and should be ignored for new work.

What is the difference between Gemini Flash and Gemini Pro, and which should I use?

Flash is fast and cheap, and it covers extraction, classification, chat and high-volume jobs. Pro reasons better on multi-step problems, dense code and long analytical chains, but it is slower and costs more per token. Start with Flash and escalate to Pro only when Flash visibly struggles; Flash-Lite variants exist for the highest-volume, lowest-cost cases.

How do I send an image or a PDF to Gemini in the same call as my prompt?

Gemini is natively multimodal, so you pass text and bytes together in the `contents` list as parts. Use `inline_data` with a MIME type for small images, and upload large PDFs, video or reused files with the Files API (`client.files.upload`) then pass the handle. No base64 handling, separate vision endpoint or OCR step is needed.

How do I force Gemini to return valid JSON instead of free text?

Set `response_mime_type` to `application/json` and pass a `response_schema` in the config, for example a Pydantic model. Gemini then returns JSON matching that schema, and `response.parsed` gives you a typed Python object rather than a string to `json.loads`. Asking for JSON in the prompt alone gives no such guarantee.

When should I switch from the Gemini Developer API key to Vertex AI?

Switch when you need org-level IAM, data residency guarantees, VPC Service Controls, customer-managed encryption or consolidated Google Cloud billing. The same `google-genai` code carries over: you set `vertexai=True` with a project and location, and authenticate through Google Cloud instead of an API key. The models are the same family, so the move is about governance and scale, not capability.

Why does the Gemini API return a 429 or INVALID_ARGUMENT error?

A `429` with `RESOURCE_EXHAUSTED` means you hit the rate limit or quota, typical on free-tier keys under load, so add retry with backoff. `INVALID_ARGUMENT` means the request is malformed, most often a wrong MIME type or an inline payload that is too large; switching that payload to the Files API fixes a large share of early failures. Also check that the model ID you call has not been retired.

What to do, from this lesson

These actions are compiled in the role's Playbook.

  • Send PDFs, images, audio, and video natively in one request
  • Prototype visually in AI Studio, then Get code with an env-var key
See the full action playbook →