# The Gemini APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition →: your first real calls
Here is a complete, multimodal Gemini call in Python: one image, one question, one response, with each piece doing something specific to Gemini.
from google import genai
client = genai.Client() # reads GEMINI_API_KEY from the environment
with open("invoice.png", "rb") as f:
image_bytes = f.read()
response = client.models.generate_content(
model="gemini-2.5-flash",
contents=[
{"text": "Extract the total amount and due date. Reply as JSON."},
{"inline_data": {"mime_type": "image/png", "data": image_bytes}},
],
)
print(response.text)That is the whole thing. No base64 dance, no separate vision endpoint, no OCR preprocessor. You hand the model bytes and text together, and it reads both. Let us unpack what is Gemini-specific here, because that is where the value is.
The package is google-genai
pip install google-genaigoogle.generativeaipip install google-genai
export GEMINI_API_KEY="your-key-from-aistudio"Grab the key from aistudio.google.com under "Get APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition → key". The genai.Client() constructor reads GEMINI_API_KEY automatically, so you rarely pass it in code. Full reference lives at ai.google.dev.
One SDK, two backends. The same client talks to either the Gemini Developer API (the AI Studio key, fast to start) or Vertex AI (Google Cloud, with IAM, VPC controls, and enterprise billing). You switch by setting vertexai=True and a project plus location, not by rewriting your code. Prototype on the Developer APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition →, graduate to Vertex when you need governance. We cover that move later in this path.
The model string is a real decision, not a formality. Gemini ships in tiers:
Use a pinned, dated alias like gemini-2.5-flash rather than chasing whatever is newest. Model IDs evolve, so check the live list in AI Studio or the models page before you ship. The pattern holds even as version numbers tick up: reachreachThe number of unique people exposed to your message in a given period. Unlike impressions, reach counts each person once, no matter how often they see it.View full definition → for Flash first, escalate to Pro only when Flash visibly struggles.
Gemini is natively multimodal, meaning text, images, audio, video, and PDFs go through the same model rather than a bolted-on vision module. So the cost and latency tradeoff between Flash and Pro applies to image and document work too, not just text. A Flash model reading a 40-page PDF is often all you need.
contents structurecontents is a list of parts. Each part is a chunk of input: a text part, an inline_data part (raw bytes plus a MIME type), or a reference to an uploaded file. The model sees them in order, so prompt placement matters. Putting the instruction before the image, as above, tends to work well for "do X to this thing" tasks.
For small images, inline_data is fine. For anything large (long video, big PDFs, files you reuse across calls), upload once with the Files API and pass a handle instead:
uploaded = client.files.upload(file="contract.pdf")
response = client.models.generate_content(
model="gemini-2.5-pro",
contents=["Summarize the indemnification clauses.", uploaded],
)
print(response.text)Notice you can pass plain strings and file objects directly; the SDK wraps them into parts for you. The explicit dict form from the first example is just the same thing spelled out.
Gemini's long context windowcontext windowThe context window is the maximum amount of text (measured in tokens) a language model can process at once, including both the input prompt and the generated output.View full definition → is large enough that you can drop entire documents, codebases, or transcripts straight into contents and skip retrieval for many tasks. This is a genuine shift in how you design: sometimes the simplest "RAG" is no RAG, just the whole corpus in the prompt.
But it is not free. More tokenstokensA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition → means more cost and more latency, and very long contexts can dilute attention on the one detail you care about. The instinct to paste everything is a trap when a tight, relevant slice would do. Treat long context as a tool you choose, not a default you lean on.
Pass a config to control generation. Two settings earn their keep immediately.
Structured output. Instead of begging the model for JSON in the prompt and hoping, you can constrain it to a schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition →. Gemini will return valid JSON matching it.
from google import genai
from pydantic import BaseModel
class Invoice(BaseModel):
total: float
due_date: str
client = genai.Client()
response = client.models.generate_content(
model="gemini-2.5-flash",
contents=["Extract total and due date.", uploaded],
config={
"response_mime_type": "application/json",
"response_schema": Invoice,
},
)
invoice = response.parsed # a typed Invoice object
print(invoice.total, invoice.due_date)response.parsed hands you a real Python object, not a string you have to json.loads and pray over. This is how you make Gemini calls safe to wire into downstream code.
System instructions. Set persistent behavior with system_instruction in the config rather than burying it in every prompt:
config={"system_instruction": "You are a terse financial analyst. Cite figures exactly as written."}The temperature, max_output_tokens, and thinking_config settings also live here. That last one is Gemini-specific: on reasoning-capable models you can adjust the thinking budget, the amount of internal reasoning the model spends before answering. Lower it for speed on easy tasks, raise it for hard ones.
Here is a capability you will not find on most APIs: you can let Gemini ground its answers in live Google Search results, with citations, in a single config flag.
from google import genai
from google.genai import types
client = genai.Client()
response = client.models.generate_content(
model="gemini-2.5-flash",
contents="What changed in the latest Gemini API pricing?",
config=types.GenerateContentConfig(
tools=[types.Tool(google_search=types.GoogleSearch())]
),
)
print(response.text)The model decides when to search, runs queries, and synthesizes an answer with sources attached in the response metadata. This is the cleanest way to fight stale knowledge for factual, time-sensitive questions, and it is built in rather than something you assemble yourself. Read the details on the grounding docs.
Knowledge check
1. Which package should you install to use the current unified Google GenAI SDK shown in the lesson?
2. How does the `genai.Client()` constructor obtain your API key when called with no arguments?
3. In the multimodal example, why is no base64 encoding or separate vision endpoint required to send an image?
4. Select ALL correct answers about the two backends the unified SDK can talk to.
Sélectionnez toutes les réponses correctes.
5. Select ALL correct answers about the structure of the example Gemini API call.
Sélectionnez toutes les réponses correctes.
Three practical things before you ship.
Streaming. For anything a human waits on, stream tokenstokensA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition → as they arrive instead of blocking for the full response:
for chunk in client.models.generate_content_stream(
model="gemini-2.5-flash",
contents="Explain native multimodality in two sentences.",
):
print(chunk.text, end="", flush=True)Chat sessions. For multi-turn conversations, client.chats.create(model=...) keeps history for you, so you call chat.send_message(...) and it remembers prior turns. You do not rebuild the full transcript by hand each time.
Errors and limits. Free-tier keys have rate limits, and you will hit 429 responses under load. Build in retry with backoff. Watch for RESOURCE_EXHAUSTED (quota) versus INVALID_ARGUMENT (your request is malformed, often a bad MIME type or oversized inline payload). When inline data gets large, switch to the Files APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition →; that fixes a surprising share of early failures.
Do not write Python to explore. Open AI Studio, paste your prompt, drop in an image, toggle structured output and grounding, tune temperature, and watch it work. When the prompt behaves, click "Get code" and AI Studio generates the exact google-genai call, model ID and config included. Your loop becomes: experiment in AI Studio, export, then refine in your editor.
This is also where you sanity-check tokentokenA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition → usage and try models side by side before committing one to production.
The Developer APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition → key is perfect for prototypes and small apps. Move to Vertex AI when you need any of: org-level IAM and access control, data residency guarantees, VPC Service Controls, customer-managed encryption, or consolidated Google Cloud billing. The same google-genai code carries over; you flip the client to Vertex mode and authenticate through Google Cloud instead of an APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition → key. See cloud.google.com/vertex-ai. The decision is about governance and scale, not capability, since the underlying models are the same family.
The APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition → is one of several ways to reachreachThe number of unique people exposed to your message in a given period. Unlike impressions, reach counts each person once, no matter how often they see it.View full definition → Gemini, and they target different jobs. The Gemini app and Gems are end-user surfaces. Gemini in Workspace lives inside Docs and Gmail. Gemini CLI and Code Assist serve developers in the terminal and IDE. The API
genai.Client(), and the same code runs against both the AI Studio Developer APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition → and Vertex AI.inline_data for small images and the Files APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition → for large PDFs, video, or anything you reuse, and remember Gemini reads them natively in one call.