Images, voice, and vision

> Same layout, but make Pro the highlighted "most popular" tier and tighten the vertical spacing.

Because the model retains the previous image as context, follow-ups like "now in dark mode" or "swap teal for indigo" actually work. This is where ChatGPT beats a one-shot image tool: the conversation is the version history.

Two practical limits to internalize. First, generated text inside images is much better than it used to be but still not pixel-perfect, so treat headings as placeholders, not final copy. Second, this is a mockup, not a Figma file. You get a flat raster, not editable layers. Use it to align a team on direction in minutes, then rebuild the chosen direction in your real design tool.

You can also feed an image *in* and ask for a variation: upload a competitor's screenshot and say "design something in this spirit but for a developer audience." The model reasons over the uploaded pixels, which is the bridge to our next mode.

Vision: reading screenshots, whiteboards, and errors

Vision is the inverse of generation. You give ChatGPT an image and it reads it. This is the mode most professionals underuse, and it is the one that saves the most time.

The killer use case is debugging from a screenshot. Suppose your terminal throws a wall of red and you cannot be bothered to read it carefully. Screenshot it, drop it in, and ask:

> Here's the error from my build. What's the root cause and the smallest fix? Assume Node 20.

ChatGPT reads the stack trace as text, identifies the failing module, and usually points at the real cause rather than the topmost line (which is often noise). For best results, pair the image with the relevant file. Vision plus a pasted snippet beats either alone, because the model can cross-reference what it sees with what you wrote.

The whiteboard case is just as strong. Photograph a messy architecture sketch after a meeting and ask:

> Transcribe this whiteboard into a structured markdown doc: the boxes are services, the arrows are data flowdata flowAn automated sequence of steps that moves data from source to destination: ingestion, transformation, validation, and loading, so it arrives clean and ready to use.Voir la définition complète →. Flag anything ambiguous.

It will turn arrows and scrawl into a clean component list and call out the two arrows it could not read. That "flag what's ambiguous" instruction matters: it stops the model from confidently inventing the parts it cannot see.

A few accuracy notes. Vision reads handwriting, diagrams, charts, UI, and dense text, but very low-resolution or skewed photos degrade results, so crop tight and shoot straight on. For sensitive documents, remember the image is sent to the model like any other input; treat it with the same care as pasted text. The official rundown of what the vision-capable models can and cannot do lives in the OpenAI vision guide.

Vision plus advanced data analysis

Vision gets dramatically more useful when you chain it into Advanced Data Analysis (the Code Interpreter sandbox you met earlier). Upload a screenshot of a chart from a PDF report, and ask ChatGPT to *extract* the underlying numbers and rebuild the chart properly:

> Read the bar values off this chart, put them in a table, then plot a cleaner version with sorted bars and labeled axes.

The model reads the image, writes Python to recreate the data, and runs it in the sandbox. You get a real, downloadable figure plus the data behind it. That is a workflow no single-purpose tool gives you.

Voice: hands-free, real-time conversation

Voice mode lets you talk to ChatGPT and have it talk back, with low enough latency to feel like a conversation rather than a walkie-talkie. Under the hood this is powered by OpenAI's realtime, speech-to-speech models, which is why it can be interrupted mid-sentence and pick up your tone, not just your words.

Where voice shines is anything where your hands or eyes are busy. The canonical case is the hands-free brainstorm:

> I'm going for a walk. Be my thinking partner. I'm naming a B2B analytics product. Throw me five directions, ask me what resonates, and don't dump a giant list, keep it conversational.

That last instruction is the whole game with voice. By default models want to enumerate. In voice, a fifteen-item list is useless because you cannot scan it. Explicitly ask for short turns, one idea at a time, and the ability to challenge you. Treat it like a colleague, not a search box.

Two things make voice far more powerful than it first appears:

• It shares your memory and custom instructions. A voice brainstorm knows your projects and preferences just like a typed chat, so it can reference "the onboarding flow we discussed" without you re-explaining.
• It can be combined with the camera on mobile. You can point your phone at a machine, a menu, or a whiteboard and ask about it out loud. That is vision and voice fused: "What does this dashboard warning mean?" while looking at the screen.

When you are done walking, the entire spoken conversation is there as text in the thread, so you can switch to Canvas and ask ChatGPT to turn the rambling brainstorm into a tight one-pager. The mode you *start* in does not lock you in.

Choosing the right mode

The mistake is treating these as novelties. MapMapUsing software to automate repetitive marketing tasks and campaigns, enabling personalisation at scale across channels like email, web, and social.Voir la définition complète → them to the shape of the task:

• Generate an image when you need to *externalize* a visual idea fast: a mockup, a diagram draft, a concept to align people. Low stakes on pixel precision.
• Use vision when the information is *trapped in pixels*: an error screenshot, a whiteboard, a chart in a PDF, a photo of a form. You are converting image to structured text or analysis.
• Use voice when your *hands or eyes are occupied* or when thinking out loud beats typing: brainstorming, rubber-ducking a problem, dictating while commuting.

Most real workflows chain them. Photograph a whiteboard (vision), discuss it on a walk (voice), then have ChatGPT generate a cleaned-up architecture diagram (image), all in one thread that remembers the whole arc.

Bringing multimodal into the APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.Voir la définition complète →

Everything above is also available programmatically, which is how you bake it into a product instead of doing it by hand. The Responses API accepts text and images in the same request, so the screenshot-debug workflow becomes a function your app can call.

Here is the vision case as code: send an image plus a question and get structured analysis back.

from openai import OpenAI

client = OpenAI()

response = client.responses.create(
    model="gpt-4.1",
    input=[{
        "role": "user",
        "content": [
            {"type": "input_text", "text": "What's the root cause of this build error? One sentence."},
            {"type": "input_image", "image_url": "https://example.com/build-error.png"},
        ],
    }],
)

print(response.output_text)

You can pass a hosted URL as shown, or a base64-encoded local file, which is what you would do for a user's uploaded screenshot. The same input array pattern handles multiple images, so a "compare these two UI states" feature is just two input_image entries.

For image *generation* from the APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.Voir la définition complète →, you call the image generation tool (or the dedicated images endpoint) rather than the chat models, and you get back image data you can store or display. Combine that with structured outputs and you can build a pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.Voir la définition complète → that, say, reads a receipt photo and returns clean JSON line items with guaranteed schemaschema, no fragile parsing.

Voice in the APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.Voir la définition complète → is its own surface: the realtime models power speech-to-speech, while separate transcription and text-to-speech endpoints handle the simpler "just convert audio" cases. If you only need a transcript, use transcription; reachreachThe number of unique people exposed to your message in a given period. Unlike impressions, reach counts each person once, no matter how often they see it.Voir la définition complète → for realtime when you need a true interactive voice agent. The decision mirrors the ChatGPT side: full conversation versus one-shot conversion.

A note on what stays in ChatGPT

Not everything needs the APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.Voir la définition complète →. If your goal is a repeatable *personal* multimodal workflow, a Custom GPT with clear instructions ("Whenever I upload a chart, extract the data, rebuild it cleanly, and give me the table") captures it without a line of code. The APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.Voir la définition complète → is for when you are shipping the capability to other people inside a product.

Key Takeaways

• Pick the mode by where the information lives. Pixels in, use vision; idea out, generate an image; hands busy, use voice. Most strong workflows chain all three in one thread.
• For vision, pair the image with context and ask it to flag ambiguity. A screenshot plus a code snippet, plus "tell me what you can't read," beats the image alone and prevents confident hallucinationhallucinationA hallucination is when an AI model generates output that is fluent and confident but factually wrong, fabricated, or unsupported by its source data.Voir la définition complète →.
• In voice, explicitly demand short conversational turns. Models default to long lists, which are useless out loud. Tell it to give one idea at a time and challenge you.
• Chain vision into Advanced Data Analysis to turn a chart screenshot into real, downloadable data and a clean rebuilt figure.
• Reach for the [Responses API](https://platform.openai.com/docs/api-reference/responses) with `input_image` when you need to productize the screenshot-to-analysis workflow, and add structured outputs for reliable JSON.