Block 5

Building with AI

How to structure an AI project, navigate the tools landscape (RAG, agents, vector databases, APIs), write basic code, and evaluate outputs against cost and latency.

3 Modules·9 Lessons

Everyone talks about building with AI. Most people are still stuck admiring the demo. This block moves you from spectator to builder, and it does it without pretending you need a computer science degree to have an opinion that matters in the room.

You start where every serious project should start, before a single line of code. Framing the problem. Deciding whether you need a prompt, a RAG setup, a fine-tune, or an agent. Setting scope, data, and success criteria so you know what winning looks like. This is the discipline that separates real initiatives from expensive science experiments.

Then you go into the machinery. You map the tools landscape, from APIs to no-code platforms to vector databases. You learn how retrieval-augmented generation puts your own data behind the model, so it stops guessing and starts answering. You see how agents take actions instead of just producing text, and where that power gets dangerous if nobody scoped it properly.

Finally, you get your hands dirty. Your first API call in Python, so you understand what your teams actually do. Real evaluation, because a system that feels smart and a system that works are two different things. And the tradeoffs that decide budgets, cost, latency, and model selection, the choices that quietly make or break your unit economics.

By the end you can sit across from any technical team and hold the conversation as a peer. You will know what is easy, what is hard, what is a red flag, and what is a excuse. That is leverage. Leaders who understand how AI gets built stop being sold to and start directing the work.

What you'll master

  • Frame an AI problem before anyone touches a keyboard
  • Choose between prompting, RAG, fine-tuning, and agents with confidence
  • Scope a project with clear data needs and success criteria
  • Design a RAG system that puts your own data behind the model
  • Evaluate outputs to prove a system actually works
  • Weigh cost, latency, and model tradeoffs like an operator
  • Hold a peer-level conversation with any technical team

Modules

Frequently asked questions

What does it actually take to build with AI if I'm not an engineer?

It takes a method more than it takes code. Framing the problem, choosing between prompting, RAG, fine-tuning and agents, defining data needs and success criteria, then evaluating outputs against cost and latency. Writing a first API call in Python helps you understand what your teams do, but the decisions that make or break an AI project are made before anyone opens an editor.

What's the difference between RAG and fine-tuning, and when do I use each?

Retrieval-augmented generation puts your own documents behind the model at query time, so the answer is grounded in your data without retraining anything. Fine-tuning modifies the model itself to adopt a format, a tone, or a specialised behaviour. RAG is the default when the issue is knowledge the model doesn't have; fine-tuning addresses how the model responds, not what it knows.

How do I know if an AI system actually works and isn't just a convincing demo?

You define success criteria before building, then evaluate outputs systematically against them. A demo shows the system on cases chosen to flatter it; an evaluation runs it on cases that represent real usage, including the awkward ones. Without that, you're measuring how smart the system feels, which is a different thing from whether it works.

When does an AI agent make sense instead of a simple prompt?

An agent makes sense when the task requires taking actions across several steps or tools, not just producing text. A prompt returns an answer; an agent can call an API, query a system, and chain decisions. That capability is also where the risk concentrates, which is why scope and permissions have to be defined before deployment, not after an incident.

Why does my AI project need a vector database?

A vector database stores embeddings, the numerical representations of your documents, so the system can retrieve the passages most relevant to a question. It's the retrieval half of a RAG setup: without it, the model has no way to find your data before answering. If you aren't grounding answers in your own content, you probably don't need one.

How do cost and latency influence which model I choose?

They usually decide it. A larger model gives better answers but costs more per call and responds more slowly, which matters enormously at volume or in a user-facing flow. The practical approach is to set the quality threshold your use case actually requires, then pick the cheapest and fastest model that clears it. These tradeoffs quietly determine the unit economics of the whole project.