How to Learn Agentic AI: 50 Real Questions, Answered

Before the Advanced Agentic AI Workshop, we asked everyone registering a single open question: what do you want to learn?

We got dozens of answers. Some were four words. One was a paragraph about guardrails on proprietary enterprise code that I've now read about nine times. One was just the word "Abby."

I read all of them and something became obvious: the room wasn't one audience. It was three, sitting in the same chairs.

Group one wants to understand it. They typed "Basics," "Expand knowledge," "Learn tokens and agentic ai." They're not behind — they're being honest.

Group two wants to ship it. "Build an agentic system in production." "Run agents while I'm away." "Port my code to Unity." They've done the tutorials. The tutorials stopped helping.

Group three wants to govern it. "AI governance and ensuring accuracy." "Hidden risks and where you need the human in the loop." "Roadblocks for AI rollouts." They're the ones who'll be in the room when it goes wrong.

Here's the thing I keep coming back to: everyone eventually needs all three. The person asking for basics will be asked about governance within a year. The person shipping to production will discover they skipped the fundamentals about tokens and now their costs are insane.

Stripped of duplicates, they came to 50 distinct questions. I answered every one. Individually, grouped into 11 sections. Skip to yours — or read the whole thing and find out which group you're actually in.

---

Part 1 — Foundations: The Vocabulary Nobody Explains

From the people who wrote: "Basics." "Expand knowledge." "Learn tokens and agentic ai." "One or more useful AI tips." "How to use AI at best of my abilities." And one beautifully complete request covering prompting, model mechanics, workflows, and context management.

What actually is an agent — and what's just a chatbot with extra steps?

This is the most useful distinction in the whole field, and almost nobody draws it cleanly.

A chatbot is a function. You put text in, you get text out, and the loop is closed by you. You read the answer, you decide what to do next, you type again. You are the control flow.

An agent is a loop that closes itself. It gets a goal, decides on an action, takes that action against the real world, observes what happened, and decides again — until it's done or it gives up. The model isn't producing the answer. The model is producing the next move.

The practical test: if you removed the human from the middle, would anything still happen? If yes, it's an agent. If the whole thing stops, it's a very good chatbot.

Concrete example. "Summarize this contract" — chatbot. "Watch this folder, and when a contract lands, extract the renewal date, check it against our CRM, and if it's under 60 days out, draft the renewal email and put it in my drafts" — agent. Same model underneath. Completely different system around it.

That system around it is where all the actual engineering lives. The model is maybe 20% of the work.

Tokens, context windows, and training cutoffs — what they mean once you're building, not chatting

When you're chatting, tokens are trivia. When you're building, they're your budget, your latency, and your failure mode. Three things worth internalizing:

A token is roughly ¾ of a word. That's it. "Agentic" is probably two tokens. A 10-page PDF is maybe 6,000. The reason this matters is that you pay per token, in both directions, on every loop iteration. An agent that runs 15 steps re-reads its context 15 times. A sloppy prompt isn't a style problem — it's a cost multiplier with an exponent.

The context window is working memory, not storage. It's what the model can see right now. Big windows made people lazy: "just stuff everything in." But models attend unevenly across a long context — material in the middle of a huge window gets less reliable treatment than material at the edges. A tightly curated 8,000-token context routinely outperforms a lazy 200,000-token one. Curation is a feature.

The training cutoff is a hard wall, and agents are how you get past it. A model knows the world up to a date. It does not know your Q3 numbers, your codebase, or what happened last Tuesday. This is exactly why tool use exists — search, retrieval, and API calls aren't bonus features, they're the model's only route to current and private truth. Hallucination is very often just a model being asked a question it had no way to answer, and being too agreeable to say so.

Practical rule I use: anything that can be looked up should be looked up, not remembered. Reserve the context window for reasoning, not for facts.

Does prompt structure — role, context, command, format — still matter in 2026?

One attendee laid out the classic four-part structure. It's a good structure. Here's my honest read on where it stands.

For chat, it matters less than it used to. Models are far better at inferring what you want. You don't need to say "you are a helpful expert" to get expert help.

For agents, it matters more than ever — but it's been renamed. What used to be "prompting" is now system design, and the four parts map almost exactly:

• Role → the agent's scope and permissions. What is this thing allowed to do?
• Context → retrieval strategy. What gets loaded, when, from where?
• Command → the task decomposition. What's one step versus one run?
• Format → the output contract. Structured schema, not vibes.

That last one is the highest-leverage change most people can make today. Stop asking for prose you'll parse with a regex. Ask for a defined schema and validate it. If the output doesn't validate, retry automatically. You've just turned a soft failure into a caught error, which is the entire game.

The tip I'd actually give: write the failure instruction. Most prompts describe success only. Add the line that says what to do when it can't — "if the document doesn't contain a renewal date, return null, do not infer one." That single sentence eliminates a large share of hallucinations, because you've given the model a legitimate way to say "I don't know."

How do I stop an agent's context from rotting mid-task?

Context rot is the thing that makes a demo work brilliantly for six steps and then go strange on step nine. The agent starts contradicting itself, forgets the constraint you gave it at the start, or re-does work it already did.

The cause is almost always the same: context is being appended to, not managed. Every step adds tool output, reasoning, errors, and retries. By step nine the actual instruction is buried under 40,000 tokens of transcript.

Treat context as a system with four distinct layers, each with a different lifetime:

The layer people skip is task state. Keep an explicit, structured scratchpad — a plan with checkboxes — that the agent rewrites after each step rather than a transcript it appends to. Now step nine reads a clean 200-token status instead of excavating its own history.

For the attendee who asked about uploading reference files and maintaining project history: same principle. Don't paste the reference material into the conversation. Index it, and retrieve the relevant three paragraphs per step. Your project history should live in a file the agent reads and updates — not in the chat.

Using AI as a build partner instead of an answer machine

The difference is whether you bring it a question or a problem.

An answer machine gets: "write me a function that dedupes a list." You get a function. Fine.

A build partner gets: "here's the shape of what I'm making, here's the constraint I'm stuck on, here's what I already tried and why it failed — what am I not seeing?" Now you get pushback, alternatives, and occasionally the observation that your constraint is the actual problem.

Three habits that make this real:

Give it the failures, not just the goal. What you already ruled out is the most information-dense thing you have. Most people withhold it and then get suggestions they already rejected.

Ask for the argument against. After you get a plan you like, ask what breaks it. Models will happily agree with you all day; you have to explicitly invite the disagreement. "What would a skeptical reviewer say about this design?" is a genuinely useful prompt.

Work in a persistent artifact, not a chat. Keep a living document — the spec, the design, the plan — that both of you edit. Chats scroll away. Documents accumulate.

The tell that you've got a build partner rather than an answer machine: you occasionally change your mind because of it.

"I want to use AI to the best of my abilities" — where's the ceiling, and how do I find mine?

I love this question because it's really asking: how do I know what I don't know?

Here's the honest map. Most people's ceiling isn't the model's capability — it's their task selection. They're using a system that can run a six-hour research project to write emails slightly faster.

Four levels, and you can locate yourself in about ten seconds:

Level 1 — Faster typing. Drafting, summarizing, rewriting. Real value, small value. Most people live here permanently.

Level 2 — Thinking partner. Analysis, critique, exploring options, being argued with. This is where the quality of your input starts mattering more than the tool.

Level 3 — Delegated tasks. You hand over a whole job with a clear definition of done — "research these 12 competitors and build me a comparison table with sources" — and you review output, not process.

Level 4 — Standing systems. Something runs on a schedule or a trigger without you. You wrote it once. It works while you're asleep. (Part 5 is entirely about getting here.)

The jump that changes people's lives is 2 → 3, and it's not a skill jump. It's a trust jump, and it's gated on one thing: can you check the output faster than you could have made it? If yes, delegate it. If no, don't — you'll spend more time verifying than doing, and you'll conclude the tool is bad when actually the task was wrong.

So the answer to "how do I find my ceiling": for one week, write down every task you didn't delegate, and next to it, why. The list of reasons is your ceiling, in your own handwriting.

---

Part 2 — Architecture: How an Agent Is Actually Built

From: "Building agents" (×2). "Agent building" (×2). "Agentic AI" (×2). "Agent training." "Advanced concepts I might not know about yet." "Agent-to-agent communication." "Fully autonomous agents." "The advanced methods for AI agents management."

The five components every agent has — and the one everyone forgets

Whatever framework you use, whatever model you're on, every agent decomposes into the same five parts. Learn these and you can read any agent codebase in about twenty minutes.

1 — The model. The reasoning engine. Swappable, and you should design assuming you'll swap it. Pick per-task, not per-project: a cheap fast model for classification steps, an expensive one for the hard reasoning step.

2 — Tools. Functions the agent can call. This is where most agent quality actually comes from. A mediocre model with excellent tools beats a great model with vague ones. Good tools are narrow, well-named, and return structured results with clear errors. get_customer_by_email beats query_database. Every tool you add expands capability and expands the surface area for mistakes.

3 — Memory. Split it: short-term (the current run's state) and long-term (what carries across runs — preferences, facts, prior decisions). Most people build the first and skip the second, which is why their agent has amnesia every morning.

4 — Orchestration. The control loop. When to keep going, when to stop, what to do on failure, how many steps are too many, when to escalate to a person. This is where your actual product logic lives, and it's mostly boring engineering: state machines, retries, timeouts, idempotency.

5 — Evaluation. This is the forgotten one. Nearly everyone builds components 1–4, ships, and then has no way to answer "is it getting better or worse?" Without eval, every change is a guess and every regression is a surprise from a customer. Part 4 goes deep on this — but note that it's a component, not a phase. It belongs in the architecture diagram.

If you're starting today: build 2 and 5 first. Tools and eval. The rest is comparatively easy.

Do I need to train anything? The ladder from prompt to fine-tune

Short answer for the person who wrote "Agent training": almost certainly not, and if you think you do, you're probably one rung too high on this ladder.

Climb in order. Only go up when the rung below genuinely fails.

Rung 1 — Better instructions. Clearer scope, examples of good output, explicit failure handling. Free, instant, and it solves a genuinely embarrassing share of problems people plan to solve with fine-tuning.

Rung 2 — Tools. The model isn't wrong, it just can't see your data. Give it a way to look. Solves most "the model doesn't know our stuff" complaints.

Rung 3 — Retrieval (RAG). Too much proprietary knowledge to fit in context. Index it, search it, inject the relevant slices. This is where most serious enterprise systems live and stay.

Rung 4 — Fine-tuning. Now you're teaching style, format, or a narrow classification behavior — not facts. Fine-tuning is bad at teaching knowledge and good at teaching shape. Use it when you need consistent output structure at scale, or a small cheap model to imitate a big expensive one on one specific task.

The trap: people try to fine-tune facts in. It works badly, it's expensive, and every fact change means retraining. Facts belong in retrieval. Behavior belongs in fine-tuning. Keep the line clean and you'll skip a very costly quarter.

The advanced concepts that actually change your design

For the attendee who asked what they might not know about yet — here are the five ideas that, in my experience, meaningfully change how you build. Not the buzzwords. The ones with consequences.

Tool design is prompt engineering. The tool's name, its parameter names, and its description are read by the model on every single call. A badly described tool is a badly written instruction repeated a thousand times. Rewriting tool descriptions is often the highest ROI hour available to you.

Error messages are inputs. When a tool fails, the error text goes back into the model's context and shapes the next attempt. Error: 400 teaches nothing. Error: date must be YYYY-MM-DD, you sent "next Tuesday" produces a correct retry. Write your errors for the model, not for a log file.

Context is a budget, not a bucket. Covered in the context-rot answer above, but the design consequence: build a compaction strategy before you need it, because you'll need it exactly when the system is under load and hardest to change.

Determinism where you can, model where you must. Every step you can do in normal code — validation, routing, math, formatting — should be. Models are for judgment. Using an LLM to do arithmetic or route by keyword is paying reasoning prices for a lookup, and adding variance for free.

Idempotency and replay. Agents retry. If your tool sends an email on every call, retries send four emails. Design tools so calling twice is safe, and log every step so you can replay a failed run to see exactly what it saw. Debugging a non-deterministic system without a replay log is genuinely miserable.

Multi-agent and A2A: real leverage or theater?

One attendee asked specifically about agent-to-agent communication and advanced multi-agent workflows. Here's my balanced take, and I'll try to be fair to both sides because this one gets tribal.

The case against most multi-agent systems: many are a single agent wearing a hat collection. If your "team" of five agents all use the same tools, share the same context, and hand off in a straight line, you've built one agent with extra latency and five times the token cost. The org-chart metaphor is seductive and often just cosplay.

The case for it, which is real: multi-agent earns its cost in three specific situations.

Fan-out for parallelism. Genuinely independent subtasks — search five sources, review eight files — run concurrently. Wall-clock time drops. Unambiguous win.

Adversarial verification. A second agent whose only job is to attack the first one's output. This works because it has no sunk cost in the answer. It's the single most reliable quality technique I know, and it's the basis of the judge layers in Part 4.

Genuine specialization by permission. Not by personality — by access. The agent that can read HR records shouldn't be the one browsing the web. Splitting them is a security boundary, and security boundaries are a legitimate reason to split anything.

On A2A protocols specifically: standardized agent-to-agent communication is where the field is heading, and it matters most for agents crossing organizational boundaries — your agent talking to a vendor's agent. Inside your own system, you don't need a protocol; you need function calls. Watch the space, don't build your internal architecture around it yet.

My rule: start with one agent. Split only when you can name the specific constraint the split relieves — time, objectivity, or permissions. "It felt more organized" is not a constraint.

Fully autonomous agents — where the honest ceiling sits

The attendee who wanted to "exchange ideas on building fully autonomous agents" deserves a straight answer rather than either hype or dismissal.

Full autonomy is achievable today — but only inside a bounded domain with three properties:

Verifiable outcomes. You can check the result mechanically. Code that compiles and passes tests. A form that validates. A number that reconciles. Where verification is automatic, autonomy is safe, because failure is caught by the system rather than by a customer.

Reversible actions. Everything the agent does can be undone. Drafts, branches, staged changes, sandboxes. Autonomy plus irreversibility is how you get an incident report.

Bounded blast radius. Worst realistic case is annoying, not catastrophic. Ask literally: if this ran wrong 100 times overnight, what's the damage? If the answer involves money leaving, data being deleted, or customers being contacted, you don't want autonomy there — you want a queue and an approval.

Where autonomy is not close: anything where the definition of "correct" is contested, taste-dependent, or political. An agent can autonomously refactor a module. It cannot autonomously decide your pricing strategy, and the failure there won't look like an error — it'll look like a confident, well-written, wrong answer.

So the realistic frontier isn't "autonomous vs. supervised." It's autonomous execution with supervised commitment. The agent does 95% of the work unattended and stops at the one gate that's expensive to get wrong. That's not a compromise. That's how you'd manage a talented new hire, and for the same reasons.

Managing many agents: orchestration, shared memory, and cost

For the person asking about advanced management methods — this is the problem you hit at agent number four, and it's an operations problem, not an AI problem.

Orchestration. Once you have several agents, something has to own scheduling, dependencies, and failure. Resist building a clever meta-agent to manage other agents — you've added a non-deterministic layer to your control plane, which is exactly where you want determinism. Use a normal workflow engine. Let the agents be the smart part and the orchestrator be the dumb, reliable part.

Shared memory. Agents that don't share state duplicate work and contradict each other. Agents that share everything leak context and blow up costs. The pattern that works: a shared write-once record of decisions and facts, plus private working memory per agent. Anyone can read the decision log. Nobody reads anyone else's scratch work.

Cost. Costs scale multiplicatively and quietly. Steps × agents × context size × retries. The controls, in order of impact: cap steps per run, compress context aggressively between steps, route easy steps to cheap models, and cache anything you compute twice. Instrument cost per completed task, not per token — per-token is a bill, per-task is a metric you can manage.

The thing to build early: a single dashboard showing runs started, runs completed, runs escalated, average steps, and cost per task. You will not manage what you cannot see, and by agent number six you'll be flying blind without it.

---

Part 3 — Production: Strategy Through Deployment

From: "Agentic AI — end to end understanding, strategy thru deployment." "Agentic AI end to end deployment." "The whole lifecycle of setting up and managing the AI agent system." "Learn how building production grade systems work." "I want to learn how to build an agentic system in production." "End to end demos." "How to take the problem and solve end 2 end — FDE." "Current challenges and efficient processes."

The complete lifecycle, on one page

Eight people asked some version of "show me the whole thing, start to finish." Here it is. Six stages, and the two most-skipped are marked.

1 — Problem selection. More projects die here than anywhere else. You want high volume, tolerable error, and a clear definition of done. If you can't write down what "correct" looks like in a sentence, stop; you'll never know if it works.

2 — Baseline. Do the task manually 20 times and record how long it takes, how often a human gets it wrong, and what the edge cases are. Skipping this means you'll later have no idea whether 82% accuracy is a triumph or a disaster. (Humans are often at 85%. This surprises people.)

3 — Build. Thinnest possible version. One tool if you can. Resist the framework buffet in week one.

4 — Evaluate. A fixed test set — 50 to 200 real cases with known-good answers — that you run on every change. Without this you're not engineering, you're vibing at scale.

5 — Deploy in three gears. Shadow: it runs but nothing it produces is used, and you compare against humans. Assist: output goes to a human who approves or edits. Auto: it acts, with monitoring. Most teams jump straight to gear three and then get to explain themselves in a meeting.

6 — Operate. Models change, data drifts, the world moves. Every production failure becomes a new case in your eval set. That feedback arrow is the whole product.

What separates a demo from a production-grade system?

Two attendees asked this almost verbatim. The gap is bigger than people expect and it's not about the model at all — it's the six things a demo gets to assume.

Failure handling. The demo path is happy. Production is 30% unhappy paths: the API is down, the PDF is a scan, the field is empty, the response is malformed. Production-grade means every one of those has a defined behavior that isn't "crash" or "hallucinate something plausible."

Determinism where it counts. Same input should mean substantially the same output. That means pinned model versions, controlled sampling on critical steps, and validated output schemas. A demo that gives a different answer each run is charming. A system that does is a support ticket generator.

Observability. Every run logged with full inputs, tool calls, outputs, cost, and duration — replayable. When someone says "it did something weird on Tuesday," you need to see exactly what it saw. This is non-negotiable and it's the thing people bolt on too late.

Cost and latency ceilings. A demo can take 90 seconds and cost $2. If that's your unit economics at 10,000 runs a day, you have a business problem masquerading as a technical one.

Security boundaries. What can this thing touch? Least privilege, real credentials management, and a very clear answer to "what happens if someone puts malicious instructions in a document the agent reads." Prompt injection is not theoretical once your agent reads untrusted input.

A rollback. A switch that turns it off and routes back to the old process, testable, that someone who isn't you knows how to use at 2am.

Demo is 20% of the work. That's not pessimism — it's the ratio, and knowing it upfront is what stops projects from stalling at 80% "done" for three months.

What a real end-to-end demo should show you

For whoever wrote "End to end demos" — here's what to demand, because most demos are theater.

Ask to see: a failure, handled gracefully. The eval numbers, on a test set, with the baseline next to them. The cost per run. The escalation path when the agent is unsure. And the logs for a run that went wrong.

If a demo only shows the happy path on curated inputs, you've learned that the model works, which you already knew. You've learned nothing about the system. The good news: asking these five questions makes you the most useful person in any AI vendor meeting for the rest of your career.

The FDE playbook: take a problem and solve it end to end

The attendee who wrote "FDE" is describing the forward-deployed engineer pattern — sitting with the people who have the problem and building until it's actually solved. It's the highest-leverage way to build agent systems right now, and it inverts the normal process.

Week one: don't build. Watch. Sit with the person doing the work. Record the steps. You will find that the documented process and the real process differ substantially, and the difference is where all the value is. The exceptions everyone handles from memory — that's the actual system.

Find the bottleneck step, not the annoying step. People will point you at what irritates them. That's rarely the constraint. Look for where work queues.

Build in the open, in days not sprints. Show something rough on day three. The feedback from a rough working thing is worth more than a month of requirements gathering, because people can't describe their own tacit knowledge — but they can absolutely tell you when your version is wrong.

Take responsibility for the outcome, not the deliverable. The FDE mindset is "this problem is solved," not "my component shipped." That means you own the data cleanup, the awkward integration, and the training session nobody scoped.

Then generalize — carefully. Solve it deeply for one team, then look for the second team with the same shape. The mistake is generalizing after one instance and building a platform nobody asked for.

Where agent projects actually stall right now

Someone asked about current challenges. From what I see, in order of frequency:

Evaluation debt. Teams ship without a test set, iterate on vibes, and hit a wall where every fix breaks something else invisibly. This is the number one killer and it's entirely preventable.

Integration, not intelligence. The model works fine. The blocker is that the data lives in a system with no API, owned by a team that's busy until Q4. Most "AI projects" are integration projects with a model at the end.

The 80% plateau. Getting to 80% takes two weeks. Getting to 95% takes six months, because the last 15% is a long tail of weird cases. Teams that budgeted for the two weeks quietly die here. The fix is design, not effort: build the escalation path so 80% is shippable, with the rest routed to a person.

Trust collapse. One bad output in front of an executive and the project is politically dead regardless of aggregate accuracy. This is why shadow mode exists — accumulate a track record before anyone's reputation is attached.

Ownership vacuum. It launches, the project team disbands, nobody owns it, quality drifts, it gets switched off in six months. Agents are operated, not delivered. Someone has to have their name on it.

---

Part 4 — Trust: Guardrails, Evals, and Governance

From: "AI governance and ensuring accuracy." "New possibilities to optimize tasks at work and understand the hidden risks / points where you need the human in the loop." "Use cases and common roadblocks for AI rollouts." And the longest, most specific submission we received — twice.

Governance and accuracy without standing up a governance committee

The word "governance" makes people picture a policy document nobody reads. Let me reframe it as three questions you can answer this week.

What is this agent allowed to touch? Write it down. Systems, data, actions. Not aspirationally — actually enforce it with credentials and scoped permissions. An agent's blast radius should be a configuration, not a hope.

How do we know it's right? This is accuracy, and it needs a number attached to a test set. "It seems good" is not governance. "94% on 200 held-out cases, measured weekly, with these three known failure modes" is.

Who is accountable when it's wrong? A named person. Not "the AI team." The moment a human name is attached, all the other governance questions get answered quickly, because someone has an incentive.

That's genuinely most of it at small scale. Add a decision log — every material design choice, why, and what you'd need to see to reverse it — and a change process so nobody edits a production prompt at 11pm without an eval run. At enterprise scale you'll add model risk review, data residency, and audit trails. But if you're not doing the three questions above, the heavier apparatus is decoration.

On accuracy specifically, the underrated move: make the agent's uncertainty visible. A system that says "I'm confident about the date, unsure about the amount" is dramatically more governable than one that presents everything with the same flat certainty. Ask for confidence, route low confidence to a human, and you've converted an accuracy problem into a workflow problem — which is a much better problem to have.

The anchor question: guardrails, eval and judge layers on legacy proprietary enterprise code

Two people submitted this. It's the most sophisticated question we got, so it gets the longest answer. Paraphrasing: how are guardrails built, how do eval and judge layers actually work to keep specialized dynamic agents on rails — even against legacy, proprietary production code with in-house design patterns — how do guardrails iterate, and what metrics actually measure success in an enterprise?

Let's take it in four parts.

Part one: guardrails are four layers, not one thing

"Guardrail" gets used for four genuinely different mechanisms with different costs and different reliability. Confusing them is why teams end up with expensive, slow systems that still go off the rails.

The governing principle: every check you can move up the stack is cheaper, faster, and more reliable. Teams reach for a judge model first because it's the interesting layer. It should be the last resort, reserved for things that genuinely require judgment.

Part two: the legacy proprietary code problem

This is the hard part of the question, and it's a genuinely different problem from "agent writes code." The agent has strong priors from public code and your codebase doesn't follow them. It'll write idiomatic modern patterns into a system with fifteen years of in-house conventions, and everything will look plausible and be wrong.

Four things work here, roughly in order of impact:

Extract the conventions into a written constitution. Not a wiki — a compact document the agent reads on every run. "We use the Xyz repository pattern, never direct SQL. All errors go through AppError. Never import from internal/legacy outside its own package." Ten to fifty rules. This is the single highest-leverage artifact you can build, and the useful side effect is that it's also the best onboarding doc your team has ever had.

Make examples the specification. Better than describing your pattern: point at three files that exemplify it and say "match these." Models are much stronger at imitation than at following abstract descriptions. Curate a small set of canonical files and pin them in retrieval.

Turn conventions into mechanical checks wherever possible. Every convention you can express as a custom lint rule moves from layer 4 to layer 3 — from a judge's opinion to a pass/fail. Writing custom AST rules for your house patterns feels like a detour. It's the main road. It gives both your agents and your humans the same enforceable definition of correct.

Constrain the working set. Don't hand an agent the whole monolith. Give it one module, its tests, its direct dependencies, and the conventions doc. A tightly scoped agent with 5,000 tokens of the right code beats one drowning in 300,000 tokens of the whole repo.

And the safety valve that makes this survivable: agents propose, they don't merge. Output goes to a branch or a diff. Existing code review is your final guardrail and it already exists.

Part three: how judge layers actually work

A judge is a model evaluating output against a rubric. Three things determine whether it's useful or expensive noise:

The rubric must be specific and criterion-by-criterion. "Is this good code?" produces mush. "Does this follow the repository pattern as shown in the reference file? Yes/no, cite the line." produces signal. Score criteria separately — a combined 1-10 score averages away the information you need.

The judge must be independent. Fresh context, no visibility into the reasoning that produced the output, ideally a different model. A judge that reads the maker's justification gets talked into things, exactly like a person would.

The judge must be calibrated against humans. This is the step everyone skips. Take 50 outputs, have your senior engineers grade them, have the judge grade them, and measure agreement. If the judge agrees with your experts 60% of the time, it is not a quality gate — it's a random number generator with good grammar. Iterate the rubric until agreement is high enough to act on, and re-check it quarterly.

Judges work well for: convention adherence, completeness against a spec, tone, whether the reasoning supports the conclusion. Judges work badly for: correctness that a test could determine, anything requiring context they don't have, and anything where your own experts disagree with each other. If humans can't agree, a judge can't arbitrate.

Part four: how guardrails iterate, and the metrics that hold up

Guardrails aren't authored, they're accumulated. The loop:

1. Something goes wrong in production.
2. Add that exact case to the eval set — permanently.
3. Ask: which layer should have caught this? Fix at the highest layer that could.
4. Re-run the full eval to make sure the fix didn't break anything else.

Over a year this produces a test set that encodes everything your system has learned. It becomes your most valuable asset, more than the prompts.

On metrics — the ones I've seen actually drive decisions in enterprise settings:

Task completion rate without human intervention. The headline number. Percentage of runs that finish correctly with nobody stepping in. Everything else explains this one.

Escalation precision. Of the cases the agent flagged for human review, what fraction genuinely needed it? Low precision means humans stop trusting the flags, and then the whole safety design quietly stops working.

Miss rate — the one that matters most. Cases the agent handled confidently and got wrong. This is the number that ends projects. Track it separately from overall accuracy, because a system that's 95% accurate with a 5% silent-error rate is far more dangerous than one that's 85% accurate and flags every uncertainty.

Review time per output. If a human spends 15 minutes checking what took 20 to do manually, you've built an expensive lateral move. Falling review time over months is the real adoption signal.

Cost per completed task, versus the manual baseline. Not cost per token. The comparison unit executives actually respond to.

Rework rate. How often does accepted output come back later as a defect? Catches the failure mode where reviewers rubber-stamp because the output looks confident.

The two vanity metrics to avoid: raw model benchmark scores (irrelevant to your domain) and usage volume (measures enthusiasm, not value).

One last thing worth saying plainly: in enterprise settings the binding constraint is usually not model capability. It's that nobody wrote down what correct looks like. The guardrail work forces that conversation, and that's often worth more than the agent.

Where to put the human in the loop — and where a human just adds latency

One attendee asked about hidden risks and the points where you need a human. Here's the framework I use, which is deliberately unsentimental.

Put a human where the cost of being wrong is high and asymmetric. Money leaving. Data being deleted. A customer being contacted. Anything legally binding. In these places the human isn't checking quality — they're accepting accountability, which is a different job.

Put a human where the agent is uncertain — but only if the agent can express uncertainty usefully. Confidence-based routing is the highest-value human-in-the-loop pattern available, and it only works if you designed for it.

Put a human where taste is the deliverable. Brand voice, sensitive communication, strategy. An agent can draft; the judgment is the product.

Now, where humans add nothing but delay:

Rubber-stamp approvals. If your reviewer approves 99% of items without changes, they're not a control — they're a queue. Sample-audit instead: check 5% properly, which catches drift and costs almost nothing.

Checks a machine does better. Humans are worse than compilers at finding syntax errors and worse than diff tools at spotting changes. Don't spend attention there.

Reviews with no authority to reject. If the reviewer can't realistically say no — no time, no context, no standing — the review is ceremony. Remove it or empower it.

The hidden risks, since they were asked for directly: automation complacency (people stop reading carefully after two good months — this is the big one), silent drift (quality degrades gradually and nothing alerts), skill atrophy (nobody remembers how to do it manually when the agent is down), prompt injection (untrusted input containing instructions), and accountability diffusion (everyone assumed someone else checked). Four of those five are organizational, which tells you where to spend your worry.

The rollout roadblocks, ranked by how often they kill the project

An attendee asked for use cases and common roadblocks. Use cases run through Part 8; here are the roadblocks in the order they actually take projects down.

1 — No owner after launch. Covered under where projects stall, and still the top killer. Delivered, then orphaned.

2 — Data access. The agent needs information locked in a system with no API and a protective owner. Budget real months for this and start it on day one, in parallel with the build.

3 — The trust incident. One visible bad output early, before a track record exists. Preventable with shadow mode and staged rollout. Almost never recoverable after the fact.

4 — Solving the wrong problem. Automating a workflow that shouldn't exist. Sometimes the right answer is to delete the process, and an agent is an expensive way to avoid that conversation.

5 — Change management. The people whose work changes weren't involved and don't want it. Involve them as co-designers, not recipients. The forward-deployed approach above exists partly to prevent this.

6 — Compliance late. Legal and security find out in month five. Bring them in during week two, when their concerns are cheap to design around instead of expensive to retrofit.

Notice that exactly one of these six is technical. That's the actual lesson of this section.

---

Part 5 — Autonomy: Getting Off the Chat Window

From: "Look forward on how to start automating vs just doing chat session initiated tasks." "Streamlining AI agents in the workflow." "How to get agents to work over night on dashboards and analysis tools when I am asleep." "Run ai agents so they work while I am away." "Automate workflows" (×2). "Automation and AI workflows." "How can I think when it comes to building or automating workflows." "New possibilities to optimize tasks at work." "Building a workflow agent that I can talk about in interviews."

Chat is a demo. Triggers are a product.

The attendee who described "automating vs just doing chat session initiated tasks" has identified the single most important transition in this whole field. Everything valuable is on the other side of it.

Here's the shift in one line: stop being the trigger.

In a chat session, you are the scheduler, the input provider, the error handler, and the delivery mechanism. The agent is a very capable component in a system where you're doing four jobs. Automation means replacing each of those four with something that isn't you.

The practical on-ramp, which takes about a week:

Take one chat task you do repeatedly. Write down the prompt you actually use. Then answer four questions: what event should start this? Where does the input come from without me? What would "obviously wrong" look like? Where should the answer land?

Answer those four and you have a specification. The building is the easy part.

Start with read-only. The first automated version should produce something and put it somewhere you'll see it. No actions taken, nothing sent. Run it for two weeks. You'll be surprised how often it's wrong in the first week, and grateful nothing shipped.

On "streamlining agents in the workflow" specifically: the streamlining is mostly deletion. Every handoff, every copy-paste, every "and then I check it" is a seam. Automation is the process of removing seams, and you remove them one at a time, from the end you understand best.

The overnight agent: architecture for runs nobody is watching

Two people asked this — one wanting dashboards and analysis built while asleep, one wanting agents to work while away. It's my favorite question in the set, because it's completely achievable and almost nobody does it.

An unattended run has one property that changes everything: nobody will notice the failure for eight hours. Every design decision follows from that.

The five rules that make this work:

Snapshot your inputs before you start. If the data changes mid-run, you can't reproduce what happened. Freeze it, run against the frozen copy, log the snapshot ID.

Cap everything. Steps, wall-clock time, dollars. An unattended agent in a loop is a genuinely expensive way to learn this lesson, and people learn it exactly once.

Write to drafts, never to production. The overnight analysis lands as a draft dashboard, a branch, a proposed update. You approve in the morning in ninety seconds. This is what makes the whole pattern safe enough to actually deploy.

Deliver a brief, not a dump. The morning output should be three things: what it did, what it found, what it couldn't handle. If you have to read a log to understand your overnight run, the run isn't finished.

Build the dead-man switch. If the agent doesn't report by 7am, that's an alert. The failure mode of unattended systems is silence, and silence looks exactly like success until someone checks.

For the dashboards-and-analysis case specifically: the highest-value overnight pattern isn't "build me a dashboard." It's "look at yesterday's data, compare to the trailing pattern, and tell me the three things that changed." Anomaly narration. You wake up to a paragraph about what's different, not a chart you still have to interpret. That's the version that changes how you work.

A thinking model for decomposing any workflow

One attendee asked, quite precisely, how to think when building or automating workflows. That's the right question and here's the model I use.

Every workflow decomposes into four kinds of steps, and each kind has a correct implementation. Getting this mapping right is most of the design work.

The method, four passes over the workflow:

Pass one — list the steps as they actually happen, including the informal ones. "And then Priya eyeballs it" is a step. Write it down.

Pass two — label each step MOVE, JUDGE, DECIDE, or CHECK. Be strict. Most steps people plan to give a model are actually MOVE steps, and coding them is cheaper, faster, and correct every time.

Pass three — find the queues. Where does work sit waiting? That's your bottleneck and it's where the value is, regardless of which step is most annoying.

Pass four — draw the boundary. Automate a contiguous run of MOVE/JUDGE/CHECK steps that ends at a DECIDE. That's your v1. The DECIDE step is your natural human checkpoint, and it means you never have to argue about whether the agent is "allowed" to do something consequential.

The most common design error I see: automating a single step in the middle. You've added an integration on both sides and saved thirty seconds. Automate a contiguous span, ending at a decision. That's where the compounding is.

How to pick your first workflow — and the ones to refuse

Three attendees asked simply to "automate workflows." The selection matters more than the build, so here's the filter.

Score a candidate on four things:

Frequency. Daily beats monthly. You need repetition both for ROI and for enough examples to know if it's working.

Tolerance for error. Can a wrong answer be caught and fixed cheaply? Internal drafts, yes. Outbound customer communication, not for v1.

Clarity of done. Can you describe correct output in one sentence? If three people on your team would define it differently, fix that before automating it.

Data reachability. Is the input somewhere a program can get it today? A perfect use case behind an inaccessible system is a six-month project wearing a two-week costume.

Refuse these, at least at first: anything where correctness is contested, anything requiring context that only lives in someone's head, anything with legal exposure, anything you do twice a year, and anything where the current process is broken — fix the process first, or you'll automate the mess and make it permanent.

The unglamorous truth: the best first workflow is usually boring, internal, high-frequency, and slightly embarrassing that a human ever did it. Look for the thing someone does every Monday morning that they'd describe as "just admin."

Which everyday work tasks pay off first

An attendee asked about new possibilities for optimizing tasks at work. Sorted by payoff-to-effort, from what I've watched actually stick:

Reading things so you don't have to. Long documents, meeting transcripts, research, inboxes — reduced to "here's what's relevant to you." Highest hit rate, lowest risk, immediate.

Monitoring for change. Watching a competitor page, a metric, a regulatory feed, a repo, a channel — and telling you only when something meaningfully moves. Massively underused, because it replaces a task most people don't realize they're doing badly.

First drafts of structured documents. Reports, updates, specs, summaries that follow a known shape. The shape is the leverage: with a good template, the draft is 80% there.

Cross-referencing. Comparing two sources that should agree and flagging where they don't. Invoices against contracts, docs against code, spec against implementation. Tedious for humans, error-prone, and mechanically checkable — a perfect fit.

Preparation. Everything that should happen before a meeting, a review, or a decision: gathering context, pulling last quarter's numbers, listing open items. Nobody has time for it, which is exactly why it's valuable.

The pattern across all five: they're reduction tasks, not creation tasks. Take a large amount of input and produce a small, correct, relevant output. Reduction is where verification is easy and value is immediate. Creation is where the risk lives.

Building a workflow agent you can defend in an interview

Great question, and the framing tells me you already understand something important: the interview isn't about the agent, it's about your judgment.

Build something small with a real trigger and a real destination. Don't build a chatbot. Build something that runs on its own and produces output someone would actually use — even if that someone is only you.

Then be ready for the five questions that separate people who built something from people who followed a tutorial:

"What did it get wrong?" Have a real answer. Failure modes you found and how you handled them. Anyone who says it worked perfectly is telling you they never tested it.

"How do you know it works?" Show a test set, even a small one. Thirty cases with expected outputs makes you unusual.

"What did you deliberately not automate?" This is the judgment question. Naming the DECIDE step you kept human — and why — demonstrates more seniority than any amount of building.

"What does it cost per run?" Knowing your unit economics signals you think in production terms.

"What would break at 100x?" Rate limits, cost, context size, the human review step becoming the bottleneck. You don't need to have solved it, just to have seen it.

A small agent you can discuss at this depth beats an impressive one you can only demo. Interviewers are trying to find out how you think when it goes wrong — so build something that went wrong, and fix it.

---

Part 6 — Agentic AI for Product Managers

From: "Product management." "Agentic PM." "AI for PMs." "Agentic ai for PMs." "How to improve Product Mgmt with AI" (×2). "How to leverage AI to develop strong, detailed Product Requirements." "Product building Experience."

Seven submissions. Product managers were the single largest identifiable group in the room, which tracks — the job is being reshaped faster than the tooling is settling.

What actually changes about the PM job when agents enter the product

Two distinct things are happening and people conflate them constantly. Separate them and the whole picture gets clearer.

Change one: agents as a tool you use. Faster research, faster drafts, faster analysis. Real, useful, and honestly the smaller change. It makes you a faster version of the PM you already are.

Change two: agents as a thing you ship. This is the one that rewrites the job description, because you're now shipping a product whose behavior is probabilistic. Four specific consequences:

Your spec can't enumerate behavior. You can't write "when the user clicks X, Y happens" for a system that decides. You specify goals, boundaries, and failure behavior instead. The artifact shifts from a flowchart to something closer to a policy.

Quality becomes a distribution. Not "does it work" but "how often, how badly does it fail, and what happens when it does." You need to be fluent in accuracy tradeoffs the way you're fluent in latency tradeoffs. A PM who can't reason about a precision/recall tradeoff is going to struggle.

You own the failure experience. What the product does when it's unsure is now a core design surface — arguably the core design surface. The difference between a trusted AI product and an abandoned one is almost never the happy path. It's whether it degrades honestly.

Eval design becomes a PM responsibility. Deciding what "correct" means is a product decision, not an engineering one. If you delegate the eval set entirely to engineering, you've delegated the definition of your product's quality. That's the biggest new responsibility on the list, and the most commonly abdicated.

The PMs pulling ahead right now aren't the ones using the most tools. They're the ones who got comfortable specifying a system that will be wrong sometimes.

Where AI genuinely improves PM output — and where it just makes more artifacts

Balanced answer, because the honest version is mixed.

Genuinely better:

Synthesis across many inputs. Fifty support tickets, thirty interview transcripts, six months of feedback — finding the themes. This is a reduction task with verifiable output, and it's the strongest use case in the PM toolkit by a distance.

Pre-mortems and edge cases. "Here's the feature — give me twenty ways this fails and the users who'd hate it." Models are excellent at enumerating possibilities, which is exactly where individual humans are weakest and where PM blind spots live.

Competitive and market research with sources. With citations you can check. Without citations you're generating confident fiction and putting it in a deck.

Turning a rough thought into a structured argument. You know what you think; you need it legible to six stakeholders by Thursday.

Just more artifacts:

Generating PRDs from a one-line prompt. You get a plausible document nobody has thought about. The document isn't the value — the thinking that produced it is, and you've just skipped it. (The PRD loop further down is the version that works.)

Prioritization scores. A model doesn't know your strategy, your politics, your tech debt, or what your CEO said in the hallway. It'll produce a confident RICE table that launders a guess into a number.

User personas from nothing. Personas built from imagination rather than research are exactly as useful as before — which is to say, not.

The test: does this reduce input I couldn't have read, or create output I could have written? Reduction is where the leverage is. Creation is where the busywork multiplies. A PM producing three times as many documents is not three times more effective; they're generating reading for everyone else.

Writing PRDs that an agent — and an engineer — can build from

The attendee asked how to use AI to develop strong, detailed product requirements. My answer is a bit contrarian: don't ask AI to write your PRD. Ask it to attack your PRD.

Here's the loop that actually produces better requirements.

Step 1 — You write the thin version yourself. The problem, who has it, what success looks like, what's out of scope. Half a page. Nobody else can do this part, because it's the part that requires knowing things.

Step 2 — Interrogation. "Act as the engineer who has to build this. List every question you'd need answered before you could start." You'll get twenty questions, and roughly twelve will be things you hadn't decided. That gap is your actual PRD work.

Step 3 — Answer them, then adversarial review. "Act as a skeptical staff engineer. What's underspecified, contradictory, or going to cause a rewrite in month three?"

Step 4 — Edge case generation. "What are the unusual inputs and states? What happens on empty, duplicate, expired, concurrent, or malicious input?" This is where models genuinely outperform an individual PM, because they don't get bored on item fifteen.

Step 5 — Round-trip test. New session, no context: "Read this PRD and describe what you'd build." If the description doesn't match your intent, your PRD is ambiguous — and you've found out for free instead of at demo time. This step alone is worth the whole loop.

Then, specifically for agent features, add four sections most PRD templates don't have:

• Definition of correct — the actual rubric, with examples of good and bad output
• Failure behavior — what it does when unsure, when a tool fails, when input is bad
• Escalation path — who gets it, with what context, how quickly
• Quality bar — the accuracy target and the measurement method

If those four are missing, engineering will invent them, and they'll invent them from a technical viewpoint rather than a user one. That's not their failure — it's an unspecified product decision landing on whoever's closest.

What building an AI product feels like compared to a deterministic one

Someone just wrote "Product building Experience," which I read as: what's it actually like? Four honest differences.

The first 70% arrives in a weekend and it's intoxicating. You'll demo something in three days that would have taken a quarter. This is real, and it's also a trap, because your timeline instincts get calibrated on the easiest part of the project.

Then progress becomes non-monotonic. You fix one case, another breaks. There's no compiler telling you what you damaged. This is genuinely disorienting for people used to deterministic systems — you can work for a week and be unsure whether the product got better. This is exactly why the eval set exists: it's the only thing that converts "feels better" into "is better," and without it, teams demoralize.

Your users' mental model is unstable. They don't know what it can do, so they either under-ask or wildly over-ask, and both produce disappointment. Onboarding for AI products is mostly expectation-setting. Show the boundaries early and explicitly.

Trust is asymmetric and unforgiving. Nine great outputs build a little trust. One confidently wrong output destroys a lot. Deterministic software doesn't have this property; a bug is a bug. Here, a wrong answer feels like a betrayal, because the system sounded sure. Design accordingly: visible uncertainty, easy correction, and never let it sound confident when it isn't.

The teams that navigate this well treat the first demo as the start of the project rather than 70% of it. The teams that struggle promised a date based on the weekend.

---

Part 7 — Coding with Agents

From: "Vibe codeing." "Vibe coding." "How to leverage Agentic tools beyond simple data analysis, competitive research, and vibe coding." "I am looking to automate some coding workflows for my Indie startup to port my code to Unity."

Vibe coding: what it's genuinely good at, and the exact point it breaks

Two people wrote it, one with a typo that I found charming. Let's be precise about it, because "vibe coding" is doing a lot of work as a term.

Vibe coding is describing what you want and accepting code without fully reading it. That's the definition, and the "without fully reading it" is the load-bearing part.

Where it's genuinely excellent:

Greenfield and disposable. Prototypes, internal tools, scripts, one-off analyses. Nothing depends on it, and if it's wrong you find out immediately.

Unfamiliar territory. A language or framework you don't know. You'd have spent four hours on Stack Overflow anyway, and you'll learn faster by reading working code than by reading docs.

The boring 80%. Boilerplate, CRUD, config, test scaffolding, glue. Code that's tedious rather than hard.

Exploring a solution shape. Three rough versions of an approach to see which feels right, then write the real one deliberately.

The exact point it breaks — and it's a sharp line, not a gradient: the moment code you didn't read becomes code you depend on.

Concretely, four failure modes:

Plausible-but-wrong. It compiles, it runs, and it's subtly incorrect in a case you didn't test. Deterministic bugs announce themselves. These don't.

Architectural drift. Each generation is locally reasonable. Twenty of them and you have four state management patterns and no coherent design, because nothing held a view of the whole.

Debugging debt comes due at the worst time. You can't debug what you never understood. It works until it doesn't, and then you're reading 2,000 lines of unfamiliar code under time pressure.

Silent security holes. Auth, input handling, secrets, injection. Generated code is often functionally right and defensively naive.

The professional version isn't abstinence — it's vibe-code the shape, deliberately review the seams. Read every line that touches data, auth, money, or state. Skim the rest. And keep the tests hand-written, because tests are how you verify the code you didn't read.

Past the starter tasks — three harder classes of work

An attendee asked, pointedly, how to use agentic tools beyond data analysis, competitive research, and vibe coding. They've noticed those three are where everyone stops. Here's what's on the other side.

Class one: continuous work rather than requested work. Everything in the starter set is request-response — you ask, it answers. The next tier runs continuously against a changing world. An agent that watches your error logs and opens a ticket with a hypothesis and a suspected commit. An agent that reads every incoming contract and flags deviations from your standard terms. An agent that reviews every PR against your conventions before a human sees it. Nobody asked; it noticed. (Part 5 is the architecture for this.)

Class two: work that spans systems and would otherwise need a meeting. The high-value tasks in most organizations are cross-system reconciliation. Does what sales promised match what the contract says match what engineering built? Does the roadmap match the actual commit activity? Do our docs match our API? These involve three sources, a judgment call, and enough tedium that nobody does them until something breaks. Agents are extremely good here, and the work is nearly invisible until you automate it.

Class three: work that was never done because it wasn't worth a person's time. This is the biggest category and the hardest to see, because it doesn't appear on any process map. Nobody reads every support ticket. Nobody checks every dependency's changelog. Nobody writes a post-mortem for small incidents. Nobody keeps the internal wiki accurate. These weren't skipped because they're worthless — they were skipped because the value never quite cleared the cost of a human hour. Agents move that threshold, and the space below the old threshold is enormous.

The mental move: stop asking "what do I do that AI could do?" That question caps you at your current process. Ask "what would we do if attention were 100x cheaper?" Everything on that second list is currently unbuilt.

Agent-assisted porting and migration — the Unity case, generalized

One attendee wants to port their indie startup's code to Unity and is experimenting to see if it's possible. Short answer: yes, and porting is genuinely one of the best-fit tasks for agents — but only if you build it as a pipeline rather than as a conversation.

Why it fits: the source of truth exists (your current code), correctness is mechanically checkable (it compiles, tests pass, behavior matches), and the work is high-volume and repetitive. That's the ideal profile from picking your first workflow.

Why the naive version fails: "port this file to Unity" file-by-file in a chat produces a thousand locally-plausible translations with no shared conventions, and you spend longer reconciling them than you'd have spent writing it.

The stages in words:

Port one thing by hand, properly. Not the easiest file — a representative one. You're not saving time here; you're producing the reference.

Write the mapping rules. Explicitly: this pattern becomes that pattern, this lifecycle maps to Awake/Start, coordinates convert like so, this dependency has no equivalent so we do X. Ten to thirty rules, extracted from the file you just did. This document is the project. Everything downstream is execution.

Batch by module, never by file. Give the agent the rules, the reference implementation, and one coherent module. Coherence is what prevents architectural drift.

Verify mechanically, always. Compiles, tests pass, behavior matches. For a game specifically, this is where you need to invest — write characterization tests against the original before you port, so you have something to diff against. That upfront cost is what makes the whole pipeline trustworthy.

Human reviews the seams. Not every line — the interfaces between modules, anything touching state, timing, or platform-specific behavior.

And the honest caveat: the parts that won't port are the parts that were never really about the code. Engine-specific behavior, timing, physics, feel. An agent will produce something that compiles and plays wrong, and no test will catch "the jump feels bad." Plan for those by hand, and use the agent to buy yourself the time to do them properly.

---

Part 8 — Enterprise and Business Adoption

From: "How can I use AI AGENTS more effectively in my company." "Able to understand capabilities and use cases in a corp." "How to leverage AI in business." "Simple ways for CEO to learn about AI" (×2). "How to build commercial AI agents."

From scattered individual usage to a real internal capability

The pattern in almost every company right now: forty people using AI privately, at wildly different levels, sharing nothing, and the organization capturing approximately none of it. Individual productivity gains that evaporate at the team boundary.

The fix is four layers, built in this order. Skipping to layer three is the most common and most expensive mistake.

Layer 1 — Access. Everyone has capable tools and explicit permission. The blocker is usually not budget; it's that nobody said out loud that it's allowed, so cautious people abstain and your usage skews toward the least cautious. Say it out loud, in writing.

Layer 2 — Practice. The gap between your best and median user is enormous and almost entirely invisible. Make it visible: a channel where people post what worked, with the actual prompt. A monthly 30-minute internal show-and-tell. This is the cheapest, highest-return intervention available to most companies and it costs one recurring calendar invite.

Layer 3 — Systems. Now shared infrastructure: a prompt and pattern library, approved tools with real data access, an eval harness people can reuse. This is where individual gains start compounding into organizational ones.

Layer 4 — Embedded. Agents inside actual workflows, with owners, metrics, and budgets. This is where the real numbers are — but attempting it before layers 1–3 gives you a project nobody knows how to run.

The common failure: a company jumps to layer 4 with a flagship AI initiative, no internal practice, no eval capability, and no one who's built anything. It launches, it underwhelms, and the organization concludes AI doesn't work for them. Build the base.

The corporate use-case map: which functions, which order

An attendee wanted to understand capabilities and use cases in a corporate setting. Here's the map, ordered by how quickly value shows up.

Start here — internal, high-volume, error-tolerant:

Support triage. Classify, route, draft a suggested response for an agent to approve. High volume, immediate measurement, human still in the loop.

Sales research. Pre-call briefings assembled from CRM, news, and the website. Saves 20 minutes per call and reps adopt it immediately, because it's obviously in their interest.

Document processing. Extraction from invoices, contracts, forms, reports. Mechanically checkable, high volume, and the baseline human accuracy is lower than people assume.

Internal knowledge search. "How do we handle X?" answered from your own policies with citations. Universally requested; needs your documents to actually be good, which is the hidden cost.

Then — cross-system and continuous:

Compliance and contract review against a standard playbook. Reconciliation between systems that should agree. Monitoring of metrics, competitors, and regulatory feeds with narrated anomalies. Reporting assembled and drafted, human-edited.

Later, carefully — customer-facing and consequential:

Autonomous customer communication. Only after a long shadow period and a demonstrated miss rate. Anything that moves money or makes commitments. Anything with regulatory exposure.

The ordering principle: error tolerance descending, volume descending. Earn the right to the consequential work with a track record on the safe work. Every organization that inverted this order has a story about it.

Where the margin actually shows up in the business case

"How to leverage AI in business" is broad, so let me answer the question underneath it: where does this become money?

Cost per unit of work. The obvious one and the most oversold. Real, but be careful — headcount rarely reduces, work absorption increases. Frame it as capacity, not savings, or you'll be held to a number you won't hit.

Cycle time. Often the bigger lever, and consistently underrated. Quote in an hour instead of three days. Support resolved in ten minutes instead of a day. Speed changes win rates and retention in ways that don't show up in a cost model but absolutely show up in revenue.

Work that wasn't happening at all. The largest pool by far. Not doing existing work cheaper — doing valuable work that never cleared the cost threshold. Every customer getting a personal follow-up. Every contract actually being read. This is where the durable advantage is, because it changes what your company does, not just what it spends.

Quality variance reduction. Your best salesperson's briefing quality, available to everyone. Your best support answer, consistently. Raising the floor is usually worth more than raising the ceiling, and it's much easier to measure.

Where the margin does not show up: an AI feature bolted onto your product because competitors have one. Customers don't pay for AI. They pay for a job done better, and if the job isn't better they'll notice within a week.

The honest CFO framing: most of the value in year one is cycle time and capacity, not cost reduction. Promise cost reduction and you'll be explaining yourself at the next board meeting.

The CEO's version: 30 days, no jargon

Two people asked for simple ways for a CEO to learn about AI. Here's the version I'd actually give, and it involves less reading than expected.

Week one — use it yourself, badly, for real work. Not a demo. Take something on your actual plate — a board deck outline, a competitor question, a long document — and do it with AI. Thirty minutes a day. You cannot lead this from a briefing; the intuition only comes from the friction.

Week two — find your own best users. Ask internally: who's using this well? Have three of them show you what they do for twenty minutes each. You'll learn more than from any consultant, you'll find out how far ahead of you your company already is, and you'll signal that this is safe to talk about.

Week three — ask five questions of your leaders. What repetitive work does your team do most? Where does work sit waiting? What do we not do because we lack the hours? Where would a mistake be cheap? What data do we have that we don't use? The answers are your use-case pipeline, in the words of the people who own the work.

Week four — fund one small thing, and one habit. One narrow project with a named owner and a 90-day measurable outcome. Plus a standing 30-minute monthly session where people show what they built. The project might fail. The habit compounds.

Three things worth knowing plainly, no jargon:

It's confident when it's wrong. Every process needs a check where being wrong is expensive.

It doesn't know your business unless you connect it to your systems. Most of the work is that connection, not the AI.

The bottleneck is your processes, not the technology. If nobody can say what "correct" means for a task, no tool will fix it.

The CEO-specific trap: over-investing in the big platform initiative and under-investing in the boring internal habit. The habit is what produces the people who'll run the initiative.

Building commercial agents — pricing, unit economics, and the moat

For the attendee asking how to build commercial AI agents — meaning agents you sell — this is a different game from internal tooling in three specific ways.

Your costs are variable and usage-scaled. Traditional software has near-zero marginal cost. Agents don't: every run costs real money, and heavy users can be unprofitable. Know your cost per run before you price. Seat-based pricing on a variable-cost product is how you end up with a gross margin problem that grows with your success.

Pricing follows verifiability. Three models, and the right one depends on how checkable your output is. Seats — simple, familiar, breaks under heavy usage. Usage — aligns cost with revenue, but customers hate unpredictable bills. Outcome-based — the most defensible and the hardest: you charge per resolved ticket, per processed invoice. Only works when the outcome is unambiguously measurable, and when it is, it's a very strong position because you're selling results rather than software.

The moat is not the model. Everyone has the same models. Four things are actually defensible:

Workflow depth. Being genuinely integrated into how work gets done, with the ugly edge cases handled. Boring, hard to copy, and the most common real moat.

Proprietary evaluation. Your accumulated test set of real cases with known-correct answers. A competitor can copy your prompts in a day; they can't copy three years of production failures you turned into eval cases.

Data and feedback loops. Every correction makes the product better for everyone, which is only a moat if you actually engineered the loop.

Trust and compliance posture. In regulated industries, the audit trail and the certifications are the product as much as the agent is.

Two practical warnings. Beware the demo-to-deployment gap — enterprise buyers love the demo and then spend nine months on security review; price and plan for that cycle. And beware being a thin wrapper on a capability the platform will absorb — if your entire product is one prompt and a nice UI, ask what remains when that becomes a checkbox in a tool your customer already pays for. The answer should be workflow depth, data, or trust. If it's "we were first," that's not a moat, that's a head start.

---

Part 9 — Tools and the Ecosystem

From: "Tools" (×2). "Learn about agentic ai tools." "Google" (×2). "AI resources and use cases in my domain."

How to choose tools once instead of re-choosing every quarter

Three people just wrote "Tools." I understand the impulse — but I'm going to resist giving you a list, because a list of tools is the fastest-expiring content in this field, and you'd be back here in four months.

Instead: the stack has five slots. Learn the slots, and you can evaluate anything.

Four rules that have aged well:

Optimize for the layers with high switching costs. Your model choice barely matters — you'll change it three times and it'll take an afternoon each time. Your integration and retrieval decisions are where you'll actually be living. Spend your evaluation effort there, not on framework comparisons.

Choose boring for orchestration. Frameworks are the most churn-prone layer and the easiest to write yourself. An agent loop is genuinely about 200 lines. If a framework saves you a week but obscures what's happening when it misbehaves, it's a bad trade — debuggability is worth more than convenience in a probabilistic system.

Never let a tool own your eval data. Your test set is your most durable asset (see the guardrails answer). Keep it in your own repo, in a plain format, portable to anything.

Prefer the thing your team already runs. An adequate tool your infra team supports beats an excellent one nobody can operate at 3am.

And for keeping current: track the categories, not the products. When a new tool appears, ask which slot it's in and what it does better in that slot. That question survives every release cycle.

Where Google fits in the agent stack

Two people just wrote "Google," which — fair enough, it's a workshop form. I'll read it the most useful way: what's Google's position in agentic AI, and does it change your architecture?

The short version: Google is making a full-stack bet — frontier models, an enterprise agent platform on Google Cloud, deep integration with Workspace, and a strong push on agent interoperability standards so agents from different vendors and organizations can talk to each other. That last piece is the strategically interesting one and it connects directly back to the multi-agent answer above.

What this means for your architecture, in three points:

If your data already lives in Google Workspace or Google Cloud, the integration argument is strong. The high-switching-cost layers in the stack above — tools and retrieval — are exactly where being native to your existing data estate pays off. That's a better reason to pick a vendor than model benchmarks.

Interoperability standards matter more than they look. A2A-style protocols are aimed at agents crossing organizational boundaries — your agent negotiating with a supplier's agent. Nobody needs this for internal architecture today. It's likely to matter a lot in two years. Track it; don't build your internal design around it now.

Don't pick a vendor on capability. The frontier labs leapfrog each other continuously, and any capability lead has a short shelf life. Pick on data gravity, operational fit, procurement reality, and compliance posture. Then keep your model layer swappable, as per the tool-stack rule above — that's the hedge that makes the decision reversible.

The general principle, which outlives any specific vendor: the model layer is a market, and you're a customer in it, not a partner to one company. Architect so that being wrong about a vendor is an afternoon's work, not a rewrite.

Finding domain resources that aren't recycled listicles

An attendee asked about AI resources and use cases in their specific domain. Three filters that work regardless of the domain:

Look for people posting failures. Anyone writing about what didn't work, with specifics, is doing the actual work. Success posts are marketing; post-mortems are engineering.

Prefer practitioner communities over content sites. The useful material in most domains is in a Slack, a Discord, a subreddit, or a small mailing list — not on the first page of search results. Find the three people in your field who are actually building and read what they read.

Read adjacent domains and translate. Your industry's AI content is probably thin. But the underlying patterns — extraction, reconciliation, triage, monitoring — are identical across industries. Legal contract review and medical coding are the same shape. Steal the architecture from a mature domain and port the vocabulary.

---

Part 10 — Verticals: Agents in the Wild

From: "I have been constructing something good to help health care professionals and doctors." "Build AI agents for YouTubers." "Building a Startup." "How others are using Agentic AI for different industries and products."

Healthcare agents: the regulated-domain pattern

One attendee is building for healthcare professionals and doctors, which is one of the highest-value and highest-difficulty domains there is. Everything in Part 4 applies with the volume turned up, plus four domain-specific realities.

The regulatory line is about claims, not technology. What determines your compliance burden is whether your output is framed as clinical decision-making or as information and workflow support. Summarizing a chart, drafting documentation, surfacing relevant literature, reducing administrative load — very different regulatory posture than suggesting a diagnosis. Most successful healthcare AI products are deliberately positioned on the administrative side of that line, and that's a product decision made on day one, not a legal review done at the end. Get an actual regulatory expert early; this is not a place for inference from blog posts.

The buyer, the user, and the beneficiary are three different people. The hospital buys, the clinician uses, the patient benefits. Your product has to satisfy all three and they have conflicting priorities. Products that only delight the clinician don't get purchased; products that only satisfy procurement don't get used.

Time is the currency, and the bar is brutal. Clinicians are measured in minutes. A tool that saves ten minutes of documentation per day is transformative. A tool that adds one click to an existing workflow is dead, regardless of how good it is. Integration into the existing system is not a nice-to-have — it's the entire product.

Verification must be structural. In a domain where confident errors have real consequences, uncertainty has to be visible, sources have to be citable, and the human has to be able to check in seconds rather than minutes. "Here's the summary, and here's the line in the chart it came from" is the pattern. Never present a synthesis the clinician can't trace.

The highest-value, lowest-risk starting point in healthcare is nearly always administrative burden — documentation, prior authorization, coding, referral letters, inbox triage. It's where the pain is, where the volume is, where the error tolerance is highest, and where nobody has to trust an algorithm with a clinical judgment to get value.

Creator-economy agents — the YouTuber case

Someone wants to build agents for YouTubers. Good market: high volume of repetitive work, individual operators with no staff, and clear willingness to pay for time. Here's where the value actually sits.

Not in content generation. Creators' entire competitive position is their voice. An agent that writes their script is attacking the one thing they can't outsource. Products in this space that lead with generation tend to disappoint, because the output is generically fine and their audience notices.

Yes in everything around the content. The unglamorous surround is where creators drown: comment triage and surfacing the ones worth answering, repurposing one video into ten platform-appropriate clips with captions, sponsor research and outreach admin, competitive monitoring, thumbnail and title A/B analysis against actual performance data, and turning analytics into "here's what changed and probably why."

The killer feature is the standing brief. The overnight-agent pattern maps perfectly here: every morning, what happened on your channel, what's different, which three comments deserve a reply, what your peers published. Creators are running a business with no analyst. Be the analyst.

Business model reality: individual creators are price-sensitive and churn hard. The economics work better for creators-as-businesses — those with an editor, a manager, a schedule. Same product, very different willingness to pay for time saved.

Starting an agent company right now

For the attendee building a startup — the field is crowded at the surface and empty underneath. Four things I'd hold onto.

Depth beats breadth, decisively. "An agent for X" where X is broad is competing with every platform's default feature. "An agent that handles the specific reconciliation nightmare that mid-size logistics brokers face every Tuesday" is a business. Narrow enough that you can be complete — handling the ugly edge cases is the product, and it's what a horizontal tool will never do.

Sell the outcome, not the agent. Customers do not want an agent. They want the invoices processed, the tickets resolved, the compliance check passed. Price and position on the outcome (see the commercial-agents answer), and it also protects you from the model-commoditization problem, because you're not selling access to a capability anyone can buy.

Your unfair advantage is domain knowledge, not AI skill. AI skill is now widely distributed and getting more so. Knowing exactly how claims adjusters actually work, including the informal parts, is rare and slow to acquire. If you don't have that, get a co-founder who does — it's a harder gap to close than the technical one.

Build the eval set as a company asset from week one. Every customer correction, every failure, captured as a test case. In two years, that's the thing a competitor can't replicate. It's also the artifact that will make your product visibly better than a well-funded newcomer's, which is the position you want to be in.

The most common startup failure mode here: building a genuinely impressive demo for a problem nobody has budget for. Talk to twenty potential customers about what they currently pay people to do. Budget lines reveal real pain; enthusiasm doesn't.

Cross-industry patterns worth stealing

One attendee wanted to know how others are using agentic AI across different industries and products. The useful finding, having looked at a lot of these: there are about six patterns, wearing different costumes.

Extract-and-structure. Unstructured input to structured data. Insurance claims, medical records, invoices, resumes, legal documents, research papers. Same architecture every time.

Triage-and-route. Classify incoming work, prioritize, send to the right place with context attached. Support tickets, security alerts, applications, leads, patient messages.

Reconcile-and-flag. Compare two sources that should agree, report the differences. Finance, compliance, inventory, documentation-versus-code, contract-versus-delivery.

Monitor-and-narrate. Watch a stream, report what meaningfully changed and why. Metrics, competitors, regulations, infrastructure, channel performance.

Draft-from-template. Produce a structured document from context and a known shape. Reports, letters, proposals, specs, briefings.

Research-and-brief. Gather from many sources, synthesize with citations, deliver as a short answer. Sales prep, due diligence, literature review, market analysis.

That's most of it. The implication is practical and freeing: when you're stuck, find a mature implementation of your pattern in a completely different industry and copy its architecture. The domain vocabulary changes. The components don't. A logistics reconciliation agent and a clinical coding audit agent are the same system with different nouns — and the logistics team has probably already solved the failure mode you're staring at.

---

Part 11 — Learning From the Room

From: the AI student spending the summer at Stanford. "Share & learn latest trends in agentic ai." "Learn about how others are solving the agentic AI problem." "How folks use agentic AI differently." And "Abby."

For the student: practical building, industry reality, and how to actually meet people

One attendee wrote the most complete answer we got: an AI student at Stanford for the summer, wanting practical approaches to building agentic systems, an understanding of current industry practice, and to meet people working on AI products. Three parts, three answers.

On practical building: the gap between academic and industry AI work is not sophistication — it's that industry spends 80% of its time on things that don't appear in papers. Data access, integration, evaluation on messy real cases, cost, latency, and what happens when it fails. If you want to be immediately valuable, get good at evaluation. It's the least glamorous and most in-demand skill in the field right now, because almost nobody arrives knowing how to do it and every serious team needs it. Build one agent end to end, with a real trigger, a real test set, and real failures you fixed. One of those beats five course projects.

On industry practice: the single most useful thing to internalize is how conservative good production practice is compared to the discourse. Small models where they suffice, deterministic code wherever possible, humans on the consequential steps, extensive logging, boring infrastructure. The teams doing the most impressive work are usually running the least exotic architectures. If that sounds anticlimactic — that's the insight.

On meeting people: the reliable move is to arrive with an artifact, not a question. "Can I pick your brain about agents?" gets a polite decline. "I built this thing, here's what broke, here's my theory about why — does that match what you see?" gets a real conversation, because you've given them something interesting rather than asked for a favor. Everything follows from that: build small, publish what you learned, and specifically publish your failures. The AI community rewards visible learning-in-public more than almost any other field, and you're in the right city for it. Go to the small events, not the big ones.

Keeping up without drowning

Someone wanted to share and learn the latest trends. The honest advice is mostly about what to ignore.

Follow capabilities, not releases. New model versions arrive constantly. What matters is whether a class of thing became possible — reliable long-horizon tasks, cheap reasoning, native computer use. Those shifts happen a few times a year and are worth restructuring around. The rest is patch notes.

Give it a weekly slot and a hard stop. Two hours, once a week. Continuous monitoring feels productive and mostly generates anxiety and a sense that you're behind. You're not behind; the field is only about three years old in its current form and nobody is more than eighteen months ahead of anyone.

Prefer primary sources and practitioners. Documentation, technical blogs, and people posting real failure detail. Skip anything with "everything changed" in the headline.

Build something every quarter. Reading about agents produces the illusion of understanding with remarkable efficiency. One small build teaches more than fifty articles, and it also tells you which of the fifty articles were nonsense.

Where teams are converging — and where they're genuinely diverging

Two attendees asked how others are solving this and how people use it differently. Having looked at a lot of implementations, there's a clear split.

Converging on:

Tools over fine-tuning for domain knowledge. This argument is largely over. Structured outputs and validation as standard. Eval sets as infrastructure — the teams shipping seriously all have one. Staged rollout through shadow and assist modes. Humans on consequential steps — near-universal, and mostly learned the hard way.

Still genuinely contested:

How much orchestration to buy versus build. Reasonable, experienced people strongly disagree. Single agent versus multi-agent — see the multi-agent answer; the field hasn't settled. How much to trust model self-assessment. Some teams weight confidence scores heavily, others treat them as noise, and both have evidence. Where the human checkpoint goes — early gating versus late review is a live argument with real tradeoffs. How much to invest in prompt engineering now that models are more forgiving.

And the widest divergence isn't technical at all — it's organizational placement. Some companies run a central AI team that builds for everyone. Some embed engineers in each function. Some just give everyone tools and let it emerge. These produce wildly different outcomes and nobody has a definitive answer, because it depends on your existing culture more than on anything about AI.

The reassuring read: if you feel unsure about the contested list, you're not behind. You're accurately perceiving an unsettled field. The unreassuring read: the converged list is now table stakes, and if you're not doing those five, that's the gap.

"Abby"

One person, asked what they wanted to learn from an advanced agentic AI workshop, wrote: Abby.

I have no idea. It might be a name, a typo, an autocomplete accident, or the world's most confident one-word answer.

I've kept it in because it's a decent metaphor for working with these systems. You will regularly get an output that is confident, well-formed, and completely uninterpretable without context you don't have. The correct response is not to guess and build on it. The correct response is to ask.

So — Abby, if you're reading this: come find me at the next one. Everyone else: this is your reminder that "I don't know what you mean, can you say more?" is a valid and underused response, for humans and agents alike.

---

What the Answers Actually Told Us

Three postures showed up in that room. Understand it. Ship it. Govern it.

The pattern worth noticing is that they're not stages of maturity — they're the same problem viewed from three distances. The person asking for "Basics" and the person asking about judge-layer calibration metrics on legacy enterprise code are asking the same question: how do I know this thing is doing what I think it's doing? One is asking about tokens. One is asking about evals. It's the same question.

If I had one sentence for each group:

To the ones who want to understand it: the model is the least important part — learn the loop around it, and you'll understand every product you see for the next five years.

To the ones who want to ship it: your test set is the product; build it before you build anything else, and everything downstream gets easier.

To the ones who want to govern it: write down what "correct" means, attach a human name to it, and you'll have done more than most governance programs achieve in a year.

And to everyone: the thing separating people who get value from this from people who don't is not technical skill. It's the willingness to pick one real, boring, repetitive task and follow it all the way through to something that runs without you.

Pick one. Start tonight. It'll be broken by Thursday, and that's when you'll actually start learning.

---

Have a question that didn't get answered here, or think one of these answers is wrong? That's the more interesting conversation — bring it to the next session.