Generative AI Interview Questions: Name the Trade-Off

Generative AI interview questions are not usually a test of how many model names you can recite. They are a test of how you reason when two good goals pull against each other: quality against latency, control against flexibility, freshness against simplicity. If you can explain a concept in plain English and then say what it costs you, you can answer most of what gets asked — and stay calm when the follow-ups arrive.

This guide covers the concepts worth being able to explain simply, the trade-offs interviewers expect you to name, a reusable answer shape, and how to handle the second and third question without freezing. It sits alongside the same trade-off reasoning used in other technical interviews, such as the approach in Python Interview Questions: Concepts and Trade-Offs.

Why generative AI interview questions test trade-offs, not definitions

An interviewer who asks "what is a transformer?" is not collecting vocabulary. They are opening a door. The useful signal comes next, when they ask what you would change if the answer had to be faster, cheaper, safer, or more current.

Almost every question in this space is really asking: what did you give up to get this behaviour? A system that answers more questions confidently also makes more mistakes. A system that never guesses is safer and less useful. Naming that tension out loud is the skill being measured.

A simple habit makes this easy. After any definition, add one sentence about what it costs or constrains. "Attention lets the model weigh earlier tokens against each other; the cost is that longer inputs get expensive, so context length becomes a budget decision." You have now said something a definition alone cannot.

Set expectations honestly for yourself. You are not expected to have read every paper. You are expected to reason from principles you do know, and to be clear about where your knowledge stops. That is a stronger position than bluffing, and it is far easier to defend under follow-up questioning.

The core concepts worth being able to explain simply

You do not need deep mathematics for most generative AI interview questions. You need a clean mental model and one honest limitation per idea.

Tokens, embeddings and sequence length

Tokens are the chunks of text a model reads and writes. Embeddings turn those chunks into vectors so that similar meanings sit close together. Sequence length matters because longer inputs mean more computation, which shows up as cost and latency.

The non-obvious part is that tokenisation is not the same as words. A rare word, a code snippet or a string of digits can split into several tokens, so two inputs that look the same length on screen can cost very different amounts. Plain version: "Tokens are how the model counts text, and length is a budget I have to manage — and the count is not the same as the word count."

Attention and context windows

Attention lets the model weigh earlier tokens when producing the next one. The context window is how much it can hold at once. What that buys you is coherence across a long document or conversation; what it costs is money and time per call.

The detail worth knowing: fitting inside the window is not the same as being used well. Some evaluations suggest models use information near the start or end of a long context more reliably than material buried in the middle, though this varies by model and task and is not a settled rule. That is one reason teams test rather than assume, and why retrieval or summarising often beats simply extending the window — "just put everything in the prompt" is not a free fix.

Pretraining, fine-tuning and prompting

These are three different levers. Pretraining builds general capability and is out of reach for most teams. Fine-tuning adjusts a model toward your data or style, which buys consistency and control but costs data preparation, compute and re-tuning when things change. Prompting is fast and flexible but less reliable for strict formats.

The concrete failure to be able to describe: if you need output that always matches an exact schema, prompting alone tends to drift — a field gets renamed, a value comes back as a string instead of a number, or an extra sentence appears around the JSON. Fine-tuning on many correct examples usually makes the format more reliable, though teams often pair it with output validation or constrained decoding. It also freezes behaviour, so a change in requirements means collecting new examples and tuning again. A good answer says which lever you would reach for and why.

Decoding choices

Temperature, top-k and top-p shape how the model picks the next token. Lower temperature gives more repeatable output; higher temperature gives more variety and more risk.

The subtlety is that these settings interact rather than stack neatly. Top-k and top-p both cut down the candidate set before sampling, so in many setups the more restrictive one ends up doing most of the cutting — though exactly how they combine depends on the implementation and the values you choose. Tuning one while ignoring the other can give less change than expected. And a low temperature does not guarantee a correct answer, only a more repeatable one. If a task needs a stable, checkable answer, say so, and say what you lose in creativity.

Retrieval

Retrieval fetches relevant passages from your own data and puts them in the prompt, so the model grounds its answer in sources rather than relying only on what it memorised during training. It buys freshness and attribution. It costs you a retrieval pipeline to build, tune and monitor.

The part candidates often miss: retrieval quality is a separate problem from model quality. If the right passage is never fetched, no amount of prompting will recover it, and a confident-sounding answer built on the wrong passage is worse than an obvious "I could not find that." So the honest limitation is not just "more moving parts" — it is that you now have two systems to evaluate, and the weaker one sets the ceiling.

For each concept, prepare a one-line plain-English explanation plus one limitation. Two or three sentences each is enough. If you want a broader set of technical examples to practise with, our blog has related material on concept-and-trade-off interviews.

A reusable answer shape for generative AI interview answers

Strong generative AI interview answers tend to follow the same five-part shape. It works for conceptual questions and for "have you used X" questions alike.

Name the concept in one sentence. "Retrieval means fetching relevant documents and putting them in the prompt."
Say what problem it solves and where. "It helps when answers must reflect your own, changing data rather than the model's training snapshot."
State the trade-off. "The gain is freshness and traceability; the cost is an extra pipeline and more moving parts."
Give a concrete example. From a project, a course, or a small side build. Even a modest example beats a hypothetical one, as long as it is true.
Close with what you would check or measure. "Before choosing it, I would check retrieval quality and whether answers actually cite the right passage."

Here is the shape in full, spoken aloud in about forty seconds:

"Fine-tuning adjusts a model toward your data or style. It helps when you need consistent formatting or domain phrasing that prompting cannot hold reliably. The trade-off is control and consistency versus the time and data it takes to prepare, plus re-tuning when your data changes. I used it on a small classification side project where the output format had to be stable. Before choosing it again, I would measure whether prompting with a few good examples already gets close enough."

Notice the last sentence. It shows you think in terms of evidence, not enthusiasm.

Trade-offs interviewers expect you to name

Many generative AI interview questions are really an invitation to name one of these tensions. Practise saying each in one sentence, then the situation where you would pick each side.

Quality versus latency. Bigger models and longer contexts usually cost more time and money. Pick quality when the answer is the product; pick speed when it is a step inside a larger flow.
Fine-tuning versus prompting. Control and consistency versus flexibility and speed of iteration. Pick fine-tuning when format or tone must hold; pick prompting when requirements are still moving.
Retrieval versus long context. Freshness and attribution versus the simplicity of a single call. Pick retrieval when data changes or sources must be shown.
Hallucination versus coverage. A system that refuses more often is safer but less useful. Pick the cautious side when errors are costly; accept more coverage when a human reviews output.
Cost versus evaluation depth. How much testing you can afford before shipping. More evaluation buys confidence and costs time.
Privacy and data handling. What can go to a hosted model versus what must stay in-house. This is a constraints question first and a modelling question second.

A useful drill: take one trade-off per day and say both sides out loud in under thirty seconds. You are building the reflex to find the axis of the trade-off before you start talking.

Handling generative AI follow-up questions without freezing

Generative AI follow-up questions are normal, not a sign you failed. Expect the second and third question to probe the edge of your answer. That is the interviewer doing their job.

If you do not know, say what you do know and how you would find out. Reasoning aloud is the signal. "I have not tuned that myself. I would start by checking whether prompting with examples gets close, then measure before spending on fine-tuning."
Pause briefly to pick the axis. Before you speak, decide whether the question is about quality, cost, latency, control or freshness. Then answer along that axis.
Ask a clarifying question when scope is ambiguous. "Are we optimising for cost or for answer quality here?" is a good question, not a weakness.
Keep answers to a minute or two, then invite the follow-up. Stopping gives the interviewer room to steer, and it stops you from filling silence with guesses.
Stay inside your real experience. If you have only read about something, say that, and describe what you would test first.

Common traps in generative AI technical interviews

Reciting architecture names without being able to say what they are for.
Claiming hands-on experience you cannot describe in specifics. Describe what you actually built or studied instead.
Ignoring evaluation. Interviewers like candidates who ask how you would know the output improved.
Treating "it depends" as a complete answer. Always say what it depends on.
Overloading an answer with jargon when a plain explanation would land better.

Each trap has the same fix: replace the label with a purpose, and the claim with a specific. Generative AI interview questions reward the candidate who can be plain and precise at the same time.

How to practise these answers honestly

Generic advice — say it out loud, record yourself — helps a little. What moves the needle for generative AI interview questions is drilling the specific moves this subject rewards.

The axis-first drill. Keep the six trade-offs above in front of you. Read a question, then say only the axis before you answer: "this is a cost-versus-quality question." Do twenty of these without answering. You are training the pause that stops you rambling.
The limitation drill. For each concept, say the plain explanation, then force yourself to add a limitation you have not used before. If you keep reaching for the same one, you do not know the concept as well as you think.
The two-system drill. For any retrieval answer, describe how you would evaluate the retriever and the generator separately. This is the follow-up most candidates are unprepared for.
The format drill. Explain out loud what specifically breaks when you ask a model for a strict output format without fine-tuning, and what you would do about it.
The honesty drill. Take one thing you have only read about and say it that way, then describe what you would test first. Practise the sentence until it sounds natural rather than apologetic.
The measurement drill. End every practice answer with one sentence on what you would measure. If you cannot name a measure, your answer is not finished.

Rehearse out loud and review how your answers land, keeping everything you say accurate to your real experience. If you have a question about your practice, our support page is the place to ask. Then repeat with a friend or a mock session and ask them to interrupt with follow-ups.

A short pre-interview checklist

Before you walk in, make sure you can handle the most common generative AI interview questions in your own words.

Pick five concepts you can explain simply, and name one limitation for each — ideally one that is not the obvious one.
Prepare two honest project stories where a generative AI choice involved a trade-off.
Have one sentence ready for "what would you measure?" on any answer you give.
Decide how you will handle a question you cannot answer: what you know, and what you would check.
Review the basics of adjacent topics so follow-ups do not catch you out.

The takeaway is simple: definitions get you started, trade-offs carry the conversation. If you can explain a concept plainly and say what it costs, you can handle the follow-ups calmly — and you will sound like someone who has actually thought about the work.

Frequently asked questions

Do I need to know how transformers work to pass a generative AI interview?

Usually not at the level of mathematics. You should be able to explain what attention does in plain English, why context length is a budget, and what that costs in latency and money. If a question goes deeper than you can go, say what you do know and how you would find out.

How deep should I go on fine-tuning versus prompting?

Enough to say which lever you would reach for and why. Prompting is fast to iterate and flexible but less reliable for strict formats; fine-tuning buys consistency and control but costs data preparation, compute and re-tuning when requirements change. One sentence on each side, plus the situation where you would pick it, is usually enough.

What if I have never shipped a generative AI feature in production?

Say so plainly, then show how you reason. Course work, side projects and small experiments count as long as you describe them accurately. Interviewers are often testing your judgement about trade-offs and evaluation, not the size of your deployment history.

How do I answer 'how would you reduce hallucinations' without overclaiming?

Describe approaches rather than guarantees: grounding answers in retrieved sources, asking the system to say when it cannot find support, and evaluating outputs against a set of known cases. Be honest that these reduce risk rather than eliminate it, and say what you would measure to know whether it improved.

Should I mention specific models or tools by name in my answers?

Mention them when they are genuinely part of your experience, and always explain why you chose them. Naming a tool without the reasoning adds little. Explaining the trade-off you were weighing is what the interviewer is listening for.

How long should a generative AI interview answer be?

A minute or two is usually right for a conceptual question. Long enough to name the concept, the problem it solves, the trade-off and one example; short enough to invite a follow-up. If the interviewer wants more depth, they will ask.

Rehearse these answers out loud and review how they land.