interviewDeck

Your one-stop platform to prepare, practice and ace your interviews.

Loading your questions…

interviewDeck

All Questions

Filters & tools

LLMs & GenAI Interview Questions and Answers

26 hand-picked LLMs & GenAI interview questions with detailed answers. Open the interactive version above to search, filter by difficulty, run code, bookmark questions and track your progress.

What is a Large Language Model (LLM)?

An LLM is a neural network trained on massive text corpora to predict the next token. By scaling model size, data, and compute, these models learn grammar, facts, reasoning patterns, and instruction-following — enabling them to generate, summarise, translate, and answer questions.

Examples: GPT-4, Claude, Gemini, Llama, Mistral. They are the foundation of modern GenAI applications.

What is the Transformer architecture and why did it matter?

Introduced in Attention Is All You Need (2017). Key ideas:

  • Self-attention — each token attends to all others, capturing long-range dependencies without recurrence.
  • Parallelisation — unlike RNNs, all positions processed simultaneously → faster training on GPUs.
  • Encoder-decoder — original design; modern LLMs are mostly decoder-only (GPT-style) for text generation.

Transformers are the architecture behind virtually all current LLMs.

What are tokens and how does tokenization work?

LLMs don't read words — they process tokens, subword units from a fixed vocabulary (typically 32k–128k tokens). Tokenization splits text via algorithms like BPE (Byte Pair Encoding) or SentencePiece.

Why it matters: pricing is per token, context windows are token-limited, and token count ≠ word count ("ChatGPT" might be 1 token; "antidisestablishmentarianism" might be several).

import tiktoken
enc = tiktoken.encoding_for_model("gpt-4o")
len(enc.encode("Hello, how are you?"))  # token count

What is a context window?

The maximum number of tokens an LLM can process in a single request — includes both input (prompt + history) and output (completion). Examples: 128k (GPT-4o), 200k (Claude), 1M+ (Gemini).

Exceeding it causes truncation or errors. Long contexts increase latency and cost. Techniques like RAG avoid stuffing everything into the prompt.

Explain temperature, top-p, and top-k sampling.

LLMs output a probability distribution over the next token. Sampling parameters control randomness:

  • Temperature — scales logits before softmax. Low (0–0.3) = deterministic, focused. High (0.8–1.0) = creative, varied.
  • Top-k — sample only from the k most likely tokens.
  • Top-p (nucleus) — sample from the smallest set of tokens whose cumulative probability ≥ p (e.g. 0.9).

Use low temperature for code/facts; higher for creative writing.

response = client.chat.completions.create(
  model="gpt-4o",
  messages=[...],
  temperature=0.2,   # factual tasks
  top_p=0.9
)

What is prompt engineering?

The practice of crafting inputs to get reliable, high-quality outputs from an LLM without changing model weights. Techniques include:

  • Clear instructions and role assignment (system prompt)
  • Few-shot examples (show input→output pairs)
  • Chain-of-thought ("think step by step")
  • Structured output formats (JSON schema, XML tags)
  • Breaking complex tasks into steps
messages = [
  {"role": "system", "content": "You are a SQL expert. Return only valid SQL."},
  {"role": "user", "content": "List users who signed up last week"}
]

Zero-shot vs few-shot prompting.

Zero-shot — no examples in the prompt; the model relies entirely on pre-training and instructions. Works for common tasks the model has seen.

Few-shot — include 1–5 input→output examples in the prompt to demonstrate the desired pattern. Improves accuracy on niche formats, classification, or domain-specific tasks.

More examples help up to a point — then you hit context limits and diminishing returns.

# few-shot example
prompt = """
Classify sentiment:
Text: "Love it!" -> Positive
Text: "Terrible experience" -> Negative
Text: "It's okay I guess" -> """

What is chain-of-thought (CoT) prompting?

Asking the model to reason step by step before giving the final answer. Dramatically improves performance on math, logic, and multi-step reasoning tasks.

Variants: zero-shot CoT ("Let's think step by step"), few-shot CoT (include worked examples with reasoning), and native reasoning models (o1, DeepSeek-R1) that internalise this process.

prompt = "A store has 23 apples. They sell 17 and receive 8 more. How many now? Think step by step."

System, user, and assistant message roles.

Chat APIs structure conversations as a list of messages with roles:

  • system — sets behaviour, persona, constraints. Usually one per conversation.
  • user — the human's input.
  • assistant — the model's previous responses (conversation history).
  • tool — function call results (in tool-use flows).

The model sees the full message history (within context limits) to maintain conversational coherence.

messages = [
  {"role": "system", "content": "You are a helpful coding tutor."},
  {"role": "user", "content": "Explain recursion"},
  {"role": "assistant", "content": "Recursion is when..."},
  {"role": "user", "content": "Give an example in Python"}
]

When do you use prompting vs fine-tuning vs RAG?

ApproachBest forTradeoff
PromptingGeneral tasks, quick iteration, changing requirementsLimited by context window; no private data unless in prompt
RAGQ&A over private/docs data, reducing hallucinationsRetrieval quality is the bottleneck; added latency
Fine-tuningConsistent style/format, domain language, classificationNeeds training data, compute, retraining on model updates

Often combined: RAG for knowledge + fine-tuning for behaviour + prompting for task instructions.

What are hallucinations and how do you reduce them?

A hallucination is when an LLM generates confident but factually incorrect or fabricated information. Causes: training data gaps, probabilistic generation, no grounding in real data.

Mitigations:

  • RAG — ground answers in retrieved documents
  • Citations — require the model to quote sources
  • Lower temperature — reduce creative guessing
  • Output validation — check against a knowledge base
  • "I don't know" instructions — allow the model to refuse

What is fine-tuning and what is LoRA?

Fine-tuning — continue training a pre-trained model on domain-specific data to adapt its behaviour. Full fine-tuning updates all weights (expensive).

LoRA (Low-Rank Adaptation) — a PEFT (Parameter-Efficient Fine-Tuning) method that trains small adapter matrices instead of the full model. ~1% of parameters updated → much less GPU memory and faster training. Adapters can be swapped per task.

What are embeddings and how are they used?

An embedding is a dense vector (e.g. 1536 floats) representing the semantic meaning of text. Similar meanings → vectors close together in vector space (measured by cosine similarity).

Uses: semantic search, RAG retrieval, clustering, recommendation, deduplication. Generated by embedding models (text-embedding-3-small, Cohere embed, open-source models like BGE).

response = client.embeddings.create(
  model="text-embedding-3-small",
  input="How do I reset my password?"
)
vector = response.data[0].embedding  # list of 1536 floats

What is RAG (Retrieval-Augmented Generation)?

RAG combines retrieval with generation:

  1. Index — chunk documents, embed chunks, store in a vector database.
  2. Retrieve — on a user query, embed the query and find the most similar chunks.
  3. Generate — pass retrieved chunks + query to the LLM as context; the model answers grounded in those documents.

Reduces hallucinations and enables Q&A over private, up-to-date data without fine-tuning.

What is function calling (tool use) in LLMs?

The LLM can request execution of predefined functions instead of only generating text. You describe tools (name, parameters, description) in the API call; the model returns a structured tool call; your code executes it and returns the result; the model continues with the result in context.

Enables: API integration, database queries, calculations, sending emails — the model decides when and which tool to use.

tools = [{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Get current weather for a city",
    "parameters": {
      "type": "object",
      "properties": { "city": { "type": "string" } },
      "required": ["city"]
    }
  }
}]

What is an AI agent?

An AI agent is an LLM-powered system that autonomously plans, uses tools, and iterates toward a goal — not just single prompt→response.

Loop: observe → think → act (tool call) → observe result → repeat until done. Frameworks: LangChain Agents, LangGraph, AutoGPT, CrewAI, OpenAI Assistants API.

Risks: runaway loops, incorrect tool calls, cost explosion. Mitigate with max iterations, human-in-the-loop, and tool sandboxing.

How do you evaluate LLM application quality?

LLM outputs are non-deterministic — traditional unit tests aren't enough.

  • Human eval — gold standard; expensive and slow.
  • LLM-as-judge — use a strong model to score outputs against criteria.
  • Automated metrics — BLEU/ROUGE for summarisation; exact match for structured output; retrieval recall@k for RAG.
  • Regression sets — curated input→expected output pairs; run on every deploy.
  • Frameworks — RAGAS (RAG eval), LangSmith, Phoenix, DeepEval.

How do you optimize LLM API costs?

  • Model selection — use smaller/cheaper models for simple tasks (gpt-4o-mini vs gpt-4o).
  • Prompt compression — shorter system prompts, summarise conversation history.
  • Caching — prompt caching (Anthropic/OpenAI) for repeated prefixes; cache embeddings.
  • Batching — batch API for non-real-time workloads (50% discount).
  • RAG over long context — retrieve relevant chunks instead of sending full documents.
  • Self-hosting — open-source models (Llama, Mistral) for high-volume, predictable workloads.

How do you implement guardrails for LLM applications?

Layers of protection:

  • Input filtering — moderation API, block prompt injection patterns, PII detection.
  • System prompt constraints — define allowed scope and refusal behaviour.
  • Output validation — schema validation (JSON), content filters, fact-checking against retrieved docs.
  • Rate limiting — prevent abuse.
  • Human review — for high-stakes outputs (medical, legal, financial).

Frameworks: Guardrails AI, NeMo Guardrails, LlamaGuard.

Open-source vs closed-source LLMs — tradeoffs.

Closed (GPT-4, Claude)Open (Llama, Mistral)
QualityGenerally highestClosing the gap fast
CostPer-token API pricingSelf-host infra cost
PrivacyData sent to providerFull data control on-prem
CustomizationLimited fine-tuningFull fine-tuning, weights access
Ops burdenLow (managed API)High (GPU, scaling, updates)

What are multimodal LLMs?

Models that accept and generate multiple modalities — text, images, audio, video — in a single model. Examples: GPT-4o (text + vision + audio), Gemini (text + image + video), Claude (text + image).

Use cases: image description, document OCR, visual Q&A, chart analysis, voice assistants. Images are typically encoded into token-like representations the transformer processes alongside text.

How do you get structured (JSON) output from an LLM?

Methods:

  • Prompting — "Respond in JSON: {name, age}" — unreliable alone.
  • JSON mode — API flag forcing valid JSON output.
  • Structured output / response format — provide a JSON Schema; model constrained to match it (OpenAI, Anthropic).
  • Function calling — define the schema as a tool; model returns structured args.
  • Post-processing — parse + validate with Pydantic/Zod; retry on failure.
response = client.chat.completions.create(
  model="gpt-4o",
  response_format={ "type": "json_schema", "json_schema": {...} },
  messages=[...]
)

Inference vs training — what's the difference?

Training — learning model weights from data. Requires massive GPU clusters, days/weeks, millions of dollars for frontier models. Done once (or periodically) by model providers.

Inference — running the trained model to generate outputs. What your application does on every API call. Optimised for latency and throughput (quantisation, batching, KV-cache).

As an application developer, you almost always work with inference — training is for ML researchers and model labs.

What is prompt injection and how do you defend against it?

Prompt injection — an attacker crafts input that overrides the system prompt or tricks the model into unintended actions. Types:

  • Direct — user says "ignore previous instructions".
  • Indirect — malicious instructions hidden in retrieved documents, emails, or web pages the model processes.

Defenses: input/output filtering, privilege separation (don't give the LLM direct tool access to sensitive systems), human approval for actions, treat all external content as untrusted.

What is MCP (Model Context Protocol)?

An open protocol (Anthropic, 2024 — now industry-wide) standardizing how AI applications connect to external tools and data. Instead of every app hand-wiring every integration, an MCP server exposes capabilities in a uniform JSON-RPC contract, and any MCP client (Claude, IDEs, your own chat app) can use them — "USB for AI integrations".

A server exposes three primitive types:

  • Tools — model-invokable actions with JSON-schema'd inputs (query a DB, call an API);
  • Resources — readable data/context (files, records);
  • Prompts — reusable prompt templates.

Transports: stdio for local servers, HTTP + SSE/streamable HTTP for remote — with sessions binding a client to server state across calls.

// tool definition an MCP server advertises
{
  "name": "search_orders",
  "description": "Search customer orders by status and date range",
  "inputSchema": {
    "type": "object",
    "properties": {
      "status": { "enum": ["PLACED", "SHIPPED", "CANCELLED"] },
      "from":   { "type": "string", "format": "date" }
    },
    "required": ["status"]
  }
}

How do you stream an LLM response into a UI (SSE parsing, incremental render)?

LLM APIs stream tokens as SSE — data: lines each carrying a JSON delta, ending with a done signal. The UI pipeline:

  1. Open the stream (EventSource for GET, or fetch + ReadableStream reader for POST bodies — the common case).
  2. Parse incrementally: decode chunks, split on the double-newline event boundary, keep the trailing partial line in a buffer (chunks don't align with events!).
  3. Append each delta to state and render — throttle re-renders (rAF/batching) so 50 tokens/sec doesn't mean 50 renders/sec; render markdown safely as it grows.
  4. Handle the tail: done signal, errors mid-stream (show partial + retry), and cancellation via AbortController when the user navigates away.
const res = await fetch('/api/chat', { method: 'POST', body, signal: ctrl.signal });
const reader = res.body.getReader();
const dec = new TextDecoder();
let buf = '';
for (;;) {
  const { done, value } = await reader.read();
  if (done) break;
  buf += dec.decode(value, { stream: true });
  const events = buf.split('\n\n');
  buf = events.pop();                    // keep the incomplete tail!
  for (const ev of events) {
    const data = ev.replace(/^data: /, '');
    if (data === '[DONE]') break;
    answer += JSON.parse(data).delta;    // incremental render
  }
}