LLM Interview Projects

Each project below is a complete, runnable system. For each, I provide the architecture, key code, and the interview questions you'll be asked.

Project 1: Document Q&A System (RAG)

Architecture

Key Code

from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_chroma import Chroma
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import PyPDFLoader
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough

class DocQA:
    def __init__(self):
        self.embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
        self.llm = ChatOpenAI(model="gpt-4o", temperature=0)
        self.splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
    
    def ingest(self, pdf_path: str):
        docs = PyPDFLoader(pdf_path).load()
        chunks = self.splitter.split_documents(docs)
        self.vectorstore = Chroma.from_documents(chunks, self.embeddings)
        return len(chunks)
    
    def query(self, question: str) -> dict:
        retriever = self.vectorstore.as_retriever(search_kwargs={"k": 5})
        prompt = ChatPromptTemplate.from_template(
            "Answer based on context:\n{context}\n\nQuestion: {question}"
        )
        chain = (
            {"context": retriever | (lambda docs: "\n".join(d.page_content for d in docs)),
             "question": RunnablePassthrough()}
            | prompt | self.llm | StrOutputParser()
        )
        return {"answer": chain.invoke(question), "sources": retriever.invoke(question)}

Interview Questions

Q: How would you handle a document with tables and images? A: Use document-aware chunking (Unstructured loader), extract tables as separate chunks with metadata, use multimodal embeddings for images (CLIP), and include table structure in chunk metadata.

Q: How do you prevent hallucination? A: Faithfulness prompting ("use ONLY context"), citation requirements, self-RAG to decide when to answer vs. say "I don't know", and Ragas faithfulness metric for evaluation.

Project 2: Customer Support Chatbot

Architecture

Key Code

from langgraph.prebuilt import create_react_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings

@tool
def search_faq(query: str) -> str:
    """Search company FAQ database."""
    vectorstore = Chroma(persist_directory="./faq_db", embedding_function=OpenAIEmbeddings())
    results = vectorstore.similarity_search(query, k=3)
    return "\n".join([doc.page_content for doc in results])

@tool
def create_ticket(category: str, description: str, priority: str = "medium") -> str:
    """Create a support ticket for human review."""
    return f"Ticket created: {category}/{priority} - {description}"

@tool
def check_order_status(order_id: str) -> str:
    """Look up order status in the database."""
    # In production: query actual database
    return f"Order {order_id}: Shipped, arriving tomorrow"

# Create agent with tools
llm = ChatOpenAI(model="gpt-4o")
agent = create_react_agent(
    model=llm,
    tools=[search_faq, create_ticket, check_order_status],
    prompt="You are a helpful customer support agent. Be professional and empathetic."
)

Interview Questions

Q: How do you handle a user who is frustrated or abusive? A: Implement sentiment detection, set escalation thresholds, provide de-escalation prompts, and always offer human handoff. Log negative interactions for review.

Q: How do you measure the chatbot's success? A: Resolution rate (no human needed), CSAT scores, average conversation length, escalation rate, cost per query, and time to resolution.

Project 3: Code Review Agent

Architecture

Key Code

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import JsonOutputParser
from pydantic import BaseModel, Field

class CodeIssue(BaseModel):
    line: int
    severity: str = Field(description="error, warning, info")
    category: str = Field(description="bug, performance, security, style, readability")
    message: str
    suggestion: str

class ReviewResult(BaseModel):
    issues: list[CodeIssue]
    score: int = Field(ge=0, le=100)
    summary: str

llm = ChatOpenAI(model="gpt-4o", temperature=0)
structured_llm = llm.with_structured_output(ReviewResult)

review_prompt = ChatPromptTemplate.from_messages([
    ("system", "Review code for bugs, performance, security, and style. Return structured issues."),
    ("human", "Review this {language} code:\n```{language}\n{code}\n```"),
])

chain = review_prompt | structured_llm
result = chain.invoke({"language": "python", "code": "def f(x): return x+1"})

Interview Questions

Q: How do you handle false positives in code review? A: Confidence scoring, allow developers to dismiss with reason, feedback loop to improve accuracy, and tiered severity (errors need fixing, warnings are suggestions).

Q: How would you handle a 10,000-line PR? A: Split into files, review each independently, use context window strategically, prioritize changed lines, and provide summary + detailed per-file reviews.

Projects 4-10 (Summary)

Project 4: Email Auto-Responder

• Stack: LangChain + Gmail API + RAG
• Key Feature: Intent classification + templated responses
• Interview Q: How do you handle ambiguous emails?

Project 5: Meeting Summarizer

• Stack: Whisper + LangChain + Notion API
• Key Feature: Action item extraction + follow-up scheduling
• Interview Q: How do you handle multiple speakers?

Project 6: Data Analyst Agent

• Stack: LangGraph + pandas + matplotlib
• Key Feature: Natural language to SQL + visualization
• Interview Q: How do you prevent SQL injection?

Project 7: Content Generator

• Stack: LangChain + SEO tools + CMS API
• Key Feature: Brand voice consistency + A/B testing
• Interview Q: How do you maintain consistent quality?

Project 8: Translation Memory System

• Stack: LangChain + TM database + quality estimation
• Key Feature: Context-aware translation + terminology consistency
• Interview Q: How do you handle domain-specific terminology?

Project 9: Medical Triage Bot

• Stack: LangGraph + medical knowledge base + safety guardrails
• Key Feature: Symptom assessment + escalation logic
• Interview Q: How do you ensure patient safety?

Project 10: Legal Document Analyzer

• Stack: LangChain + clause extraction + risk scoring
• Key Feature: Contract comparison + obligation tracking
• Interview Q: How do you handle jurisdiction-specific rules?

General Interview Tips

System Design Questions

1. Start with requirements — clarify scope, constraints, scale
2. Draw the architecture — components, data flow, integrations
3. Discuss trade-offs — cost vs quality, latency vs accuracy
4. Plan for production — monitoring, evaluation, fallbacks
5. Show iteration — how you'd improve over time

Coding Questions

1. Start simple — get a working prototype first
2. Add error handling — LLMs are unpredictable
3. Test edge cases — empty input, very long input, adversarial
4. Optimize for cost — token usage, caching, model selection
5. Measure everything — latency, accuracy, cost

Behavioral Questions

1. "Tell me about an LLM project" — use STAR format with metrics
2. "How do you handle LLM failures?" — fallbacks, guardrails, monitoring
3. "How do you evaluate LLM quality?" — offline + online metrics, A/B testing
4. "How do you reduce LLM costs?" — caching, smaller models, prompt optimization