LLM Frameworks
LangChain Fundamentals â Build Production LLM Applications
LangChain is the most popular framework for building LLM-powered applications. It provides composable building blocks for chains, prompts, parsers, memory, and tool integration.
- Composable Chains â Link LLM calls, parsers, and tools into pipelines
- Prompt Templates â Reusable, parameterized prompts with few-shot examples
- Output Parsers â Convert LLM text into structured JSON, Pydantic, or SQL
- Real Projects â Build a customer support bot, code reviewer, and data analyst
"LangChain is not just a library â it's a way of thinking about LLM applications."
LangChain Fundamentals
LangChain provides a framework for building applications powered by language models. It standardizes the interface between LLMs and external data sources, tools, and memory systems.
Architecture Overview
1. Prompt Templates
Prompt templates are reusable, parameterized prompts that separate the prompt structure from the data.
Basic Prompt Templates
from langchain_core.prompts import ChatPromptTemplate
# Simple prompt template
prompt = ChatPromptTemplate.from_template(
"You are a {role}. Answer the question about {topic}.\n\nQuestion: {question}"
)
# Create messages
messages = prompt.invoke({
"role": "data scientist",
"topic": "machine learning",
"question": "What is gradient boosting?"
})
print(messages)
# [SystemMessage(content='You are a data scientist...'), HumanMessage(content='...')]
Few-Shot Prompt Templates
from langchain_core.prompts import FewShotChatMessagePromptTemplate
# Define examples
examples = [
{"input": "The movie was terrible", "output": "negative"},
{"input": "Absolutely loved it!", "output": "positive"},
{"input": "It was okay, nothing special", "output": "neutral"},
]
# Format example prompt
example_prompt = ChatPromptTemplate.from_messages([
("human", "{input}"),
("ai", "{output}"),
])
# Create few-shot prompt
few_shot_prompt = FewShotChatMessagePromptTemplate(
example_prompt=example_prompt,
examples=examples,
)
# Final prompt
final_prompt = ChatPromptTemplate.from_messages([
("system", "Classify the sentiment of the review."),
few_shot_prompt,
("human", "{input}"),
])
# Use it
chain = final_prompt | llm
result = chain.invoke({"input": "This film was a masterpiece"})
Dynamic Few-Shot Selection
from langchain_core.prompts import SemanticFewShotPromptTemplate
from langchain_chroma import Chroma
# Store examples in vector database
vectorstore = Chroma.from_texts(
training_examples,
embedding=OpenAIEmbeddings()
)
# Select semantically similar examples at runtime
dynamic_prompt = SemanticFewShotPromptTemplate(
example_selector=SemanticSimilarityExampleSelector(
vectorstore=vectorstore,
k=3,
),
example_prompt=example_prompt,
input_variables=["input"],
)
2. Output Parsers
Output parsers convert raw LLM text into structured data.
JSON Output Parser
from langchain_core.output_parsers import JsonOutputParser
from langchain_core.prompts import ChatPromptTemplate
from pydantic import BaseModel, Field
class MovieReview(BaseModel):
title: str = Field(description="Movie title")
rating: float = Field(description="Rating from 1-10")
summary: str = Field(description="Brief review summary")
recommended: bool = Field(description="Whether to recommend")
parser = JsonOutputParser(pydantic_object=MovieReview)
prompt = ChatPromptTemplate.from_messages([
("system", "Analyze the movie. {format_instructions}"),
("human", "{review}"),
])
chain = prompt | llm | parser
result = chain.invoke({
"review": "Inception was a mind-bending sci-fi masterpiece...",
"format_instructions": parser.get_format_instructions()
})
# Returns: {"title": "Inception", "rating": 9.5, "summary": "...", "recommended": true}
Pydantic Output Parser (Structured Output)
from langchain_openai import ChatOpenAI
from pydantic import BaseModel, Field
class CodeReview(BaseModel):
"""Structured code review output."""
language: str = Field(description="Programming language")
issues: list[str] = Field(description="List of code issues found")
suggestions: list[str] = Field(description="Improvement suggestions")
score: int = Field(description="Code quality score 1-100")
summary: str = Field(description="One-line summary")
# Use with_structured_output for type-safe parsing
llm = ChatOpenAI(model="gpt-4o")
structured_llm = llm.with_structured_output(CodeReview)
review = structured_llm.invoke("""
Review this Python code:
def calc(x):
return x + 1
""")
# Returns CodeReview instance with typed fields
print(review.language) # "Python"
print(review.score) # 85
Pydantic Tools Parser
from langchain_core.tools import tool
from pydantic import BaseModel, Field
class SearchInput(BaseModel):
query: str = Field(description="Search query")
max_results: int = Field(default=5, description="Max results")
@tool(args_schema=SearchInput)
def web_search(query: str, max_results: int = 5) -> str:
"""Search the web for information."""
# Implementation here
return f"Results for: {query}"
# LangChain auto-generates the tool schema
print(web_search.args_schema.model_json())
# {'properties': {'query': {'type': 'string', 'description': '...'}, ...}}
3. Chains (LCEL â LangChain Expression Language)
LCEL is LangChain's composable pipeline syntax using the pipe | operator.
Basic Chain
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
# Define components
prompt = ChatPromptTemplate.from_template(
"Write a {style} poem about {topic}"
)
llm = ChatOpenAI(model="gpt-4o", temperature=0.7)
parser = StrOutputParser()
# Chain with pipe operator
chain = prompt | llm | parser
# Invoke
poem = chain.invoke({"style": "haiku", "topic": "artificial intelligence"})
Parallel Chains
from langchain_core.runnables import RunnableParallel
# Run multiple chains in parallel
analysis_chain = prompt_analysis | llm | parser
summary_chain = prompt_summary | llm | parser
# Both run simultaneously, results merged into dict
parallel_chain = RunnableParallel(
analysis=analysis_chain,
summary=summary_chain,
)
result = parallel_chain.invoke({"document": long_text})
# {"analysis": "...", "summary": "..."}
Sequential Chains with Transform
from langchain_core.runnables import RunnableLambda
def word_count(output: dict) -> dict:
"""Add word count to output."""
output["word_count"] = len(output["summary"].split())
return output
chain = (
prompt
| llm
| parser
| RunnableLambda(word_count) # Transform step
)
result = chain.invoke({"topic": "quantum computing"})
Conditional Chains
from langchain_core.runnables import RunnableBranch
# Route based on input
classifier = (
ChatPromptTemplate.from_template("Classify as technical or casual: {input}")
| llm
| StrOutputParser()
)
technical_chain = (
ChatPromptTemplate.from_template("Give technical answer: {input}")
| llm
)
casual_chain = (
ChatPromptTemplate.from_template("Give casual answer: {input}")
| llm
)
# Branch based on classification
router = RunnableBranch(
(lambda x: "technical" in classifier.invoke(x), technical_chain),
(lambda x: "casual" in classifier.invoke(x), casual_chain),
casual_chain, # Default
)
4. Real Project: Customer Support Bot
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_chroma import Chroma
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser, JsonOutputParser
from langchain_core.tools import tool
from langchain.agents import AgentExecutor, create_tool_calling_agent
from pydantic import BaseModel
from typing import Optional
# --- Data Models ---
class SupportTicket(BaseModel):
category: str
priority: str
summary: str
resolution_steps: list[str]
# --- Tools ---
@tool
def search_knowledge_base(query: str) -> str:
"""Search company knowledge base for support articles."""
vectorstore = Chroma.from_documents(docs, OpenAIEmbeddings())
results = vectorstore.similarity_search(query, k=3)
return "\n".join([doc.page_content for doc in results])
@tool
def create_ticket(category: str, priority: str, description: str) -> str:
"""Create a support ticket in the system."""
return f"Ticket created: {category}/{priority} - {description}"
# --- Prompt ---
support_prompt = ChatPromptTemplate.from_messages([
("system", """You are a helpful customer support agent.
Available tools:
- search_knowledge_base: Search for articles
- create_ticket: Create support tickets
When a customer asks a question:
1. First search the knowledge base
2. If found, provide the answer
3. If not found, create a ticket for human review
Be professional, empathetic, and concise."""),
("human", "{input}"),
("placeholder", "{agent_scratchpad}"),
])
# --- Agent ---
llm = ChatOpenAI(model="gpt-4o", temperature=0)
agent = create_tool_calling_agent(llm, [search_knowledge_base, create_ticket], support_prompt)
executor = AgentExecutor(agent=agent, tools=[search_knowledge_base, create_ticket], verbose=True)
# --- Run ---
result = executor.invoke({
"input": "How do I reset my password? I've tried the normal flow but it's not sending emails."
})
print(result["output"])
Architecture SVG
5. Real Project: Code Review Agent
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.tools import tool
from langchain.agents import AgentExecutor, create_tool_calling_agent
from pydantic import BaseModel, Field
from typing import Optional
import subprocess
class CodeIssue(BaseModel):
line: int = Field(description="Line number")
severity: str = Field(description="error, warning, info")
message: str = Field(description="Issue description")
suggestion: str = Field(description="How to fix")
class CodeReviewResult(BaseModel):
language: str
score: int = Field(ge=0, le=100)
issues: list[CodeIssue]
summary: str
improvements: list[str]
# --- Tools ---
@tool
def run_linter(code: str, language: str) -> str:
"""Run linter on code and return issues."""
if language == "python":
result = subprocess.run(
["python", "-m", "py_compile", "-"],
input=code, capture_output=True, text=True
)
return result.stderr or "No issues found"
return "Linter not available for this language"
@tool
def check_complexity(code: str) -> str:
"""Analyze code complexity and suggest refactoring."""
lines = code.split("\n")
long_functions = []
for i, line in enumerate(lines):
if "def " in line and len(line) > 60:
long_functions.append(i + 1)
return f"Long function definitions at lines: {long_functions}" if long_functions else "Functions are well-sized"
@tool
def search_best_practices(topic: str) -> str:
"""Search for language-specific best practices."""
practices = {
"python": "Use type hints, context managers, f-strings, list comprehensions",
"javascript": "Use const/let, async/await, optional chaining",
"rust": "Use Result instead of unwrap, prefer &str over String",
}
return practices.get(topic.lower(), "No specific practices found")
# --- Review Agent ---
llm = ChatOpenAI(model="gpt-4o", temperature=0)
structured_llm = llm.with_structured_output(CodeReviewResult)
review_prompt = ChatPromptTemplate.from_messages([
("system", """You are an expert code reviewer. Analyze the code for:
1. Bugs and errors
2. Performance issues
3. Security vulnerabilities
4. Code style and readability
5. Best practices violations
Use tools when needed. Provide a structured review."""),
("human", "Review this {language} code:\n\n```{language}\n{code}\n```"),
])
tools = [run_linter, check_complexity, search_best_practices]
agent = create_tool_calling_agent(llm, tools, review_prompt)
executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
# Run review
result = executor.invoke({
"language": "python",
"code": """
def process_data(items):
results = []
for item in items:
try:
value = item['value']
results.append(value * 2)
except:
pass
return results
"""
})
Interview Prep: Common Questions
Q: When would you use LangChain vs raw API calls?
Answer: Use LangChain when you need:
- Chaining multiple LLM calls with complex logic
- Tool integration (RAG, APIs, databases)
- Output parsing into structured formats
- Memory management across conversations
- Agent patterns with multi-step reasoning
Use raw APIs when:
- Simple single-call completion
- Maximum control over every parameter
- Minimal dependencies required
Q: What is LCEL and why does it exist?
Answer: LCEL (LangChain Expression Language) is a declarative way to compose LangChain components using the pipe | operator. Benefits:
- Streaming support out of the box
- Parallel execution via
RunnableParallel - Async support without code changes
- Composability â chains are themselves runnables
- Batch processing for efficiency
# LCEL chain
chain = prompt | llm | parser
# Equivalent to:
# chain = SequentialChain([prompt, llm, parser])
Q: How do you handle LLM output parsing failures?
Answer: Use with_fallbacks and retry mechanisms:
from langchain_core.runnables import RunnableWithFallbacks
# Try structured output first, fallback to string parsing
chain = structured_llm.with_fallbacks(
[llm | JsonOutputParser()] # Fallback if structured output fails
)
# With retry
from langchain_core.runnables import RunnableRetry
chain = RunnableRetry(
runnable=base_chain,
max_attempts=3,
retry_if_exception_type=(OutputParserException,),
)
Q: Explain the difference between invoke, batch, and stream.
Answer:
invoke(input)â Single synchronous call, returns full resultbatch([input1, input2])â Multiple calls in parallel, returns liststream(input)â Yields chunks as they're generated, first-token latency
# Single call
result = chain.invoke({"query": "hello"})
# Batch processing
results = chain.batch([{"query": "hello"}, {"query": "world"}])
# Streaming
for chunk in chain.stream({"query": "tell me a story"}):
print(chunk, end="", flush=True)
Q: How do you test LangChain applications?
Answer: Use LangSmith for tracing and Evals for evaluation:
# Enable tracing
import os
os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_API_KEY"] = "your-key"
# Test with assertions
def test_support_bot():
result = executor.invoke({"input": "How do I reset password?"})
assert "reset" in result["output"].lower()
assert len(result["output"]) > 50
assert "tool" in str(result.get("intermediate_steps", []))
Key Takeaways
- Start with prompts â Good prompts are the foundation of good LLM apps
- Use structured outputs â Always parse LLM output into typed data
- Chain with LCEL â Use pipe operators for composable, testable pipelines
- Add tools incrementally â Start with one tool, add more as needed
- Evaluate continuously â Use LangSmith to trace and evaluate every call