LLM Capstone
LLM Capstone Project â Building Production-Ready AI Applications
This capstone project guides you through building an end-to-end LLM application, from conception to deployment. You'll apply everything learned throughout the course to create a real-world system.
- Project Design â Requirements gathering and architecture
- Implementation â Building the complete system
- Deployment â Production deployment and monitoring
The best way to learn is by doing.
LLM Capstone Project
This capstone project challenges you to build a complete LLM application that solves a real-world problem. You'll go through the full software development lifecycle, from requirements gathering to production deployment.
Project Selection
Choosing a Project
Select a project that:
- Solves a real problem: Addresses a genuine need
- Is achievable: Can be completed in the time frame
- Demonstrates skills: Showcases LLM knowledge
- Has data available: Has accessible training/evaluation data
- Can be deployed: Can run in a production environment
Project Ideas
| Category | Project | Complexity |
|---|---|---|
| Education | Adaptive tutoring system | High |
| Healthcare | Medical document summarizer | High |
| Finance | Financial news analyzer | Medium |
| Legal | Contract analysis tool | High |
| Customer Service | Multi-channel support bot | Medium |
| Content | Automated content pipeline | Medium |
| Research | Literature review assistant | Medium |
| Productivity | Meeting summarizer | Low-Medium |
Project Design Phase
Requirements Gathering
Requirements template:
## Functional Requirements
- [ ] Core feature 1: Description
- [ ] Core feature 2: Description
- [ ] User interaction model: Chat/API/Batch
- [ ] Output format: Text/JSON/Structured
## Non-Functional Requirements
- [ ] Latency: < X seconds
- [ ] Throughput: X requests/minute
- [ ] Accuracy: X% on evaluation set
- [ ] Availability: X% uptime
## Constraints
- [ ] Budget: $X
- [ ] Timeline: X weeks
- [ ] Team size: X people
- [ ] Technology stack: Specific tools/frameworks
## Ethical Considerations
- [ ] Bias mitigation strategies
- [ ] Privacy requirements
- [ ] Safety measures
- [ ] Transparency needs
<div className="my-6 flex justify-center">
<svg viewBox="0 0 1270 2120" width="100%" style={{maxWidth: 1270}} xmlns="http://www.w3.org/2000/svg">
<defs>
<linearGradient id="gg95-llm-capstone-project_mdx-0" x1="0" y1="0" x2="0" y2="1"><stop offset="0%" stopColor="#6366f1"/><stop offset="100%" stopColor="#4338ca"/></linearGradient>
<filter id="gs95-llm-capstone-project_mdx-0"><feDropShadow dx="0" dy="1" stdDeviation="2" floodOpacity="0.08"/></filter>
</defs>
<rect x="0" y="0" width="1270" height="2120" rx="10" fill="#f8fafc" stroke="#e2e8f0" strokeWidth="1" filter="url(#gs95-llm-capstone-project_mdx-0)"/>
<text x="17" y="32" fill="#1e293b" fontFamily="monospace" fontSize="10">### Architecture Design</text>
<text x="17" y="48" fill="#1e293b" fontFamily="monospace" fontSize="10"><MathDefinition title="LLM System Architecture"></text>
<text x="17" y="64" fill="#1e293b" fontFamily="monospace" fontSize="10">**LLM system architecture** defines the components, their interactions, dat</text>
<text x="17" y="80" fill="#1e293b" fontFamily="monospace" fontSize="10"></MathDefinition></text>
<text x="17" y="96" fill="#1e293b" fontFamily="monospace" fontSize="10">Typical architecture components:</text>
<text x="17" y="112" fill="#1e293b" fontFamily="monospace" fontSize="10"><div className="my-6 flex justify-center"></text>
<text x="17" y="128" fill="#1e293b" fontFamily="monospace" fontSize="10"><svg viewBox="0 0 465 296" width="100%" style={"maxWidth": 465} xmlns="http</text>
<text x="17" y="144" fill="#1e293b" fontFamily="monospace" fontSize="10"><defs></text>
<text x="17" y="160" fill="#1e293b" fontFamily="monospace" fontSize="10"><linearGradient id="ng95-llm-capstone-project_mdx-1" x1="0" y1="0" x2="0" y</text>
<text x="17" y="176" fill="#1e293b" fontFamily="monospace" fontSize="10"><filter id="ns95-llm-capstone-project_mdx-1"><feDropShadow dx="0" dy="1" st</text>
<text x="17" y="192" fill="#1e293b" fontFamily="monospace" fontSize="10"></defs></text>
<text x="17" y="208" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="0" y="0" width="465" height="296" rx="10" fill="#f8fafc" stroke="#</text>
<text x="17" y="224" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="12" y="48" fill="#1e293b" fontFamily="monospace" fontSize="10">Use</text>
<text x="17" y="240" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="12" y="80" fill="#1e293b" fontFamily="monospace" fontSize="10">API</text>
<text x="17" y="256" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="12" y="128" fill="#1e293b" fontFamily="monospace" fontSize="10">Pr</text>
<text x="17" y="272" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="12" y="144" fill="#1e293b" fontFamily="monospace" fontSize="10">En</text>
<text x="17" y="288" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="12" y="192" fill="#1e293b" fontFamily="monospace" fontSize="10">LL</text>
<text x="17" y="304" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="12" y="240" fill="#1e293b" fontFamily="monospace" fontSize="10">Mo</text>
<text x="17" y="320" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="12" y="256" fill="#1e293b" fontFamily="monospace" fontSize="10">Se</text>
<text x="17" y="336" fill="#1e293b" fontFamily="monospace" fontSize="10"></svg></text>
<text x="17" y="352" fill="#1e293b" fontFamily="monospace" fontSize="10"></div></text>
<text x="17" y="368" fill="#1e293b" fontFamily="monospace" fontSize="10">### Technology Selection</text>
<text x="17" y="384" fill="#1e293b" fontFamily="monospace" fontSize="10"><MathFormula</text>
<text x="17" y="400" fill="#1e293b" fontFamily="monospace" fontSize="10">title="Technology Selection Score"</text>
<text x="17" y="416" fill="#1e293b" fontFamily="monospace" fontSize="10">tex="S = \\sum_{i=1}^{n} w_i \\cdot s_i"</text>
<text x="17" y="432" fill="#1e293b" fontFamily="monospace" fontSize="10">/></text>
<text x="17" y="448" fill="#1e293b" fontFamily="monospace" fontSize="10">Criteria:</text>
<text x="17" y="464" fill="#1e293b" fontFamily="monospace" fontSize="10">1. **Performance**: Latency, throughput</text>
<text x="17" y="480" fill="#1e293b" fontFamily="monospace" fontSize="10">2. **Scalability**: Ability to handle growth</text>
<text x="17" y="496" fill="#1e293b" fontFamily="monospace" fontSize="10">3. **Cost**: Infrastructure and operational costs</text>
<text x="17" y="512" fill="#1e293b" fontFamily="monospace" fontSize="10">4. **Community**: Support and ecosystem</text>
<text x="17" y="528" fill="#1e293b" fontFamily="monospace" fontSize="10">5. **Team expertise**: Existing knowledge</text>
<text x="17" y="544" fill="#1e293b" fontFamily="monospace" fontSize="10">## Implementation Phase</text>
<text x="17" y="560" fill="#1e293b" fontFamily="monospace" fontSize="10">### Project Structure</text>
<text x="17" y="576" fill="#1e293b" fontFamily="monospace" fontSize="10"><div className="my-6 flex justify-center"></text>
<text x="17" y="592" fill="#1e293b" fontFamily="monospace" fontSize="10"><svg viewBox="0 0 730 695" width="100%" style={"maxWidth": 730} xmlns="http</text>
<text x="17" y="608" fill="#1e293b" fontFamily="monospace" fontSize="10"><defs></text>
<text x="17" y="624" fill="#1e293b" fontFamily="monospace" fontSize="10"><linearGradient id="tg95-llm-capstone-project_mdx-0" x1="0" y1="0" x2="0" y</text>
<text x="17" y="640" fill="#1e293b" fontFamily="monospace" fontSize="10"><filter id="ts95-llm-capstone-project_mdx-0"><feDropShadow dx="0" dy="1" st</text>
<text x="17" y="656" fill="#1e293b" fontFamily="monospace" fontSize="10"></defs></text>
<text x="17" y="672" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="0" y="0" width="730" height="695" rx="10" fill="#f8fafc" stroke="#</text>
<text x="17" y="688" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="0" y="0" width="730" height="30" rx="10" fill="url(#tg95-llm-capst</text>
<text x="17" y="704" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="0" y="15" width="730" height="15" fill="url(#tg95-llm-capstone-pro</text>
<text x="17" y="720" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="365" y="20" textAnchor="middle" fill="#fff" fontFamily="Inter,syst</text>
<text x="17" y="736" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="50" width="348" height="28" rx="4" fill="url(#tg95-llm-caps</text>
<line x1="12" y1="744" x2="1258" y2="744" stroke="#e2e8f0" strokeWidth="0.5"/>
<text x="17" y="752" fill="#6366f1" fontFamily="Inter,system-ui,sans-serif" fontWeight="700" fontSize="11"><text x="190" y="69" textAnchor="middle" fill="#fff" fontFamily="Inter</text>
<text x="17" y="768" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="50" width="348" height="28" rx="4" fill="url(#tg95-llm-cap</text>
<text x="17" y="784" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="540" y="69" textAnchor="middle" fill="#fff" fontFamily="Inter,syst</text>
<text x="17" y="800" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="80" width="348" height="28" rx="4" fill="#fff" stroke="#e2e</text>
<line x1="12" y1="808" x2="1258" y2="808" stroke="#e2e8f0" strokeWidth="0.5"/>
<text x="17" y="816" fill="#6366f1" fontFamily="Inter,system-ui,sans-serif" fontWeight="700" fontSize="11"><text x="190" y="99" textAnchor="middle" fill="#1e293b" fontFamily="In</text>
<text x="17" y="832" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="80" width="348" height="28" rx="4" fill="#fff" stroke="#e2</text>
<text x="17" y="848" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="540" y="99" textAnchor="middle" fill="#1e293b" fontFamily="Inter,s</text>
<text x="17" y="864" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="110" width="348" height="28" rx="4" fill="#f8fafc" stroke="</text>
<line x1="12" y1="872" x2="1258" y2="872" stroke="#e2e8f0" strokeWidth="0.5"/>
<text x="17" y="880" fill="#6366f1" fontFamily="Inter,system-ui,sans-serif" fontWeight="700" fontSize="11"><text x="190" y="129" textAnchor="middle" fill="#1e293b" fontFamily="I</text>
<text x="17" y="896" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="110" width="348" height="28" rx="4" fill="#f8fafc" stroke=</text>
<text x="17" y="912" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="540" y="129" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="928" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="140" width="348" height="28" rx="4" fill="#fff" stroke="#e2</text>
<text x="17" y="944" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="190" y="159" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="960" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="140" width="348" height="28" rx="4" fill="#fff" stroke="#e</text>
<line x1="12" y1="968" x2="1258" y2="968" stroke="#e2e8f0" strokeWidth="0.5"/>
<text x="17" y="976" fill="#6366f1" fontFamily="Inter,system-ui,sans-serif" fontWeight="700" fontSize="11"><text x="540" y="159" textAnchor="middle" fill="#1e293b" fontFamily="I</text>
<text x="17" y="992" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="170" width="348" height="28" rx="4" fill="#f8fafc" stroke="</text>
<text x="17" y="1008" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="190" y="189" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1024" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="170" width="348" height="28" rx="4" fill="#f8fafc" stroke=</text>
<text x="17" y="1040" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="540" y="189" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1056" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="200" width="348" height="28" rx="4" fill="#fff" stroke="#e2</text>
<line x1="12" y1="1064" x2="1258" y2="1064" stroke="#e2e8f0" strokeWidth="0.5"/>
<text x="17" y="1072" fill="#6366f1" fontFamily="Inter,system-ui,sans-serif" fontWeight="700" fontSize="11"><text x="190" y="219" textAnchor="middle" fill="#1e293b" fontFamily="I</text>
<text x="17" y="1088" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="200" width="348" height="28" rx="4" fill="#fff" stroke="#e</text>
<text x="17" y="1104" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="540" y="219" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1120" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="230" width="348" height="28" rx="4" fill="#f8fafc" stroke="</text>
<text x="17" y="1136" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="190" y="249" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1152" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="230" width="348" height="28" rx="4" fill="#f8fafc" stroke=</text>
<line x1="12" y1="1160" x2="1258" y2="1160" stroke="#e2e8f0" strokeWidth="0.5"/>
<text x="17" y="1168" fill="#6366f1" fontFamily="Inter,system-ui,sans-serif" fontWeight="700" fontSize="11"><text x="540" y="249" textAnchor="middle" fill="#1e293b" fontFamily="I</text>
<text x="17" y="1184" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="260" width="348" height="28" rx="4" fill="#fff" stroke="#e2</text>
<text x="17" y="1200" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="190" y="279" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1216" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="260" width="348" height="28" rx="4" fill="#fff" stroke="#e</text>
<line x1="12" y1="1224" x2="1258" y2="1224" stroke="#e2e8f0" strokeWidth="0.5"/>
<text x="17" y="1232" fill="#6366f1" fontFamily="Inter,system-ui,sans-serif" fontWeight="700" fontSize="11"><text x="540" y="279" textAnchor="middle" fill="#1e293b" fontFamily="I</text>
<text x="17" y="1248" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="290" width="348" height="28" rx="4" fill="#f8fafc" stroke="</text>
<text x="17" y="1264" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="190" y="309" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1280" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="290" width="348" height="28" rx="4" fill="#f8fafc" stroke=</text>
<text x="17" y="1296" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="540" y="309" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1312" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="320" width="348" height="28" rx="4" fill="#fff" stroke="#e2</text>
<line x1="12" y1="1320" x2="1258" y2="1320" stroke="#e2e8f0" strokeWidth="0.5"/>
<text x="17" y="1328" fill="#6366f1" fontFamily="Inter,system-ui,sans-serif" fontWeight="700" fontSize="11"><text x="190" y="339" textAnchor="middle" fill="#1e293b" fontFamily="I</text>
<text x="17" y="1344" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="320" width="348" height="28" rx="4" fill="#fff" stroke="#e</text>
<text x="17" y="1360" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="540" y="339" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1376" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="350" width="348" height="28" rx="4" fill="#f8fafc" stroke="</text>
<text x="17" y="1392" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="190" y="369" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1408" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="350" width="348" height="28" rx="4" fill="#f8fafc" stroke=</text>
<line x1="12" y1="1416" x2="1258" y2="1416" stroke="#e2e8f0" strokeWidth="0.5"/>
<text x="17" y="1424" fill="#6366f1" fontFamily="Inter,system-ui,sans-serif" fontWeight="700" fontSize="11"><text x="540" y="369" textAnchor="middle" fill="#1e293b" fontFamily="I</text>
<text x="17" y="1440" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="380" width="348" height="28" rx="4" fill="#fff" stroke="#e2</text>
<text x="17" y="1456" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="190" y="399" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1472" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="380" width="348" height="28" rx="4" fill="#fff" stroke="#e</text>
<text x="17" y="1488" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="540" y="399" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1504" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="410" width="348" height="28" rx="4" fill="#f8fafc" stroke="</text>
<text x="17" y="1520" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="190" y="429" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1536" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="410" width="348" height="28" rx="4" fill="#f8fafc" stroke=</text>
<text x="17" y="1552" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="540" y="429" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1568" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="440" width="348" height="28" rx="4" fill="#fff" stroke="#e2</text>
<line x1="12" y1="1576" x2="1258" y2="1576" stroke="#e2e8f0" strokeWidth="0.5"/>
<text x="17" y="1584" fill="#6366f1" fontFamily="Inter,system-ui,sans-serif" fontWeight="700" fontSize="11"><text x="190" y="459" textAnchor="middle" fill="#1e293b" fontFamily="I</text>
<text x="17" y="1600" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="440" width="348" height="28" rx="4" fill="#fff" stroke="#e</text>
<text x="17" y="1616" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="540" y="459" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1632" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="470" width="348" height="28" rx="4" fill="#f8fafc" stroke="</text>
<text x="17" y="1648" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="190" y="489" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1664" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="470" width="348" height="28" rx="4" fill="#f8fafc" stroke=</text>
<text x="17" y="1680" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="540" y="489" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1696" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="500" width="348" height="28" rx="4" fill="#fff" stroke="#e2</text>
<line x1="12" y1="1704" x2="1258" y2="1704" stroke="#e2e8f0" strokeWidth="0.5"/>
<text x="17" y="1712" fill="#6366f1" fontFamily="Inter,system-ui,sans-serif" fontWeight="700" fontSize="11"><text x="190" y="519" textAnchor="middle" fill="#1e293b" fontFamily="I</text>
<text x="17" y="1728" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="500" width="348" height="28" rx="4" fill="#fff" stroke="#e</text>
<text x="17" y="1744" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="540" y="519" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1760" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="530" width="348" height="28" rx="4" fill="#f8fafc" stroke="</text>
<line x1="12" y1="1768" x2="1258" y2="1768" stroke="#e2e8f0" strokeWidth="0.5"/>
<text x="17" y="1776" fill="#6366f1" fontFamily="Inter,system-ui,sans-serif" fontWeight="700" fontSize="11"><text x="190" y="549" textAnchor="middle" fill="#1e293b" fontFamily="I</text>
<text x="17" y="1792" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="530" width="348" height="28" rx="4" fill="#f8fafc" stroke=</text>
<text x="17" y="1808" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="540" y="549" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1824" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="560" width="348" height="28" rx="4" fill="#fff" stroke="#e2</text>
<text x="17" y="1840" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="190" y="579" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1856" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="560" width="348" height="28" rx="4" fill="#fff" stroke="#e</text>
<text x="17" y="1872" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="540" y="579" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1888" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="590" width="348" height="28" rx="4" fill="#f8fafc" stroke="</text>
<line x1="12" y1="1896" x2="1258" y2="1896" stroke="#e2e8f0" strokeWidth="0.5"/>
<text x="17" y="1904" fill="#6366f1" fontFamily="Inter,system-ui,sans-serif" fontWeight="700" fontSize="11"><text x="190" y="609" textAnchor="middle" fill="#1e293b" fontFamily="I</text>
<text x="17" y="1920" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="590" width="348" height="28" rx="4" fill="#f8fafc" stroke=</text>
<text x="17" y="1936" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="540" y="609" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="1952" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="620" width="348" height="28" rx="4" fill="#fff" stroke="#e2</text>
<line x1="12" y1="1960" x2="1258" y2="1960" stroke="#e2e8f0" strokeWidth="0.5"/>
<text x="17" y="1968" fill="#6366f1" fontFamily="Inter,system-ui,sans-serif" fontWeight="700" fontSize="11"><text x="190" y="639" textAnchor="middle" fill="#1e293b" fontFamily="I</text>
<text x="17" y="1984" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="620" width="348" height="28" rx="4" fill="#fff" stroke="#e</text>
<text x="17" y="2000" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="540" y="639" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="2016" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="15" y="650" width="348" height="28" rx="4" fill="#f8fafc" stroke="</text>
<text x="17" y="2032" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="190" y="669" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="2048" fill="#1e293b" fontFamily="monospace" fontSize="10"><rect x="365" y="650" width="348" height="28" rx="4" fill="#f8fafc" stroke=</text>
<text x="17" y="2064" fill="#1e293b" fontFamily="monospace" fontSize="10"><text x="540" y="669" textAnchor="middle" fill="#1e293b" fontFamily="Inter,</text>
<text x="17" y="2080" fill="#1e293b" fontFamily="monospace" fontSize="10"></svg></text>
<text x="17" y="2096" fill="#1e293b" fontFamily="monospace" fontSize="10"></div></text>
<text x="17" y="2112" fill="#1e293b" fontFamily="monospace" fontSize="10">### Configuration Management</text>
</svg>
</div>python
from pydantic import BaseModel
from typing import Optional
from enum import Enum
class Environment(str, Enum):
DEVELOPMENT = "development"
STAGING = "staging"
PRODUCTION = "production"
class LLMConfig(BaseModel):
model_name: str = "meta-llama/Llama-3-8B-Instruct"
temperature: float = 0.7
max_tokens: int = 512
top_p: float = 0.9
class RAGConfig(BaseModel):
chunk_size: int = 1000
chunk_overlap: int = 200
retrieval_top_k: int = 5
class AppConfig(BaseModel):
environment: Environment = Environment.DEVELOPMENT
llm: LLMConfig = LLMConfig()
rag: RAGConfig = RAGConfig()
debug: bool = False
Model Implementation
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
from typing import List, Dict, Any
class LLMEngine:
def __init__(self, config: LLMConfig):
self.config = config
self.tokenizer = AutoTokenizer.from_pretrained(config.model_name)
self.model = AutoModelForCausalLM.from_pretrained(
config.model_name,
torch_dtype=torch.float16,
device_map="auto"
)
def generate(self, prompt: str, **kwargs) -> str:
inputs = self.tokenizer(prompt, return_tensors="pt").to(self.model.device)
generation_kwargs = {
"max_new_tokens": self.config.max_tokens,
"temperature": self.config.temperature,
"top_p": self.config.top_p,
"do_sample": self.config.temperature > 0
}
generation_kwargs.update(kwargs)
outputs = self.model.generate(**inputs, **generation_kwargs)
response = self.tokenizer.decode(
outputs[0][inputs.shape[-1]:],
skip_special_tokens=True
)
return response
def generate_with_history(
self,
messages: List[Dict[str, str]],
**kwargs
) -> str:
prompt = self.tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
return self.generate(prompt, **kwargs)
Retrieval System
from langchain_community.vectorstores import Chroma
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain.text_splitter import RecursiveCharacterTextSplitter
from typing import List, Dict, Any
class RetrievalSystem:
def __init__(self, config: RAGConfig):
self.config = config
self.embeddings = HuggingFaceEmbeddings(
model_name="sentence-transformers/all-MiniLM-L6-v2"
)
self.text_splitter = RecursiveCharacterTextSplitter(
chunk_size=config.chunk_size,
chunk_overlap=config.chunk_overlap
)
self.vectorstore = None
def index_documents(self, documents: List[Dict[str, Any]]):
texts = [doc["content"] for doc in documents]
metadatas = [doc["metadata"] for doc in documents]
splits = self.text_splitter.create_documents(texts, metadatas)
self.vectorstore = Chroma.from_documents(
splits,
self.embeddings
)
def retrieve(self, query: str) -> List[Dict[str, Any]]:
if self.vectorstore is None:
return []
results = self.vectorstore.similarity_search_with_score(
query,
k=self.config.retrieval_top_k
)
return [
{
"content": doc.page_content,
"metadata": doc.metadata,
"score": score
}
for doc, score in results
]
API Implementation
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from typing import List, Optional
import uvicorn
app = FastAPI(title="LLM Application API")
class GenerateRequest(BaseModel):
prompt: str
max_tokens: Optional[int] = 512
temperature: Optional[float] = 0.7
class GenerateResponse(BaseModel):
text: str
tokens_used: int
latency_ms: float
@app.post("/generate", response_model=GenerateResponse)
async def generate_text(request: GenerateRequest):
try:
start_time = time.time()
# Generate response
response = llm_engine.generate(
request.prompt,
max_new_tokens=request.max_tokens,
temperature=request.temperature
)
latency = (time.time() - start_time) * 1000
return GenerateResponse(
text=response,
tokens_used=len(tokenizer.encode(response)),
latency_ms=latency
)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@app.post("/rag/generate")
async def rag_generate(request: GenerateRequest):
# Retrieve relevant documents
documents = retrieval_system.retrieve(request.prompt)
# Construct prompt with context
context = "\n\n".join([doc["content"] for doc in documents])
prompt = f"""Context: {context}
Question: {request.prompt}
Answer:"""
# Generate response
response = llm_engine.generate(prompt)
return {
"answer": response,
"sources": documents
}
Evaluation Phase
Evaluation Framework
Evaluation dimensions:
- Functional: Does it do what it should?
- Performance: Does it meet latency/throughput requirements?
- Quality: Are outputs accurate and useful?
- Usability: Is it easy to use?
- Robustness: Does it handle edge cases?
Evaluation Metrics
class ProjectEvaluator:
def __init__(self, test_cases: List[Dict]):
self.test_cases = test_cases
self.results = []
def evaluate_functional(self, system):
correct = 0
for test_case in self.test_cases:
output = system.process(test_case["input"])
if self.check_output(output, test_case["expected"]):
correct += 1
return correct / len(self.test_cases)
def evaluate_performance(self, system, num_requests=100):
latencies = []
for _ in range(num_requests):
start = time.time()
system.process("Test input")
latencies.append(time.time() - start)
return {
"avg_latency": sum(latencies) / len(latencies),
"p95_latency": sorted(latencies)[int(0.95 * len(latencies))],
"p99_latency": sorted(latencies)[int(0.99 * len(latencies))]
}
def evaluate_quality(self, system, evaluator_llm):
scores = []
for test_case in self.test_cases:
output = system.process(test_case["input"])
score = evaluator_llm.evaluate(
test_case["input"],
output,
test_case.get("criteria", [])
)
scores.append(score)
return sum(scores) / len(scores)
User Testing
User testing protocol:
- Recruit users: Find representative users
- Define tasks: Create realistic usage scenarios
- Observe usage: Watch how users interact
- Gather feedback: Collect qualitative and quantitative feedback
- Iterate: Improve based on findings
Deployment Phase
Deployment Architecture
Deployment options:
- Cloud: AWS, GCP, Azure
- On-premises: Self-hosted infrastructure
- Edge: Local deployment
- Hybrid: Combination of cloud and edge
Containerization
# Dockerfile
FROM python:3.10-slim
WORKDIR /app
# Install dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Copy application code
COPY src/ ./src/
COPY models/ ./models/
# Expose port
EXPOSE 8000
# Run application
CMD ["uvicorn", "src.api.endpoints:app", "--host", "0.0.0.0", "--port", "8000"]
Monitoring and Observability
Key metrics:
- Latency: Response time percentiles
- Throughput: Requests per second
- Error rate: Failed requests percentage
- Quality: Output quality scores
- Cost: Token usage and cost per request
Continuous Improvement
Improvement cycle:
- Monitor: Collect metrics and feedback
- Analyze: Identify issues and opportunities
- Plan: Prioritize improvements
- Implement: Make changes
- Verify: Test and validate changes
Project Documentation
README Template
# LLM Capstone Project: [Project Name]
## Overview
Brief description of the project and its goals.
## Architecture
High-level architecture diagram and component descriptions.
## Setup
Instructions for setting up the development environment.
## Usage
Examples of how to use the system.
## Evaluation
Results of evaluation against requirements.
## Deployment
Instructions for deploying to production.
## Future Work
Potential improvements and extensions.
Model Card
Best Practices
Project Management
- Agile methodology: Use sprints and iterative development
- Version control: Use Git for code and documentation
- Regular reviews: Demo progress regularly
- Risk management: Identify and mitigate risks early
Code Quality
- Code review: Review all code before merging
- Testing: Maintain high test coverage
- Documentation: Document code and decisions
- Refactoring: Continuously improve code quality
Learning
- Reflection: Reflect on what you learned
- Knowledge sharing: Share findings with others
- Portfolio: Document the project for your portfolio
- Presentation: Prepare a presentation of your work
Practice Exercises
-
Project Planning: Create a detailed project plan for your capstone project, including milestones, risks, and mitigation strategies.
-
Architecture Design: Design the system architecture for your project, including component diagrams and data flows.
-
Implementation Sprint: Implement the core functionality of your project in a time-boxed sprint.
-
Evaluation: Conduct a comprehensive evaluation of your system against requirements.
What to Learn Next
-> LLM Research Paper Guide Key papers, reading guides, and research methodology for LLMs.
-> LLM Glossary Comprehensive glossary of LLM terms and concepts.
-> LLM Tool Ecosystem Overview of HuggingFace, LangChain, LlamaIndex, and other tools.
-> LLM Best Practices Best practices for common LLM tasks and applications.
-> LLM Roadmap Learning roadmap, skill progression, and career paths in LLMs.
-> Back to LLM Overview Return to the beginning of the LLM course.