The 8-Phase Core Curriculum
Binge the Core items in order (they're listed first in each phase) for the intuition. Optional items go deeper or explain the same idea another way; use them when something doesn't click. Tick items off as you go.
The Project Lab: Build Working Software
AI competence is proven by shipping working applications, not collecting certificates. Reuse a unified backend and frontend across these six systems.
8–10 Week Study Timeline
A sustainable, non-burnout operating plan: 3–4 video sessions (30–60 min), 1–2 note & Feynman compression sessions, 2 practical coding sessions, and 1 weekly whiteboard review.
Architectures You Must Be Able to Draw
Before writing code, you should be able to sketch these four systems from memory on a whiteboard.
1. Production RAG Architecture
How documents are processed, embedded, indexed, retrieved, reranked, and injected into the LLM context.
[ Raw Documents (PDF / MD) ]
│
┌──────┴──────┐
│ Parse & Clean│
│ Chunk (512T)│
└──────┬──────┘
▼
[ Dense Embedding API ]
│
▼
┌───────────────┐
│ Vector Index │ ◄─── User Query ─── [ Query Embed ]
│ (HNSW/Chroma) │ │
└───────┬───────┘ │ (Keyword search)
▼ ▼
[ Dense Top-50 ] [ BM25 Sparse Top-50 ]
│ │
└────────┬─────────┘
▼
[ Hybrid Reciprocal Rank Fusion ]
▼
[ Cross-Encoder Reranker (Top-5) ]
▼
┌───────────────────────────────────┐
│ Grounded Prompt Context │
│ "Answer strictly using sources:" │
└─────────────────┬─────────────────┘
▼
[ LLM Generation with Citations ]
2. Autonomous ReAct Agent Loop
The execution loop connecting model reasoning, tool selection, observations, memory, and termination.
┌───────────────────────────────────┐
│ User Goal / Prompt │
└─────────────────┬─────────────────┘
▼
┌───────────────────────┐
│ Model State & Context │
└───────────┬───────────┘
▼
┌───────────────────────────────────┐
│ THOUGHT: Decide next action │
└─────────────────┬─────────────────┘
▼
Is tool call required?
┌───────┴───────┐
YES│ NO│ (Task complete)
▼ ▼
[ ACTION: Call Tool ] [ FINAL RESULT ]
[ (Name, JSON Args) ]
│
▼
[ TOOL EXECUTION ] ─── (Sandbox / MCP Server)
│
▼
[ OBSERVATION ]
[ (Tool Output / Error) ]
│
▼
[ Inject into Context ] ───► (Loop back to Model)
3. Transformer Decoder-Only Block
The internal matrix transformation inside each layer of modern autoregressive models (Llama, GPT, DeepSeek).
Input Tokens [B, T]
│
▼
[ Token Embed ] + [ RoPE Positional Encoding ]
│
├─────────────────────────────┐ (Residual)
▼ │
[ RMSNorm / LayerNorm ] │
│ │
▼ │
[ Multi-Head Attention (Q, K, V) ] │
• Scaled Dot-Product: │
Softmax( (Q @ K^T)/sqrt(d) + Mask ) @ V
• FlashAttention Tile Optimization │
│ │
▼ │
[ Add (+) ] ◄──────────────────────┘
│
├─────────────────────────────┐ (Residual)
▼ │
[ RMSNorm / LayerNorm ] │
│ │
▼ │
[ Feed-Forward Network (SwiGLU / FFN) ]│
• Linear1 → Swish × Linear2 → Proj │
│ │
▼ │
[ Add (+) ] ◄──────────────────────┘
│
▼
[ Final Normalization (RMSNorm) → LM Head (Linear Projection) → Logits ]
• Softmax converts logits into token probabilities when needed
4. Pre-Training vs Post-Training (SFT + DPO)
How broad statistical web-text patterns are transformed into safe, instruction-following assistants.
[ Massive Web Text (Trillions of Tokens) ]
│
▼
┌───────────────────────────┐
│ 1. PRE-TRAINING │
│ Next-Token Prediction │
│ Unsupervised Loss: -log P │
└─────────────┬─────────────┘
▼
[ Base Model Checkpoint ]
(Completes text, not an assistant)
│
▼
┌───────────────────────────┐
│ 2. SUPERVISED FINE-TUNING │
│ High-Quality Pairs (Q&A) │
│ Instruction Following │
└─────────────┬─────────────┘
▼
[ SFT Assistant Model ]
│
▼
┌───────────────────────────┐
│ 3. PREFERENCE ALIGNMENT │
│ Direct Preference (DPO) │
│ or GRPO Reasoning Policy │
│ Chosen vs Rejected Pairs │
└─────────────┬─────────────┘
▼
[ Production Aligned Model ]
The 7-Question Mental Model Framework
For every algorithm or model in this roadmap, you should be able to answer these seven questions without referring to notes: