Tutorials Roadmap

Hands-on tutorials that turn vault theory into working code. Each tutorial is self-contained — copy-paste and run.

Philosophy: Implement algorithms from scratch first, then use libraries. You don’t understand it until you’ve built it.

Prerequisites

• Python 3.10+, NumPy, Matplotlib
• For DL tutorials: PyTorch
• For NLP tutorials: transformers, datasets (HuggingFace)

pip install numpy matplotlib scikit-learn torch transformers datasets

---

From-Scratch ML Implementations

Build core algorithms using only NumPy. No sklearn, no magic.

#	Tutorial	Vault Foundation	Difficulty
01	01 - Linear Regression from Scratch	Linear Regression, Gradient Descent	Beginner
02	02 - Logistic Regression from Scratch	Logistic Regression, Loss Functions	Beginner
03	03 - Decision Tree from Scratch	Decision Trees, Entropy	Intermediate
04	04 - K-Means from Scratch	K-Means Clustering	Beginner
05	05 - Neural Network from Scratch	Backpropagation, Neurons and Activation Functions	Intermediate
06	06 - Gradient Descent Variants	Gradient Descent, Optimizers	Intermediate

Deep Learning Internals

Understand what happens inside neural networks before using frameworks.

#	Tutorial	Vault Foundation	Difficulty
07	07 - Backpropagation Step by Step	Backpropagation, Chain Rule	Intermediate
08	08 - Attention Mechanism from Scratch	Attention Mechanism, Transformers	Advanced
15	15 - Build an Autograd Engine	Backpropagation, Chain Rule	Intermediate
18	18 - Activations and Initialization Deep Dive	Vanishing and Exploding Gradients, Batch Normalization	Intermediate
19	19 - Becoming a Backprop Ninja	Backpropagation, Batch Normalization	Advanced
22	22 - Build a CNN from Scratch	Convolutional Neural Networks	Advanced

Language Modeling (Karpathy’s makemore → nanoGPT Path)

Build language models from the simplest (bigrams) to GPT, step by step.

#	Tutorial	Vault Foundation	Difficulty
16	16 - Bigram Language Model	Language Models, Probability Basics	Beginner
17	17 - MLP Language Model	Embeddings, Language Models	Intermediate
23	23 - RNN Text Generator	Recurrent Neural Networks	Intermediate
20	20 - Build GPT from Scratch	Transformers, Attention Mechanism	Advanced
21	21 - Build a BPE Tokenizer	Text Preprocessing	Intermediate

Data Science Workflows

Real-world patterns that the theory notes don’t cover.

#	Tutorial	Vault Foundation	Difficulty
09	09 - Feature Engineering Cookbook	Feature Engineering, Exploratory Data Analysis	Intermediate
10	10 - Cross-Validation Done Right	Cross-Validation, Train-Test Split	Intermediate
11	11 - Handling Class Imbalance	Evaluation Metrics, Loss Functions	Intermediate

Modern AI Workflows

End-to-end pipelines with pretrained models.

#	Tutorial	Vault Foundation	Difficulty
12	12 - Transfer Learning End-to-End	Transfer Learning, Convolutional Neural Networks	Intermediate
13	13 - Fine-Tune BERT for Classification	BERT and Masked Language Models, Text Classification	Advanced
14	14 - Build a RAG Pipeline	Retrieval Augmented Generation, Embeddings	Advanced

Framework Internals (tinygrad-inspired)

How DL frameworks actually work — from tensor ops to GPU execution.

#	Tutorial	Key Insight	Difficulty
25	25 - How DL Frameworks Work	Lazy eval → scheduling → kernel fusion → codegen → execution	Advanced
26	26 - The Minimal Op Set	ALL of deep learning is ~25 operations	Intermediate
27	27 - Tensor Internals Strides and Views	Strides, views, why reshape doesn’t copy data	Intermediate
28	28 - Build a Tensor Library	Broadcasting + autograd on arrays, not scalars	Advanced

Modern Generative Models

Understand how diffusion models generate samples.

#	Tutorial	Vault Foundation	Difficulty
29	29 - Diffusion Models from Scratch	Generative Models, VAE	Advanced

Reinforcement Learning

From policy gradients to PPO — the workhorse algorithms of RL.

#	Tutorial	Vault Foundation	Difficulty
30	30 - PPO from Scratch	Reinforcement Learning, Policy Gradients	Advanced

Meta: How to Train

#	Tutorial	Vault Foundation	Difficulty
24	24 - Neural Network Training Recipes	Gradient Descent, Bias-Variance Tradeoff	All levels

---

Suggested Learning Paths

Path A: “I want to understand how ML works” (from-scratch)

01 → 02 → 03 → 04 → 05 → 06 → 15 → 07

Path B: “I want to build language models” (Karpathy track)

15 → 16 → 17 → 18 → 19 → 23 → 21 → 08 → 20

Path C: “I want to train models on real data” (practical)

09 → 10 → 11 → 24 → 12 → 13 → 14

Path D: “I want to understand deep learning deeply” (internals)

07 → 15 → 18 → 19 → 22 → 08 → 20

Path E: “I want to understand how frameworks work” (tinygrad track)

15 → 27 → 26 → 28 → 25

---

How to Use These Tutorials

1. Read the vault note first — understand the concept
2. Work through the tutorial — run every code block, modify values, break things
3. Do the exercises — they force you to think, not just copy
4. Connect back — each tutorial links to related vault notes and next tutorials

The goal is not to type code. It’s to understand what every line does and why.