Artificial Intelligence (ML, DL, RAG, Agentic) Engineering

No upfront math wall. One Math Friday session per week, each timed to land exactly one week before the content that needs it - so every formula used later in the course is derived or visualized here first. Nothing is hand-waved.

• M1 · Linear Algebra I: vectors, dot products (three views), norms, cosine similarity, matrices as transformations of space, matrix multiplication four ways
• M2 · Linear Algebra II: rank, eigenvalues & eigenvectors, SVD and low-rank approximation (the idea that later becomes LoRA), projections and least squares
• M3 · Calculus: partial derivatives, the gradient, Jacobians, and the chain rule on computational graphs - which IS backpropagation, taught before backprop
• M4 · Probability & Statistics I: Bayes' rule with worked examples, random variables, the key distributions, covariance, the Central Limit Theorem demonstrated
• M5 · Probability & Statistics II: maximum likelihood derives MSE and cross-entropy, MAP derives L2 regularization; entropy, KL divergence, softmax and temperature
• M6 · Optimization: gradient descent to AdamW - momentum, RMSProp, learning-rate schedules and warmup, and why non-convex deep learning works anyway

You know Python. This week converts Python programmers into array programmers - vectorized thinking is the single biggest unlock for everything after.

• L1.1 · Kickoff, environment & NumPy I: the full course map, environment setup (uv, Jupyter, Colab, GPU check), the ndarray - shapes, strides, indexing, views vs copies
• L1.2 · NumPy II - vectorization & broadcasting: why Python loops are 100-1000x slower, broadcasting rules decoded, einsum introduced, numerical gotchas (overflow, log-sum-exp)
• L1.3 · pandas & data wrangling: DataFrames, groupby split-apply-combine, merging, datetime handling, a full cleaning workflow on a deliberately filthy dataset
• L1.4 · Visualization & EDA: Matplotlib's object-oriented API, seaborn, an EDA methodology (distributions, outliers, correlations, leakage hunting), and how charts lie
• L1.5 · Data for ML - the pipeline mindset: train/validation/test discipline, data leakage (six real stories), scaling, encoding, imbalanced data, leak-proof sklearn Pipelines

Skipping this is why most AI-course graduates can't debug a model. We don't skip it. Every algorithm: intuition, math, from-scratch NumPy implementation, scikit-learn, and when it breaks.

• L2.1 · What learning means + linear regression: loss minimization, hypothesis spaces, MSE derived from maximum likelihood, the normal equation derived two ways
• L2.2 · Gradient descent in anger: derive the gradients by hand, write the training loop, overfitting vs underfitting, bias-variance, Ridge and Lasso
• L2.3 · Classification - logistic regression: sigmoid, cross-entropy derived from MLE, softmax regression, MNIST solved with no neural network
• L2.4 · Evaluation: why accuracy lies - confusion matrices, precision/recall/F1, ROC and AUC from scratch, cross-validation, tuning discipline
• L2.5 · Trees, forests & boosting: Gini impurity by hand, random forests, XGBoost, and when gradient boosting still beats deep learning on tabular data
• L2.6 · kNN, SVM & Naive Bayes: the curse of dimensionality, kNN as the conceptual ancestor of RAG, the kernel trick, a spam filter built end-to-end
• L3.1 · Unsupervised learning - clustering: k-means from scratch, DBSCAN, Gaussian mixtures, segmentation and anomaly detection
• L3.2 · PCA & the embedding idea: PCA via eigendecomposition, eigenfaces, t-SNE/UMAP, and the central abstraction of modern AI - everything is a vector in a learned space
• L3.3 · End-to-end ML project: frame, EDA, baseline, iterate, tune, error analysis, write-up - done live by the instructor, then by you, assessed

The Karpathy progression: build an autograd engine from scratch in pure Python, then meet PyTorch and discover it's the same thing, industrial-strength. Nothing in a deep net remains a black box after these two weeks.

• L4.1 · From neuron to network: perceptrons and the XOR failure, why nonlinearity is everything (proved), MLP architecture with every shape annotated
• L4.2 · Backpropagation I - micrograd: build your own autograd engine in pure Python - computational graphs, local x upstream gradients, backward(), topological sort
• L4.3 · Backpropagation II & training dynamics: backprop through matrix ops, softmax + cross-entropy gradients, vanishing/exploding gradients, initialization, GELU, clipping
• L4.4 · PyTorch I - tensors & autograd: PyTorch tensor = NumPy array + GPU + autograd; everything maps 1:1 onto your micrograd, gradient-checked against your own engine
• L4.5 · PyTorch II - the real training stack: nn.Module, losses, optimizers, Datasets & DataLoaders, the canonical training loop from memory, checkpointing
• L4.6 · Making deep nets actually train well: dropout, BatchNorm and LayerNorm, AdamW, schedules and warmup, plus the debugging recipe - overfit one batch first
• L5.1 · Embeddings & recommender mini-project: nn.Embedding as learned representations, word2vec intuition, a MovieLens recommender with nearest-neighbour exploration

Tight by design: enough CNN depth to be dangerous and to make Vision Transformers meaningful later - convolution built by hand before any framework touches it.

• L5.2 · Convolutions from first principles: the sliding window worked by hand, Sobel edge detectors computed manually, stride, padding, receptive fields
• L5.3 · CNN architectures - the lineage: LeNet, AlexNet, VGG, Inception, ResNet - and why residual connections fix gradient flow (an idea that returns in transformers)
• L6.1 · Training CV systems in practice: augmentation as regularization, transfer learning, mixed precision, Grad-CAM to see what your network looks at
• L6.2 · Beyond classification: detection (YOLO deep dive), segmentation (U-Net), Vision Transformer preview, GANs & diffusion conceptual tour

The road to GPT. Every component of the transformer is built from scratch - tokenizer, attention, the block - before the full architecture is assembled from your own parts.

• L7.1 · Text as data + classical NLP baseline: TF-IDF and n-gram language models, perplexity - the baseline your GPT must beat
• L7.2 · Tokenization - the underrated layer: Byte-Pair Encoding stepped through by hand, byte-level BPE (GPT-2 style), special tokens, and tokenizer-driven failure modes
• L7.3 · Sequence models - RNN to LSTM: hidden state as memory, vanishing gradients, and the bottleneck problem that motivated attention
• L7.4 · Attention from scratch: queries, keys, values as soft dictionary lookup, why QK-transpose, why the scale factor, causal masks, multi-head - verified numerically
• L8.1 · The transformer, assembled: the Attention Is All You Need paper as your first full guided read, positional encodings and RoPE, pre-norm blocks, BERT vs GPT vs T5
• L8.2 · Pretrained transformers & Hugging Face: the pretrain/fine-tune paradigm, fine-tuning DistilBERT, sentence embeddings and semantic search - which becomes RAG

The centerpiece. You leave this phase having pretrained a GPT on data you chose and instruction-tuned it to follow orders - with every line of the model written by you.

• L9.1 · Language modeling - the objective: why next-token prediction is enough, LM data pipelines, context windows, perplexity, a bigram LM as the minimal start
• L9.2 · nanoGPT I - the model: the full GPT architecture in under 300 lines, written live and explained line by line - embeddings, causal attention, MLPs, residuals
• L9.3 · nanoGPT II - training for real: AdamW, cosine schedule with warmup, gradient accumulation, mixed precision, reading loss curves, scaling laws in one slide
• L9.4 · Sampling & evaluation: greedy, temperature, top-k and top-p implemented from scratch, why beam search is wrong for open-ended text, the KV cache built and benchmarked
• L10.1 · From GPT to ChatGPT - fine-tuning & alignment: SFT, chat templates and prompt masking, RLHF and DPO at intuition level, then instruction-tune your own GPT
• L10.2 · Fine-tuning real open models - LoRA & QLoRA: the open-model landscape (Llama, Mistral, Qwen), LoRA as the SVD callback, quantization, single-GPU fine-tuning
• L10.3 · Inside the model - interpretability: logit lens, attention-pattern reading, induction heads, and why hallucination is guaranteed by the training objective

Now - and only now - the API and agent layer. You know what's inside the box, so nothing here is magic: RAG, agents and evals built from scratch before any framework.

• L11.1 · LLM APIs like an engineer: streaming, structured output, tool-calling mechanics, retries and cost tracking, prompt engineering with engineering discipline
• L11.2 · RAG I: the full pipeline built raw - chunking, embedding models, FAISS vector search - with every intermediate artifact inspected, no frameworks
• L11.3 · RAG II - production quality: why naive RAG fails, hybrid BM25 + dense search, cross-encoder re-ranking, query rewriting, honest evaluation with golden test sets
• L11.4 · Agents & tool use: a minimal agent loop written from scratch, function calling, MCP, memory, and the failure modes - loops, cascades, runaway cost
• L12.1 · Evaluation & safety for LLM systems: LLM-as-judge and its biases, regression suites for prompts, prompt injection and defensive red-teaming, PII handling, tracing

Where this course beats every other AI course on the market: you deploy. Docker, CI/CD with eval gates, Kubernetes, monitoring - your models running as real, observable services.

• L12.2 · Experiment tracking & reproducibility: MLflow and Weights & Biases, data versioning, seeds and env pinning - reproduce a teammate's run from their config alone
• L12.3 · Serving models: FastAPI model services, LLM serving with vLLM and Ollama, quantized CPU inference, load testing p50/p95/p99
• L13.1 · Containers, CI/CD & Kubernetes for ML: CUDA images and layer-caching discipline, CI pipelines that gate deployment on eval thresholds, GPU scheduling
• L13.2 · Monitoring, drift & production reality: Prometheus and Grafana dashboards on your model service, data and concept drift, retraining strategy, five real postmortems
• L13.3 · Capstone working session + the field ahead: capstone build time, how to read papers and keep up, and what AI-engineer vs ML-engineer interviews actually test