A code-first course on building scalable data systems with distributed algorithms. Part of the Hands-On AI Science series, designed around Innovation-First Learning principles.
Modern AI has outgrown a single machine. Training frontier models, processing web-scale data, and coordinating many agents that act and learn in parallel all demand distributed infrastructure. This course fuses theoretical concepts with engineering ideas, so students can design systems that stay correct, fast, and cost-efficient as problems grow.
Core concepts, models, and ideas behind scaling intelligence: the MapReduce computation model, data partitioning and shuffling, distributed SGD and parameter servers, model and data parallelism, and multi-agent coordination, communication, and emergent behavior.
PySpark, Databricks, Hadoop MapReduce, Horovod, DeepSpeed, Ray, PettingZoo, Gymnasium, Stable Baselines3, and cloud compute (AWS/Azure).
A specific course syllabus is built for each audience: graduate or undergraduate, across engineering, digital health, or computer science.
As AI and mature libraries handle standard tasks, professional developers must focus on innovation. Student projects tackle new use cases by generating unique data and fine-tuning task-specific scalable AI systems.
Students begin their projects while learning the material and enrich them as new concepts arrive. Each team gives several in-class presentations for discussion and feedback.
MapReduce Fundamentals
Hadoop, word count, mapper/reducer
Spark & Distributed DataFrames
PySpark, Databricks, transformations
Advanced MapReduce Algorithms
Joins, sorting, graph algorithms at scale
Distributed ML Pipelines
Spark MLlib, feature engineering
Project Proposal Presentations
Student proposals, peer feedback
Distributed Deep Learning Training
Horovod, DeepSpeed, data parallelism
Parameter Servers & Model Parallelism
Ray, gradient synchronization
Interim Project Presentations
Progress demos, instructor feedback
Multi-Agent RL Foundations
Gymnasium, Stable Baselines3
Multi-Agent Environments
PettingZoo, cooperative/competitive
Communication & Emergent Behavior
Agent coordination, shared rewards
Scalable Inference & Optimization
Model distillation, quantization, ONNX
Final Project Presentations
Live demos, peer evaluation
