1. Foundations

This chapter goes from running a model to rebuilding it. You run and serve SmolLM2-135M-Instruct with MAX, then rewrite it from scratch, one part per section, until your version produces the same logits as MAX’s and generates text with a KV cache.

SectionYou learn
1.1 Run and serve an LLMAutoPipeline, AutoModel, max generate, max serve, and the two numbers that describe speed.
1.2 Tensors and eager executionmax.tensor, max.functional, and max.nn: eager execution and compilation.
1.3 Tokens, embeddings, and weightsWhat a config and a checkpoint hold, and how to build the token embedding from them.
1.4 Building blocksRMSNorm, the SwiGLU MLP, and rotary position embeddings.
1.5 AttentionCausal grouped-query attention, first from basic operations and then with flash attention.
1.6 The full modelAssembling 30 layers and matching MAX’s logits.
1.7 Generation and samplingThe generation loop, temperature, top-k, and top-p.
1.8 The KV cacheCaching keys and values so that decoding stops redoing work.

Every section is a notebook that runs in a few minutes, on a GPU if you have one and on a CPU otherwise. Sections 1.2 to 1.8 add code to the minillm package as they go, so each section can start from the previous ones without rerunning them.

By the end, you have a working LLM inference stack that you wrote yourself, and you know the parts of MAX it is built from. Chapter 2 splits that model across several devices.