Stanford CS221 | Autumn 2025 | Lecture 3: Learning II

Why It Matters

Linear classifiers provide fast, interpretable baselines for large‑scale prediction tasks, and mastering their loss functions is essential for building reliable, scalable AI products.

Key Takeaways

• Linear classification predicts discrete labels using weighted sums and bias.
• Decision boundary is defined where the logit equals zero.
• 0‑1 loss measures misclassification without considering confidence magnitude.
• Linear classifiers remain expressive in high‑dimensional feature spaces.
• Choosing appropriate loss functions shapes learning algorithm behavior.

Summary

The lecture introduces linear classification, extending the regression framework to predict discrete class labels. By representing inputs as vectors and applying a weighted sum plus bias, the model outputs a logit whose sign determines the predicted class, typically encoded as +1 or ‑1.

Key concepts include the hypothesis class of all possible weight‑bias pairs, the geometric decision boundary where the logit equals zero, and the role of loss functions. The instructor contrasts squared loss—unsuitable for classification because it penalizes confidence magnitude—with the 0‑1 loss that simply counts misclassifications, and later rewrites it in terms of the margin to facilitate optimization.

Illustrative examples plot two‑dimensional points (e.g., (1,2) and (2,0)) and show how shifting the bias moves the straight‑line decision boundary x₀‑x₁‑1=0. The speaker demonstrates computing logits, applying the sign test, and evaluating loss on individual examples, emphasizing that loss design directly shapes the learning algorithm.

The discussion underscores why linear classifiers remain a staple in high‑dimensional settings: they are computationally cheap, convex, and surprisingly expressive when features number in the thousands. Understanding loss selection and margin formulation lays groundwork for more advanced models such as logistic regression and neural networks, which dominate modern AI applications.

Original Description

For more information about Stanford’s Artificial Intelligence professional and graduate programs, visit: https://stanford.io/ai

To learn more about enrolling in this course, visit: https://online.stanford.edu/courses/cs221-artificial-intelligence-principles-and-techniques

Please follow along with the course schedule: https://stanford-cs221.github.io/autumn2025/

Follow the playlist: https://youtube.com/playlist?list=PLoROMvodv4rMeDqwS1yFl3j3sR_-MQNEN&si=bVivXjDfVEQKky1D

Teaching Team

Percy Liang, Associate Professor of Computer Science (and courtesy in Statistics)