Probability and Machine Learning

About this event

Why do we train classifiers on cross-entropy? Last time we showed that MSE isn't an arbitrary rule. It falls out of assuming Gaussian noise on a continuous target. So what happens when the target isn't a number at all, but a label: cat or dog, spam or not, one of ten digits? Same recipe, new distribution. Once you ask "what distribution generates a label?", cross-entropy basically derives itself, and so does the reason it has "entropy" in its name. In this focused hour, we'll trace exactly where cross-entropy comes from: • How binary cross-entropy emerges from a Bernoulli likelihood, and multiclass cross-entropy from a categorical one • Why "cross-entropy" is the right name: what minimizing it says about the gap between your model's predictions and the real data • Why plain MSE on 0/1 labels is the wrong tool, and how logistic regression and softmax fit into the picture Come with basic ML familiarity and your questions. This is meant to be interactive, not a lecture. Having been at the MSE session helps but isn't required. You'll leave seeing the loss behind nearly every classifier you've trained in a completely different light. One focused hour. Bring a notebook.