DSML / 03 / CLASSIFICATION
Same points. Different decisions.
A classifier divides feature space into predicted categories. Compare the shapes those divisions take—and ask whether a good fit to training data carries over to unseen points.
Start here: Blue circles are class 0 and orange triangles are class 1. The pale background shows the model’s predicted class. A dark ring marks an incorrect prediction. Switch between training and test points; test observations were held out when fitting the models.
Loading the interactive activity… JavaScript is needed for the controls; you can still read the explanations below.
All models use the same 120 training and 80 test observations for each example. Changing the classifier keeps that split fixed.
Explore the numerical data
Challenge the boundary
- On Concentric circles, compare Logistic regression with SVM with RBF kernel. Which boundary shapes can separate an inner group from an outer ring?
- Choose Two moons with higher noise. Compare decision-tree depths 1, 3 and 8. Record both training and test accuracy. Does a deeper tree always improve the test result?
- Switch between training and test points for the same model. Find the ringed errors. Where do they concentrate, and what might make those cases ambiguous?
Training performance is only part of the story
A model can learn details of the training observations that do not generalise. Test accuracy measures the fraction of held-out points classified correctly in this particular split. It is a useful check, but one split of a small synthetic dataset does not establish a reliable ranking of algorithms.
These boundaries are reproducible examples computed with scikit-learn, rather than models retrained in your browser. Each example has 200 synthetic observations and a fixed, stratified 60/40 train/test split. Feature scaling is fitted on the training set only. The coloured regions show hard class predictions on a grid, not calibrated confidence.
What changes between the models?
Logistic regression makes a straight boundary with these two features. The RBF-kernel SVM can make curved boundaries. Decision trees split along feature axes; greater depth permits finer partitions. The random forest combines 50 trees, each limited to depth 5. Other settings are fixed so the comparisons stay manageable.
For the moons and circles, noise perturbs point coordinates around the shapes. The separated-groups example adds coordinate noise to a fixed two-class dataset.
Connect it to movement
A classifier might distinguish predefined gestures using wearable sensor features. In a real study, splitting recordings from the same person across training and test sets may give an optimistic picture of performance on new users. The relevant validation depends on how the model will be used.
Developed from my decision-boundary teaching app and the scikit-learn classifier comparison used in class. The scikit-learn example is credited to its developers.
Original scikit-learn classifier comparison · Third-party licence