Two groups of children are playing in a park: some are playing football, others are skipping rope. You need to stretch a rope between them to separate the two groups. You could put the rope in lots of places, and all of them would separate the groups for now. But if you stretch it too close to the footballers, the first child who drifts sideways chasing the ball ends up on the wrong side.
The smart choice is to put the rope as far as possible from both groups: right down the middle of the widest road between them.
Support Vectors
Support vector machines (SVMs) look for exactly this widest road. The interesting part is that it’s not everyone in the groups who decides where the road goes, only the few children closest to the border. Wherever the children at the far end of the park stand, the rope stays where it is. Those few examples at the border are called support vectors, which is where the method gets its name.
This gives SVMs an important advantage: because they base their decision on a handful of critical examples, they can generalise well even when data is scarce.
When A Straight Line Isn’t Enough
But what if the children are mixed together? Say the rope skippers form a ring in the middle and the footballers run around them. No straight rope can separate the two groups.
SVMs have an elegant answer. Picture red and blue marbles mixed on a table. If you hit the table from below and the red ones bounce a little higher than the blue ones, you could slide a flat sheet of paper between them in mid-air. An SVM mathematically “throws” the data into a higher-dimensional space and looks for a flat separator there. This trick is called the kernel trick.
SVMs And EEG
In EEG studies, the number of participants is often limited while the number of features is high. Because SVMs generalise well even with little data, they have long been one of the standard baselines in EEG research, from brain–computer interfaces to emotion recognition. When we try out a new method, an SVM is often our first point of comparison too.
References
- Cortes, C., Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297. DOI: 10.1007/BF00994018