Imagine you’re watching a long TV series. When a character in the new episode says “You were the one who told me”, you need to remember what happened three episodes ago to make sense of the scene. On the other hand, you don’t need to remember the colour of someone’s jacket in that episode. Your brain picks what matters, keeps it, and lets the rest go.
Sequential data, such as words in a sentence or EEG measurements arriving one after another in time, works the same way. To understand the current step, you need to know what came before. LSTM networks (Long Short-Term Memory) were designed for this.
A Notebook And Three Gates
Think of an LSTM as a viewer holding a notebook. With every new scene, it makes three decisions about that notebook:
- Forget gate: Which notes no longer matter? The jacket colour can go.
- Input gate: What happened in this scene that’s worth writing down? The argument between two characters should be noted.
- Output gate: Which notes should I look at to understand the current scene?
These “gates” are really dials. During training, the network learns for itself when and how far to open or close each one.
Why Does It Matter?
Simpler sequential networks before LSTM had a big problem: forgetfulness. Information from just a few steps back faded quickly. The notebook idea let important information survive for a long time. That’s why LSTMs were widely used for years in speech recognition, translation and weather forecasting.
LSTMs And EEG
EEG is a long time series with hundreds of measurements per second. To catch an emotional reaction or the moment attention slips, you can’t look at a single instant; you need to see how the signal changes over time. LSTMs can learn such patterns by reading the signal step by step and “remembering” the important changes.
In the next post, we’ll look at the Transformer architecture, which looks at the whole sequence at once instead of reading it in order.
References
- Hochreiter, S., Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9(8), 1735–1780. DOI: 10.1162/neco.1997.9.8.1735