LibriBrain Word Classification

Sandbox competition. This is a demonstration of the platform built on synthetic data with the same shape as the PNPL 2026 challenge. Submit anything — nothing here counts.

The challenge

Can we tell which word someone heard from a fraction of a second of brain activity, recorded non-invasively?

Participants receive magnetoencephalography (MEG) recordings of people listening to audiobooks, cut into one-second windows around each spoken word. For every window, predict a probability distribution over a fixed 50-word vocabulary. Models are ranked by how often the true word lands in their top ten guesses — balanced across words, so the frequent "the" and "and" don't dominate.

Tracks

Track	Subjects	What it tests
Deep	1 deeply-sampled subject	How far can within-subject decoding go with lots of data?
Broad	32 held-out subjects	Do models generalise to brains they have never seen?

You can compete in either or both. Teams are shared across tracks and each track has its own leaderboard.

Why it matters

Non-invasive speech decoding is a stepping stone towards communication aids for people who cannot speak. Shared benchmarks with honest, held-out evaluation are how the field makes reliable progress — which is why the final ranking uses a private split that nobody has seen.

Getting started

1. Read the Rules and join the competition.
2. Grab the data and the sample submission from the Data tab.
3. Install the helper library: pip install pnpl and follow the quick-start.
4. Upload your first CSV from Submit predictions, or use the API.

Metric

Submissions are ranked by top-10 balanced accuracy (BalAcc@10). For each word $w$ in the vocabulary $V$ ($|V| = 50$):

$$\text{recall@10}(w) = \frac{\#\{\, i : y_i = w \text{ and } w \in \text{top}_{10}(\hat p_i) \}}{\#\{\, i : y_i = w \}}$$ $$\text{BalAcc@10} = \frac{1}{|V|} \sum_{w \in V} \text{recall@10}(w)$$

Ties are broken by BalAcc@1, then by the earlier submission. Hosts also track OVMI (out-of-vocabulary mutual information, in bits) for analysis.

Chance level for BalAcc@10 is $10/50 = 0.20$.

Public and private splits

The test set is split into a public part (≈⅓ of rows), scored on every submission and shown on the public leaderboard, and a private part (≈⅔) used for the final ranking. The private leaderboard is published by the hosts once winning solutions have been verified.

Submission format

One row per test window, in the canonical order from LibriBrainCompetitionHoldout.indices:

index,the,and,of,to,a,...,then
0,0.021,0.034,0.018,...
1,0.017,0.009,0.044,...

• index — the canonical row index (integer).
• One probability column per vocabulary word (column names are case-insensitive).
• Probabilities need not sum to one; only their ranking matters.
• .csv, .csv.gz and .zip (containing a single CSV) are accepted.

Final submissions

Select up to two submissions per track for final scoring on the My submissions tab. If you select none, your two best public submissions are used automatically.

• Start — data and leaderboards open.
• Entry & team-merger deadline — one week before the end. You must accept the rules before this date to take part.
• Final submission deadline — 23:59 UTC on the end date.
• Verification — the top teams are asked to share code and checkpoints so we can reproduce their results.
• Results — the private leaderboard is published after verification, together with the winners' write-ups.

All deadlines are at 23:59 UTC unless stated otherwise.

Prizes are awarded per track to the top three verified teams:

Place	Deep	Broad
1st	£750	£750
2nd	£500	£500
3rd	£250	£250

Winners are invited to present at the workshop and to co-author the competition paper. One prize per team per track.