ROAMM: A Benchmark Dataset for Multimodal Human Attention Decoding and EEG-to-Text Modeling During Naturalistic Reading

Abstract (source: OpenReview · © authors)

We present Reading Observed At Mindless Moments (ROAMM), a multimodal dataset comprising 50 hours of simultaneous EEG and eye-tracking recordings collected during naturalistic multi-page reading from 44 participants. ROAMM includes synchronized physiological recordings, eye-movement events, page-level comprehension scores, and span-level mind-wandering (MW) annotations obtained using a retrospective self-report paradigm. We introduce a standardized leave-one-subject-out benchmark for MW detection and achieve up to 0.609 AUROC using supervised models. We additionally evaluate EEG-to-text decoding on reading segments with and without MW labels, showing that decoding performance decreases during MW episodes. ROAMM enables research on MW detection, EEG-to-text decoding, multimodal representation learning, and attention-related degradation of language representations during naturalistic reading.

Keywords