Multimodal Recognition of Reading Activity in Transit Using Body-Worn Sensors
Andreas Bulling, Jamie A. Ward, Hans Gellersen
ACM Transactions on Applied Perception (TAP), 9 (1), pp. 1--21, 2012.
AbstractLinksBibTeXProject
Reading is one of the most well studied visual activities. Vision research traditionally focuses on understanding the perceptual and cognitive processes involved in reading. In this work we recognise reading activity by jointly analysing eye and head movements of people in an everyday environment. Eye movements are recorded using an electrooculography (EOG) system; body movements using body-worn inertial measurement units. We compare two approaches for continuous recognition of reading: String matching (STR) that explicitly models the characteristic horizontal saccades during reading, and a support vector machine (SVM) that relies on 90 eye movement features extracted from the eye movement data. We evaluate both methods in a study performed with eight participants reading while sitting at a desk, standing, walking indoors and outdoors, and riding a tram. We introduce a method to segment reading activity by exploiting the sensorimotor coordination of eye and head movements during reading. Using person-independent training, we obtain an average precision for recognising reading of 88.9% (recall 72.3%) using STR and of 87.7% (recall 87.9%) using SVM over all participants. We show that the proposed segmentation scheme improves the performance of recognising reading events by more than 24%. Our work demonstrates that the joint analysis of multiple modalities is beneficial for reading recognition and opens up discussion on the wider applicability of this recognition approach to other visual and physical activities.
@article{bulling12_tap,
title = {Multimodal {R}ecognition of {R}eading {A}ctivity in {T}ransit {U}sing {B}ody-{W}orn {S}ensors},
author = {Bulling, Andreas and Ward, Jamie A. and Gellersen, Hans},
year = {2012},
journal = {ACM Transactions on Applied Perception (TAP)},
volume = {9},
number = {1},
pages = {1--21},
doi = {10.1145/2134203.2134205},
}
Robust Recognition of Reading Activity in Transit Using Wearable Electrooculography
Andreas Bulling, Jamie A. Ward, Hans Gellersen, Gerhard Tröster
Proc. International Conference on Pervasive Computing (Pervasive), pp. 19-37, 2008.
AbstractLinksBibTeXProject
In this work we analyse the eye movements of people in transit in an everyday environment using a wearable electrooculographic (EOG) system. We compare three approaches for continuous recognition of reading activities: a string matching algorithm which exploits typical characteristics of reading signals, such as saccades and fixations; and two variants of Hidden Markov Models (HMMs) - mixed Gaussian and discrete. The recognition algorithms are evaluated in an experiment performed with eight subjects reading freely chosen text without pictures while sitting at a desk, standing, walking indoors and outdoors, and riding a tram. A total dataset of roughly 6 hours was collected with reading activity accounting for about half of the time. We were able to detect reading activities over all subjects with a top recognition rate of 80.2% (71.0% recall, 11.6% false positives) using string matching. We show that EOG is a potentially robust technique for reading recognition across a number of typical daily situations.
@inproceedings{bulling08_pervasive,
title = {Robust {R}ecognition of {R}eading {A}ctivity in {T}ransit {U}sing {W}earable {E}lectrooculography},
author = {Bulling, Andreas and Ward, Jamie A. and Gellersen, Hans and Tr{\"{o}}ster, Gerhard},
year = {2008},
booktitle = {Proc. International Conference on Pervasive Computing (Pervasive)},
pages = {19-37},
doi = {10.1007/978-3-540-79576-6_2},
keywords = {Activity Recognition, Electrooculography (EOG), Reading Activity, Recognition of Reading, Transit, wearable},
}
