Learning Representations from Gaze Dynamics

Description: Representation learning has been shown to be an effective approach to capture semantic patterns from input data. The learnt representation or embeddings can generalise to different scenarios, datasets and downstream tasks, reducing the effort researchers have to put into collecting huge datasets for each specific task. Representation learning has been widely investigated in NLP and CV. For example, word2vec uses a vector to represent each word.

Eye gaze data contains rich information of human affective intention as well as spontaneous actions. However, learning the representation of eye gaze data still remains under-explored. This project aims to build gaze representation learning models based on deep learning techniques, such as transformer and autoencoder architecture.

Supervisor: Guanhua Zhang

Distribution: 10% Literature, 20% Data preparation, 40% Implementation, 30% Analysis and Evaluation

Requirements: Strong programming skills, practical experience in data processing and deep learning

Literature: Ashish Vaswani et al. 2017. Attention is all you need. Advances in neural information processing systems 30.

Zecheng He et al. 2020. Actionbert: Leveraging user actions for semantic understanding of user interfaces. arXiv:2012.12350.