Publications
2026
Unsupervised Partner Design Enables Robust Ad-hoc Teamwork
Constantin Ruhdorfer, Matteo Bortoletto, Victor Oei, Anna Penzkofer, Andreas Bulling
Proc. International Conference on Machine Learning (ICML), 2026.
AbstractLinksBibTeXProject Spotlight
We introduce Unsupervised Partner Design (UPD), a population-free multi-agent reinforcement learning method for robust ad-hoc teamwork. UPD generates training partners on-the-fly and selects them adaptively based on a learnability criterion, removing the need for pre-trained partner populations or manual parameter tuning. We show that this simple mechanism enables effective partner diversity and can be extended to joint partner-environment selection when a procedural level generator is available. Across Level-Based Foraging, Overcooked-AI, and the Overcooked Generalisation Challenge, UPD consistently achieves strong performance compared to both population-based and population-free baselines. In a human-AI user study, agents trained with UPD achieve higher returns and are rated as more adaptive, more human-like, and less frustrating than all evaluated baseline methods.
@inproceedings{ruhdorfer26_icml,
title = {Unsupervised Partner Design Enables Robust Ad-hoc Teamwork},
author = {Ruhdorfer, Constantin and Bortoletto, Matteo and Oei, Victor and Penzkofer, Anna and Bulling, Andreas},
year = {2026},
booktitle = {Proc. International Conference on Machine Learning (ICML)},
shorttitle = {{UPD}},
}
2025
SSPictR: A Biologically-plausible Image Representation
Anna Penzkofer, Karim Habashy, Chris Eliasmith, Andreas Bulling
ECAI Workshop on Artificial Intelligence and Cognition (AIC), pp. 1--13, 2025.
AbstractLinksBibTeXProject Oral Presentation
Finding interpretable and generalisable representations of natural images has the potential to increase performance on various computer vision tasks, particularly those that require semantic information and spatial understanding, such as image segmentation or scene recognition. Drawing inspiration from a cognitive modelling framework, we propose SSPictR - a biologically plausible image representation based on spatial semantic pointers (SSPs). SSPictR encodes semantic labels of objects and their spatial locations extracted from segmentation maps. It only requires a single vector to capture a compressed but fully decodable neuro-symbolic representation of an image. We demonstrate the biological plausibility of SSPictR by performing representation similarity analysis, finding a significant correlation with fMRI data recorded from the early visual cortex. We further highlight the effectiveness and out-of-domain generalisability of SSPictR representations by training a compact model for scene recognition on standard benchmark datasets. Our simple neural network achieves performance on par with previous work, while having more than three times fewer trainable parameters. Taken together, SSPictR bridges the gap between biological plausibility and effective representations for tasks in computer vision and beyond.
@inproceedings{penzkofer25_aic,
title = {SSPictR: A Biologically-plausible Image Representation},
author = {Penzkofer, Anna and Habashy, Karim and Eliasmith, Chris and Bulling, Andreas},
year = {2025},
booktitle = {ECAI Workshop on Artificial Intelligence and Cognition (AIC)},
pages = {1--13},
}
A Cognitively Plausible Visual Working Memory Model
Anna Penzkofer, Michael Furlong, Chris Eliasmith, Andreas Bulling
Proc. Annual Meeting of the Cognitive Science Society (CogSci), pp. 1-6, 2025.
AbstractLinksBibTeXProject
Visual working memory (VWM) plays a fundamental role in cognitive processes, such as perception, attention, and reasoning. However, existing approaches to modelling VWM are not integrated into cognitive architectures and lack interpretability with respect to their parameters. To address this limitation, we propose a novel VWM model based on the well-established Semantic Pointer Architecture (SPA). In contrast to previous works, our model is the first to integrate a VWM model with a cognitive attention model. It only requires three interpretable hyper-parameters: spatial capacity, feature certainty, and memory decay. We experimentally show that our base model without memory decay replicates the set-size effect and swap errors of human data on a continuous reproduction task. More importantly, we show that by introducing a memory decay, we can achieve a statistically significant (p << 0.001) improvement in model fit, suggesting a potentially important role of memory decay in VWM. Further, our VWM model can be easily extended to model pre- and post-cue conditions, consistently achieving KL divergence between modelled and human performance of less than 0.05.
@inproceedings{penzkofer25_cogsci,
title = {A {Cognitively} {Plausible} {Visual} {Working} {Memory} {Model}},
author = {Anna Penzkofer and Michael Furlong and Chris Eliasmith and Andreas Bulling},
year = {2025},
booktitle = {Proc. Annual Meeting of the Cognitive Science Society (CogSci)},
pages = {1-6},
doi = {https://escholarship.org/uc/item/3928d5s4},
}
The Overcooked Generalisation Challenge: Evaluating Cooperation with Novel Partners in Unknown Environments Using Unsupervised Environment Design
Constantin Ruhdorfer, Matteo Bortoletto, Anna Penzkofer, Andreas Bulling
Transactions on Machine Learning Research (TMLR), pp. 1-25, 2025.
AbstractLinksBibTeXProject
We introduce the Overcooked Generalisation Challenge (OGC) - a new benchmark for evaluating reinforcement learning (RL) agents on their ability to cooperate with unknown partners in unfamiliar environments. Existing work typically evaluated cooperative RL only in their training environment or with their training partners, thus seriously limiting our ability to understand agents' generalisation capacity - an essential requirement for future collaboration with humans. The OGC extends Overcooked-AI to support dual curriculum design (DCD). It is fully GPU-accelerated, open-source, and integrated into the minimax DCD benchmark suite. Compared to prior DCD benchmarks, where designers manipulate only minimal elements of the environment, OGC introduces a significantly richer design space: full kitchen layouts with multiple objects that require the designer to account for interaction dynamics between agents. We evaluate state-of-the-art DCD algorithms alongside scalable neural architectures and find that current methods fail to produce agents that generalise effectively to novel layouts and unfamiliar partners. Our results indicate that both agents and curriculum designers struggle with the joint challenge of partner and environment generalisation. These findings establish OGC as a demanding testbed for cooperative generalisation and highlight key directions for future research.
@article{ruhdorfer25_tmlr,
title = {The Overcooked Generalisation Challenge: Evaluating Cooperation with Novel Partners in Unknown Environments Using Unsupervised Environment Design},
author = {Constantin Ruhdorfer and Matteo Bortoletto and Anna Penzkofer and Andreas Bulling},
year = {2025},
journal = {Transactions on Machine Learning Research (TMLR)},
pages = {1-25},
url = {https://openreview.net/forum?id=K2KtcMlW6j},
}
2024
VSA4VQA: Scaling A Vector Symbolic Architecture To Visual Question Answering on Natural Images
Anna Penzkofer, Lei Shi, Andreas Bulling
Proc. Annual Meeting of the Cognitive Science Society (CogSci), 2024.
AbstractLinksBibTeXProject Oral Presentation
While Vector Symbolic Architectures (VSAs) are promising for modelling spatial cognition, their application is currently limited to artificially generated images and simple spatial queries. We propose VSA4VQA – a novel 4D implementation of VSAs that implements a mental representation of natural images for the challenging task of Visual Question Answering (VQA). VSA4VQA is the first model to scale a VSA to complex spatial queries. Our method is based on the Semantic Pointer Architecture (SPA) to encode objects in a hyper-dimensional vector space. To encode natural images, we extend the SPA to include dimensions for object’s width and height in addition to their spatial location. To perform spatial queries we further introduce learned spatial query masks and integrate a pre-trained vision-language model for answering attribute-related questions. We evaluate our method on the GQA benchmark dataset and show that it can effectively encode natural images, achieving competitive performance to state-of-the-art deep learning methods for zero-shot VQA.
@inproceedings{penzkofer24_cogsci,
title = {{VSA4VQA}: {Scaling} {A} {Vector} {Symbolic} {Architecture} {To} {Visual} {Question} {Answering} on {Natural} {Images}},
author = {Anna Penzkofer and Lei Shi and Andreas Bulling},
year = {2024},
booktitle = {Proc. Annual Meeting of the Cognitive Science Society (CogSci)},
volume = {46},
url = {https://escholarship.org/uc/item/26j7v1nf.},
}
Int-HRL: Towards Intention-based Hierarchical Reinforcement Learning
Anna Penzkofer, Simon Schaefer, Florian Strohm, Mihai Bâce, Stefan Leutenegger, Andreas Bulling
Neural Computing and Applications (NCAA), 36, pp. 1--7, 2024.
AbstractLinksBibTeXProject
While deep reinforcement learning (RL) agents outperform humans on an increasing number of tasks, training them requires data equivalent to decades of human gameplay. Recent hierarchical RL methods have increased sample efficiency by incorporating information inherent to the structure of the decision problem but at the cost of having to discover or use human-annotated sub-goals that guide the learning process. We show that intentions of human players, i.e., the precursor of goal-oriented decisions, can be robustly predicted from eye gaze even for the long-horizon sparse rewards task of Montezuma’s Revenge – one of the most challenging RL tasks in the Atari2600 game suite. We propose Int-HRL: Hierarchical RL with intention-based sub-goals that are inferred from human eye gaze. Our novel sub-goal extraction pipeline is fully automatic and replaces the need for manual sub-goal annotation by human experts. Our evaluations show that replacing hand-crafted sub-goals with automatically extracted intentions leads to an HRL agent that is significantly more sample efficient than previous methods.
@article{penzkofer24_ncaa,
title = {Int-HRL: Towards Intention-based Hierarchical Reinforcement Learning},
author = {Anna Penzkofer and Simon Schaefer and Florian Strohm and Mihai Bâce and Stefan Leutenegger and Andreas Bulling},
year = {2024},
journal = {Neural Computing and Applications (NCAA)},
volume = {36},
pages = {1--7},
doi = {10.1007/s00521-024-10596-2},
}
MultiMediate'24: Multi-Domain Engagement Estimation
Philipp Müller, Michal Balazia, Tobias Baur, Michael Dietz, Alexander Heimerl, Anna Penzkofer, Dominik Schiller, François Brémond, Jan Alexandersson, Elisabeth André, Andreas Bulling
Proceedings of the 32nd ACM International Conference on Multimedia, pp. 11377 - 11382, 2024.
AbstractLinksBibTeXProject
Estimating the momentary level of participant's engagement is an important prerequisite for assistive systems that support human interactions. Previous work has addressed this task in within-domain evaluation scenarios, i.e. training and testing on the same dataset. This is in contrast to real-life scenarios where domain shifts between training and testing data frequently occur. With MultiMediate'24, we present the first challenge addressing multi-domain engagement estimation. As training data, we utilise the NOXI database of dyadic novice-expert interactions. In addition to within-domain test data, we add two new test domains. First, we introduce recordings following the NOXI protocol but covering languages that are not present in the NOXI training data. Second, we collected novel engagement annotations on the MPIIGroupInteraction dataset which consists of group discussions between three to four people. In this way, MultiMediate'24 evaluates the ability of approaches to generalise across factors such as language and cultural background, group size, task, and screen-mediated vs. face-to-face interaction. This paper describes the MultiMediate'24 challenge and presents baseline results. In addition, we discuss selected challenge solutions.
@inproceedings{mueller24_mm,
title = {MultiMediate'24: Multi-Domain Engagement Estimation},
author = {Philipp M{\"{u}}ller and Michal Balazia and Tobias Baur and Michael Dietz and Alexander Heimerl and Anna Penzkofer and Dominik Schiller and François Brémond and Jan Alexandersson and Elisabeth André and Andreas Bulling},
year = {2024},
booktitle = {Proceedings of the 32nd ACM International Conference on Multimedia},
pages = {11377 - 11382},
doi = {10.1145/3664647.3689004},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
}
2023
Int-HRL: Towards Intention-based Hierarchical Reinforcement Learning
Anna Penzkofer, Simon Schaefer, Florian Strohm, Mihai Bâce, Stefan Leutenegger, Andreas Bulling
Proc. Adaptive and Learning Agents Workshop (ALA), pp. 1--7, 2023.
AbstractLinksBibTeXProject
While deep reinforcement learning (RL) agents outperform humans on an increasing number of tasks, training them requires data equivalent to decades of human gameplay. Recent hierarchical RL methods have increased sample efficiency by incorporating information inherent to the structure of the decision problem but at the cost of having to discover or use human-annotated sub-goals that guide the learning process. We show that intentions of human players, i.e. the precursor of goal-oriented decisions, can be robustly predicted from eye gaze even for the long-horizon sparse rewards task of Montezuma’s Revenge – one of the most challenging RL tasks in the Atari2600 game suite. We propose Int-HRL: Hierarchical RL with intention-based sub-goals that are inferred from human eye gaze. Our novel sub-goal extraction pipeline is fully automatic and replaces the need for manual sub-goal annotation by human experts. Our evaluations show that replacing hand-crafted sub-goals with automatically extracted intentions leads to a HRL agent that is significantly more sample efficient than previous methods.
@inproceedings{penzkofer23_ala,
title = {Int-HRL: Towards Intention-based Hierarchical Reinforcement Learning},
author = {Anna Penzkofer and Simon Schaefer and Florian Strohm and Mihai Bâce and Stefan Leutenegger and Andreas Bulling},
year = {2023},
booktitle = {Proc. Adaptive and Learning Agents Workshop (ALA)},
pages = {1--7},
}
2021
ConAn: A Usable Tool for Multimodal Conversation Analysis
Anna Penzkofer, Philipp Müller, Felix Bühler, Sven Mayer, Andreas Bulling
Proc. ACM International Conference on Multimodal Interaction (ICMI), pp. 341-351, 2021.
AbstractLinksBibTeXProject
Multimodal analysis of group behavior is a key task in human-computer interaction, as well as the social and behavioral sciences, but is often limited to more easily controllable laboratory settings or requires elaborate multi-sensor setups and time-consuming manual data annotation. We present ConAn – a usable tool to explore and automatically analyze non-verbal behavior of multiple persons during natural group conversations. In contrast to traditional multi-sensor setups, our tool only requires a single 360° camera and uses state-of-the-art computer vision methods to automatically extract behavioral indicators, such as gaze direction, facial expressions, and speaking activity. Thus, our tool allows for easy and fast deployment supporting researchers in understanding both individual behavior and group interaction dynamics, but also in quantifying user-object interactions. We illustrate the benefits of our tool on three sample use cases: general conversation analysis, assessment of collaboration quality, and impact of technology on audience behavior. Taken together, ConAn represents an important step towards democratizing automatic conversation analysis in HCI and beyond.
@inproceedings{penzkofer21_icmi,
title = {ConAn: A Usable Tool for Multimodal Conversation Analysis},
author = {Penzkofer, Anna and Müller, Philipp and Bühler, Felix and Mayer, Sven and Bulling, Andreas},
year = {2021},
booktitle = {Proc. ACM International Conference on Multimodal Interaction (ICMI)},
pages = {341-351},
doi = {10.1145/3462244.3479886},
video = {https://www.youtube.com/watch?v=H2KfZNgx6CQ},
}
