Publications

Explore a collection of publications and projects, from diverse fields, that cite Pupil Labs and use Pupil Labs eye tracking hardware and software in their research.

Filters

Year

42
62
54
47
40
49
25
9

Fields

72
44
40
36
27
24
23
23
22
21
17
15
14
13
13
12
11
10
10
9
8
6
6
5
5
5
5
5
4
3
3
3
3
3
2
2
2
2
2
2
2
2
2
2
2
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
0-0 of 328 publications
What Are You Looking At? Using Eye Tracking Glasses to Monitor Toddler Attention in Natural Learning Situations.
2021
Psychology
Altvater-Mackensen, Nicole
In ACM Symposium on Eye Tracking Research and Applications, pp. 1-4. 2021
Investigating eye gaze offers a window into the learning mind as eye movements are linked to attention and cognitive processing (Yarbus, 1967; Hyönä et al., 2003). Because eye tracking is non-invasive and does not require and overt behavioral response, it is a suitable tool to use with infants and toddlers (for a review on the use of eye tracking in infancy research see Gredebäck et al., 2010). Researchers developed widely applied paradigms in which infants’ looking is used to infer the discrimination, categorization or recognition of different stimuli (Michnick Golinkoff et al., 2013; Oakes, 2010). A prominent example in research on language development is the looking while listening procedure (Fernald et al., 2008), in which a child is presented with two images side-by-side of which one is labelled. Measuring how long it takes the child to orient towards the labelled object is taken as a measure of word recognition. Within the different paradigms employed, researchers assume that gaze direction coincides with attention and that it informs us about what children are interested in, what information they pick up, which predictions they make and which expectations they have (for a critical review on the conceptual foundations of preferential looking paradigms see Tafreshi et al., 2014). Most eye tracking studies to date use remote, stationary eye trackers, providing objective measures of eye gaze in a pre-defined, calibrated space. They mostly present stimulus material such as pictures and videos on screen and monitor infants’ eye movements and/or pupil dilation to assess their perception and understanding of these stimuli. This research allows for tight experimental control and provides important insights into the specifics of cognitive processing. However, the controlled set-up comes at a cost: it allows to automatically assess how infants explore a predefined visual space, but gaze mapping is limited to this visual space and the child is required to keep a certain distance and angle to the eye tracker to allow automatic detection of the pupil. Also, when the child grabs an object or points at something, gaze might be lost due to the arm occluding the eye tracker. Such constraints restrict the use of remote eye tracking in behavioral research. In addition, studies usually use very controlled – and compared to the real world – reduced stimuli. This might limit the ecological validity of the results obtained (for a detailed discussion see Ladouce et al., 2017). Recent approaches further highlight the embodiment of cognition in development (Laakso, 2011) and the role of social interaction for cognitive processing and learning (Csibra & Gergely, 2009). Against this background, it seems necessary to investigate infants’ processing in more naturalistic and complex settings to fully describe and understand the mechanisms and processes of cognitive development. Yet, recording eye gaze from children in natural situations is a difficult endeavor. While recent developments include eye tracking glasses which allow adult participants to freely move around and interact, the commercially available systems do not fit on an infant or toddler head. Some researchers have turned to small head-mounted cameras (e.g., Smith et al., 2011) to study infant cognition in interaction with objects and people. In this approach, the child’s field of view is recorded and from what is visible in the scene, it is inferred what the child is looking at. This research has, for instance, illustrated that children are most likely to learn a novel label when it is uttered at a time when its referent is dominant in the child’s view (Yu & Smith, 2012). However, head-mounted cameras do not track the child’s gaze. Image processing algorithms allow to automatically detect, e.g., objects of interest and facesin the scene and to calculate their relative (visual) salience. But if several objects (or faces or both) are visible in close proximity, it remains unclear what exactly the child is fixating at a specific point in time. Neither can it be determined at which part of an object or face the child is looking, for instance whether the child is attending to the eyes or the mouth (as, e.g., investigated using stationary eye tracking in Lewkowicz & Hansen-Tift, 2012). Other researchers have therefore developed mobile eye tracking systems employing similar components as adult eye tracking glasses but attaching them to a light-weight headgear (Franchak et al., 2011). One major challenge, however, remains in these systems: How can we determine that a look falls within a specific area of interest? In screen-based systems, areas of interest can be defined in terms of fixed coordinates because the eye tracker and the screen on which stimuli are presented are stationary. In contrast, the field of view constantly changes in mobile systems due to movement of the participant (or movement of objects/people in the scene). So far, this mapping problem has been solved through manual coding of scene data – an approach similar to the coding of looking data from video cameras that is time- and resource-intensive.
An enhanced Bouma model fits a hundred people's visual crowding.
2021
Psychology
Kurzawski, Jan W., Augustin Burchell, Darshan Thapa, Najib J. Majaj, Jonathan A. Winawer, Denis G
bioRxiv
Crowding is the failure to recognize an object due to surrounding clutter. Its strength varies across the visual field and individuals. To characterize the statistics of crowding—ultimately to relate psychophysics of crowding to physiology—we measured radial crowding distance and acuity of 105 observers along the four cardinal meridians of the visual field. Fitting the well-known Bouma law — crowding distance depends linearly on radial eccentricity — explains 52% of the variance in log crowding distance, cross-validated. Our enhanced Bourma model, with factors for observer, meridian, and target kind, explains 72% of the variance, again cross-validated. The meridional factors confirm previously reported asymmetries. We find a 0.62 horizontal:vertical advantage, a 0.92 lower:upper advantage, and a 0.82 right:left advantage. Crowding distance and acuity have a correlation of 0.41 at the fovea, which drops to 0.23 at ±5 deg along the horizontal midline. Acuity and crowding represent the size and spacing limits of perception. Since they are dissociated in clinical populations (Song et al., 2014; Strappini et al., 2017) and shown here to be only moderately correlated in our sample of mostly university students, clinical testing to predict real-world performance should consider measuring both. In sum, enhancing the Bouma law with terms for meridian, observer, and target kind provides an excellent fit to our 105-person survey of crowding.
Towards Robust Robot Control in Cartesian Space Using an Infrastructureless Head-and Eye-Gaze Interface.
2021
HRI
Wöhle, Lukas, Marion Gebhard
Sensors 21, no. 5
This paper presents a lightweight, infrastructureless head-worn interface for robust and real-time robot control in Cartesian space using head- and eye-gaze. The interface comes at a total weight of just 162 g. It combines a state-of-the-art visual simultaneous localization and mapping algorithm (ORB-SLAM 2) for RGB-D cameras with a Magnetic Angular rate Gravity (MARG)-sensor filter. The data fusion process is designed to dynamically switch between magnetic, inertial and visual heading sources to enable robust orientation estimation under various disturbances, e.g., magnetic disturbances or degraded visual sensor data. The interface furthermore delivers accurate eye- and head-gaze vectors to enable precise robot end effector (EFF) positioning and employs a head motion mapping technique to effectively control the robots end effector orientation. An experimental proof of concept demonstrates that the proposed interface and its data fusion process generate reliable and robust pose estimation. The three-dimensional head- and eye-gaze position estimation pipeline delivers a mean Euclidean error of 19.0 ± 15.7 mm for head-gaze and 27.4 ± 21.8 mm for eye-gaze at a distance of 0.3–1.1 m to the user. This indicates that the proposed interface offers a precise control mechanism for hands-free and full six degree of freedom (DoF) robot teleoperation in Cartesian space by head- or eye-gaze and head motion.
Gaze-based intention estimation for shared autonomy in pick-and-place tasks.
2021
HRI
Fuchs, Stefan, Anna Belardinelli
Frontiers in Neurorobotics 15
Shared autonomy aims at combining robotic and human control in the execution of remote, teleoperated tasks. This cooperative interaction cannot be brought about without the robot first recognizing the current human intention in a fast and reliable way so that a suitable assisting plan can be quickly instantiated and executed. Eye movements have long been known to be highly predictive of the cognitive agenda unfolding during manual tasks and constitute, hence, the earliest and most reliable behavioral cues for intention estimation. In this study, we present an experiment aimed at analyzing human behavior in simple teleoperated pick-and-place tasks in a simulated scenario and at devising a suitable model for early estimation of the current proximal intention. We show that scan paths are, as expected, heavily shaped by the current intention and that two types of Gaussian Hidden Markov Models, one more scene-specific and one more action-specific, achieve a very good prediction performance, while also generalizing to new users and spatial arrangements. We finally discuss how behavioral and model results suggest that eye movements reflect to some extent the invariance and generality of higher-level planning across object configurations, which can be leveraged by cooperative robotic systems.
A DIGITAL MICRO SCREEN FOR THE ENHANCED APPEARANCE OF OCULAR PROSTHETIC MOTILITY (AN AMERICAN OPHTHALMOLOGICAL SOCIETY THESIS).
2021
Medicine
Tao, Jeremiah P., Emily S. Charlson, Yinheng Zhu, Zonglin Guo, Wanli Chen, Xun Zhan, Hongjian Shi, Ian G
American Journal of Ophthalmology
Purpose This study aims to improve the apparent motility of ocular prosthetic devices using technology. Prevailing ocular prostheses are acrylic shells with a static eye image rendered on the convex surface. A limited range of ocular prosthetic movement and lack of natural saccadic movements commonly causes the appearance of eye misalignment that may be disfiguring. Digital screens and computational systems may obviate current limitations in eye prosthetic motility and help prosthetic wearers feel less self-conscious about their appearance. Methods We applied convoluted neural networks (CNNs) to track pupil location in various conditions. These algorithms were coupled to a microscreen digital prosthetic eye (DPE) prototype to assess the ability of the system to capture full ocular ductions and saccadic movements in a miniaturized, portable, and wearable system. Results The CNNs captured pupil location with high accuracy. Pupil location data were transmitted to a miniature screen ocular prosthetic prototype that displayed a dynamic contralateral eye image. The transmission achieved a full range of ocular ductions and with grossly undetectable latency. Lack of iris and sclera color and detail, as well as constraints in luminosity, dimensionality and image stability limited the real eye appearance. Yet, the digitally rendered eye moved in the same amplitude and velocity as the native, tracked eye. Conclusions Real-time image processing using CNNs coupled to microcameras and a miniscreen DPE may offer improvements in amplitude and velocity of apparent prosthetic eye movement. These developments, along with ocular image precision, may offer a next-generation eye prosthesis.
Gaze Coordination of Groups in Dynamic Events–A Tool to Facilitate Analyses of Simultaneous Gazes Within a Team.
2021
Psychology
Fasold, Frowin, André Nicklas, Florian Seifriz, Karsten Schul, Benjamin Noël, Paula Aschendorf, Stefanie Klatt
Frontiers in Psychology 12
The performance and the success of a group working as a team on a common goal depends on the individuals’ skills and the collective coordination of their abilities. On a perceptual level, individual gaze behavior is reasonably well investigated. However, the coordination of visual skills in a team has been investigated only in laboratory studies and the practical examination and knowledge transfer to field studies or the applicability in real-life situations have so far been neglected. This is mainly due to the fact that a methodological approach along with a suitable evaluation procedure to analyze the gaze coordination within a team in highly dynamic events outside the lab, is still missing. Thus, this study was conducted to develop a tool to investigate the coordinated gaze behavior within a team of three human beings acting with a common goal in a dynamic real-world scenario. This team was a (three-person) basketball referee team adjudicating a game. Using mobile eye-tracking devices and an indigenously designed software tool for the simultaneous analysis of the gaze data of three participants, allowed, for the first time, the simultaneous investigation of the coordinated gaze behavior of three people in a highly dynamic setting. Overall, the study provides a new and innovative method to investigate the coordinated gaze behavior of a three-person team in specific tasks. This method is also applicable to investigate research questions about teams in dynamic real-world scenarios and get a deeper look at interactions and behavior patterns of human beings in group settings (for example, in team sports).
Classifying Excavator Collisions Based on Users’ Visual Perception in the Mixed Reality Environment.
2021
HCI
Forsman, Viking, Markus Wallmyr, Taufik Akbar Sitompul, Rikard Lindell
In 5th International Conference on Human Computer Interaction Theory and Applications
Visual perception plays an important role for recognizing possible hazards. In the context of heavy machinery, relevant visual information can be obtained from the machine’s surrounding and from the human-machine interface that exists inside the cabin. In this paper, we propose a method that classifies the occurring collisions by combining the data collected by the eye tracker and the automatic logging mechanism in the mixed reality simulation. Thirteen participants were asked to complete a test scenario in the mixed reality simulation, while wearing an eye tracker. The results demonstrate that we could classify the occurring collisions based on two visual perception conditions: (1) whether the colliding objects were visible from the participants’ field of view and (2) whether the participants have seen the information presented on the human-machine interface before the collisions occurred. This approach enabled us to interpret the occurring collisions differently, compared to the traditional approach that uses the total number of collisions as the representation of participants’ performance.
Eyeblink Detection in the Field: A Proof of Concept Study of Two Mobile Optical Eye-Trackers.
2021
Eye Tracking
Schweizer, Theresa, Thomas Wyss, Rahel Gilgen-Ammann
Military Medicine
Introduction: High physical and cognitive strain, high pressure, and sleep deficit are part of daily life for military professionals and civilians working in physiologically demanding environments. As a result, cognitive and physical capacities decline and the risk of illness, injury, or accidents increases. Such unfortunate outcomes could be prevented by tracking realtime physiological information, revealing individuals’ objective fatigue levels. Oculometrics, and especially eyeblinks, have been shown to be promising biomarkers that reflect fatigue development. Head-mounted optical eye-trackers are a common method to monitor these oculometrics. However, studies measuring eyeblink detection in real-life settings have been lacking in the literature. Therefore, this study aims to validate two current mobile optical eye-trackers in an unrestrained military training environment. Materials and Method: Three male participants (age 20.0 ± 1.0) of the Swiss Armed Forces participated in this study by wearing three optical eye-trackers, two VPS16s (Viewpointsystem GmbH, Vienna, Austria) and one Pupil Core (Pupil Labs GmbH, Berlin, Germany), during four military training events: Healthcare education, orienteering, shooting, and military marching. Software outputs were analyzed against a visual inspection (VI) of the video recordings of participants’ eyes via the respective software. Absolute and relative blink numbers were provided. Each blink detected by the software was classified as a “true blink” (TB) when it occurred in the software output and the VI at the same time, as a “false blink” (FB) when it occurred in the software but not in the VI, and as a “missed blink” (MB) when the software failed to detect a blink that occurred in the VI. The FBs were further examined for causes of the incorrect recordings, and they were divided into four categories: “sunlight,” “movements,” “lost pupil,” and “double-counted”. Blink frequency (i.e., blinks per minute) was also analyzed. Results: Overall, 49.3% and 72.5% of registered eyeblinks were classified as TBs for the VPS16 and Pupil Core, respectively. The VPS16 recorded 50.7% of FBs and accounted for 8.5% of MBs, while the Pupil Core recorded 27.5% of FBs and accounted for 55.5% of MBs. The majority of FBs—45.5% and 73.9% for the VPS16 and Pupil Core, respectively—were erroneously recorded due to participants’ eye movements while looking up, down, or to one side. For blink frequency analysis, systematic biases (±limits of agreement) stood at 23.3 (±43.5) and −4.87 (±14.1) blinks per minute for the VPS16 and Pupil Core, respectively. Significant differences in systematic bias between devices and the respective VIs were found for nearly all activities (P < .05). Conclusion: An objective physiological monitoring of fatigue is necessary for soldiers as well as civil professionals who are exposed to higher risks when their cognitive or physical capacities weaken. However, optical eye-trackers’ accuracy has not been specified under field conditions—especially not in monitoring fatigue. The significant overestimation and underestimation of the VPS16 and Pupil Core, respectively, demonstrate the general difficulty of blink detection in the field.
Eye, Robot: Calibration Challenges and Potential Solutions for Wearable Eye Tracking in Individuals with Eccentric Fixation.
2021
Eye Tracking
Love, Kassia, Anca Velisar, Natela Shanidze
In ACM Symposium on Eye Tracking Research and Applications, pp. 1-3. 2021
Loss of the central retina, including the fovea, can lead to a loss of visual acuity and oculomotor deficits, and thus have profound effects on day-to-day tasks. Recent advances in head-mounted, 3D eye tracking have allowed researchers to extend studies in this population to a broader set of daily tasks and more naturalistic behaviors and settings. However, decreases in fixational stability, multiple fixational loci and their uncertain role as oculomotor references, as well as eccentric fixation all provide additional challenges for calibration and collection of eye movement data. Here we quantify reductions in calibration accuracy relative to fixation eccentricity, and suggest a robotic calibration and validation tool that will allow for future developments of calibration and tracking algorithms designed with this population in mind.
Noise in the Machine: Sources of Physical and Computation Error in Eye Tracking with Pupil Core Wearable Eye Tracker: Wearable Eye Tracker Noise in Natural Motion Experiments.
2021
Eye Tracking
Velisar, Anca, Natela Shanidze
In ACM Symposium on Eye Tracking Research and Applications, pp. 1-3. 2021
Developments in wearable eye tracking devices make them an attractive solution for studies of eye movements during naturalistic head/body motion. However, before these systems’ potential can be fully realized, a thorough assessment of potential sources of error is needed. In this study, we examine three possible sources for the Pupil Core eye tracking goggles: camera motion during head/body motion, choice of calibration marker configuration, and eye movement estimation. In our data, we find that up to 36% of reported eye motion may be attributable to camera movement; choice of appropriate calibration routine is essential for minimizing error; and the use of a secondary calibration for eye position remapping can improve eye position errors estimated from the eye tracker.