We are excited to announce another round of updates for Pupil Cloud. This release focuses mostly on stability, but we’re also adding a few nice UI updates as well!

Enrichment timeline visualization

We have put more design work into the enrichment timelines to make it easier for researchers to discover the results of enrichments. Timelines for enrichments now visualize three states:

1. Processing: The processing states of the enrichment. Ready, Processing, Done.

2. Localization: Visualizes where the reference image or surface detected in the recording.

3. Gaze on reference: Visualizes where the subject is looking at the reference image or surface in the recording.

Automated recording recovery

We developed an automatic recovery tool that fixes recordings that were not playable on the Companion Device or that were directly exported from the Companion Device to desktop.

AOI Heatmap Metrics: Fixation Count

We’ve added average fixation count metric to the AOI Heatmap visualization.

Marker Mapper Image

The Marker Mapper image now automatically selects an image where all markers are visible from within the enrichment section.

Workspace management

You can now leave workspaces. This can be useful if you are no longer contributing to a workspace and want to clean up the list of workspaces associated with your account. You can always join a workspace again by accepting an invite from a workspace admin.

Follow mode for playback as default

We’ve added a new feature so that the playhead is always in view during playback. This is now the default playback mode. If you want to switch back to the old style, you can click the “follow mode” button.

Improved Fullscreen Experience

We removed the top application bars when in fullscreen so that we can maximize the available space for fullscreen viewing.

We are excited to announce another round of updates for Pupil Cloud! We are introducing Area of Interest (AOI) analysis, improvements for the recording timeline, and several other smaller changes.

Areas of Interest

We are excited to launch AOI tools in Pupil Cloud! Use the AOI Editor in your Reference Image Mapper and Marker Mapper enrichments to draw AOIs on top of the reference image or surface. You can draw anything from simple polygons to multiple disconnected shapes.

AOI Metrics

After you have drawn AOIs you will automatically get CSV files of standard metrics of fixations on AOIs: total fixation duration, average fixation duration, time to first fixation, and reach.

AOI Heatmaps

Create visualizations of AOI metrics using an AOI Heatmap!

Enrichment Results in the Timeline

The recording timeline now contains visualizations for enrichments that indicate when they successfully tracked the reference image, surface, or face respectively.

This provides you with a high-level overview of when the stimuli were in the subject’s field of view.

Timeline Zoom

You can now zoom in the timeline. Zoom out for an overview of the entire recording and zoom in up to 1-second intervals to inspect the details.

Project Layout Changes

Enrichments now have a dedicated table. This enables you to search/sort enrichments and view more information about each enrichment in a single view. It also cleans up the sidebar!

Create Button

We added a “create” button in the enrichment and visualization views. Each button opens a modal that also contains more information about each enrichment and visualization. The aim is to make it easier to understand the setup and output of enrichments and visualizations.

Faster Downloads

Downloads of Marker Mapper and Reference Image Mapper exports are now 20x faster!

Search

You can now search for workspaces and projects by their ID.

Native Recording Data

If you want to download recording data from Pupil Cloud, in the same format it was saved on the Neon Companion Device, it is now called Native Recording Data.

Timestamps

We have switched to hardware based timestamps for video feeds for increased accuracy. If you need old recordings in Pupil Cloud updated to hardware timestamps reach out info@pupil-labs.com

Introducing the Analysis View

We are excited to introduce the Analysis View in Pupil Cloud! This is where high-level analysis, data aggregation, and advanced visualization will take place.

While this is just the first release and we are starting small, we have many features in the pipeline that will make the Analysis View one of the most important areas in Pupil Cloud in the near future!

What is available today?

Here is what has changed in this release:

• In this first release, we are mostly rearranging existing features in preparation for bigger upcoming releases.

• Analysis View: In each Project, you will see “Analysis” as a new menu item in the sidebar.

• Heatmaps: We moved heatmaps out of enrichments and into Analysis. A Heatmap visualization is automatically added for each Reference Image Mapper or Marker Mapper enrichment.

• Video Renderer: We renamed the Gaze Overlay Enrichment to Video Renderer and moved it into the Analysis view.

What’s next?

We have many exciting features planned for the Analysis View in the near future. This includes:

• Areas of Interest (AOIs): We’re putting the finishing touches on an AOI tool. Draw AOIs on reference images and surfaces. Aggregate data on AOIs.

• Metrics: Automatically calculate standard metrics for data on AOIs like e.g. “Average Fixation Duration”. Visualize them using various graphing options.

• Video Renderer: More gaze visualization options (think cross-hair, eye video overlay). Abilities to visualize and render data from multiple enrichments.

• Sharing: Share links to videos and visualizations with others.

We will be adding more and more of these features in the coming weeks, so stay tuned for updates!

Updates and Fixes

• Workspace collaboration: Now when you create a new workspace, you can immediately invite other editors to collaborate. Hopefully this makes this feature more discoverable! You can also invite users from workspace settings.

• Enrichment recordings: The enrichment track now has a tooltip and visual style to clearly communicate if a recording is not matched/included in the enrichment.

Pupil Cloud UI Overhaul

Since the first release of Pupil Cloud in 2019 (time flies!), we have received a lot of feedback from the community and gained tons of experience using it ourselves. This led us to rethink the entire UI and do a major overhaul, which is now ready for release.

The new UI is available starting today at cloud.pupil-labs.com and we encourage you to check it out! In case you are not ready to make the switch yet, you can still access the old Pupil Cloud UI at old.cloud.pupil-labs.com.

We compiled a shortlist of the most important changes and the motivation behind them below. If you'd prefer a short video introduction check out our new onboarding video!

There are some significant changes, and we would love to hear what you think about them! If you have any feedback or feature requests, please let us know. If you have any issues or questions, reach out via via chat!

Navigation

The new navigation bar lets you move between workspaces and projects faster and easier. The breadcrumb structure keeps you informed of your current location.

Project Editor

We have completely redesigned the project and enrichment interaction. In the old design, there was a lot of back-and-forth between different views. With this update we aim to provide a streamlined pipeline.

Enrichment View

The new Enrichment View lets you see everything about your enrichment in one place, including its definition and visualizations of the results.

Downloads

We have added a new Downloads View to projects, which enables you to download all recording and enrichment data from a single place. The overview now includes a summary of the files that will be included in each download. In a future update, you will also be able to customize which files you wish to download.

Global Events

We are making changes to how events work to create more consistency. Previously, there were two types of events in Pupil Cloud:

• Recording events: These events were created at recording time and uploaded to Pupil Cloud with the recording. They were then available in all projects to which the recording was added.

• Project events: These events were created post hoc as part of a project and were only available within that project.

This system was confusing because events did not behave consistently. It also meant additional work for many users, as they had to define the same events repeatedly if they used the same recording in multiple projects.

With this update, we are introducing Global Events. Every event, no matter when or where it was created, will always be available with the respective recording. This allows you to define an event once and use it in every project.

As part of this change, we are promoting all existing events to global events. Users who previously defined the same event multiple times in different projects will now see duplicate events in their projects. We apologize for any inconvenience caused.

Customizing and Filtering Recording Tables

We added Filters to make it even easier to find recordings in your workspace. You can use filters together with search. Filter based on attributes like wearer, duration, or any other recording attribute.Furthermore, you can now customize the columns displayed in the recording table. This means you can add columns that are important to you and remove those that are less relevant.

Hotkeys

We have introduced several new hotkeys to help power users move more quickly. To view the hotkey overview, access the help menu in the top-right corner of the screen.

These hotkeys will enable you to move more swiftly through recordings and speed up the event annotation process. If there are any additional hotkeys you would like to see implemented let us know.

Resizable Panels

The panels in the new user interface are resizable. You can choose what you want to see most of: the recording table, the player, or the timeline!

Templates and Wearers in Workspace Settings

All functionality to create, edit and delete Templates and Wearers has been moved to the workspace settings.

Big updates for Pupil Cloud! A Demo Workspace with sample recordings, projects, and enrichments for everyone to explore. Fixation scanpaths can now visualized in all video playback in Cloud.

Demo Workspace

Every Pupil Cloud user now has access to our new Demo Workspace. It contains recordings and an example project with enrichments. We encourage everyone to explore it to help understand cloud features, best practices, and to get hands-on on with a real world dataset recorded with Pupil Invisible. We will continue to add more projects over time. Have a use-case you'd like to see as a demo? Get in touch!

Learn more about the demo workspace here.

Fixation Scanpath Visualization

We have added a new visualization for fixation scanpaths to all video playback in Pupil Cloud. It shows the sequence of fixations for the last two seconds in a recording. The visualization compensates for head movements to ensure fixations remain in the right location even when the viewpoint changes. (We've got a white paper on the new fixation detection algorithm coming soon - stay tuned!)

We are excited to announce our latest update for Pupil Cloud including a new blink detector for Pupil Invisible, visualizations for the Reference Image Mapper enrichment and quicker access to data downloads in convenient formats!

Blink Detector

We built a brand new blink detection algorithm and are making it available in Pupil Cloud. Blinks will be calculated for recordings automatically on upload to Pupil Cloud and be available in exports. The algorithm analyzes motion patterns in the eye videos to robustly detect blink events including blink duration. We are planning on open sourcing the blink detection algorithm in the near future.

Learn more about the algorithm here.

Checkout the export format of blink data here.

Reference Image Mapper Visualizations

The Reference Image Mapper enables you to automatically map gaze from the scene video onto a reference image. We want to give users a glimpse into how the algorithm works and a way to inspect the results.

To facilitate that, we added two new visualizations for the Reference Image Mapper to the project editor.

If you select a Reference Image Mapper enrichment in the project editor sidebar you can enable a side-by-side view of the reference image and the scene video. You can play back gaze on them simultaneously and verify the correctness of the mapping.

Internally, the Reference Image Mapper is generating a 3D point cloud representation of the recorded environment. You can now enable a visualization of this point cloud projected onto the scene video. This allows you to verify that the 3D scene camera motion has been estimated correctly.

CSV Data Downloads In Drive

You can already use the Raw Data Exporter enrichment to download recordings in convenient formats like CSV and MP4 files. Now you can download recordings in convenient formats directly from Drive! We did this to help speed up the exploration of data for those who don’t need to create a project.

In the Drive view you can click on the Download button and now see two options. Download binary recording data which downloads the raw data as recorded on the Companion Device (plus 200Hz gaze data). And the new option called Download Recording which will result in CSV data files and videos in convenient formats (including 200 Hz gaze, fixation and blink data).

We are excited to announce our latest release for Pupil Cloud including a novel fixation detector, advanced privacy features in workspaces, and more! All recordings uploaded after this release will automatically have fixation data.

Fixation Detection

We have developed a novel fixation detector for Pupil Invisible! It was designed to cope with the challenges of head-mounted eye tracking and is one of the most robust algorithms out there!

Traditional fixation detection algorithms are designed for stationary settings which assume minimal head movement. Our algorithm actively compensates for head movements and can detect fixations more reliably in dynamic scenarios. This includes robustness to vestibulo–ocular reflex (VOR) movements.

Fixations will be calculated automatically on upload to Pupil Cloud and be available in all exports. We are planning on open sourcing our new fixation detection algorithm in the near future along with a white paper and integration into Pupil Player for offline support.

Check out documentation on fixations exported for enrichments in the docs

Advanced Workspace Privacy Settings

We are introducing additional privacy settings for workspaces to cover a few specialized use cases that were requested by the community.

You can now disable scene video upload for the entire workspace. This allows users to make use of Pupil Cloud features like the calculation of 200 Hz gaze or fixations, while complying with strict privacy policies that would not allow scene video uploads to our servers.

In a future release we will introduce the ability to automatically blur faces on-upload of a recording to a workspace, such that the potentially sensitive original version is never stored in Pupil Cloud.

“Created by” Column in Recordings List

You can now see who uploaded each recording in the Drive view “Created by” column. We hope this makes collaboration easier within your workspace.

Optimized Project Editor Layout

We made more space for the video player at the center, and added a toolbar above the video player. The toolbar contains contextual enrichment related functions. Currently only for the Marker Mapper enrichment - more in the near future!

Big release! We added two powerful new features. Workspaces enable collaboration with your colleagues. Face Mapper automatically detects faces in recordings!

Workspaces

Workspaces enable you to collaborate with colleagues through role based access from data collection with Pupil Invisible Companion App to data annotation and enrichment in Pupil Cloud. Workspaces act as isolated spaces that contain recordings, wearers, templates, labels, projects and enrichments.

Note: Update the Pupil Invisible Companion app to the most recent version (v1.3.0) to make use of workspaces.

Learn more about Workspaces

Face Mapper

Face Mapper is a new enrichment that automatically and robustly detects faces in the scene video and maps gaze data onto the detected faces. This enables you to easily determine when a subject has been looking at a face and to compute aggregate statistics (example: how much time was spent looking at faces?).

Learn more about Face Mapper

Share your thoughts

Have feedback, questions, feature requests - send us your thoughts!

Head Pitch & Roll Estimates are now available!

The Inertial Measurment Unit (IMU) sensor embedded in the Pupil Invisible Glasses frame provides measurements that can tell us how the wearer’s head is moving. This can be valuable if you want to measure when your subject is looking downwards vs upwards for example.

In this update we have made the outputs of the IMU much easier to work with. Instead of only reporting the raw outputs of the IMU (rotational speed and translational acceleration), we now include estimates of the absolute pitch and roll of the head.

This is part of the Raw Data Exporter. You can find more details about how the estimates are calculated in the docs!

Information on when the Glasses are worn now included in Raw Data Export

The gaze.csv file included in the Raw Data Export now contains a new column called worn that indicates whether the Pupil Invisible Glasses have been worn by a subject at the respective time point (1.0 = worn, 0.0 = not worn).

This data has previously only been available as part of the binary recording data.

Improved Export Format of Marker Mapper

We have updated the export format of the Marker Mapper to be easier to use:

We have added a new column detected markers to the aoi_positions.csv file that contains the IDs of all the markers detected in the respective frame.We have split the corner coordinates in image column in the aoi_positions.csv file in to a separate column per coordinate to make it easier to parse.

Do you have feedback you would like to share?

Please do so!

Reference Image Mapper

The Reference Image Mapper is a powerful enrichment that enables you to automatically map gaze to an object of interest. This enrichment works best when you provide optimal reference image and scanning recording, and understand the limitations of the tool. To make this easier we have created an explainer video and integrated it directly into the enrichment creator.

We also added the reference image that was used to create the enrichment in the download. This ensures that this important image is always available for further analysis.

Undistorted Gaze Overlay Videos

Pupil Invisible's world camera uses a wide-angle lens to capture a large field of view which introduces optical distortion. We have introduced a new setting to the Gaze Overlay enrichment that enables you to removing this distortion from the video.

In-App Feature Notifications

We are regularly adding new features to Pupil Cloud and want to make sure every user stays on top of what is currently available and possible. New features will be announced within Pupil Cloud itself.

• Improved: Enrichment statuses have been renamed for more clarity:

• ‘Processing’ is now ‘In progress’

• ‘Ready’ is now ‘Ready to Compute’

• ‘Computed’ is now ‘Done’

• Improved: Reference Image Mapper now accepts PNG images as well as JPG images.

• New: Volume controls! Now all video players in app have a volume slider. Tip: Click the volume icon to mute.

• New: Reference Image Mapper enrichment. This enrichment enables you to track areas of interest in your environment based on a reference image of them. See our blog post on this and the docs.

• Enrichment: Gaze Overlay Export enrichment. This enrichment enables you to export world video with gaze overlay directly from Pupil Cloud. See docs.

• New: Raw Data Export enrichment. This enrichment allows you to export all recording data in easy to use CSV and MP4 file formats for further processing. See docs.

• New: All data formats for all enrichments are now documented in detail in the docs.

• Improved: Help menu in left side bar has been updated for quicker access to all important resources.

• Improved: Lots of UI tweaks and improvements.

• Improved: The entire app has been tuned to improve the performance especially for accounts with a large number of recordings.

• New: Drive view has a projects column that shows which projects a recording belongs to.

Event name suggestions: When adding events in the project editor, the text field now allows to quickly reuse previous event names.

Various bug fixes and performance improvements across the app.

New features to power analysis! Check out blog post for an overview or check out new features directly in Pupil Cloud 🐱

• New: Projects enable you to group and analyze a collection of recordings. For more see docs.

• New: Marker Mapper enrichment for analyzing AOIs using Apriltag Markers. See docs.

• New: Now all recordings in the drive view have thumbnails. Tip: hover over the thumbnail and drag left/right to get a rough preview the recording.