All software releases
Here you will find a log of all features, changes, bug fixes, and developer notes for Pupil Labs software.
April 1, 2024
We are excited to announce another round of updates for Pupil Cloud. This release focuses mostly on stability, but we’re also adding a few nice UI updates as well!
Enrichment timeline visualization
![](https://framerusercontent.com/images/5bQOkr2OzTwosILfgrG6eYOJA.png)
We have put more design work into the enrichment timelines to make it easier for researchers to discover the results of enrichments. Timelines for enrichments now visualize three states:
Processing: The processing states of the enrichment. Ready, Processing, Done.
Localization: Visualizes where the reference image or surface detected in the recording.
Gaze on reference: Visualizes where the subject is looking at the reference image or surface in the recording.
Automated recording recovery
We developed an automatic recovery tool that fixes recordings that were not playable on the Companion Device or that were directly exported from the Companion Device to desktop.
AOI Heatmap Metrics: Fixation Count
We’ve added average fixation count metric to the AOI Heatmap visualization.
Marker Mapper Image
The Marker Mapper image now automatically selects an image where all markers are visible from within the enrichment section.
Workspace management
You can now leave workspaces. This can be useful if you are no longer contributing to a workspace and want to clean up the list of workspaces associated with your account. You can always join a workspace again by accepting an invite from a workspace admin.
Follow mode for playback as default
We’ve added a new feature so that the playhead is always in view during playback. This is now the default playback mode. If you want to switch back to the old style, you can click the “follow mode” button.
Improved Fullscreen Experience
We removed the top application bars when in fullscreen so that we can maximize the available space for fullscreen viewing.
April 1, 2024
We are excited to announce another round of updates for Pupil Cloud. This release focuses mostly on stability, but we’re also adding a few nice UI updates as well!
Enrichment timeline visualization
![](https://framerusercontent.com/images/5bQOkr2OzTwosILfgrG6eYOJA.png)
We have put more design work into the enrichment timelines to make it easier for researchers to discover the results of enrichments. Timelines for enrichments now visualize three states:
Processing: The processing states of the enrichment. Ready, Processing, Done.
Localization: Visualizes where the reference image or surface detected in the recording.
Gaze on reference: Visualizes where the subject is looking at the reference image or surface in the recording.
Automated recording recovery
We developed an automatic recovery tool that fixes recordings that were not playable on the Companion Device or that were directly exported from the Companion Device to desktop.
AOI Heatmap Metrics: Fixation Count
We’ve added average fixation count metric to the AOI Heatmap visualization.
Marker Mapper Image
The Marker Mapper image now automatically selects an image where all markers are visible from within the enrichment section.
Workspace management
You can now leave workspaces. This can be useful if you are no longer contributing to a workspace and want to clean up the list of workspaces associated with your account. You can always join a workspace again by accepting an invite from a workspace admin.
Follow mode for playback as default
We’ve added a new feature so that the playhead is always in view during playback. This is now the default playback mode. If you want to switch back to the old style, you can click the “follow mode” button.
Improved Fullscreen Experience
We removed the top application bars when in fullscreen so that we can maximize the available space for fullscreen viewing.
January 24, 2024
We are excited to announce another round of updates for Pupil Cloud! We are introducing Area of Interest (AOI) analysis, improvements for the recording timeline, and several other smaller changes.
Areas of Interest
We are excited to launch AOI tools in Pupil Cloud! Use the AOI Editor in your Reference Image Mapper and Marker Mapper enrichments to draw AOIs on top of the reference image or surface. You can draw anything from simple polygons to multiple disconnected shapes.
AOI Metrics
After you have drawn AOIs you will automatically get CSV files of standard metrics of fixations on AOIs: total fixation duration, average fixation duration, time to first fixation, and reach.
AOI Heatmaps
Create visualizations of AOI metrics using an AOI Heatmap!
![](https://framerusercontent.com/images/9Z6KeERWFFBmWHjdSA73HdMkho.png)
Enrichment Results in the Timeline
The recording timeline now contains visualizations for enrichments that indicate when they successfully tracked the reference image, surface, or face respectively.
This provides you with a high-level overview of when the stimuli were in the subject’s field of view.
![](https://framerusercontent.com/images/gs5j7uCpGIpxJl0LcFpjHRoCBs.png)
Timeline Zoom
You can now zoom in the timeline. Zoom out for an overview of the entire recording and zoom in up to 1-second intervals to inspect the details.
Project Layout Changes
Enrichments now have a dedicated table. This enables you to search/sort enrichments and view more information about each enrichment in a single view. It also cleans up the sidebar!
![](https://framerusercontent.com/images/Vq6MpTaLPAi8n8SKwUsNf8VeNk.png)
Create Button
We added a “create” button in the enrichment and visualization views. Each button opens a modal that also contains more information about each enrichment and visualization. The aim is to make it easier to understand the setup and output of enrichments and visualizations.
![](https://framerusercontent.com/images/ph0AsjiRohAlcLik1MhxgYtfBk4.png)
![](https://framerusercontent.com/images/pSeIjW2sMnbRKURQEGK8HpREVU.png)
Faster Downloads
Downloads of Marker Mapper and Reference Image Mapper exports are now 20x faster!
Search
You can now search for workspaces and projects by their ID.
Native Recording Data
If you want to download recording data from Pupil Cloud, in the same format it was saved on the Neon Companion Device, it is now called Native Recording Data
.
Timestamps
We have switched to hardware based timestamps for video feeds for increased accuracy. If you need old recordings in Pupil Cloud updated to hardware timestamps reach out info@pupil-labs.com
January 24, 2024
We are excited to announce another round of updates for Pupil Cloud! We are introducing Area of Interest (AOI) analysis, improvements for the recording timeline, and several other smaller changes.
Areas of Interest
We are excited to launch AOI tools in Pupil Cloud! Use the AOI Editor in your Reference Image Mapper and Marker Mapper enrichments to draw AOIs on top of the reference image or surface. You can draw anything from simple polygons to multiple disconnected shapes.
AOI Metrics
After you have drawn AOIs you will automatically get CSV files of standard metrics of fixations on AOIs: total fixation duration, average fixation duration, time to first fixation, and reach.
AOI Heatmaps
Create visualizations of AOI metrics using an AOI Heatmap!
![](https://framerusercontent.com/images/9Z6KeERWFFBmWHjdSA73HdMkho.png)
Enrichment Results in the Timeline
The recording timeline now contains visualizations for enrichments that indicate when they successfully tracked the reference image, surface, or face respectively.
This provides you with a high-level overview of when the stimuli were in the subject’s field of view.
![](https://framerusercontent.com/images/gs5j7uCpGIpxJl0LcFpjHRoCBs.png)
Timeline Zoom
You can now zoom in the timeline. Zoom out for an overview of the entire recording and zoom in up to 1-second intervals to inspect the details.
Project Layout Changes
Enrichments now have a dedicated table. This enables you to search/sort enrichments and view more information about each enrichment in a single view. It also cleans up the sidebar!
![](https://framerusercontent.com/images/Vq6MpTaLPAi8n8SKwUsNf8VeNk.png)
Create Button
We added a “create” button in the enrichment and visualization views. Each button opens a modal that also contains more information about each enrichment and visualization. The aim is to make it easier to understand the setup and output of enrichments and visualizations.
![](https://framerusercontent.com/images/ph0AsjiRohAlcLik1MhxgYtfBk4.png)
![](https://framerusercontent.com/images/pSeIjW2sMnbRKURQEGK8HpREVU.png)
Faster Downloads
Downloads of Marker Mapper and Reference Image Mapper exports are now 20x faster!
Search
You can now search for workspaces and projects by their ID.
Native Recording Data
If you want to download recording data from Pupil Cloud, in the same format it was saved on the Neon Companion Device, it is now called Native Recording Data
.
Timestamps
We have switched to hardware based timestamps for video feeds for increased accuracy. If you need old recordings in Pupil Cloud updated to hardware timestamps reach out info@pupil-labs.com
August 7, 2023
Introducing the Analysis View
We are excited to introduce the Analysis View in Pupil Cloud! This is where high-level analysis, data aggregation, and advanced visualization will take place.
While this is just the first release and we are starting small, we have many features in the pipeline that will make the Analysis View one of the most important areas in Pupil Cloud in the near future!
What is available today?
Here is what has changed in this release:
In this first release, we are mostly rearranging existing features in preparation for bigger upcoming releases.
Analysis View: In each Project, you will see “Analysis” as a new menu item in the sidebar.
Heatmaps: We moved heatmaps out of enrichments and into Analysis. A Heatmap visualization is automatically added for each Reference Image Mapper or Marker Mapper enrichment.
Video Renderer: We renamed the Gaze Overlay Enrichment to Video Renderer and moved it into the Analysis view.
What’s next?
We have many exciting features planned for the Analysis View in the near future. This includes:
Areas of Interest (AOIs): We’re putting the finishing touches on an AOI tool. Draw AOIs on reference images and surfaces. Aggregate data on AOIs.
Metrics: Automatically calculate standard metrics for data on AOIs like e.g. “Average Fixation Duration”. Visualize them using various graphing options.
Video Renderer: More gaze visualization options (think cross-hair, eye video overlay). Abilities to visualize and render data from multiple enrichments.
Sharing: Share links to videos and visualizations with others.
We will be adding more and more of these features in the coming weeks, so stay tuned for updates!
Updates and Fixes
Workspace collaboration: Now when you create a new workspace, you can immediately invite other editors to collaborate. Hopefully this makes this feature more discoverable! You can also invite users from workspace settings.
Enrichment recordings: The enrichment track now has a tooltip and visual style to clearly communicate if a recording is not matched/included in the enrichment.
August 7, 2023
Introducing the Analysis View
We are excited to introduce the Analysis View in Pupil Cloud! This is where high-level analysis, data aggregation, and advanced visualization will take place.
While this is just the first release and we are starting small, we have many features in the pipeline that will make the Analysis View one of the most important areas in Pupil Cloud in the near future!
What is available today?
Here is what has changed in this release:
In this first release, we are mostly rearranging existing features in preparation for bigger upcoming releases.
Analysis View: In each Project, you will see “Analysis” as a new menu item in the sidebar.
Heatmaps: We moved heatmaps out of enrichments and into Analysis. A Heatmap visualization is automatically added for each Reference Image Mapper or Marker Mapper enrichment.
Video Renderer: We renamed the Gaze Overlay Enrichment to Video Renderer and moved it into the Analysis view.
What’s next?
We have many exciting features planned for the Analysis View in the near future. This includes:
Areas of Interest (AOIs): We’re putting the finishing touches on an AOI tool. Draw AOIs on reference images and surfaces. Aggregate data on AOIs.
Metrics: Automatically calculate standard metrics for data on AOIs like e.g. “Average Fixation Duration”. Visualize them using various graphing options.
Video Renderer: More gaze visualization options (think cross-hair, eye video overlay). Abilities to visualize and render data from multiple enrichments.
Sharing: Share links to videos and visualizations with others.
We will be adding more and more of these features in the coming weeks, so stay tuned for updates!
Updates and Fixes
Workspace collaboration: Now when you create a new workspace, you can immediately invite other editors to collaborate. Hopefully this makes this feature more discoverable! You can also invite users from workspace settings.
Enrichment recordings: The enrichment track now has a tooltip and visual style to clearly communicate if a recording is not matched/included in the enrichment.
May 10, 2023
Pupil Cloud UI Overhaul
Since the first release of Pupil Cloud in 2019 (time flies!), we have received a lot of feedback from the community and gained tons of experience using it ourselves. This led us to rethink the entire UI and do a major overhaul, which is now ready for release.
The new UI is available starting today at cloud.pupil-labs.com and we encourage you to check it out! In case you are not ready to make the switch yet, you can still access the old Pupil Cloud UI at old.cloud.pupil-labs.com.
We compiled a shortlist of the most important changes and the motivation behind them below. If you'd prefer a short video introduction check out our new onboarding video!
There are some significant changes, and we would love to hear what you think about them! If you have any feedback or feature requests, please let us know. If you have any issues or questions, reach out via via chat!
Navigation
The new navigation bar lets you move between workspaces and projects faster and easier. The breadcrumb structure keeps you informed of your current location.
Project Editor
We have completely redesigned the project and enrichment interaction. In the old design, there was a lot of back-and-forth between different views. With this update we aim to provide a streamlined pipeline.
Enrichment View
The new Enrichment View lets you see everything about your enrichment in one place, including its definition and visualizations of the results.
Downloads
We have added a new Downloads View to projects, which enables you to download all recording and enrichment data from a single place. The overview now includes a summary of the files that will be included in each download. In a future update, you will also be able to customize which files you wish to download.
![Screenshot of the downloads view.](https://framerusercontent.com/images/n3oC0UOQRKx9OCHYn7i6NKUVnbQ.png)
Global Events
We are making changes to how events work to create more consistency. Previously, there were two types of events in Pupil Cloud:
Recording events: These events were created at recording time and uploaded to Pupil Cloud with the recording. They were then available in all projects to which the recording was added.
Project events: These events were created post hoc as part of a project and were only available within that project.
This system was confusing because events did not behave consistently. It also meant additional work for many users, as they had to define the same events repeatedly if they used the same recording in multiple projects.
With this update, we are introducing Global Events. Every event, no matter when or where it was created, will always be available with the respective recording. This allows you to define an event once and use it in every project.
As part of this change, we are promoting all existing events to global events. Users who previously defined the same event multiple times in different projects will now see duplicate events in their projects. We apologize for any inconvenience caused.
Customizing and Filtering Recording Tables
We added Filters to make it even easier to find recordings in your workspace. You can use filters together with search. Filter based on attributes like wearer, duration, or any other recording attribute.Furthermore, you can now customize the columns displayed in the recording table. This means you can add columns that are important to you and remove those that are less relevant.
![Screenshot illustrating the filters.](https://framerusercontent.com/images/pL14bxwWvD3vW2k7FB1dJch4v18.png)
Hotkeys
We have introduced several new hotkeys to help power users move more quickly. To view the hotkey overview, access the help menu in the top-right corner of the screen.
These hotkeys will enable you to move more swiftly through recordings and speed up the event annotation process. If there are any additional hotkeys you would like to see implemented let us know.
![Screenshot illustrating the filters.](https://framerusercontent.com/images/wm5cG5CjDIGOK42uAl370wp5PNc.png)
Resizable Panels
The panels in the new user interface are resizable. You can choose what you want to see most of: the recording table, the player, or the timeline!
Templates and Wearers in Workspace Settings
All functionality to create, edit and delete Templates and Wearers has been moved to the workspace settings.
![Screenshot illustrating the filters.](https://framerusercontent.com/images/50q3MZwiIvdoM0aOu0sX203IrU.png)
May 10, 2023
Pupil Cloud UI Overhaul
Since the first release of Pupil Cloud in 2019 (time flies!), we have received a lot of feedback from the community and gained tons of experience using it ourselves. This led us to rethink the entire UI and do a major overhaul, which is now ready for release.
The new UI is available starting today at cloud.pupil-labs.com and we encourage you to check it out! In case you are not ready to make the switch yet, you can still access the old Pupil Cloud UI at old.cloud.pupil-labs.com.
We compiled a shortlist of the most important changes and the motivation behind them below. If you'd prefer a short video introduction check out our new onboarding video!
There are some significant changes, and we would love to hear what you think about them! If you have any feedback or feature requests, please let us know. If you have any issues or questions, reach out via via chat!
Navigation
The new navigation bar lets you move between workspaces and projects faster and easier. The breadcrumb structure keeps you informed of your current location.
Project Editor
We have completely redesigned the project and enrichment interaction. In the old design, there was a lot of back-and-forth between different views. With this update we aim to provide a streamlined pipeline.
Enrichment View
The new Enrichment View lets you see everything about your enrichment in one place, including its definition and visualizations of the results.
Downloads
We have added a new Downloads View to projects, which enables you to download all recording and enrichment data from a single place. The overview now includes a summary of the files that will be included in each download. In a future update, you will also be able to customize which files you wish to download.
![Screenshot of the downloads view.](https://framerusercontent.com/images/n3oC0UOQRKx9OCHYn7i6NKUVnbQ.png)
Global Events
We are making changes to how events work to create more consistency. Previously, there were two types of events in Pupil Cloud:
Recording events: These events were created at recording time and uploaded to Pupil Cloud with the recording. They were then available in all projects to which the recording was added.
Project events: These events were created post hoc as part of a project and were only available within that project.
This system was confusing because events did not behave consistently. It also meant additional work for many users, as they had to define the same events repeatedly if they used the same recording in multiple projects.
With this update, we are introducing Global Events. Every event, no matter when or where it was created, will always be available with the respective recording. This allows you to define an event once and use it in every project.
As part of this change, we are promoting all existing events to global events. Users who previously defined the same event multiple times in different projects will now see duplicate events in their projects. We apologize for any inconvenience caused.
Customizing and Filtering Recording Tables
We added Filters to make it even easier to find recordings in your workspace. You can use filters together with search. Filter based on attributes like wearer, duration, or any other recording attribute.Furthermore, you can now customize the columns displayed in the recording table. This means you can add columns that are important to you and remove those that are less relevant.
![Screenshot illustrating the filters.](https://framerusercontent.com/images/pL14bxwWvD3vW2k7FB1dJch4v18.png)
Hotkeys
We have introduced several new hotkeys to help power users move more quickly. To view the hotkey overview, access the help menu in the top-right corner of the screen.
These hotkeys will enable you to move more swiftly through recordings and speed up the event annotation process. If there are any additional hotkeys you would like to see implemented let us know.
![Screenshot illustrating the filters.](https://framerusercontent.com/images/wm5cG5CjDIGOK42uAl370wp5PNc.png)
Resizable Panels
The panels in the new user interface are resizable. You can choose what you want to see most of: the recording table, the player, or the timeline!
Templates and Wearers in Workspace Settings
All functionality to create, edit and delete Templates and Wearers has been moved to the workspace settings.
![Screenshot illustrating the filters.](https://framerusercontent.com/images/50q3MZwiIvdoM0aOu0sX203IrU.png)
July 8, 2022
Big updates for Pupil Cloud! A Demo Workspace with sample recordings, projects, and enrichments for everyone to explore. Fixation scanpaths can now visualized in all video playback in Cloud.
Demo Workspace
Every Pupil Cloud user now has access to our new Demo Workspace. It contains recordings and an example project with enrichments. We encourage everyone to explore it to help understand cloud features, best practices, and to get hands-on on with a real world dataset recorded with Pupil Invisible. We will continue to add more projects over time. Have a use-case you'd like to see as a demo? Get in touch!
Learn more about the demo workspace here.
![Screenshot of the Demo Workspace.](https://framerusercontent.com/images/xgLWSm1JFFOzmVJ6s5AmUabgyU.png)
Fixation Scanpath Visualization
We have added a new visualization for fixation scanpaths to all video playback in Pupil Cloud. It shows the sequence of fixations for the last two seconds in a recording. The visualization compensates for head movements to ensure fixations remain in the right location even when the viewpoint changes. (We've got a white paper on the new fixation detection algorithm coming soon - stay tuned!)
![Screenshot of the recording player with the fixation scanpath visualization enabled.](https://framerusercontent.com/images/nJrxxKP4OpM7CL3tdyKzFqQ1w.png)
July 8, 2022
Big updates for Pupil Cloud! A Demo Workspace with sample recordings, projects, and enrichments for everyone to explore. Fixation scanpaths can now visualized in all video playback in Cloud.
Demo Workspace
Every Pupil Cloud user now has access to our new Demo Workspace. It contains recordings and an example project with enrichments. We encourage everyone to explore it to help understand cloud features, best practices, and to get hands-on on with a real world dataset recorded with Pupil Invisible. We will continue to add more projects over time. Have a use-case you'd like to see as a demo? Get in touch!
Learn more about the demo workspace here.
![Screenshot of the Demo Workspace.](https://framerusercontent.com/images/xgLWSm1JFFOzmVJ6s5AmUabgyU.png)
Fixation Scanpath Visualization
We have added a new visualization for fixation scanpaths to all video playback in Pupil Cloud. It shows the sequence of fixations for the last two seconds in a recording. The visualization compensates for head movements to ensure fixations remain in the right location even when the viewpoint changes. (We've got a white paper on the new fixation detection algorithm coming soon - stay tuned!)
![Screenshot of the recording player with the fixation scanpath visualization enabled.](https://framerusercontent.com/images/nJrxxKP4OpM7CL3tdyKzFqQ1w.png)
May 11, 2022
Gaze Super Speed!
We have drastically increased the framerate of the real-time gaze signal to 120+ Hz. This was made possible by the hard work of our R&D and engineering teams in optimizing the neural network that runs on the Companion Device. Accuracy remains with drastic improvements to inference speed!
The high speed gaze data is available both in the recording data as well as via the real-time API. As usual, the full 200 Hz signal is available after uploading to Pupil Cloud.
May 11, 2022
Gaze Super Speed!
We have drastically increased the framerate of the real-time gaze signal to 120+ Hz. This was made possible by the hard work of our R&D and engineering teams in optimizing the neural network that runs on the Companion Device. Accuracy remains with drastic improvements to inference speed!
The high speed gaze data is available both in the recording data as well as via the real-time API. As usual, the full 200 Hz signal is available after uploading to Pupil Cloud.
April 29, 2022
Pupil Invisible Monitor
Monitor your data collection straight from the web-browser - on any device!
Just type pi.local:8080
into your address bar and the Pupil Invisible Monitor app will open, which allows you to view video and gaze data from all the Pupil Invisible Glasses in your network!
It also allows you to remote control them, see storage and battery levels, and save events with the press of a button.
A more detailed introduction can be found here.
April 29, 2022
Pupil Invisible Monitor
Monitor your data collection straight from the web-browser - on any device!
Just type pi.local:8080
into your address bar and the Pupil Invisible Monitor app will open, which allows you to view video and gaze data from all the Pupil Invisible Glasses in your network!
It also allows you to remote control them, see storage and battery levels, and save events with the press of a button.
A more detailed introduction can be found here.
March 24, 2022
We are excited to announce our latest update for Pupil Cloud including a new blink detector for Pupil Invisible, visualizations for the Reference Image Mapper enrichment and quicker access to data downloads in convenient formats!
Blink Detector
We built a brand new blink detection algorithm and are making it available in Pupil Cloud. Blinks will be calculated for recordings automatically on upload to Pupil Cloud and be available in exports. The algorithm analyzes motion patterns in the eye videos to robustly detect blink events including blink duration. We are planning on open sourcing the blink detection algorithm in the near future.
Learn more about the algorithm here.
Checkout the export format of blink data here.
Reference Image Mapper Visualizations
The Reference Image Mapper enables you to automatically map gaze from the scene video onto a reference image. We want to give users a glimpse into how the algorithm works and a way to inspect the results.
To facilitate that, we added two new visualizations for the Reference Image Mapper to the project editor.
If you select a Reference Image Mapper enrichment in the project editor sidebar you can enable a side-by-side view of the reference image and the scene video. You can play back gaze on them simultaneously and verify the correctness of the mapping.
Internally, the Reference Image Mapper is generating a 3D point cloud representation of the recorded environment. You can now enable a visualization of this point cloud projected onto the scene video. This allows you to verify that the 3D scene camera motion has been estimated correctly.
![Screenshot of Reference Image Mapper Visualizations.](https://framerusercontent.com/images/kJRQPb9FNAp5iXcabeMgN8fCgXo.png)
CSV Data Downloads In Drive
You can already use the Raw Data Exporter enrichment to download recordings in convenient formats like CSV and MP4 files. Now you can download recordings in convenient formats directly from Drive! We did this to help speed up the exploration of data for those who don’t need to create a project.
In the Drive view you can click on the Download button and now see two options. Download binary recording data which downloads the raw data as recorded on the Companion Device (plus 200Hz gaze data). And the new option called Download Recording which will result in CSV data files and videos in convenient formats (including 200 Hz gaze, fixation and blink data).
![Screenshot of available download options.](https://framerusercontent.com/images/z9j6cl1u7h8x72ASZDFghxfeYs4.png)
March 24, 2022
We are excited to announce our latest update for Pupil Cloud including a new blink detector for Pupil Invisible, visualizations for the Reference Image Mapper enrichment and quicker access to data downloads in convenient formats!
Blink Detector
We built a brand new blink detection algorithm and are making it available in Pupil Cloud. Blinks will be calculated for recordings automatically on upload to Pupil Cloud and be available in exports. The algorithm analyzes motion patterns in the eye videos to robustly detect blink events including blink duration. We are planning on open sourcing the blink detection algorithm in the near future.
Learn more about the algorithm here.
Checkout the export format of blink data here.
Reference Image Mapper Visualizations
The Reference Image Mapper enables you to automatically map gaze from the scene video onto a reference image. We want to give users a glimpse into how the algorithm works and a way to inspect the results.
To facilitate that, we added two new visualizations for the Reference Image Mapper to the project editor.
If you select a Reference Image Mapper enrichment in the project editor sidebar you can enable a side-by-side view of the reference image and the scene video. You can play back gaze on them simultaneously and verify the correctness of the mapping.
Internally, the Reference Image Mapper is generating a 3D point cloud representation of the recorded environment. You can now enable a visualization of this point cloud projected onto the scene video. This allows you to verify that the 3D scene camera motion has been estimated correctly.
![Screenshot of Reference Image Mapper Visualizations.](https://framerusercontent.com/images/kJRQPb9FNAp5iXcabeMgN8fCgXo.png)
CSV Data Downloads In Drive
You can already use the Raw Data Exporter enrichment to download recordings in convenient formats like CSV and MP4 files. Now you can download recordings in convenient formats directly from Drive! We did this to help speed up the exploration of data for those who don’t need to create a project.
In the Drive view you can click on the Download button and now see two options. Download binary recording data which downloads the raw data as recorded on the Companion Device (plus 200Hz gaze data). And the new option called Download Recording which will result in CSV data files and videos in convenient formats (including 200 Hz gaze, fixation and blink data).
![Screenshot of available download options.](https://framerusercontent.com/images/z9j6cl1u7h8x72ASZDFghxfeYs4.png)
March 21, 2022
Real-Time Network API and python library
Check out Pupil Invisible’s new real-time network API! This makes it easier than ever to get real-time eye tracking data for your research and applications.
Use our new Python library to instantly receive video and gaze data on your computer and remote control all your Pupil Invisible devices.
Implement gaze-interaction methodologies or track the progress of your data collection by starting recordings and saving events programmatically in no-time!
To learn more, please check out our Real-Time API Guides.
A new and improved version of Pupil Invisible Monitor based on this API is also coming very soon!
Pupil Invisible & Pupil Cloud Docs
Big update to the Pupil Invisible & Pupil Cloud docs. We improved the structure and added new tutorials and guides. Take a look!
If you are new to Pupil Invisible & Pupil Cloud, check out our new getting started guides.
Looking to dive deep? Check out the explainer sections on. the available data streams or enrichments.
More guides and other content additions are already in the works!
March 21, 2022
Real-Time Network API and python library
Check out Pupil Invisible’s new real-time network API! This makes it easier than ever to get real-time eye tracking data for your research and applications.
Use our new Python library to instantly receive video and gaze data on your computer and remote control all your Pupil Invisible devices.
Implement gaze-interaction methodologies or track the progress of your data collection by starting recordings and saving events programmatically in no-time!
To learn more, please check out our Real-Time API Guides.
A new and improved version of Pupil Invisible Monitor based on this API is also coming very soon!
Pupil Invisible & Pupil Cloud Docs
Big update to the Pupil Invisible & Pupil Cloud docs. We improved the structure and added new tutorials and guides. Take a look!
If you are new to Pupil Invisible & Pupil Cloud, check out our new getting started guides.
Looking to dive deep? Check out the explainer sections on. the available data streams or enrichments.
More guides and other content additions are already in the works!
December 10, 2021
We are excited to announce our latest release for Pupil Cloud including a novel fixation detector, advanced privacy features in workspaces, and more! All recordings uploaded after this release will automatically have fixation data.
Fixation Detection
We have developed a novel fixation detector for Pupil Invisible! It was designed to cope with the challenges of head-mounted eye tracking and is one of the most robust algorithms out there!
Traditional fixation detection algorithms are designed for stationary settings which assume minimal head movement. Our algorithm actively compensates for head movements and can detect fixations more reliably in dynamic scenarios. This includes robustness to vestibulo–ocular reflex (VOR) movements.
Fixations will be calculated automatically on upload to Pupil Cloud and be available in all exports. We are planning on open sourcing our new fixation detection algorithm in the near future along with a white paper and integration into Pupil Player for offline support.
Check out documentation on fixations exported for enrichments in the docs
Advanced Workspace Privacy Settings
We are introducing additional privacy settings for workspaces to cover a few specialized use cases that were requested by the community.
You can now disable scene video upload for the entire workspace. This allows users to make use of Pupil Cloud features like the calculation of 200 Hz gaze or fixations, while complying with strict privacy policies that would not allow scene video uploads to our servers.
In a future release we will introduce the ability to automatically blur faces on-upload of a recording to a workspace, such that the potentially sensitive original version is never stored in Pupil Cloud.
“Created by” Column in Recordings List
You can now see who uploaded each recording in the Drive view “Created by” column. We hope this makes collaboration easier within your workspace.
Optimized Project Editor Layout
We made more space for the video player at the center, and added a toolbar above the video player. The toolbar contains contextual enrichment related functions. Currently only for the Marker Mapper enrichment - more in the near future!
December 10, 2021
We are excited to announce our latest release for Pupil Cloud including a novel fixation detector, advanced privacy features in workspaces, and more! All recordings uploaded after this release will automatically have fixation data.
Fixation Detection
We have developed a novel fixation detector for Pupil Invisible! It was designed to cope with the challenges of head-mounted eye tracking and is one of the most robust algorithms out there!
Traditional fixation detection algorithms are designed for stationary settings which assume minimal head movement. Our algorithm actively compensates for head movements and can detect fixations more reliably in dynamic scenarios. This includes robustness to vestibulo–ocular reflex (VOR) movements.
Fixations will be calculated automatically on upload to Pupil Cloud and be available in all exports. We are planning on open sourcing our new fixation detection algorithm in the near future along with a white paper and integration into Pupil Player for offline support.
Check out documentation on fixations exported for enrichments in the docs
Advanced Workspace Privacy Settings
We are introducing additional privacy settings for workspaces to cover a few specialized use cases that were requested by the community.
You can now disable scene video upload for the entire workspace. This allows users to make use of Pupil Cloud features like the calculation of 200 Hz gaze or fixations, while complying with strict privacy policies that would not allow scene video uploads to our servers.
In a future release we will introduce the ability to automatically blur faces on-upload of a recording to a workspace, such that the potentially sensitive original version is never stored in Pupil Cloud.
“Created by” Column in Recordings List
You can now see who uploaded each recording in the Drive view “Created by” column. We hope this makes collaboration easier within your workspace.
Optimized Project Editor Layout
We made more space for the video player at the center, and added a toolbar above the video player. The toolbar contains contextual enrichment related functions. Currently only for the Marker Mapper enrichment - more in the near future!
November 29, 2021
We are pleased to announce the release of Pupil Core software v3.5!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord! 😄
Overview
In Pupil v3.5, we cleaned up log messages, improved software stability, fixations are now cached between Pupil Player sessions, and we have fine tuned 3d pupil confidence.
Pupil v3.5 is the first release that supports the new macOS Monterey (with a small caveat). Read more below.
New
Recording-specific annotation hotkeys
Add option to filter out low-confidence values in raw data export
Add option to disable world video recording
macOS Monterey support
Improved
Reduced number of user-facing log messages
Export-in-progress file-name suffixes
Windows: Handle exceptions during driver installation gracefully
Windows: Option to skip automatic driver installation
Windows: Create start menu entries and desktop shortcuts for all users on install
Add Euler angles to headpose tracker result
Normalize IMU pitch/roll values to [-180, 180] degrees
Confidence adjustment for pye3d results
Cache post-hoc fixation detector results
Documentation
New
Recording-specific annotation hotkeys - #2170
Until now, the keyboard-shortcut setup for annotations was stored in Pupil Player's session settings. These were reset if you updated Pupil Core software to a new version.
Starting with this version, annotation keyboard shortcuts will be stored persistently within the recording.
The definitions are stored in <recording dir>/offline_data/annotation_definitions.json
with the format:
{
"version": 1,
"definitions": {
"<label>": "<hotkey>"
}
}
This file can be shared between recordings.
When Pupil Player is opened, it will attempt to load the annotation definitions from the recording-specific file. If the file is not found or is invalid, Pupil Player will fallback to the session settings which contain the last known annotation definitions.
Add option to filter out low-confidence values in raw data export - #2210
Pupil Core software assigns "confidence" values to its pupil detections and gaze estimations. They indicate the quality of the measurement. For visualization and some analysis Plugins, Pupil Player hides gaze data below the "Minimum data confidence" threshold (adjustable in the general settings menu).
Prior to this release, the raw data exporter plugin exported all data, independently of its confidence. New users, therefore, often wonder why their exported data looks noisier than what they have seen in Pupil Player.
Starting with this release, users will have the option to filter out low-confidence data (i.e. below the "Minimum data confidence" threshold) in the export. The option can be enabled in the raw data exporter's menu.
Add option to disable world video recording - #2169
If you are not interested in recording the scene video, you can now disable the scene video recording in the Recorder
menu of Pupil Capture's world
window.
macOS Monterey support
Due to new technical limitations, Pupil Capture and Pupil Service need to be started with administrator privileges to get access to the video camera feeds. To do that, copy the applications into your /Applications
folder and run the corresponding command from the terminal:
Pupil Capture:
sudo /Applications/Pupil\ Capture.app/Contents/MacOS/pupil_capture
Pupil Service:
sudo /Applications/Pupil\ Service.app/Contents/MacOS/pupil_service
Note: The terminal will prompt you for your administrator password. It will not preview any typed keys. Simply hit enter once the password has been typed.
Note: When recording with administrator privileges, the resulting folder inherits admin file permissions. Pupil Player will detect these and ask you for the administrator password to reset the file permissions. This will be only necessary once per recording.
Improved
Export-in-progress file-name suffixes - #2167, #2186
Exporting video can take a long time. Pupil Player displays the progress as a growing circle around the menu icon while writing the result incrementally to disk. Specifically, it writes the result directly to the final export location. While the export is ongoing, the video file is invalid but still available to the user. This can lead to users opening the video before it has finished exporting and then encountering playback issues.
Starting with this release, Pupil Core software changes the video exports to include a .writing
file name suffix during the export. This prevents other applications from recognizing the partially exported video file as complete. Once the export is done, the suffix is removed and other applications are able to recognize the file type as video.
Reduced number of user-facing log messages - #2190
Over the years, Pupil Capture's functionality has grown and with it the number of user-facing log messages. They can be overwhelming to the point where important log messages are overlooked or ignored. For this release, we have reviewed the log messages that are being displayed in the UI during typical actions, e.g. calibration and recordings, and have reduced them significantly.
Windows: Handle exceptions during driver installation gracefully - #2173
Previously, errors during the driver installation could cause the software to crash. Now, errors will be logged to the log file and the user will be asked to install the drivers manually.
Windows: Option to skip automatic driver installation - #2207
Using the -skip-driv
or --skip-driver-installation
flags, one can now launch Pupil Capture and Pupil Service without running the automatic driver installation on Windows. This can be helpful to speed up the start up time or to avoid unwanted driver installations.
Windows: Create start menu entries and desktop shortcuts for all users on install - #2166
Pupil Core software is installed to C:\Program Files (x86)\Pupil-Labs
by default. This directory should be accessible by all users but the start menu entry and desktop shortcut are only installed for the current user. Starting with this release, the installer will create start menu entries and desktop shortcuts for all users on install.
Add Euler angles to headpose tracker result - #2175
Next to camera poses as Rodrigues' rotation vectors, the headpose tracker now also exports the camera orientation in Euler angles.
We also fixed a typo in the exported file name.
- head_pose_tacker_model.csv
+ head_pose_tracker_model.csv
Normalize IMU pitch/roll values to [-180, 180] - #2171
Orientation values exported by the IMU Timeline plugin will now be normalized to a range of [-180, 180] degrees.
Confidence adjustment for pye3d results - #2195
In some cases, pye3d was overestimating the confidence of its pupil detections. These were visible as noisy gaze estimations in the Pupil Core applications. This release includes an updated version of pye3d with adjusted confidence estimations. As a result, the gaze signal should look more stable during lower confidence situations, e.g. while squinting or looking down. For more information, see the Developer notes below.
Cache post-hoc fixation detector results - #2194
Post-hoc fixation detector results no longer need to be recalculated if you close and reopen a recording. Pupil Player stores the results together with the used configuration (version of the gaze data and fixation detection parameters) within the recording directory. The cached data will be restored if the Player configuration matches the configuration stored on disk. If it does not match, Player will recalculate the fixations to avoid inconsistencies.
Documentation
We have improved our documentation in several places:
Pupillometry best practices - pupil-docs#453
Blink detection - pupil-docs#463
Pupil Core API - pupil-docs#455
Fixed
Fixed typo in exported headpose tracker files - #2185
Fix confidence graphs in Pupil Player for pre-2.0 recordings - #2191
Use correct socket API to retrieve local IP address for Service - #2201
Fix loading 200 Hz gaze data in Pupil Invisible recordings - #2204
Fix loading short audio streams - #2206
Known issues
Windows: Connecting remotely to Pupil Remote on Pupil Service does not work in some cases
Pupil Groups: Users have reported issues with Pupil Groups connectivity when running Pupil Capture on different devices.
We are investigating these issues and will try to resolve them with an update as soon as possible.
Developer notes
Dependencies updates
Python dependencies can be updated using pip
and the requirements.txt
file:
python -m pip install --upgrade pip wheel
pip install -r requirements.txt
pye3d v0.2.0 and v0.3.0
Version 0.2.0 of pye3d
has lowered 3d-search-result confidence by 40%. When the confidence of the input 2d pupil datum is lower than a specific threshold (0.7), pye3d attempts to reconstruct the 2d pupil ellipse based on its existing 3d eye model. We noticed that the confidence for these results was systematically overestimated given the actual detection result quality and have therefore adjusted the overall confidence assignment for them. See pye3d-detector#36 for reference.
Version 0.3.0 uses a new serialization format for the refractionizer parameters but is otherwise functionally equivalent to version 0.2.0.
pyglui v1.31.0
This new version of pyglui
introduces support for multi-line thumb and icon labels.
Example:
from pyglui import ui
context = {"state": False}
ui.Icon(
"state",
context,
label="Multi\nLine",
label_offset_size=-10, # smaller font
label_offset_x=0, # no horizontal adjustments
label_offset_y=-10, # move label upwards
label_line_height=0.8, # reduce line height
)
zeromq-pyre >= v0.3.4
Starting with version v0.3.4, zeromq-pyre
is able to handle cases gracefully where it is not able to access any network interfaces. See #2187 for reference.
Plugin API
Easy plugin icon customization - #2196
Plugins can now overwrite the following class attributes to customize the menu icon:
icon_pos_delta
: relative positioningicon_size_delta
: relative font resizingicon_line_height
(multi-line icons only): line distance
Requires pyglui v1.31.0 or higher
Real-time Network API
Add surf_to_dist_img_trans
and dist_img_to_surf_trans
to surface events #2168
These two homographies are necessary to transform points from the surface coordinate system to the 2d image space of the scene camera, and vice versa. They have been available as part of the surface location exports in Pupil Player. With this release, they also become available to other plugins in Pupil Capture or via its Network API.
Example plugin that access the new homographies in real-time:
import numpy as np
from plugin import Plugin
class Example(Plugin):
def recent_events(self, events):
# access surfaces or fallback to empty list if not available
surfaces = events.get("surfaces", [])
for surface in surfaces:
# access raw homography data, a list of lists
homography = surface["surf_to_dist_img_trans"]
# convert to a matrix
homography = np.array(homography)
# use homography, e.g. printing it
print(f"{surface['name']}: {homography}")
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, such as WinRAR or 7-Zip (both are available for free).
Supported operating systems
macOS Mojave 10.14, macOS Catalina 10.15, macOS Big Sur 11
macOS Monterey 12 requires administrator privileges to access the video cameras
Ubuntu 16.04 or newer
Windows 10
Checksums
$ shasum -a 256 pupil_v3.5*
41428173c33d07171bb7bf8f0c83b7debfabb66a1dfc7530d1ae302be70ccc9c pupil_v3.5-1-g1cdbe38_windows_x64.msi.rar
608aceaf4bf450254812b3fcaf8ea553450fcf2d626ae9c2ed0404348918d6ad pupil_v3.5-7-g651931ba_macos_x64.dmg
786feb5e937a68a21821ab6b6d86be915bcbb7c5e24c11664dbf8a95e5fcb8cc pupil_v3.5-8-g0c019f6_linux_x64.zip
November 29, 2021
We are pleased to announce the release of Pupil Core software v3.5!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord! 😄
Overview
In Pupil v3.5, we cleaned up log messages, improved software stability, fixations are now cached between Pupil Player sessions, and we have fine tuned 3d pupil confidence.
Pupil v3.5 is the first release that supports the new macOS Monterey (with a small caveat). Read more below.
New
Recording-specific annotation hotkeys
Add option to filter out low-confidence values in raw data export
Add option to disable world video recording
macOS Monterey support
Improved
Reduced number of user-facing log messages
Export-in-progress file-name suffixes
Windows: Handle exceptions during driver installation gracefully
Windows: Option to skip automatic driver installation
Windows: Create start menu entries and desktop shortcuts for all users on install
Add Euler angles to headpose tracker result
Normalize IMU pitch/roll values to [-180, 180] degrees
Confidence adjustment for pye3d results
Cache post-hoc fixation detector results
Documentation
New
Recording-specific annotation hotkeys - #2170
Until now, the keyboard-shortcut setup for annotations was stored in Pupil Player's session settings. These were reset if you updated Pupil Core software to a new version.
Starting with this version, annotation keyboard shortcuts will be stored persistently within the recording.
The definitions are stored in <recording dir>/offline_data/annotation_definitions.json
with the format:
{
"version": 1,
"definitions": {
"<label>": "<hotkey>"
}
}
This file can be shared between recordings.
When Pupil Player is opened, it will attempt to load the annotation definitions from the recording-specific file. If the file is not found or is invalid, Pupil Player will fallback to the session settings which contain the last known annotation definitions.
Add option to filter out low-confidence values in raw data export - #2210
Pupil Core software assigns "confidence" values to its pupil detections and gaze estimations. They indicate the quality of the measurement. For visualization and some analysis Plugins, Pupil Player hides gaze data below the "Minimum data confidence" threshold (adjustable in the general settings menu).
Prior to this release, the raw data exporter plugin exported all data, independently of its confidence. New users, therefore, often wonder why their exported data looks noisier than what they have seen in Pupil Player.
Starting with this release, users will have the option to filter out low-confidence data (i.e. below the "Minimum data confidence" threshold) in the export. The option can be enabled in the raw data exporter's menu.
Add option to disable world video recording - #2169
If you are not interested in recording the scene video, you can now disable the scene video recording in the Recorder
menu of Pupil Capture's world
window.
macOS Monterey support
Due to new technical limitations, Pupil Capture and Pupil Service need to be started with administrator privileges to get access to the video camera feeds. To do that, copy the applications into your /Applications
folder and run the corresponding command from the terminal:
Pupil Capture:
sudo /Applications/Pupil\ Capture.app/Contents/MacOS/pupil_capture
Pupil Service:
sudo /Applications/Pupil\ Service.app/Contents/MacOS/pupil_service
Note: The terminal will prompt you for your administrator password. It will not preview any typed keys. Simply hit enter once the password has been typed.
Note: When recording with administrator privileges, the resulting folder inherits admin file permissions. Pupil Player will detect these and ask you for the administrator password to reset the file permissions. This will be only necessary once per recording.
Improved
Export-in-progress file-name suffixes - #2167, #2186
Exporting video can take a long time. Pupil Player displays the progress as a growing circle around the menu icon while writing the result incrementally to disk. Specifically, it writes the result directly to the final export location. While the export is ongoing, the video file is invalid but still available to the user. This can lead to users opening the video before it has finished exporting and then encountering playback issues.
Starting with this release, Pupil Core software changes the video exports to include a .writing
file name suffix during the export. This prevents other applications from recognizing the partially exported video file as complete. Once the export is done, the suffix is removed and other applications are able to recognize the file type as video.
Reduced number of user-facing log messages - #2190
Over the years, Pupil Capture's functionality has grown and with it the number of user-facing log messages. They can be overwhelming to the point where important log messages are overlooked or ignored. For this release, we have reviewed the log messages that are being displayed in the UI during typical actions, e.g. calibration and recordings, and have reduced them significantly.
Windows: Handle exceptions during driver installation gracefully - #2173
Previously, errors during the driver installation could cause the software to crash. Now, errors will be logged to the log file and the user will be asked to install the drivers manually.
Windows: Option to skip automatic driver installation - #2207
Using the -skip-driv
or --skip-driver-installation
flags, one can now launch Pupil Capture and Pupil Service without running the automatic driver installation on Windows. This can be helpful to speed up the start up time or to avoid unwanted driver installations.
Windows: Create start menu entries and desktop shortcuts for all users on install - #2166
Pupil Core software is installed to C:\Program Files (x86)\Pupil-Labs
by default. This directory should be accessible by all users but the start menu entry and desktop shortcut are only installed for the current user. Starting with this release, the installer will create start menu entries and desktop shortcuts for all users on install.
Add Euler angles to headpose tracker result - #2175
Next to camera poses as Rodrigues' rotation vectors, the headpose tracker now also exports the camera orientation in Euler angles.
We also fixed a typo in the exported file name.
- head_pose_tacker_model.csv
+ head_pose_tracker_model.csv
Normalize IMU pitch/roll values to [-180, 180] - #2171
Orientation values exported by the IMU Timeline plugin will now be normalized to a range of [-180, 180] degrees.
Confidence adjustment for pye3d results - #2195
In some cases, pye3d was overestimating the confidence of its pupil detections. These were visible as noisy gaze estimations in the Pupil Core applications. This release includes an updated version of pye3d with adjusted confidence estimations. As a result, the gaze signal should look more stable during lower confidence situations, e.g. while squinting or looking down. For more information, see the Developer notes below.
Cache post-hoc fixation detector results - #2194
Post-hoc fixation detector results no longer need to be recalculated if you close and reopen a recording. Pupil Player stores the results together with the used configuration (version of the gaze data and fixation detection parameters) within the recording directory. The cached data will be restored if the Player configuration matches the configuration stored on disk. If it does not match, Player will recalculate the fixations to avoid inconsistencies.
Documentation
We have improved our documentation in several places:
Pupillometry best practices - pupil-docs#453
Blink detection - pupil-docs#463
Pupil Core API - pupil-docs#455
Fixed
Fixed typo in exported headpose tracker files - #2185
Fix confidence graphs in Pupil Player for pre-2.0 recordings - #2191
Use correct socket API to retrieve local IP address for Service - #2201
Fix loading 200 Hz gaze data in Pupil Invisible recordings - #2204
Fix loading short audio streams - #2206
Known issues
Windows: Connecting remotely to Pupil Remote on Pupil Service does not work in some cases
Pupil Groups: Users have reported issues with Pupil Groups connectivity when running Pupil Capture on different devices.
We are investigating these issues and will try to resolve them with an update as soon as possible.
Developer notes
Dependencies updates
Python dependencies can be updated using pip
and the requirements.txt
file:
python -m pip install --upgrade pip wheel
pip install -r requirements.txt
pye3d v0.2.0 and v0.3.0
Version 0.2.0 of pye3d
has lowered 3d-search-result confidence by 40%. When the confidence of the input 2d pupil datum is lower than a specific threshold (0.7), pye3d attempts to reconstruct the 2d pupil ellipse based on its existing 3d eye model. We noticed that the confidence for these results was systematically overestimated given the actual detection result quality and have therefore adjusted the overall confidence assignment for them. See pye3d-detector#36 for reference.
Version 0.3.0 uses a new serialization format for the refractionizer parameters but is otherwise functionally equivalent to version 0.2.0.
pyglui v1.31.0
This new version of pyglui
introduces support for multi-line thumb and icon labels.
Example:
from pyglui import ui
context = {"state": False}
ui.Icon(
"state",
context,
label="Multi\nLine",
label_offset_size=-10, # smaller font
label_offset_x=0, # no horizontal adjustments
label_offset_y=-10, # move label upwards
label_line_height=0.8, # reduce line height
)
zeromq-pyre >= v0.3.4
Starting with version v0.3.4, zeromq-pyre
is able to handle cases gracefully where it is not able to access any network interfaces. See #2187 for reference.
Plugin API
Easy plugin icon customization - #2196
Plugins can now overwrite the following class attributes to customize the menu icon:
icon_pos_delta
: relative positioningicon_size_delta
: relative font resizingicon_line_height
(multi-line icons only): line distance
Requires pyglui v1.31.0 or higher
Real-time Network API
Add surf_to_dist_img_trans
and dist_img_to_surf_trans
to surface events #2168
These two homographies are necessary to transform points from the surface coordinate system to the 2d image space of the scene camera, and vice versa. They have been available as part of the surface location exports in Pupil Player. With this release, they also become available to other plugins in Pupil Capture or via its Network API.
Example plugin that access the new homographies in real-time:
import numpy as np
from plugin import Plugin
class Example(Plugin):
def recent_events(self, events):
# access surfaces or fallback to empty list if not available
surfaces = events.get("surfaces", [])
for surface in surfaces:
# access raw homography data, a list of lists
homography = surface["surf_to_dist_img_trans"]
# convert to a matrix
homography = np.array(homography)
# use homography, e.g. printing it
print(f"{surface['name']}: {homography}")
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, such as WinRAR or 7-Zip (both are available for free).
Supported operating systems
macOS Mojave 10.14, macOS Catalina 10.15, macOS Big Sur 11
macOS Monterey 12 requires administrator privileges to access the video cameras
Ubuntu 16.04 or newer
Windows 10
Checksums
$ shasum -a 256 pupil_v3.5*
41428173c33d07171bb7bf8f0c83b7debfabb66a1dfc7530d1ae302be70ccc9c pupil_v3.5-1-g1cdbe38_windows_x64.msi.rar
608aceaf4bf450254812b3fcaf8ea553450fcf2d626ae9c2ed0404348918d6ad pupil_v3.5-7-g651931ba_macos_x64.dmg
786feb5e937a68a21821ab6b6d86be915bcbb7c5e24c11664dbf8a95e5fcb8cc pupil_v3.5-8-g0c019f6_linux_x64.zip
September 30, 2021
Big release! We added two powerful new features. Workspaces enable collaboration with your colleagues. Face Mapper automatically detects faces in recordings!
Workspaces
Workspaces enable you to collaborate with colleagues through role based access from data collection with Pupil Invisible Companion App to data annotation and enrichment in Pupil Cloud. Workspaces act as isolated spaces that contain recordings, wearers, templates, labels, projects and enrichments.
Note: Update the Pupil Invisible Companion app to the most recent version (v1.3.0) to make use of workspaces.
Face Mapper
Face Mapper is a new enrichment that automatically and robustly detects faces in the scene video and maps gaze data onto the detected faces. This enables you to easily determine when a subject has been looking at a face and to compute aggregate statistics (example: how much time was spent looking at faces?).
Share your thoughts
Have feedback, questions, feature requests - send us your thoughts!
September 30, 2021
Big release! We added two powerful new features. Workspaces enable collaboration with your colleagues. Face Mapper automatically detects faces in recordings!
Workspaces
Workspaces enable you to collaborate with colleagues through role based access from data collection with Pupil Invisible Companion App to data annotation and enrichment in Pupil Cloud. Workspaces act as isolated spaces that contain recordings, wearers, templates, labels, projects and enrichments.
Note: Update the Pupil Invisible Companion app to the most recent version (v1.3.0) to make use of workspaces.
Face Mapper
Face Mapper is a new enrichment that automatically and robustly detects faces in the scene video and maps gaze data onto the detected faces. This enables you to easily determine when a subject has been looking at a face and to compute aggregate statistics (example: how much time was spent looking at faces?).
Share your thoughts
Have feedback, questions, feature requests - send us your thoughts!
July 22, 2021
Head Pitch & Roll Estimates are now available!
The Inertial Measurment Unit (IMU) sensor embedded in the Pupil Invisible Glasses frame provides measurements that can tell us how the wearer’s head is moving. This can be valuable if you want to measure when your subject is looking downwards vs upwards for example.
In this update we have made the outputs of the IMU much easier to work with. Instead of only reporting the raw outputs of the IMU (rotational speed and translational acceleration), we now include estimates of the absolute pitch and roll of the head.
This is part of the Raw Data Exporter. You can find more details about how the estimates are calculated in the docs!
Information on when the Glasses are worn now included in Raw Data Export
The gaze.csv file included in the Raw Data Export now contains a new column called worn that indicates whether the Pupil Invisible Glasses have been worn by a subject at the respective time point (1.0 = worn, 0.0 = not worn).
This data has previously only been available as part of the binary recording data.
Improved Export Format of Marker Mapper
We have updated the export format of the Marker Mapper to be easier to use:
We have added a new column detected markers to the aoi_positions.csv file that contains the IDs of all the markers detected in the respective frame.We have split the corner coordinates in image column in the aoi_positions.csv file in to a separate column per coordinate to make it easier to parse.
Do you have feedback you would like to share?
July 22, 2021
Head Pitch & Roll Estimates are now available!
The Inertial Measurment Unit (IMU) sensor embedded in the Pupil Invisible Glasses frame provides measurements that can tell us how the wearer’s head is moving. This can be valuable if you want to measure when your subject is looking downwards vs upwards for example.
In this update we have made the outputs of the IMU much easier to work with. Instead of only reporting the raw outputs of the IMU (rotational speed and translational acceleration), we now include estimates of the absolute pitch and roll of the head.
This is part of the Raw Data Exporter. You can find more details about how the estimates are calculated in the docs!
Information on when the Glasses are worn now included in Raw Data Export
The gaze.csv file included in the Raw Data Export now contains a new column called worn that indicates whether the Pupil Invisible Glasses have been worn by a subject at the respective time point (1.0 = worn, 0.0 = not worn).
This data has previously only been available as part of the binary recording data.
Improved Export Format of Marker Mapper
We have updated the export format of the Marker Mapper to be easier to use:
We have added a new column detected markers to the aoi_positions.csv file that contains the IDs of all the markers detected in the respective frame.We have split the corner coordinates in image column in the aoi_positions.csv file in to a separate column per coordinate to make it easier to parse.
Do you have feedback you would like to share?
July 13, 2021
We are pleased to announce the release of Pupil Core software v3.4!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord! 😄
Overview
Pupil v3.4 introduces a new color scheme for pupil detection algorithms, improved gaze-estimation stability for frozen eye models, and a new IMU visualization and export plugin for Pupil Invisible recordings within Pupil Player.
New
Dynamic pye3d model confidence
Pupil detection color scheme
Added IMU Timeline plugin
Improved
Gaze-estimation stability with frozen eye models
Annotation log message
Prevent log flood in Surface Tracker related to finding homographies
Support upgrading legacy recordings
New
Dynamic pye3d model confidence
Pupil Core software version v3.4 ships the new pye3d
version 0.1.1.
In the previous versions of pye3d model_confidence
was fixed to a value of 1.0
. Starting with version 0.1.0, pye3d assigns different model_confidence
values based on the long-term model's parameters. Specifically, model_confidence
is set to 1.0
by default and set to 0.1
if at least one model output exceeds its physiologically reasonable range. If the ranges are exceeded, it is likely that the model is either not fit well or the 2d input ellipse was a false detection.
The assumed physiological bounds are:
phi: [-80, 80] degrees
theta: [-80, 80] degrees
pupil diameter: [1.0, 9.0] mm
eyeball center x: [-10, 10] mm
eyeball center y: [-10, 10] mm
eyeball center z: [20, 75] mm
Phi and theta ranges are relative to the eye camera's optical axis. The eye ball center ranges are defined relative to the origin of the eye camera's 3d coordinate system.
The model_confidence
will be set to 0.0
if the gaze direction cannot be calculated.
Pupil detection color scheme
The new pupil detection color scheme makes use of the dynamic model confidence values by visualizing within-bounds and out-of-bounds models in different colors.
Color legend:
#FAC800
: Pupil ellipse (2d)#F92500
: Pupil ellipse (3d)#0777FA
: Long-term model outline (within-bounds)#80E0FF
: Long-term model outline (out-of-bounds)
Note: Even though the detected pupil ellipses fit the pupil well on the right-side picture, the model is outside of the physiological ranges defined above. This can lead to errors in gaze-direction and pupil-size estimates. You can improve such a model by looking into different directions.
While the 3d debug window is opened, pye3d also visualizes the short- and ultra-long-term model outlines:
#FFB876
: Short-term model outline#FF5C16
: Ultra-long-term model outline
Added IMU Timeline plugin - #2151
This is a new plugin designed to visualize and export IMU data available in Pupil Invisible recordings.
In addition to the raw angular velocity and linear acceleration, the plugin also uses Madgwick's algorithm to estimate pitch and roll.
To export the data, the plugin must be enabled and a raw data export must be triggered. Once the export is complete, there should be a imu_timeline.csv
file with the following columns
![](https://framerusercontent.com/images/WgbBis3NuygsnAHnOpxrRlKPCxA.png)
Improved
Improve gaze-estimation stability with frozen eye models
Freezing the eye models is the best way to reduce noisy data when in a controlled environment. Prior to this release, freezing the eye model only affected eye ball center and pupil diameter measurements. Starting with this version, freezing the eye model also stabilizes gaze direction estimates.
In order to adapt to slippage, pye3d
estimates eyeball positions on different time scales. These serve specific downstream purposes. By default, the short-term model, which integrates only the most recent pupil observations, is used for calculating raw gaze-direction estimates.
In highly controlled environments with few eye movements (e.g. subject uses head rest and is asked to fixate a static target), the most recent pupil observations might not be sufficient to build a stable short-term model. If this is the case, the gaze direction varies more than one would expect given the controlled environment.Starting in pye3d
version 0.1.0, the gaze direction will be inferred from the long-term model if it is frozen.
Freezing the eye model is only recommended in controlled environments as this prevents the model from adapting to slippage. In turn, it is able to provide very stable data.
Improve annotation log message - #2148
When creating a new annotation, the log message will no longer show the Pupil Timestamp. Instead, Pupil Capture will show how old the annotation is, while Pupil Player will show the frame index of the world video.
This change should help avoid the confusion associated with displaying Pupil Timestamp.
Prevent log flood in Surface Tracker related to finding homographies - #2150
Previously, failing to locate a surface because of missing homographies would print an error message in the main window. In some cases, this would result in a flood of error logs.
This change ensures that failures of such type are only issued once, and are written to the log file directly.
Support upgrading legacy recordings - #2161
Pupil Core software version v1.16 is no longer necessary to upgrade deprecated recording formats. Instead, Pupil v3.4 performs all necessary upgrades by itself. This release also includes a bug fix for an issue with pre v0.7.4 recordings.
In v1.16, recordings made with Pupil Capture v1.2
or earlier, andPupil Mobile r0.22.x
or earlier have been deprecated due to the fact that these recordings are missing meta information that is required for the upgrade to the Pupil Player Recording Format 2.0. For details see "Missing Meta Information" in the v1.16 release notes.
Fixed
Use UTF-8 decoding for known UTF-8 encoded text files - #2146
Update circle drawing to display correctly on Apple Silicon devices - #2147
Fix crash on surface tracker export if filling the marker cache is not complete - #2157
Fix surface tracker marker export to use export range - #2158
Developer notes
Dependencies updates
Python dependencies can be updated using pip
and the requirements.txt
file:
python -m pip install --upgrade pip wheel
pip install -r requirements.txt
pyglui v1.30.0
This new version of pyglui
includes the menu icon for the new IMU timeline plugin as well as the new Color_Legend
UI element.
from pyglui import ui
color_rgb_red = (1.0, 0.0, 0.0)
label = "red color label"
ui.Color_Legend(color_rgb_red, label)
pye3d v0.1.1
This version improves gaze-estimation stability for frozen eye models. For details see above and the pye3d
changelog.
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, such as WinRAR or 7-Zip (both are available for free).
Supported operating systems
Windows 10
Ubuntu 16.04+
macOS Mojave 10.14+
macOS Monterey 12 is not supported by this release
July 13, 2021
We are pleased to announce the release of Pupil Core software v3.4!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord! 😄
Overview
Pupil v3.4 introduces a new color scheme for pupil detection algorithms, improved gaze-estimation stability for frozen eye models, and a new IMU visualization and export plugin for Pupil Invisible recordings within Pupil Player.
New
Dynamic pye3d model confidence
Pupil detection color scheme
Added IMU Timeline plugin
Improved
Gaze-estimation stability with frozen eye models
Annotation log message
Prevent log flood in Surface Tracker related to finding homographies
Support upgrading legacy recordings
New
Dynamic pye3d model confidence
Pupil Core software version v3.4 ships the new pye3d
version 0.1.1.
In the previous versions of pye3d model_confidence
was fixed to a value of 1.0
. Starting with version 0.1.0, pye3d assigns different model_confidence
values based on the long-term model's parameters. Specifically, model_confidence
is set to 1.0
by default and set to 0.1
if at least one model output exceeds its physiologically reasonable range. If the ranges are exceeded, it is likely that the model is either not fit well or the 2d input ellipse was a false detection.
The assumed physiological bounds are:
phi: [-80, 80] degrees
theta: [-80, 80] degrees
pupil diameter: [1.0, 9.0] mm
eyeball center x: [-10, 10] mm
eyeball center y: [-10, 10] mm
eyeball center z: [20, 75] mm
Phi and theta ranges are relative to the eye camera's optical axis. The eye ball center ranges are defined relative to the origin of the eye camera's 3d coordinate system.
The model_confidence
will be set to 0.0
if the gaze direction cannot be calculated.
Pupil detection color scheme
The new pupil detection color scheme makes use of the dynamic model confidence values by visualizing within-bounds and out-of-bounds models in different colors.
Color legend:
#FAC800
: Pupil ellipse (2d)#F92500
: Pupil ellipse (3d)#0777FA
: Long-term model outline (within-bounds)#80E0FF
: Long-term model outline (out-of-bounds)
Note: Even though the detected pupil ellipses fit the pupil well on the right-side picture, the model is outside of the physiological ranges defined above. This can lead to errors in gaze-direction and pupil-size estimates. You can improve such a model by looking into different directions.
While the 3d debug window is opened, pye3d also visualizes the short- and ultra-long-term model outlines:
#FFB876
: Short-term model outline#FF5C16
: Ultra-long-term model outline
Added IMU Timeline plugin - #2151
This is a new plugin designed to visualize and export IMU data available in Pupil Invisible recordings.
In addition to the raw angular velocity and linear acceleration, the plugin also uses Madgwick's algorithm to estimate pitch and roll.
To export the data, the plugin must be enabled and a raw data export must be triggered. Once the export is complete, there should be a imu_timeline.csv
file with the following columns
![](https://framerusercontent.com/images/WgbBis3NuygsnAHnOpxrRlKPCxA.png)
Improved
Improve gaze-estimation stability with frozen eye models
Freezing the eye models is the best way to reduce noisy data when in a controlled environment. Prior to this release, freezing the eye model only affected eye ball center and pupil diameter measurements. Starting with this version, freezing the eye model also stabilizes gaze direction estimates.
In order to adapt to slippage, pye3d
estimates eyeball positions on different time scales. These serve specific downstream purposes. By default, the short-term model, which integrates only the most recent pupil observations, is used for calculating raw gaze-direction estimates.
In highly controlled environments with few eye movements (e.g. subject uses head rest and is asked to fixate a static target), the most recent pupil observations might not be sufficient to build a stable short-term model. If this is the case, the gaze direction varies more than one would expect given the controlled environment.Starting in pye3d
version 0.1.0, the gaze direction will be inferred from the long-term model if it is frozen.
Freezing the eye model is only recommended in controlled environments as this prevents the model from adapting to slippage. In turn, it is able to provide very stable data.
Improve annotation log message - #2148
When creating a new annotation, the log message will no longer show the Pupil Timestamp. Instead, Pupil Capture will show how old the annotation is, while Pupil Player will show the frame index of the world video.
This change should help avoid the confusion associated with displaying Pupil Timestamp.
Prevent log flood in Surface Tracker related to finding homographies - #2150
Previously, failing to locate a surface because of missing homographies would print an error message in the main window. In some cases, this would result in a flood of error logs.
This change ensures that failures of such type are only issued once, and are written to the log file directly.
Support upgrading legacy recordings - #2161
Pupil Core software version v1.16 is no longer necessary to upgrade deprecated recording formats. Instead, Pupil v3.4 performs all necessary upgrades by itself. This release also includes a bug fix for an issue with pre v0.7.4 recordings.
In v1.16, recordings made with Pupil Capture v1.2
or earlier, andPupil Mobile r0.22.x
or earlier have been deprecated due to the fact that these recordings are missing meta information that is required for the upgrade to the Pupil Player Recording Format 2.0. For details see "Missing Meta Information" in the v1.16 release notes.
Fixed
Use UTF-8 decoding for known UTF-8 encoded text files - #2146
Update circle drawing to display correctly on Apple Silicon devices - #2147
Fix crash on surface tracker export if filling the marker cache is not complete - #2157
Fix surface tracker marker export to use export range - #2158
Developer notes
Dependencies updates
Python dependencies can be updated using pip
and the requirements.txt
file:
python -m pip install --upgrade pip wheel
pip install -r requirements.txt
pyglui v1.30.0
This new version of pyglui
includes the menu icon for the new IMU timeline plugin as well as the new Color_Legend
UI element.
from pyglui import ui
color_rgb_red = (1.0, 0.0, 0.0)
label = "red color label"
ui.Color_Legend(color_rgb_red, label)
pye3d v0.1.1
This version improves gaze-estimation stability for frozen eye models. For details see above and the pye3d
changelog.
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, such as WinRAR or 7-Zip (both are available for free).
Supported operating systems
Windows 10
Ubuntu 16.04+
macOS Mojave 10.14+
macOS Monterey 12 is not supported by this release
June 15, 2021
Reference Image Mapper
The Reference Image Mapper is a powerful enrichment that enables you to automatically map gaze to an object of interest. This enrichment works best when you provide optimal reference image and scanning recording, and understand the limitations of the tool. To make this easier we have created an explainer video and integrated it directly into the enrichment creator.
We also added the reference image that was used to create the enrichment in the download. This ensures that this important image is always available for further analysis.
Undistorted Gaze Overlay Videos
![Cloud Gaze overlay](https://framerusercontent.com/images/hlGUQKtnAeHYuqsCf7cxM2VTFU.jpeg)
Pupil Invisible's world camera uses a wide-angle lens to capture a large field of view which introduces optical distortion. We have introduced a new setting to the Gaze Overlay enrichment that enables you to removing this distortion from the video.
In-App Feature Notifications
We are regularly adding new features to Pupil Cloud and want to make sure every user stays on top of what is currently available and possible. New features will be announced within Pupil Cloud itself.
June 15, 2021
Reference Image Mapper
The Reference Image Mapper is a powerful enrichment that enables you to automatically map gaze to an object of interest. This enrichment works best when you provide optimal reference image and scanning recording, and understand the limitations of the tool. To make this easier we have created an explainer video and integrated it directly into the enrichment creator.
We also added the reference image that was used to create the enrichment in the download. This ensures that this important image is always available for further analysis.
Undistorted Gaze Overlay Videos
![Cloud Gaze overlay](https://framerusercontent.com/images/hlGUQKtnAeHYuqsCf7cxM2VTFU.jpeg)
Pupil Invisible's world camera uses a wide-angle lens to capture a large field of view which introduces optical distortion. We have introduced a new setting to the Gaze Overlay enrichment that enables you to removing this distortion from the video.
In-App Feature Notifications
We are regularly adding new features to Pupil Cloud and want to make sure every user stays on top of what is currently available and possible. New features will be announced within Pupil Cloud itself.
May 12, 2021
We are pleased to announce the release of Pupil Core software v3.3!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord! 😄
Overview
Pupil v3.3 focuses on improving software usability, stability and performance. We have made improvements to how product-specific plugins are loaded in Pupil Player, updated keyboard shortcuts, and fixed several stability issues. Additionally, the surface tracker now exports marker detection results.
Note: Prior to this release,diameter_3d
data for all Pupil Core v3 recordings did not have corneal refraction applied. See pye3d v0.0.7 notes below for updates and post-hoc fixes.
Improved
pye3d v0.0.7
This release comes with our latest pye3d version including the following improvements
Performance improvements - #27
Simplification of
Conic
parameter calculation - #26Correctly apply corneal-refraction correction to
diameter_3d
result - #28
All pye3d versions prior to 0.0.7 do not correctly apply corneal refraction to the diameter_3d
field. To check if you are affected by this bug, take a look at the method
column of your pupil_positions.csv
export.
You have two options to get correctly calculated pupil size data:
Use the
circle_3d_radius
values and multiply them by 2, orRe-run the post-hoc pupil detection in Pupil Player v3.3 or later
Marker detection export - #2140
The Surface Tracker plugin requires fiducial markers for defining and tracking areas of interest. Until now, the detection result of these was only stored as an intermediate result. With this release, the data is also stored as a CSV file next to other surface-tracking related exports.
The data is exported to surfaces/marker_detections.csv
.
![](https://framerusercontent.com/images/hQWqpXsNsC04HTh90fcLBB4bKI.png)
Improve surface detection consistency - #2139
Surfaces can now be defined as long as one or more markers are detected in the scene video. Previously, this was only sufficient in some cases -- usually, at least two markers were needed.
Note: Surface tracking works best if you use multiple markers spread-out over the area of interest to define and track your surface. The surface position export files contains information about how many markers were detected for each surface position. You can use this information as an estimation of the tracking stability.
Product-specific plugin loading in Pupil Player, part 2 - #2125
We continue work on finetuning the list of enabled plugins for Pupil Invisible recordings.
Starting with this release we have removed the “Pupil from recording” option, as there is no pupillometry data to load from Pupil Invisible recordings. Pupil Player will no longer generate an empty pupil_positions.csv
file after export.
Keyboard shortcuts update - #2129
The default annotation keyboard shortcut was changed to x
in both Pupil Capture and Pupil Player to avoid collision with other plugin shortcuts.
We've posted tables of keyboard shortcuts for Pupil Capture and Pupil Player below. You can find these tables in our Capture and Player documentation.
Fixed
Fixed HMD calibration crashing Pupil Service - #2120
Fixed Pupil Groups serialization issue - #2124
Fixed toggling pupil detection on and off - #2126
Fixed calibration area calculation errors - #2128
Fixed video source replacement in eye process - #2131
Fixed gaze visualization on Apple Silicon computers - #2132
Fixed Pupil Invisible timestamp update procedure precision loss - #2134
Made pye3d plugin's debug visualization colors consistent - #2136
Fixed validation crash when not enough reference data is collected - #2137
Fixed a crash when using Time Sync on macOS - #2138
Developer notes
Dependencies updates
Python dependencies can be updated using pip
and the requirements.txt
file:
python -m pip install --upgrade pip wheel
pip install -r requirements.txt
We have specifically updated the version requirement for pye3d.
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, such as WinRAR or 7-Zip (both are available for free).
May 12, 2021
We are pleased to announce the release of Pupil Core software v3.3!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord! 😄
Overview
Pupil v3.3 focuses on improving software usability, stability and performance. We have made improvements to how product-specific plugins are loaded in Pupil Player, updated keyboard shortcuts, and fixed several stability issues. Additionally, the surface tracker now exports marker detection results.
Note: Prior to this release,diameter_3d
data for all Pupil Core v3 recordings did not have corneal refraction applied. See pye3d v0.0.7 notes below for updates and post-hoc fixes.
Improved
pye3d v0.0.7
This release comes with our latest pye3d version including the following improvements
Performance improvements - #27
Simplification of
Conic
parameter calculation - #26Correctly apply corneal-refraction correction to
diameter_3d
result - #28
All pye3d versions prior to 0.0.7 do not correctly apply corneal refraction to the diameter_3d
field. To check if you are affected by this bug, take a look at the method
column of your pupil_positions.csv
export.
You have two options to get correctly calculated pupil size data:
Use the
circle_3d_radius
values and multiply them by 2, orRe-run the post-hoc pupil detection in Pupil Player v3.3 or later
Marker detection export - #2140
The Surface Tracker plugin requires fiducial markers for defining and tracking areas of interest. Until now, the detection result of these was only stored as an intermediate result. With this release, the data is also stored as a CSV file next to other surface-tracking related exports.
The data is exported to surfaces/marker_detections.csv
.
![](https://framerusercontent.com/images/hQWqpXsNsC04HTh90fcLBB4bKI.png)
Improve surface detection consistency - #2139
Surfaces can now be defined as long as one or more markers are detected in the scene video. Previously, this was only sufficient in some cases -- usually, at least two markers were needed.
Note: Surface tracking works best if you use multiple markers spread-out over the area of interest to define and track your surface. The surface position export files contains information about how many markers were detected for each surface position. You can use this information as an estimation of the tracking stability.
Product-specific plugin loading in Pupil Player, part 2 - #2125
We continue work on finetuning the list of enabled plugins for Pupil Invisible recordings.
Starting with this release we have removed the “Pupil from recording” option, as there is no pupillometry data to load from Pupil Invisible recordings. Pupil Player will no longer generate an empty pupil_positions.csv
file after export.
Keyboard shortcuts update - #2129
The default annotation keyboard shortcut was changed to x
in both Pupil Capture and Pupil Player to avoid collision with other plugin shortcuts.
We've posted tables of keyboard shortcuts for Pupil Capture and Pupil Player below. You can find these tables in our Capture and Player documentation.
Fixed
Fixed HMD calibration crashing Pupil Service - #2120
Fixed Pupil Groups serialization issue - #2124
Fixed toggling pupil detection on and off - #2126
Fixed calibration area calculation errors - #2128
Fixed video source replacement in eye process - #2131
Fixed gaze visualization on Apple Silicon computers - #2132
Fixed Pupil Invisible timestamp update procedure precision loss - #2134
Made pye3d plugin's debug visualization colors consistent - #2136
Fixed validation crash when not enough reference data is collected - #2137
Fixed a crash when using Time Sync on macOS - #2138
Developer notes
Dependencies updates
Python dependencies can be updated using pip
and the requirements.txt
file:
python -m pip install --upgrade pip wheel
pip install -r requirements.txt
We have specifically updated the version requirement for pye3d.
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, such as WinRAR or 7-Zip (both are available for free).
May 11, 2021
Improved: Enrichment statuses have been renamed for more clarity:
‘Processing’ is now ‘In progress’
‘Ready’ is now ‘Ready to Compute’
‘Computed’ is now ‘Done’
Improved: Reference Image Mapper now accepts PNG images as well as JPG images.
May 11, 2021
Improved: Enrichment statuses have been renamed for more clarity:
‘Processing’ is now ‘In progress’
‘Ready’ is now ‘Ready to Compute’
‘Computed’ is now ‘Done’
Improved: Reference Image Mapper now accepts PNG images as well as JPG images.
May 10, 2021
New: Volume controls! Now all video players in app have a volume slider. Tip: Click the volume icon to mute.
May 10, 2021
New: Volume controls! Now all video players in app have a volume slider. Tip: Click the volume icon to mute.
April 12, 2021
Local Recording Export
Pupil Cloud is the recommended and most convenient method for moving recordings off of the Companion Device. However, we recognize that some users may not be able to use Pupil Cloud or have their own data processing pipelines running locally. Therefore, we implemented a feature to make it easier for users to export recordings from the Companion Device using a USB connection to a computer. This feature ensures that internal files required by the Pupil Invisible Companion app are excluded from the exported recordings, thereby maintaining a consistent file format. Check out the user guide in docs for more.
Watchdog Service Update
In our previous update, we have introduced the “watchdog service” that automatically detects issues like e.g. faulty USB connections at recording time and warns you about it. If there is an issue the Companion device will vibrate, a red LED on the Pupil Invisible Glasses will blink and the Companion app will let you know what the problem is.
This is an incremental update to this service improving its robustness and the range of failure modes it can detect.
info.json Update
The info.json file is saved with each recording, containing meta data about the recording. This file is primarily used by other software reading the recordings and developers. We have added two new keys to the info.json files that are saved with each recording: os_version and recovered_files. Check out the entire recording data format in docs.
April 12, 2021
Local Recording Export
Pupil Cloud is the recommended and most convenient method for moving recordings off of the Companion Device. However, we recognize that some users may not be able to use Pupil Cloud or have their own data processing pipelines running locally. Therefore, we implemented a feature to make it easier for users to export recordings from the Companion Device using a USB connection to a computer. This feature ensures that internal files required by the Pupil Invisible Companion app are excluded from the exported recordings, thereby maintaining a consistent file format. Check out the user guide in docs for more.
Watchdog Service Update
In our previous update, we have introduced the “watchdog service” that automatically detects issues like e.g. faulty USB connections at recording time and warns you about it. If there is an issue the Companion device will vibrate, a red LED on the Pupil Invisible Glasses will blink and the Companion app will let you know what the problem is.
This is an incremental update to this service improving its robustness and the range of failure modes it can detect.
info.json Update
The info.json file is saved with each recording, containing meta data about the recording. This file is primarily used by other software reading the recordings and developers. We have added two new keys to the info.json files that are saved with each recording: os_version and recovered_files. Check out the entire recording data format in docs.
March 11, 2021
We are pleased to announce the release of Pupil Core software v3.2!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord! 😄
Overview
Pupil v3.2 focuses on quality-of-life and stability.
New
We added accuracy calculation and visualisation for VR/AR calibrations.
Enable plugins in Pupil Player only if they are able to be used with the product that created the recordings.
Improved
Pupil Player has improved playback for Pupil Invisible recordings.
New
Accuracy visualisation and calculation for HMD calibrations - #2112
Prior to this release, when performing HMD calibration, accuracy was not calculated nor visualised in Pupil Capture. This release makes accuracy calculation and visualisation consistent across setups.
Product specific Pupil Player plugin loading - #2100
Load plugins in Pupil Player only if they are able to be used with the product that created the recordings.
Pupil Player is a very flexible tool. It can load, playback, visualize, and export data from recordings made with Pupil Core, Pupil Mobile, and Pupil Invisible! Each of these products has different capabilities and data formats for their recordings.
We have evaluated all Pupil Player plugins and their validity for each product. Starting with this release, Pupil Player will only enable plugins that are suitable for the active recording type.
Here is a list of what plugins are available based on the product that generated the recording:
![](https://framerusercontent.com/images/L0MFxDYcApB7pvPLpoZ1SrmrhA.png)
Improved
pye3d detection speed improvement
We are aware that pye3d requires more computational resources than its predecessor, and are actively working to improve the speed. For some users, it has become more difficult to reach a consistent detection rate of 200Hz.
The first step we have taken to speed up pye3d is to disable some optional computations that are not currently used by Pupil Capture and Pupil Player (jump to developer notes for details).
Fixed
Support recordings with decoding issues - #2107
Support recordings when an audio device fails - #2103
Fixed laggy playback after seeking on Pupil Invisible recordings - #2108
Developer notes
Dependencies updates
Python dependencies can be updated using pip
and the requirements.txt
file:
python -m pip install --upgrade pip wheel
pip install -r requirements.txt
pye3d v0.0.6
pye3d v0.0.6 allows disabling optional computations. Pupil Core is not making use of these optional computations, they have been disabled, effectively reducing the computational load of pye3d pupil detections. Read about the changes here.
Release Note Updates
15.03.2021 - v3.2-16
Fixes an issue causing Pupil Player to crash on Windows when enabling the eye video overlay or post-hoc pupil detection - #2117
21.03.2021 - v3.2-20
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, such as WinRAR or 7-Zip (both are available for free).
March 11, 2021
We are pleased to announce the release of Pupil Core software v3.2!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord! 😄
Overview
Pupil v3.2 focuses on quality-of-life and stability.
New
We added accuracy calculation and visualisation for VR/AR calibrations.
Enable plugins in Pupil Player only if they are able to be used with the product that created the recordings.
Improved
Pupil Player has improved playback for Pupil Invisible recordings.
New
Accuracy visualisation and calculation for HMD calibrations - #2112
Prior to this release, when performing HMD calibration, accuracy was not calculated nor visualised in Pupil Capture. This release makes accuracy calculation and visualisation consistent across setups.
Product specific Pupil Player plugin loading - #2100
Load plugins in Pupil Player only if they are able to be used with the product that created the recordings.
Pupil Player is a very flexible tool. It can load, playback, visualize, and export data from recordings made with Pupil Core, Pupil Mobile, and Pupil Invisible! Each of these products has different capabilities and data formats for their recordings.
We have evaluated all Pupil Player plugins and their validity for each product. Starting with this release, Pupil Player will only enable plugins that are suitable for the active recording type.
Here is a list of what plugins are available based on the product that generated the recording:
![](https://framerusercontent.com/images/L0MFxDYcApB7pvPLpoZ1SrmrhA.png)
Improved
pye3d detection speed improvement
We are aware that pye3d requires more computational resources than its predecessor, and are actively working to improve the speed. For some users, it has become more difficult to reach a consistent detection rate of 200Hz.
The first step we have taken to speed up pye3d is to disable some optional computations that are not currently used by Pupil Capture and Pupil Player (jump to developer notes for details).
Fixed
Support recordings with decoding issues - #2107
Support recordings when an audio device fails - #2103
Fixed laggy playback after seeking on Pupil Invisible recordings - #2108
Developer notes
Dependencies updates
Python dependencies can be updated using pip
and the requirements.txt
file:
python -m pip install --upgrade pip wheel
pip install -r requirements.txt
pye3d v0.0.6
pye3d v0.0.6 allows disabling optional computations. Pupil Core is not making use of these optional computations, they have been disabled, effectively reducing the computational load of pye3d pupil detections. Read about the changes here.
Release Note Updates
15.03.2021 - v3.2-16
Fixes an issue causing Pupil Player to crash on Windows when enabling the eye video overlay or post-hoc pupil detection - #2117
21.03.2021 - v3.2-20
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, such as WinRAR or 7-Zip (both are available for free).
February 26, 2021
New: Reference Image Mapper enrichment. This enrichment enables you to track areas of interest in your environment based on a reference image of them. See our blog post on this and the docs.
Enrichment: Gaze Overlay Export enrichment. This enrichment enables you to export world video with gaze overlay directly from Pupil Cloud. See docs.
New: Raw Data Export enrichment. This enrichment allows you to export all recording data in easy to use CSV and MP4 file formats for further processing. See docs.
New: All data formats for all enrichments are now documented in detail in the docs.
Improved: Help menu in left side bar has been updated for quicker access to all important resources.
February 26, 2021
New: Reference Image Mapper enrichment. This enrichment enables you to track areas of interest in your environment based on a reference image of them. See our blog post on this and the docs.
Enrichment: Gaze Overlay Export enrichment. This enrichment enables you to export world video with gaze overlay directly from Pupil Cloud. See docs.
New: Raw Data Export enrichment. This enrichment allows you to export all recording data in easy to use CSV and MP4 file formats for further processing. See docs.
New: All data formats for all enrichments are now documented in detail in the docs.
Improved: Help menu in left side bar has been updated for quicker access to all important resources.
February 10, 2021
We are pleased to announce the release of Pupil Core software v3.1!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord! 😄
Motivation
Pupil Core software is now compatible with macOS 11 Big Sur! In addition, we have added support in Pupil Player for 200Hz gaze data downloaded from Pupil Cloud.
Improvements
macOS 11 Big Sur compatibility - #2086, #2087, #2090, #2095
macOS 11 Big Sur introduced backwards-incompatible changes that caused earlier versions of Pupil Core and other software to not run properly.
There is currently only a single Python version, Python 3.9.1, that is fully compatible with macOS Big Sur. As a result, our new macOS Pupil Core bundle ships with this new Python version. This may have an effect on user plugin dependencies. For more information, check out the developer notes below.
Support for Pupil Cloud's 200Hz gaze recordings - #2082
Due to the computational limitations on the Companion Device, gaze data is recorded at 66Hz. Recent updates to Pupil Cloud automatically "upgrades" the gaze data recordings when they are uploaded to Pupil Cloud providing you with 200Hz gaze data. In version 3.1, Pupil Player can read and export the new data.
Note: Recordings that were downloaded prior to this Pupil Cloud update need to be re-downloaded in order to include the 200Hz gaze data.
Changes
Visual calibration marker appearance - #2095
The visual appearance of the calibration marker changed slightly, due to how we render the calibration marker on screen. The change was necessary to support screen-based calibrations on Apple Silicon Macs. As a result, the marker edges do not appear as smooth as the previous version. The overall design and detection process remains the same.
Bug Fixes
Gracefully handle Pupil Invisible video files starting with a null frame - #2077, #2089
Handle zero-byte video files when looking up Pupil Mobile intrinsics #2083
Developer notes
Dependencies updates
Python dependencies can be updated using pip
and the requirements.txt
file:
python -m pip install --upgrade pip wheel
pip install -r requirements.txt
Support latest GLEW 2.2 - pyglui #117
This release also adds support for the latest version of GLEW. This means that the pyglui
dependency needs to be updated to version 1.29
.
pye3d v0.0.5
We have included pye3d==0.0.5
which fixes compatibility issues with numpy==1.20
and scikit-learn==0.24.1
.
macOS bundle shipping with Python 3.9.1
To simplify the usage of the application bundles, each bundle is shipped with their own Python environment. All plugins and their dependencies will run in the context of this environment.
Especially compiled dependencies, e.g. pyrealsense2, are always compiled against a specific Python version. If you are running custom user plugins, you will have to reinstall compiled dependencies in order to make them work with the new Python version.
Bundles ship with the following Python versions:
macOS: Python 3.9.1
Windows bundle: Python 3.6
Linux bundle: Python 3.6
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, such as WinRAR or 7-Zip (both are available for free).
macOS compatibility - #2086, #2087
Note: This is the first release to support macOS 11 Big Sur. We no longer support macOS 10.13 High Sierra since the macOS package manager homebrew no longer provides pre-compiled binaries for Pupil Core software dependencies.
February 10, 2021
We are pleased to announce the release of Pupil Core software v3.1!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord! 😄
Motivation
Pupil Core software is now compatible with macOS 11 Big Sur! In addition, we have added support in Pupil Player for 200Hz gaze data downloaded from Pupil Cloud.
Improvements
macOS 11 Big Sur compatibility - #2086, #2087, #2090, #2095
macOS 11 Big Sur introduced backwards-incompatible changes that caused earlier versions of Pupil Core and other software to not run properly.
There is currently only a single Python version, Python 3.9.1, that is fully compatible with macOS Big Sur. As a result, our new macOS Pupil Core bundle ships with this new Python version. This may have an effect on user plugin dependencies. For more information, check out the developer notes below.
Support for Pupil Cloud's 200Hz gaze recordings - #2082
Due to the computational limitations on the Companion Device, gaze data is recorded at 66Hz. Recent updates to Pupil Cloud automatically "upgrades" the gaze data recordings when they are uploaded to Pupil Cloud providing you with 200Hz gaze data. In version 3.1, Pupil Player can read and export the new data.
Note: Recordings that were downloaded prior to this Pupil Cloud update need to be re-downloaded in order to include the 200Hz gaze data.
Changes
Visual calibration marker appearance - #2095
The visual appearance of the calibration marker changed slightly, due to how we render the calibration marker on screen. The change was necessary to support screen-based calibrations on Apple Silicon Macs. As a result, the marker edges do not appear as smooth as the previous version. The overall design and detection process remains the same.
Bug Fixes
Gracefully handle Pupil Invisible video files starting with a null frame - #2077, #2089
Handle zero-byte video files when looking up Pupil Mobile intrinsics #2083
Developer notes
Dependencies updates
Python dependencies can be updated using pip
and the requirements.txt
file:
python -m pip install --upgrade pip wheel
pip install -r requirements.txt
Support latest GLEW 2.2 - pyglui #117
This release also adds support for the latest version of GLEW. This means that the pyglui
dependency needs to be updated to version 1.29
.
pye3d v0.0.5
We have included pye3d==0.0.5
which fixes compatibility issues with numpy==1.20
and scikit-learn==0.24.1
.
macOS bundle shipping with Python 3.9.1
To simplify the usage of the application bundles, each bundle is shipped with their own Python environment. All plugins and their dependencies will run in the context of this environment.
Especially compiled dependencies, e.g. pyrealsense2, are always compiled against a specific Python version. If you are running custom user plugins, you will have to reinstall compiled dependencies in order to make them work with the new Python version.
Bundles ship with the following Python versions:
macOS: Python 3.9.1
Windows bundle: Python 3.6
Linux bundle: Python 3.6
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, such as WinRAR or 7-Zip (both are available for free).
macOS compatibility - #2086, #2087
Note: This is the first release to support macOS 11 Big Sur. We no longer support macOS 10.13 High Sierra since the macOS package manager homebrew no longer provides pre-compiled binaries for Pupil Core software dependencies.
February 9, 2021
Improved: Lots of UI tweaks and improvements.
Improved: The entire app has been tuned to improve the performance especially for accounts with a large number of recordings.
New: Drive view has a projects column that shows which projects a recording belongs to.
February 9, 2021
Improved: Lots of UI tweaks and improvements.
Improved: The entire app has been tuned to improve the performance especially for accounts with a large number of recordings.
New: Drive view has a projects column that shows which projects a recording belongs to.
February 9, 2021
Improved stability of video transcoding pipeline.
February 9, 2021
Improved stability of video transcoding pipeline.
February 8, 2021
Watchdog Service
We have added a “watchdog service” to further minimize data loss while recording. Stability is one of Pupil Invisible’s strongest features during data acquisition, but there is always the off-chance of something going wrong. The watchdog service detects failure cases, like e.g. a faulty USB connection, automatically and warns you already at recording time.
The warning consists of the Companion Device vibrating and a red LED blinking on the Pupil Invisible Glasses. A message will be displayed on the Companion Device about what needs to be done!
Gaze Estimation Pipeline Update
We have updated the deep learning pipeline for gaze estimation to version 1.7. This update results in an improvement to gaze estimation accuracy of about 0.5° on average compared to the accuracy originally reported in our white paper.
February 8, 2021
Watchdog Service
We have added a “watchdog service” to further minimize data loss while recording. Stability is one of Pupil Invisible’s strongest features during data acquisition, but there is always the off-chance of something going wrong. The watchdog service detects failure cases, like e.g. a faulty USB connection, automatically and warns you already at recording time.
The warning consists of the Companion Device vibrating and a red LED blinking on the Pupil Invisible Glasses. A message will be displayed on the Companion Device about what needs to be done!
Gaze Estimation Pipeline Update
We have updated the deep learning pipeline for gaze estimation to version 1.7. This update results in an improvement to gaze estimation accuracy of about 0.5° on average compared to the accuracy originally reported in our white paper.
January 7, 2021
We are pleased to announce the release of Pupil Core software v3.0!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord! 😄
Motivation
We are excited to introduce pye3d
- a complete overhaul of the 3D pupil detector employed by Pupil Core software. In building pye3d
, we have left no stone unturned, reimagining many of the algorithmic building blocks constituting the final gaze-estimation pipeline. pye3d
is the result of three years of hard work and we are excited to get it into your hands!
Improvements
Improved built-in 3D pupil detector - #2011, #2065
pye3d
implements a novel eye-model fitting strategy that incorporates a corneal refraction-correction step, accounting for nonlinear effects in ocular optics. Gaze estimates and pupil diameter estimates are now more accurate than ever before [2,3]. Thanks to a new sphere center estimation methodology [3], it is also considerably faster than our previous 3D detector [1] in regards to fitting robust and accurate eye models. Read more about how it works in our documentation.
Academic references
[1]
L. Świrski and N. A. Dodgson. A Fully-Automatic, Temporal Approach to Single Camera, Glint-Free 3D Eye Model Fitting. In Proceedings of ECEM 2013. "L. Świrski and N. A. Dodgson. A Fully-Automatic, Temporal Approach to Single Camera, Glint-Free 3D Eye Model Fitting. In Proceedings of ECEM 2013."[2]
K. Dierkes, M. Kassner, A. Bullling, A novel approach to single camera, glint-free 3D eye model fitting including corneal refraction. In ETRA ’18: Symposium on Eye Tracking Research and Applications, 2018.[3]
K. Dierkes, M. Kassner, A. Bulling. A fast approach to refraction-aware eye-model fitting and gaze prediction. In ETRA ’19: Symposium on Eye Tracking Research and Applications, 2019.
Changes
Removal of old Pupil 3D detector
With this release, we also stop the maintainance of the previous 3D detector. Starting with the new 2.0
release, our pupil-detectors
package no longer includes the previous 3D detector. This allows us to remove Ceres as a dependency, making the project much easier to maintain and build from source.
Should you rely on this detector, you can still download previous Pupil Core software releases or install previous pupil-detectors
via PyPI.
Bug Fixes
Fixed issue that would leave an unresponsive eye window due to plugin crash - #2059
Fixed crash on canceling single marker calibration window mode - #2062
Fixed crash on post-hoc pupil detection when the frame count is zero - #2063
Developer notes
Network API Changes
Reworked pupil detector plugin and network API
In this release we reworked the pupil detector plugin base class. It is now easier than ever to implement a custom pupil detector and accompanying pupil detector plugin.
We have also introduced breaking changes to our network API for pupil detector plugins. Here is a list of possible notification payload descriptions that are handled by detector plugins:
# Notification for enabling/disabling any or all pupil detector plugins
{
'topic': 'notify.pupil_detector.set_enabled',
'subject': 'pupil_detector.set_enabled',
'value': <is_enabled: bool>,
'eye_id': <eye_id: int>, # optional; possible values: 0 or 1
'detector_plugin_class_name': <detector_plugin_class_name: str>, # optional
}
# Notification for setting the Region-Of-Interest (ROI) for any or all pupil detector plugins
{
'topic': 'notify.pupil_detector.set_roi',
'subject': 'pupil_detector.set_roi',
'value': <roi: (min_x: int, min_y: int, max_x: int, max_y: int)>,
'eye_id': <eye_id: int>, # optional; possible values: 0 or 1
'detector_plugin_class_name': <detector_plugin_class_name: str>, # optional
}
# Notification for updating a partial set of pupil detector properties for a specific pupil detector plugin
{
'topic': 'notify.pupil_detector.set_properties',
'subject': 'pupil_detector.set_properties',
'values': <detector_properties: dict>,
'eye_id': <eye_id: int>, # required; possible values: 0 or 1
'detector_plugin_class_name': <detector_plugin_class_name: str>, # required
}
# Notification for requesting pupil detector properties broadcast for any or all pupil detector plugins
{
'topic': 'notify.pupil_detector.broadcast_properties',
'subject': 'pupil_detector.broadcast_properties',
'eye_id': <eye_id: int>, # optional; possible values: 0 or 1
'detector_plugin_class_name': <detector_plugin_class_name: str>, # optional
}
For more information about custom pupil detector plugins, please read the documentation. You can also see examples of custom pupil detection plugins here.
For an example script that showcases the network API, please consult this helper script.
Changed 3D pupil datum format
pye3d
uses a slightly different pupil format than our previous 3D detector. Because pye3d
continously updates a single model instead of fitting multiple parallel ones, the new 3D pupil datum no longer contains the following keys: model_id
and model_birth_timestamp
.
Our latest Hmd-eyes release handles this by falling back to "0" and 0.0, respectively.
Binocular 3D gaze data with string-only keys - #2068
In order to be compatible with the msgpack-python v1.0.0
(specifially the new strict_map_key=True
default), this release uses strings only as dictionary keys for all msgpack-encoded data that is published via the Network API.
This change affects primarily binocular 3D gaze data which used to contain integer dictionary keys.
Raw Data Exporter - Optional model_id
key - #2061
Some of the fields in the pupil datum related to the 3D model no longer exist in pye3d
(see above). This change ensures the raw data export handles missing fields and outputs a compatible csv file.
Updated msgpack
dependency to version 1.0
- #2068
We continue to streamline and simplify the installation of Pupil Core dependencies. In this release we updated the msgpack
to the latest 1.0
. To ensure you have the latest required versions of all dependencies, please install them via requirements.txt
.
python -m pip install --upgrade pip wheel
pip install -r requirements.txt
Added support for custom receiver high-water mark for HMD Streaming source - #2058
This change adds the ability to set a custom ZMQ high watermark value for HMD Streaming source. This gives users more control over the streaming behaviour when using Pupil Core with an HMD headset.
Added launcher_process.should_stop
notification handling #2070
This change adds the ability to request stopping Capture, Player or Service with the same notification.
import zmq
import msgpack
# create and connect PUB socket to IPC
pub_socket = zmq.Socket(zmq.Context(), zmq.PUB)
pub_socket.connect(ipc_pub_url)
subject = "launcher_process.should_stop"
topic = "notify." + subject
payload = {
'topic': topic,
'subject': subject,
}
pub_socket.send_string(topic, flags=zmq.SNDMORE)
pub_socket.send(msgpack.dumps(payload, use_bin_type=True))
Simplified running from source on Windows - #2073
We have removed the necessity to run Pupil from source using the run_<app>.bat
files on Windows. Instead of extending the PATH environment variable via the .bat
file, we extend it via the application launcher. This also makes it easier to attach a debugger, e.g. via Visual Studio Code.
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, such as WinRAR or 7-Zip (both are available for free).
macOS (11) Big Sur compatibility - #2079
Note: Pupil Core is not currently supported on Mac OS Big Sur. We aim to release a compatability update within the next few weeks. :construction_worker:
January 7, 2021
We are pleased to announce the release of Pupil Core software v3.0!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord! 😄
Motivation
We are excited to introduce pye3d
- a complete overhaul of the 3D pupil detector employed by Pupil Core software. In building pye3d
, we have left no stone unturned, reimagining many of the algorithmic building blocks constituting the final gaze-estimation pipeline. pye3d
is the result of three years of hard work and we are excited to get it into your hands!
Improvements
Improved built-in 3D pupil detector - #2011, #2065
pye3d
implements a novel eye-model fitting strategy that incorporates a corneal refraction-correction step, accounting for nonlinear effects in ocular optics. Gaze estimates and pupil diameter estimates are now more accurate than ever before [2,3]. Thanks to a new sphere center estimation methodology [3], it is also considerably faster than our previous 3D detector [1] in regards to fitting robust and accurate eye models. Read more about how it works in our documentation.
Academic references
[1]
L. Świrski and N. A. Dodgson. A Fully-Automatic, Temporal Approach to Single Camera, Glint-Free 3D Eye Model Fitting. In Proceedings of ECEM 2013. "L. Świrski and N. A. Dodgson. A Fully-Automatic, Temporal Approach to Single Camera, Glint-Free 3D Eye Model Fitting. In Proceedings of ECEM 2013."[2]
K. Dierkes, M. Kassner, A. Bullling, A novel approach to single camera, glint-free 3D eye model fitting including corneal refraction. In ETRA ’18: Symposium on Eye Tracking Research and Applications, 2018.[3]
K. Dierkes, M. Kassner, A. Bulling. A fast approach to refraction-aware eye-model fitting and gaze prediction. In ETRA ’19: Symposium on Eye Tracking Research and Applications, 2019.
Changes
Removal of old Pupil 3D detector
With this release, we also stop the maintainance of the previous 3D detector. Starting with the new 2.0
release, our pupil-detectors
package no longer includes the previous 3D detector. This allows us to remove Ceres as a dependency, making the project much easier to maintain and build from source.
Should you rely on this detector, you can still download previous Pupil Core software releases or install previous pupil-detectors
via PyPI.
Bug Fixes
Fixed issue that would leave an unresponsive eye window due to plugin crash - #2059
Fixed crash on canceling single marker calibration window mode - #2062
Fixed crash on post-hoc pupil detection when the frame count is zero - #2063
Developer notes
Network API Changes
Reworked pupil detector plugin and network API
In this release we reworked the pupil detector plugin base class. It is now easier than ever to implement a custom pupil detector and accompanying pupil detector plugin.
We have also introduced breaking changes to our network API for pupil detector plugins. Here is a list of possible notification payload descriptions that are handled by detector plugins:
# Notification for enabling/disabling any or all pupil detector plugins
{
'topic': 'notify.pupil_detector.set_enabled',
'subject': 'pupil_detector.set_enabled',
'value': <is_enabled: bool>,
'eye_id': <eye_id: int>, # optional; possible values: 0 or 1
'detector_plugin_class_name': <detector_plugin_class_name: str>, # optional
}
# Notification for setting the Region-Of-Interest (ROI) for any or all pupil detector plugins
{
'topic': 'notify.pupil_detector.set_roi',
'subject': 'pupil_detector.set_roi',
'value': <roi: (min_x: int, min_y: int, max_x: int, max_y: int)>,
'eye_id': <eye_id: int>, # optional; possible values: 0 or 1
'detector_plugin_class_name': <detector_plugin_class_name: str>, # optional
}
# Notification for updating a partial set of pupil detector properties for a specific pupil detector plugin
{
'topic': 'notify.pupil_detector.set_properties',
'subject': 'pupil_detector.set_properties',
'values': <detector_properties: dict>,
'eye_id': <eye_id: int>, # required; possible values: 0 or 1
'detector_plugin_class_name': <detector_plugin_class_name: str>, # required
}
# Notification for requesting pupil detector properties broadcast for any or all pupil detector plugins
{
'topic': 'notify.pupil_detector.broadcast_properties',
'subject': 'pupil_detector.broadcast_properties',
'eye_id': <eye_id: int>, # optional; possible values: 0 or 1
'detector_plugin_class_name': <detector_plugin_class_name: str>, # optional
}
For more information about custom pupil detector plugins, please read the documentation. You can also see examples of custom pupil detection plugins here.
For an example script that showcases the network API, please consult this helper script.
Changed 3D pupil datum format
pye3d
uses a slightly different pupil format than our previous 3D detector. Because pye3d
continously updates a single model instead of fitting multiple parallel ones, the new 3D pupil datum no longer contains the following keys: model_id
and model_birth_timestamp
.
Our latest Hmd-eyes release handles this by falling back to "0" and 0.0, respectively.
Binocular 3D gaze data with string-only keys - #2068
In order to be compatible with the msgpack-python v1.0.0
(specifially the new strict_map_key=True
default), this release uses strings only as dictionary keys for all msgpack-encoded data that is published via the Network API.
This change affects primarily binocular 3D gaze data which used to contain integer dictionary keys.
Raw Data Exporter - Optional model_id
key - #2061
Some of the fields in the pupil datum related to the 3D model no longer exist in pye3d
(see above). This change ensures the raw data export handles missing fields and outputs a compatible csv file.
Updated msgpack
dependency to version 1.0
- #2068
We continue to streamline and simplify the installation of Pupil Core dependencies. In this release we updated the msgpack
to the latest 1.0
. To ensure you have the latest required versions of all dependencies, please install them via requirements.txt
.
python -m pip install --upgrade pip wheel
pip install -r requirements.txt
Added support for custom receiver high-water mark for HMD Streaming source - #2058
This change adds the ability to set a custom ZMQ high watermark value for HMD Streaming source. This gives users more control over the streaming behaviour when using Pupil Core with an HMD headset.
Added launcher_process.should_stop
notification handling #2070
This change adds the ability to request stopping Capture, Player or Service with the same notification.
import zmq
import msgpack
# create and connect PUB socket to IPC
pub_socket = zmq.Socket(zmq.Context(), zmq.PUB)
pub_socket.connect(ipc_pub_url)
subject = "launcher_process.should_stop"
topic = "notify." + subject
payload = {
'topic': topic,
'subject': subject,
}
pub_socket.send_string(topic, flags=zmq.SNDMORE)
pub_socket.send(msgpack.dumps(payload, use_bin_type=True))
Simplified running from source on Windows - #2073
We have removed the necessity to run Pupil from source using the run_<app>.bat
files on Windows. Instead of extending the PATH environment variable via the .bat
file, we extend it via the application launcher. This also makes it easier to attach a debugger, e.g. via Visual Studio Code.
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, such as WinRAR or 7-Zip (both are available for free).
macOS (11) Big Sur compatibility - #2079
Note: Pupil Core is not currently supported on Mac OS Big Sur. We aim to release a compatability update within the next few weeks. :construction_worker:
December 8, 2020
Event name suggestions: When adding events in the project editor, the text field now allows to quickly reuse previous event names.
December 8, 2020
Event name suggestions: When adding events in the project editor, the text field now allows to quickly reuse previous event names.
December 3, 2020
Various bug fixes and performance improvements across the app.
December 3, 2020
Various bug fixes and performance improvements across the app.
December 3, 2020
New features to power analysis! Check out blog post for an overview or check out new features directly in Pupil Cloud 🐱
New: Projects enable you to group and analyze a collection of recordings. For more see docs.
New: Marker Mapper enrichment for analyzing AOIs using Apriltag Markers. See docs.
New: Now all recordings in the drive view have thumbnails. Tip: hover over the thumbnail and drag left/right to get a rough preview the recording.
December 3, 2020
New features to power analysis! Check out blog post for an overview or check out new features directly in Pupil Cloud 🐱
New: Projects enable you to group and analyze a collection of recordings. For more see docs.
New: Marker Mapper enrichment for analyzing AOIs using Apriltag Markers. See docs.
New: Now all recordings in the drive view have thumbnails. Tip: hover over the thumbnail and drag left/right to get a rough preview the recording.
November 16, 2020
We are pleased to announce the release of Pupil Core software v2.6!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Motivation
In v2.6, we are taking a big step towards our modularity goal. We now support custom user pupil detector plugins. This change makes it easier than ever for you to modify the existing pupil detection algorithms or to build your own! You will no longer have to run Pupil Core software from source to get started with your custom pupil detector! Read more about it in our documentation.
Improvements
Support custom pupil detection plugins - #2033
You can now run your own pupil detection plugins that run in the eye processes. These plugins can be used to implement custom algorithms for extracting pupillometry data from the eye images and feeding it to the rest of the gaze mapping pipeline.
Pupil detection plugins are supported in Pupil Capture, Pupil Player, and Pupil Service. Similar to regular user plugins, custom pupil detection plugins should be placed inside pupil_capture_settings/plugins
for Pupil Capture, pupil_player_settings/plugins
for Pupil Player, and pupil_service_settings/plugin
for Pupil Service. The plugins are automatically loaded when the applications starts.
Read more about it in our documentation.
Support HMD Streaming Source in eye process - #2052
We have added support for using HMD Streaming Source in the eye process. This also adds support for BGR and Gray frame format for HMD streams.
Support Pupil Invisible recording format v1.4 - #2054
This new recording format has been introduced in Pupil Companion 1.0.0
. This version provides support for the OnePus 8 phone and Android 11. On the OnePlus 8, eye videos will be h264-encoded.
Bug Fixes
Removed
import *
statements - #2036Fixed typo in circle detector benchmark - #2038
Fixed gaze with negative z-component - #2043
Fixed issue when minimizing 3D pupil detector debug window on Windows - #2047
Fixed UVC camera auto-selection priority in case of multiple pattern matches - #2051
Developer Notes
Added ability to stop eye process plugins via notifications - #2039
Another small improvement for eye process plugin management in this release is the ability to stop plugins via notifications. This gives better control over the list of plugins running in the eye process.
import zmq
import msgpack
# class name of the plugin to stop
plugin_name = "..."
# create and connect PUB socket to IPC
pub_socket = zmq.Socket(zmq.Context(), zmq.PUB)
pub_socket.connect(ipc_pub_url)
for eye_id in (0, 1):
subject = "stop_eye_plugin"
topic = "notify." + subject
payload = {
'topic': topic
'subject': subject,
'target': f"eye{eye_id}",
'name': plugin_name,
}
pub_socket.send_string(topic, flags=zmq.SNDMORE)
pub_socket.send(msgpack.dumps(payload, use_bin_type=True))
Added requirements.txt
file - #2029
We continue to streamline and simplify the installation of Pupil Core dependencies. You can now use the requirements.txt
file from the root of the pupil
repository to install and update all Python dependencies.
python -m pip install --upgrade pip wheel
pip install -r requirements.txt
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, e.g. WinRAR.
November 16, 2020
We are pleased to announce the release of Pupil Core software v2.6!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Motivation
In v2.6, we are taking a big step towards our modularity goal. We now support custom user pupil detector plugins. This change makes it easier than ever for you to modify the existing pupil detection algorithms or to build your own! You will no longer have to run Pupil Core software from source to get started with your custom pupil detector! Read more about it in our documentation.
Improvements
Support custom pupil detection plugins - #2033
You can now run your own pupil detection plugins that run in the eye processes. These plugins can be used to implement custom algorithms for extracting pupillometry data from the eye images and feeding it to the rest of the gaze mapping pipeline.
Pupil detection plugins are supported in Pupil Capture, Pupil Player, and Pupil Service. Similar to regular user plugins, custom pupil detection plugins should be placed inside pupil_capture_settings/plugins
for Pupil Capture, pupil_player_settings/plugins
for Pupil Player, and pupil_service_settings/plugin
for Pupil Service. The plugins are automatically loaded when the applications starts.
Read more about it in our documentation.
Support HMD Streaming Source in eye process - #2052
We have added support for using HMD Streaming Source in the eye process. This also adds support for BGR and Gray frame format for HMD streams.
Support Pupil Invisible recording format v1.4 - #2054
This new recording format has been introduced in Pupil Companion 1.0.0
. This version provides support for the OnePus 8 phone and Android 11. On the OnePlus 8, eye videos will be h264-encoded.
Bug Fixes
Removed
import *
statements - #2036Fixed typo in circle detector benchmark - #2038
Fixed gaze with negative z-component - #2043
Fixed issue when minimizing 3D pupil detector debug window on Windows - #2047
Fixed UVC camera auto-selection priority in case of multiple pattern matches - #2051
Developer Notes
Added ability to stop eye process plugins via notifications - #2039
Another small improvement for eye process plugin management in this release is the ability to stop plugins via notifications. This gives better control over the list of plugins running in the eye process.
import zmq
import msgpack
# class name of the plugin to stop
plugin_name = "..."
# create and connect PUB socket to IPC
pub_socket = zmq.Socket(zmq.Context(), zmq.PUB)
pub_socket.connect(ipc_pub_url)
for eye_id in (0, 1):
subject = "stop_eye_plugin"
topic = "notify." + subject
payload = {
'topic': topic
'subject': subject,
'target': f"eye{eye_id}",
'name': plugin_name,
}
pub_socket.send_string(topic, flags=zmq.SNDMORE)
pub_socket.send(msgpack.dumps(payload, use_bin_type=True))
Added requirements.txt
file - #2029
We continue to streamline and simplify the installation of Pupil Core dependencies. You can now use the requirements.txt
file from the root of the pupil
repository to install and update all Python dependencies.
python -m pip install --upgrade pip wheel
pip install -r requirements.txt
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, e.g. WinRAR.
October 20, 2020
We are pleased to announce the release of Pupil Core software v2.5!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Motivation
Pupil v2.5 focuses on further improvements to software stability. We have fixed multiple issues that prevented very old recordings from being opened in Pupil Player.
Improvements
Replacing internal GLFW bindings with pyGLFW - #2015
In this release we replaced the internal glfw.py
bindings to GLFW with the external pyGLFW
package. This change simplifies our dependency list, making easier to install GLFW
on Windows.
New default camera intrinsics - #2023
We use camera intrinsics in multiple places to transform between the 2d distorted image space and the 3d undistorted camera space. With this release, we have updated the default scene camera intrinsics to perform this tansformation more accuractely at the image borders.
Note: Each camera is slightly different. Therefore, it is recommended to run your own camera intrinsics estimation to get the best possible transformations for your camera.
Bug Fixes
Fixed a crash in Pupil Player when attempting to visualize invalid data - #2021
Fixed an edge-case when handling recordings with negative time jumps - #2026, #2028
Fixed a frame drop issue on macOS when windows were occluded - #2027
Fixed an issue that caused instabilities when starting the Head Pose tracker - #2022
Surface Tracker stability improvements - #2025
Developer Notes
Improved Observable class - #2022
Improvements to the Observable class enable you to observe monkey patched methods (e.g. replaced methods that were not in the original class definition). Additionally, trying to add private methods as observers now fails with a more explicit error.
Changed Requirements
Please install the new external glfw
dependency:
pip install glfw
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, e.g. WinRAR.
October 20, 2020
We are pleased to announce the release of Pupil Core software v2.5!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Motivation
Pupil v2.5 focuses on further improvements to software stability. We have fixed multiple issues that prevented very old recordings from being opened in Pupil Player.
Improvements
Replacing internal GLFW bindings with pyGLFW - #2015
In this release we replaced the internal glfw.py
bindings to GLFW with the external pyGLFW
package. This change simplifies our dependency list, making easier to install GLFW
on Windows.
New default camera intrinsics - #2023
We use camera intrinsics in multiple places to transform between the 2d distorted image space and the 3d undistorted camera space. With this release, we have updated the default scene camera intrinsics to perform this tansformation more accuractely at the image borders.
Note: Each camera is slightly different. Therefore, it is recommended to run your own camera intrinsics estimation to get the best possible transformations for your camera.
Bug Fixes
Fixed a crash in Pupil Player when attempting to visualize invalid data - #2021
Fixed an edge-case when handling recordings with negative time jumps - #2026, #2028
Fixed a frame drop issue on macOS when windows were occluded - #2027
Fixed an issue that caused instabilities when starting the Head Pose tracker - #2022
Surface Tracker stability improvements - #2025
Developer Notes
Improved Observable class - #2022
Improvements to the Observable class enable you to observe monkey patched methods (e.g. replaced methods that were not in the original class definition). Additionally, trying to add private methods as observers now fails with a more explicit error.
Changed Requirements
Please install the new external glfw
dependency:
pip install glfw
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, e.g. WinRAR.
September 17, 2020
We are pleased to announce the release of Pupil Core software v2.4!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Motivation
In Pupil v2.4, we are adding support for eye camera intrinsics. These are used to improve 3d eyeball model positioning. Additionally, we have fixed a long-standing issue that caused application windows not to appear on startup when using Windows OS.
Improvements
Better 3D Eyeball Position - #1995
We now use the focal length of the eye cameras to improve the estimate of the 3D eyeball position. Previously, the 3d eye model assumed a fixed camera focal length. This will most likely not have any noticeable effect for most use-cases, as it does not influence the gaze estimation. We assume default focal lengths for the different types of eye cameras shipped with Pupil Labs's hardware.
Caveat: 200Hz Vive Add-on Recordings
Pupil Core software now saves the eye camera intrinsics to the recording folder. On opening recordings made with previous versions in Pupil Player, we patch up the recording by inserting the appropriate eye camera intrinsics.
It is important to note that the 200Hz Pupil Core and 200Hz Vive Add-on use eye cameras with different focal lengths. Unfortunately, it is not possible for Pupil Player to differentiate between existing eye videos made by these cameras. As a result, Pupil Player will use 200Hz Pupil Core eye camera intrinsics for existing 200hz Vive Add-on eye videos.
To avoid this, we recommend placing 200Hz Vive Add-on intrinsics into your exisiting recordings. You can download the default 200Hz Vive Add-on intrinsics from the Downloads section below. Remember to remove the vive_200hz_
prefix.
Recordings made with Pupil Capture v2.4 and higher will not require any action in this regard.
Bug Fixes
Fixed issue when opening Pupil Invisible recordings with more than 10 parts - #1996
Ensure timelines update when changing calibrations for post-hoc gaze-mapping - #2000
Fixed a crash when opening a recording that contained recorded fallback intrinsics - #2002
Fixed a very rare crash of the eye window caused by malformed data - #2008
Fixed occasionally missing windows on startup on Windows in multi-screen setups - #2007
Resolved an error message when minimizing the eye windows on Windows - #2009
Developer Notes
Changed Requirements
pupil-detectors - #1995
We have updated pupil-detectors to v1.1.1, please update with
pip install -U pupil-detectors
packaging - #1993
We now require packaging>=20.0
, please update with
pip install -U packaging
Internal Version Parsing - #1993
We have switched to a more general version-parsing method by using packaging.version.parse
to parse all internally used version strings (from Pupil and external dependencies).
Black Formatter 20.8 - #2003
We have upgraded our black formatter to version 20.8, which introduces a few changes to previous versions and reformats parts of the code which were fine previously. Please make sure to also upgrade black with
pip install -U black
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, e.g. WinRAR.
September 17, 2020
We are pleased to announce the release of Pupil Core software v2.4!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Motivation
In Pupil v2.4, we are adding support for eye camera intrinsics. These are used to improve 3d eyeball model positioning. Additionally, we have fixed a long-standing issue that caused application windows not to appear on startup when using Windows OS.
Improvements
Better 3D Eyeball Position - #1995
We now use the focal length of the eye cameras to improve the estimate of the 3D eyeball position. Previously, the 3d eye model assumed a fixed camera focal length. This will most likely not have any noticeable effect for most use-cases, as it does not influence the gaze estimation. We assume default focal lengths for the different types of eye cameras shipped with Pupil Labs's hardware.
Caveat: 200Hz Vive Add-on Recordings
Pupil Core software now saves the eye camera intrinsics to the recording folder. On opening recordings made with previous versions in Pupil Player, we patch up the recording by inserting the appropriate eye camera intrinsics.
It is important to note that the 200Hz Pupil Core and 200Hz Vive Add-on use eye cameras with different focal lengths. Unfortunately, it is not possible for Pupil Player to differentiate between existing eye videos made by these cameras. As a result, Pupil Player will use 200Hz Pupil Core eye camera intrinsics for existing 200hz Vive Add-on eye videos.
To avoid this, we recommend placing 200Hz Vive Add-on intrinsics into your exisiting recordings. You can download the default 200Hz Vive Add-on intrinsics from the Downloads section below. Remember to remove the vive_200hz_
prefix.
Recordings made with Pupil Capture v2.4 and higher will not require any action in this regard.
Bug Fixes
Fixed issue when opening Pupil Invisible recordings with more than 10 parts - #1996
Ensure timelines update when changing calibrations for post-hoc gaze-mapping - #2000
Fixed a crash when opening a recording that contained recorded fallback intrinsics - #2002
Fixed a very rare crash of the eye window caused by malformed data - #2008
Fixed occasionally missing windows on startup on Windows in multi-screen setups - #2007
Resolved an error message when minimizing the eye windows on Windows - #2009
Developer Notes
Changed Requirements
pupil-detectors - #1995
We have updated pupil-detectors to v1.1.1, please update with
pip install -U pupil-detectors
packaging - #1993
We now require packaging>=20.0
, please update with
pip install -U packaging
Internal Version Parsing - #1993
We have switched to a more general version-parsing method by using packaging.version.parse
to parse all internally used version strings (from Pupil and external dependencies).
Black Formatter 20.8 - #2003
We have upgraded our black formatter to version 20.8, which introduces a few changes to previous versions and reformats parts of the code which were fine previously. Please make sure to also upgrade black with
pip install -U black
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, e.g. WinRAR.
August 25, 2020
We are pleased to announce the release of Pupil Core software v2.3!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Motivation
Pupil v2.3 is a compatibility release for Pupil Invisible, adding support for a simple gaze confidence measure.
Improvements
Pupil Invisible Gaze Confidence - #1984
With its latest 0.8.23 release, Pupil Invisible can detect if the glasses are being worn. If they are not worn, gaze output will be hidden.
We now use this information as a simple binary gaze confidence when loading Pupil Invisible recordings in Pupil Player. If the glasses are detected as being worn, the gaze confidence will be 1.0
, otherwise 0.0
.
Smaller Improvements
Added better log message for pupil mobile network issues - #1994
Renamed "camera calibration" to "camera intrinsics" to avoid confusion with gaze calibration - #1992
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, e.g. WinRAR.
August 25, 2020
We are pleased to announce the release of Pupil Core software v2.3!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Motivation
Pupil v2.3 is a compatibility release for Pupil Invisible, adding support for a simple gaze confidence measure.
Improvements
Pupil Invisible Gaze Confidence - #1984
With its latest 0.8.23 release, Pupil Invisible can detect if the glasses are being worn. If they are not worn, gaze output will be hidden.
We now use this information as a simple binary gaze confidence when loading Pupil Invisible recordings in Pupil Player. If the glasses are detected as being worn, the gaze confidence will be 1.0
, otherwise 0.0
.
Smaller Improvements
Added better log message for pupil mobile network issues - #1994
Renamed "camera calibration" to "camera intrinsics" to avoid confusion with gaze calibration - #1992
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, e.g. WinRAR.
August 5, 2020
We are pleased to announce the release of Pupil Core software v2.2!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Motivation
With v2.2, we have focused on improving the user experience when using Pupil on high-pixel-density displays. Now, Pupil scales and layouts its content correctly based on the content-scaling factor set in your operating system's settings.
Improvements
Support For High-Pixel-Density Displays - #1976
On displays with a high pixel density, content looks much smaller than expected. Operating systems have the option to scale its content appropriately. Until now, Pupil only supported content-scaling on macOS. With this release, we have added support for content-scaling on Windows and Linux as well.
Smaller Improvements
Better user feedback when
Accuracy Visualizer
reportsnan
values - #1974Disabled
Duplicate
andDelete
buttons for read-only post-hoc calibrations - #1972Use a square default eye window aspect ratio, matching the latest default cameras - #1987
Changes
Remove Option To Manually Change The Interface Size - #1988
With native support for content scaling on the operating system level, there is no need to scale the UI manually anymore. Therefore, we have removed this feature to further simplify the user interface.
Bug Fixes
Fixed non-monotonic progress visualization for post-hoc gaze mapping - #1965
Fixed flickering on macOS when resetting the window size - #1973
Fixed an empty side-menu when restarting Capture after using HMD-Eyes - #1979
Fixed a crash when attempting to log any non-string object - #1980
Fixed 2D monocular gaze calibration not working - #1983
Developer Notes
Changed Requirements
pyglui v1.28
- #1972
We updated the pyglui dependency to v1.28. On Linux and macOS, you can update with:
pip install -U git+https://github.com/pupil-labs/pyglui
On Windows, please download and install the latest wheel from GitHub.
glfw 3.3
- #1976
In order to support content-scaling across all three platforms, we make use of the glfwGetWindowContentScale
function which has been introduced in glfw 3.3
.
We recommend installing the latest glfw 3.3
release via https://www.glfw.org/download.html.
Removed g_pool.gui_user_scale
- #1976, #1988
With the removal of the Interface Size
setting, the g_pool.gui_user_scale
variable has been removed as well. If you draw custom UI content in your plugins without using pyglui, you should now instead use g_pool.gui.scale
to scale the content appropriately for different native display scaling settings.
Removed Legacy Plugin Base Classes - #1971
The following legacy plugin base classes have been removed. All plugins inheriting from them now just inherit from Plugin
:
Visualizer_Plugin_Base
Analysis_Plugin_Base
Producer_Plugin_Base
Experimental_Plugin_Base
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, e.g. WinRAR.
August 5, 2020
We are pleased to announce the release of Pupil Core software v2.2!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Motivation
With v2.2, we have focused on improving the user experience when using Pupil on high-pixel-density displays. Now, Pupil scales and layouts its content correctly based on the content-scaling factor set in your operating system's settings.
Improvements
Support For High-Pixel-Density Displays - #1976
On displays with a high pixel density, content looks much smaller than expected. Operating systems have the option to scale its content appropriately. Until now, Pupil only supported content-scaling on macOS. With this release, we have added support for content-scaling on Windows and Linux as well.
Smaller Improvements
Better user feedback when
Accuracy Visualizer
reportsnan
values - #1974Disabled
Duplicate
andDelete
buttons for read-only post-hoc calibrations - #1972Use a square default eye window aspect ratio, matching the latest default cameras - #1987
Changes
Remove Option To Manually Change The Interface Size - #1988
With native support for content scaling on the operating system level, there is no need to scale the UI manually anymore. Therefore, we have removed this feature to further simplify the user interface.
Bug Fixes
Fixed non-monotonic progress visualization for post-hoc gaze mapping - #1965
Fixed flickering on macOS when resetting the window size - #1973
Fixed an empty side-menu when restarting Capture after using HMD-Eyes - #1979
Fixed a crash when attempting to log any non-string object - #1980
Fixed 2D monocular gaze calibration not working - #1983
Developer Notes
Changed Requirements
pyglui v1.28
- #1972
We updated the pyglui dependency to v1.28. On Linux and macOS, you can update with:
pip install -U git+https://github.com/pupil-labs/pyglui
On Windows, please download and install the latest wheel from GitHub.
glfw 3.3
- #1976
In order to support content-scaling across all three platforms, we make use of the glfwGetWindowContentScale
function which has been introduced in glfw 3.3
.
We recommend installing the latest glfw 3.3
release via https://www.glfw.org/download.html.
Removed g_pool.gui_user_scale
- #1976, #1988
With the removal of the Interface Size
setting, the g_pool.gui_user_scale
variable has been removed as well. If you draw custom UI content in your plugins without using pyglui, you should now instead use g_pool.gui.scale
to scale the content appropriately for different native display scaling settings.
Removed Legacy Plugin Base Classes - #1971
The following legacy plugin base classes have been removed. All plugins inheriting from them now just inherit from Plugin
:
Visualizer_Plugin_Base
Analysis_Plugin_Base
Producer_Plugin_Base
Experimental_Plugin_Base
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, e.g. WinRAR.
July 15, 2020
We are pleased to announce the release of Pupil Core software v2.1!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Motivation
With v2.1, we have continued to focus on making Pupil Core more core. We were able to resolve a lot of stability and user experience issues, especially regarding post-hoc calibrations.
Bug Fixes
Fixed that recorded calibration were displayed as being editable - #1932, #1938
Fixed that recorded calibration were duplicatable - #1933
Fixed that immutable calibration could be deleted - #1934
Fixed inconsistencies in the naming of recorded calibrations - #1935, #1944
Fixed minimum confidence slider for post-hoc calibration - #1942
Ensured correct validation is run even when a different calibration is selected - #1921
Removed UI for heatmaps across surfaces in Capture (only available in Player) - #1917
Fixed incorrect warning about deprecated surface definitions - #1923
Prevented crashes when rendering invalid eyeball outline - #1947, #1954
Fixed that restarting background tasks would not clear already running ones - #1939
Fixed frozen eye videos in HMD-Eyes - #1949
Disabled automatic reference location detection again in post-hoc gaze - #1951
Fixed crash when opening recording with unavailable gazer - #1922
Fixed crashes of the timeline for very short recordings - #1958
Fixed crash when deleting a running post-hoc calibration multiple times - #1955
Fixed crashing post-hoc gaze mapping for an HMD-Eyes recording - #1961
Fixed Service crashing when requesting a non-existing plugin - #1962
Fixed Capture crashing when finishing a calibration without reference data - #1960
Prevented starting gaze mappers on calibration fail - #1963
Fixed an issue when running Pupil Capture bundle on macOS 10.14 Mojave - #1919
Developer Notes
Customizing Screen Marker Positions - #1953
Using custom marker positions for the screen marker calibration choreography might be a common use case. Previously this was not easy to achieve with a custom plugin, as the relevant function was a private method. We have now made it public so that you can more easily implement custom calibration choreographies.
Black Formatting Checks on Travis - #1956
We are internally using Black as automatic Python formatter for our codebase. This helps us ensure that PRs only contain relevant changes and no formatting/whitespace changes. To enforce this more strictly, we added a black formatting check to our Travis integration, which will run on all commits and PRs. Please make sure to format your code with black before submitting a PR to ensure that all Travis checks pass.
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, e.g. WinRAR.
July 15, 2020
We are pleased to announce the release of Pupil Core software v2.1!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Motivation
With v2.1, we have continued to focus on making Pupil Core more core. We were able to resolve a lot of stability and user experience issues, especially regarding post-hoc calibrations.
Bug Fixes
Fixed that recorded calibration were displayed as being editable - #1932, #1938
Fixed that recorded calibration were duplicatable - #1933
Fixed that immutable calibration could be deleted - #1934
Fixed inconsistencies in the naming of recorded calibrations - #1935, #1944
Fixed minimum confidence slider for post-hoc calibration - #1942
Ensured correct validation is run even when a different calibration is selected - #1921
Removed UI for heatmaps across surfaces in Capture (only available in Player) - #1917
Fixed incorrect warning about deprecated surface definitions - #1923
Prevented crashes when rendering invalid eyeball outline - #1947, #1954
Fixed that restarting background tasks would not clear already running ones - #1939
Fixed frozen eye videos in HMD-Eyes - #1949
Disabled automatic reference location detection again in post-hoc gaze - #1951
Fixed crash when opening recording with unavailable gazer - #1922
Fixed crashes of the timeline for very short recordings - #1958
Fixed crash when deleting a running post-hoc calibration multiple times - #1955
Fixed crashing post-hoc gaze mapping for an HMD-Eyes recording - #1961
Fixed Service crashing when requesting a non-existing plugin - #1962
Fixed Capture crashing when finishing a calibration without reference data - #1960
Prevented starting gaze mappers on calibration fail - #1963
Fixed an issue when running Pupil Capture bundle on macOS 10.14 Mojave - #1919
Developer Notes
Customizing Screen Marker Positions - #1953
Using custom marker positions for the screen marker calibration choreography might be a common use case. Previously this was not easy to achieve with a custom plugin, as the relevant function was a private method. We have now made it public so that you can more easily implement custom calibration choreographies.
Black Formatting Checks on Travis - #1956
We are internally using Black as automatic Python formatter for our codebase. This helps us ensure that PRs only contain relevant changes and no formatting/whitespace changes. To enforce this more strictly, we added a black formatting check to our Travis integration, which will run on all commits and PRs. Please make sure to format your code with black before submitting a PR to ensure that all Travis checks pass.
Downloads
To open the RAR-archive on Windows, you will need to use decompression software, e.g. WinRAR.
June 11, 2020
We are pleased to announce the release of Pupil Core software v2.0!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Motivation
With v2.0 and beyond, we will be focusing on making Pupil Core more core.
Our vision for Pupil Core emerged early on: we wanted to make eye tracking hardware and software that was accessible. Accessibility is a useful keyword for us because it encourages us to think about the diverse group of researchers that use Pupil. Pupil Core strives for the following:
Transparency: Open source is key here. Enable researchers to introspect into the algorithms. Be explicit about the methods and biases built into the software. Inspire others to make improvements and catch bugs.
Modularity: Make it easy for users to develop and run their own custom code - plugin architecture.
Connectivity: Make it easy to connect to other software - network API.
Diversity: Pupil core is flexible. Pupil Core strives to accommodate a diverse range of research and applications. Pupil Core enables researchers who want to use it as a desktop app with GUI, researchers who want to integrate with other software and sensors via Network API and plugins, and developers who want to tinker with the source code directly.
Over the years, Pupil Core software grew organically to support the community, new hardware, and novel applications. With v2.0 took a few steps back to look at the big picture and to think about the future of Pupil Core. If we want Pupil Core to continue to be a useful platform for researchers and developers, then we need to boil it down to its essentials.
How are we making Pupil Core more core? By focusing on making essential (or core) parts of Pupil Core work really well. Here are the steps we have taken with v2.0:
Restructuring the core pipeline: We have made changes to the application architecture that allows it to run multiple pupil detectors in parallel as well as easily add custom gaze mapping algorithms via user plugins.
Removing hard to maintain features: We had to make some tough decisions. We decided to remove audio capture and native Intel RealSense support. These components were too much work to maintain.
Simplifying the user interface: Over the years we have added many features and the UI has grown organically in the process. We have decided to streamline and improve the UI. Be it merging of menus or more sane default settings, we felt that new users need a bit more help getting started with Pupil Capture and Player.
Improvements
Parallel 2D and 3D Pupil Detection
To allow for using different pupil detection algorithms, we have decoupled our existing pipeline of 2D and 3D detection. Instead of running either of those, both will now run as separate plugins with only minimal overhead. This will enable researchers (including our own R&D team) to test, optimize and replace parts of the pipeline much more conveniently in the future.
Additionally this change has allowed for the following improvements to other parts of the system:
Pupil Diameter Timelines - #1854
With 3D pupil data always available we can always display the meaningful 3D pupil diameter in the timelines. Additionally, we added outlier removal and included the range for a more expressive visualization.
Blink Detection - #1900
With 2D pupil data always available we can run a more reliable blink detection in all circumstances.
Calibration Choreography and Gazer Plugins
By far the biggest change in this release was the refactoring of calibration and gaze mapping part of the core pipeline into separate, extendable components that are easy to swap with custom implementations.
Calibration Plugin in Pupil Capture
When running Pupil Capture, in the plugin list you can find the unified Calibration plugin, which is responsible for configuring both the calibration method (with the Choreography
selector), and the gaze mapping method (with the Method
selector).
Extending Calibration and Gaze Mapping with User Plugins
Now users can implement their own calibration routines and gaze mapping methods. These will automatically show up in the Choreography
and Method
selectors.
To do this, users must subclass CalibrationChoreographyPlugin
and GazerBase
respectively. See the Developer Notes section below for a code example.
Improved Installation Workflow on Windows - #1853
We have wrapped Pupil in a Windows Installer package (MSI) in order to simplify the Windows workflow.
By default, all 3 apps (Capture, Player, and Service) will be installed in C:\Program Files (x86)\Pupil-labs\Pupil v<version>\
. All apps will also get a start-menu entry, making it much easier to open Pupil. Installed versions of Pupil can be removed in the Windows Uninstall Settings.
New versions of Pupil will be installed alongside older versions and you can choose which to start. Switching versions will still overwrite your user settings as previously.
Changes
Fixation and Blink Detection by Default - #1870
In order to make Pupil Core even more useful with out-of-the-box settings, we have enabled the Fixation Detector and Blink Detector by default.
Simplified System Graphs - #1869
We have removed the configuration menu for the System Graphs plugin in order to simplify the user interface. The optional display of live graphs for the pupil diameter has been removed as well.
The confidence graphs will now always display the confidence of the 2D pupil detection, as this is a better indicator for correct positioning of the Pupil Core eye tracking headset.
Single Marker Calibration: Physical Markers - #1897
The single marker calibration is mostly intended for calibration with physical (printed) calibration markers, as opposed to the screen marker calibration. For sake of clarity, we changed the default mode for the single marker calibration to Physical Marker (previously this was called Manual).
Removed Manual Marker Calibration
With the restructuring of the calibration workflow, we have removed the deprecated Manual Marker Calibration. The Single Marker Calibration is a more general version and additionally allows for VOR-based choreographies. If you previously used the Manual Marker Calibration, try the Single Marker Calibration in Physical Marker mode (the new default)!
Parallel Pupil Detector Visualization - #1873
We have changed the color of the 2D ellipse visualization in all eye videos to blue in order to differentiate between 2D and 3D detection. You will now always see 3 ellipses on the eyes when the visualization is enabled:
Color Description
blue: 2D pupil detection result
red: 3D pupil detection result
green: 3D eye model
Flip Eye 0 Display - #1844
The eye 0 (right eye) camera sensor is physically flipped, which results in an upside-down eye image for the right eye. This is by design and does not negatively affect pupil detection or gaze estimation. However, the upside-down eye 0 image repeatedly led users to believe that something was broken or incorrect with Pupil Core headsets. We flipped the eye 0 image now by default to better match user expectations with a right-side-up eye image.
Deprecated Audio Capture - #1868
Recording synchronized audio on 3 different operating systems (macOS, Linux, Windows) has always been a challenge for Pupil Capture. (See our motivation section above.)
Unified Network API Plugin - #1881
The previous plugins Pupil Remote and Frame Publisher were commonly used to interact with Pupil Core's Network API. For the sake of simplicity and clarity, we merged both plugins into the new Network API plugin.
Removed Blink Detector Visualization - #1901
The visualization of the blink detector (brief dark flashing on the screen) could interfere with other plugins and cause visual bugs. We have removed this feature.
Bug Fixes
Fixed Player crashing when exporting while computing gaze history - #1843
Fixed Player crashing when disabling the Surface Tracker while exporting - #1848
Fixed wrong default camera selection in some cases - #1857
Fixed a rare crash in the Surface Tracker - #1879
Fixed deprecated RealSense cameras showing up in the camera selection - #1877
Fixed that offline gaze mappers without data would show as completed - #1889
Fixed corrupted gaze data when deactivating one of multiple gaze mappers - #1888
Fixed new offline pupil data not being correctly announced sometimes - #1890
Fixed a few small issues in offline pupil detection - #1899
Fixed problems with opening the macOS bundles on newer versions of macOS
Developer Notes
Changed Requirements
Scikit-learn
We have added scikit-learn as a dependency for Pupil.
pip install scikit-learn
PyAV - 1908
We updated the PyAV dependency to v0.4.5, which includes some bug fixes. On Linux and macOS, you can update with:
pip install -U git+https://github.com/pupil-labs/PyAV
On Windows, download and install the latest wheel from GitHub.
Changed data format
Both 2D and 3D pupil detector now publish their results independently of each other. To differentiate between them, we have added the origin to the topic. For a binocular setup, you will now get 4 different pupil topics:
pupil.0.2d
pupil.0.3d
pupil.1.2d
pupil.1.3d
This means that there can now be multiple pupil datums for one eye for the same timestamp (2D and 3D). When grouping by the entire topic however, the timestamps are still guaranteed to be unique.
Restarting Unique Plugins - #1871
We have added the previously missing feature to restart plugins with by_class
uniqueness via notifications.
Recording Version 2.2 - #1885
The info.player.json
meta-version has been bumped to 2.2 will be incompatible with Pupil Player version v1.
Allow specifying ZMQ HWM option - #1895
We added the option to start the eye processes with custom values for the ZMQ High Water Mark (HWM) via notification.
Custom Calibration Plugins
Below is an example with minimal implementation for a custom HMD 3D calibration choreography and gaze mapping method:
# To implement a custom calibration or gaze mapping method from scratch, subclass GazerBase and CalibrationChoreographyPlugin
from gaze_mapping.gazer_base import GazerBase
from calibration_choreography import CalibrationChoreographyPlugin
# Another option is to modify existing implementations by subclassing existing implementations
from gaze_mapping import Gazer3D
from calibration_choreography import ScreenMarkerChoreographyPlugin
class CustomGazer3D(Gazer3D):
# Must be unique across all subclasses of GazerBase
label = "Custom 3D Gazer"
class CustomChoreographyPlugin(ScreenMarkerChoreographyPlugin):
# Must be unique accross all subclasses of CalibrationChoreographyPlugin
label = "Custom Calibration"
@classmethod
def supported_gazer_classes(cls):
return [CustomGazer3D]
Release Note Updates
17.06.2020 - v2.0-175
Added compatibility for the HMD-Eyes VR integration
Fixed an issue that prevented the validation from working correctly
24.06.2020 - v2.0-177
Fixed an issue that prevented using the head pose tracker and creating new annotations in Player - #1927
25.06.2020 - v2.0-182
Fixed an issue that prevented Pupil Service from starting - #1928
Downloads
Officially minimum supported operating system versions for the bundles:
Ubuntu: 16.04macOS: 10.13 (High Sierra)Windows: 10
To open the RAR-archive on Windows, you will need a decompression software, e.g. WinRAR.
June 11, 2020
We are pleased to announce the release of Pupil Core software v2.0!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Motivation
With v2.0 and beyond, we will be focusing on making Pupil Core more core.
Our vision for Pupil Core emerged early on: we wanted to make eye tracking hardware and software that was accessible. Accessibility is a useful keyword for us because it encourages us to think about the diverse group of researchers that use Pupil. Pupil Core strives for the following:
Transparency: Open source is key here. Enable researchers to introspect into the algorithms. Be explicit about the methods and biases built into the software. Inspire others to make improvements and catch bugs.
Modularity: Make it easy for users to develop and run their own custom code - plugin architecture.
Connectivity: Make it easy to connect to other software - network API.
Diversity: Pupil core is flexible. Pupil Core strives to accommodate a diverse range of research and applications. Pupil Core enables researchers who want to use it as a desktop app with GUI, researchers who want to integrate with other software and sensors via Network API and plugins, and developers who want to tinker with the source code directly.
Over the years, Pupil Core software grew organically to support the community, new hardware, and novel applications. With v2.0 took a few steps back to look at the big picture and to think about the future of Pupil Core. If we want Pupil Core to continue to be a useful platform for researchers and developers, then we need to boil it down to its essentials.
How are we making Pupil Core more core? By focusing on making essential (or core) parts of Pupil Core work really well. Here are the steps we have taken with v2.0:
Restructuring the core pipeline: We have made changes to the application architecture that allows it to run multiple pupil detectors in parallel as well as easily add custom gaze mapping algorithms via user plugins.
Removing hard to maintain features: We had to make some tough decisions. We decided to remove audio capture and native Intel RealSense support. These components were too much work to maintain.
Simplifying the user interface: Over the years we have added many features and the UI has grown organically in the process. We have decided to streamline and improve the UI. Be it merging of menus or more sane default settings, we felt that new users need a bit more help getting started with Pupil Capture and Player.
Improvements
Parallel 2D and 3D Pupil Detection
To allow for using different pupil detection algorithms, we have decoupled our existing pipeline of 2D and 3D detection. Instead of running either of those, both will now run as separate plugins with only minimal overhead. This will enable researchers (including our own R&D team) to test, optimize and replace parts of the pipeline much more conveniently in the future.
Additionally this change has allowed for the following improvements to other parts of the system:
Pupil Diameter Timelines - #1854
With 3D pupil data always available we can always display the meaningful 3D pupil diameter in the timelines. Additionally, we added outlier removal and included the range for a more expressive visualization.
Blink Detection - #1900
With 2D pupil data always available we can run a more reliable blink detection in all circumstances.
Calibration Choreography and Gazer Plugins
By far the biggest change in this release was the refactoring of calibration and gaze mapping part of the core pipeline into separate, extendable components that are easy to swap with custom implementations.
Calibration Plugin in Pupil Capture
When running Pupil Capture, in the plugin list you can find the unified Calibration plugin, which is responsible for configuring both the calibration method (with the Choreography
selector), and the gaze mapping method (with the Method
selector).
Extending Calibration and Gaze Mapping with User Plugins
Now users can implement their own calibration routines and gaze mapping methods. These will automatically show up in the Choreography
and Method
selectors.
To do this, users must subclass CalibrationChoreographyPlugin
and GazerBase
respectively. See the Developer Notes section below for a code example.
Improved Installation Workflow on Windows - #1853
We have wrapped Pupil in a Windows Installer package (MSI) in order to simplify the Windows workflow.
By default, all 3 apps (Capture, Player, and Service) will be installed in C:\Program Files (x86)\Pupil-labs\Pupil v<version>\
. All apps will also get a start-menu entry, making it much easier to open Pupil. Installed versions of Pupil can be removed in the Windows Uninstall Settings.
New versions of Pupil will be installed alongside older versions and you can choose which to start. Switching versions will still overwrite your user settings as previously.
Changes
Fixation and Blink Detection by Default - #1870
In order to make Pupil Core even more useful with out-of-the-box settings, we have enabled the Fixation Detector and Blink Detector by default.
Simplified System Graphs - #1869
We have removed the configuration menu for the System Graphs plugin in order to simplify the user interface. The optional display of live graphs for the pupil diameter has been removed as well.
The confidence graphs will now always display the confidence of the 2D pupil detection, as this is a better indicator for correct positioning of the Pupil Core eye tracking headset.
Single Marker Calibration: Physical Markers - #1897
The single marker calibration is mostly intended for calibration with physical (printed) calibration markers, as opposed to the screen marker calibration. For sake of clarity, we changed the default mode for the single marker calibration to Physical Marker (previously this was called Manual).
Removed Manual Marker Calibration
With the restructuring of the calibration workflow, we have removed the deprecated Manual Marker Calibration. The Single Marker Calibration is a more general version and additionally allows for VOR-based choreographies. If you previously used the Manual Marker Calibration, try the Single Marker Calibration in Physical Marker mode (the new default)!
Parallel Pupil Detector Visualization - #1873
We have changed the color of the 2D ellipse visualization in all eye videos to blue in order to differentiate between 2D and 3D detection. You will now always see 3 ellipses on the eyes when the visualization is enabled:
Color Description
blue: 2D pupil detection result
red: 3D pupil detection result
green: 3D eye model
Flip Eye 0 Display - #1844
The eye 0 (right eye) camera sensor is physically flipped, which results in an upside-down eye image for the right eye. This is by design and does not negatively affect pupil detection or gaze estimation. However, the upside-down eye 0 image repeatedly led users to believe that something was broken or incorrect with Pupil Core headsets. We flipped the eye 0 image now by default to better match user expectations with a right-side-up eye image.
Deprecated Audio Capture - #1868
Recording synchronized audio on 3 different operating systems (macOS, Linux, Windows) has always been a challenge for Pupil Capture. (See our motivation section above.)
Unified Network API Plugin - #1881
The previous plugins Pupil Remote and Frame Publisher were commonly used to interact with Pupil Core's Network API. For the sake of simplicity and clarity, we merged both plugins into the new Network API plugin.
Removed Blink Detector Visualization - #1901
The visualization of the blink detector (brief dark flashing on the screen) could interfere with other plugins and cause visual bugs. We have removed this feature.
Bug Fixes
Fixed Player crashing when exporting while computing gaze history - #1843
Fixed Player crashing when disabling the Surface Tracker while exporting - #1848
Fixed wrong default camera selection in some cases - #1857
Fixed a rare crash in the Surface Tracker - #1879
Fixed deprecated RealSense cameras showing up in the camera selection - #1877
Fixed that offline gaze mappers without data would show as completed - #1889
Fixed corrupted gaze data when deactivating one of multiple gaze mappers - #1888
Fixed new offline pupil data not being correctly announced sometimes - #1890
Fixed a few small issues in offline pupil detection - #1899
Fixed problems with opening the macOS bundles on newer versions of macOS
Developer Notes
Changed Requirements
Scikit-learn
We have added scikit-learn as a dependency for Pupil.
pip install scikit-learn
PyAV - 1908
We updated the PyAV dependency to v0.4.5, which includes some bug fixes. On Linux and macOS, you can update with:
pip install -U git+https://github.com/pupil-labs/PyAV
On Windows, download and install the latest wheel from GitHub.
Changed data format
Both 2D and 3D pupil detector now publish their results independently of each other. To differentiate between them, we have added the origin to the topic. For a binocular setup, you will now get 4 different pupil topics:
pupil.0.2d
pupil.0.3d
pupil.1.2d
pupil.1.3d
This means that there can now be multiple pupil datums for one eye for the same timestamp (2D and 3D). When grouping by the entire topic however, the timestamps are still guaranteed to be unique.
Restarting Unique Plugins - #1871
We have added the previously missing feature to restart plugins with by_class
uniqueness via notifications.
Recording Version 2.2 - #1885
The info.player.json
meta-version has been bumped to 2.2 will be incompatible with Pupil Player version v1.
Allow specifying ZMQ HWM option - #1895
We added the option to start the eye processes with custom values for the ZMQ High Water Mark (HWM) via notification.
Custom Calibration Plugins
Below is an example with minimal implementation for a custom HMD 3D calibration choreography and gaze mapping method:
# To implement a custom calibration or gaze mapping method from scratch, subclass GazerBase and CalibrationChoreographyPlugin
from gaze_mapping.gazer_base import GazerBase
from calibration_choreography import CalibrationChoreographyPlugin
# Another option is to modify existing implementations by subclassing existing implementations
from gaze_mapping import Gazer3D
from calibration_choreography import ScreenMarkerChoreographyPlugin
class CustomGazer3D(Gazer3D):
# Must be unique across all subclasses of GazerBase
label = "Custom 3D Gazer"
class CustomChoreographyPlugin(ScreenMarkerChoreographyPlugin):
# Must be unique accross all subclasses of CalibrationChoreographyPlugin
label = "Custom Calibration"
@classmethod
def supported_gazer_classes(cls):
return [CustomGazer3D]
Release Note Updates
17.06.2020 - v2.0-175
Added compatibility for the HMD-Eyes VR integration
Fixed an issue that prevented the validation from working correctly
24.06.2020 - v2.0-177
Fixed an issue that prevented using the head pose tracker and creating new annotations in Player - #1927
25.06.2020 - v2.0-182
Fixed an issue that prevented Pupil Service from starting - #1928
Downloads
Officially minimum supported operating system versions for the bundles:
Ubuntu: 16.04macOS: 10.13 (High Sierra)Windows: 10
To open the RAR-archive on Windows, you will need a decompression software, e.g. WinRAR.
March 26, 2020
We are pleased to announce the release of Pupil v1.23!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Improvements
Windows Driver Installation Stability - #1833
We have greatly improved the driver installation workflow on Windows. You should not experience problems with spaces or non-english characters in the path to your Pupil installation anymore. Additionally, you do no longer have to run the application as administrator. You still need an administrator to authorize the driver installation.
Changes
On-demand Loading of Multi-part Recordings - #1835
Multiple users have reported problems when opening Pupil Mobile recordings with a large number of video parts. We were able to fix this issue by loading the video parts on demand instead of opening them all at once. As a result, Pupil Player will open Pupil Mobile or Pupil Invisible recordings with multiple parts much quicker than before.
As a side-effect, during playback in Pupil Player, you might experience short frame jumps when a new video part needs to be loaded. This does not affect the underlying data or any exports.
Bug Fixes
Fixed a crash when changing the ROI without a connected headset - #1827
Fixed a crash when opening recordings with empty video parts - #1832
Gracefully stop recordings when receiving non-monotonic timestamps - #1829
Fixed an issue with calibration file storage on macOS - #1836
Fixed an issue with gaze history settings restoration from session settings - #1836
Developer Notes
Changed Requirements - #1834
We updated the pyndsi
dependency on v1.3. On Linux and macOS, you can update with:
pip install -U git+https://github.com/pupil-labs/pyndsi
On Windows, please download and install the latest wheel from GitHub.
Removed All Direct C++ Dependencies - #1828
After the latest rework of the calibration codebase to remove C++ dependencies in v1.22, we have re-evaluated all dependencies for Pupil Core software -- especially on Windows. We are very pleased to announce that we have finally externalized or gotten rid of all C++ dependencies! :tada:
This greatly simplifies the developer setup on Windows. The biggest changes include:
You no longer have to install Visual Studio to run from source
You no longer need to download OpenCV
You are no longer bound to a fixed path
C:\work
We have updated the developer docs with the updated setup instructions. We hope that this simplifies your setup and enables you to develop awesome plugins for Pupil Capture and Pupil Player that can be shared with the community!
Release Note Updates
30.03.2020 14:00: Fix driver installation issue in Windows bundle - #1842
In Pupil v1.23-4
, we fixed an issue that prevented users from installing the camera drivers on Windows.
06.04.2020 11:50: Include missing libuvc.dylib
in macOS bundle
In Pupil v1.23-5
, we have included missing library files for Pupil Player and Pupil Service on macOS.
08.04.2020 17:00: Include missing libusb.dylib
in macOS bundle
In Pupil v1.23-5-fix0
, we have included missing library files for Pupil Player and Pupil Service on macOS.
24.04.2020 14:00: Improve driver installation stability on Windows - #1856
In Pupil v1.23-10
, we have made changes to the driver installation procedure that ensure a successful installation on a larger variety of Windows setups.
March 26, 2020
We are pleased to announce the release of Pupil v1.23!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Improvements
Windows Driver Installation Stability - #1833
We have greatly improved the driver installation workflow on Windows. You should not experience problems with spaces or non-english characters in the path to your Pupil installation anymore. Additionally, you do no longer have to run the application as administrator. You still need an administrator to authorize the driver installation.
Changes
On-demand Loading of Multi-part Recordings - #1835
Multiple users have reported problems when opening Pupil Mobile recordings with a large number of video parts. We were able to fix this issue by loading the video parts on demand instead of opening them all at once. As a result, Pupil Player will open Pupil Mobile or Pupil Invisible recordings with multiple parts much quicker than before.
As a side-effect, during playback in Pupil Player, you might experience short frame jumps when a new video part needs to be loaded. This does not affect the underlying data or any exports.
Bug Fixes
Fixed a crash when changing the ROI without a connected headset - #1827
Fixed a crash when opening recordings with empty video parts - #1832
Gracefully stop recordings when receiving non-monotonic timestamps - #1829
Fixed an issue with calibration file storage on macOS - #1836
Fixed an issue with gaze history settings restoration from session settings - #1836
Developer Notes
Changed Requirements - #1834
We updated the pyndsi
dependency on v1.3. On Linux and macOS, you can update with:
pip install -U git+https://github.com/pupil-labs/pyndsi
On Windows, please download and install the latest wheel from GitHub.
Removed All Direct C++ Dependencies - #1828
After the latest rework of the calibration codebase to remove C++ dependencies in v1.22, we have re-evaluated all dependencies for Pupil Core software -- especially on Windows. We are very pleased to announce that we have finally externalized or gotten rid of all C++ dependencies! :tada:
This greatly simplifies the developer setup on Windows. The biggest changes include:
You no longer have to install Visual Studio to run from source
You no longer need to download OpenCV
You are no longer bound to a fixed path
C:\work
We have updated the developer docs with the updated setup instructions. We hope that this simplifies your setup and enables you to develop awesome plugins for Pupil Capture and Pupil Player that can be shared with the community!
Release Note Updates
30.03.2020 14:00: Fix driver installation issue in Windows bundle - #1842
In Pupil v1.23-4
, we fixed an issue that prevented users from installing the camera drivers on Windows.
06.04.2020 11:50: Include missing libuvc.dylib
in macOS bundle
In Pupil v1.23-5
, we have included missing library files for Pupil Player and Pupil Service on macOS.
08.04.2020 17:00: Include missing libusb.dylib
in macOS bundle
In Pupil v1.23-5-fix0
, we have included missing library files for Pupil Player and Pupil Service on macOS.
24.04.2020 14:00: Improve driver installation stability on Windows - #1856
In Pupil v1.23-10
, we have made changes to the driver installation procedure that ensure a successful installation on a larger variety of Windows setups.
February 20, 2020
We are pleased to announce the release of Pupil v1.22!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Features and Changes
Video Source Rework - #1792
We have reworked the video source input system to greatly improve stability, user experience, and maintainability of the related features:
Single Plugin UI
All previous plugins for managing the video source (managers and sources) have been unified under a single plugin menu: Video Source, which handles device/camera selection as well as adjusting camera settings.
Automatic vs Manual Camera Selection
The new source system offers two ways for selecting cameras: automatic and manual:
In automatic mode (default) you can activate source devices, e.g. Local USB or Pupil Mobile. Activating a device selects the best matching available cameras as input for all windows in Pupil Capture.
Enabling manual mode allows you to select cameras individually for every window.
UI Cleanup
The following non-essential user interface elements have been removed to provide a better overall user experience:
The UI for selecting a file source has been removed. Starting a file source via network notification or via drag-and-drop is still possible.
Manually selecting HMD-Eyes as input source has been removed, as the plugin is started from Unity via network notification.
The Capture plugin UI in the eye windows has been removed entirely from Pupil Player as the displayed information was incomplete and the interactivity was limited.
The Test Image source has been removed as it was only intended for development use.
Removed Built-in RealSense Video Backend
Intel RealSense sensors are designed as a prototyping/development platform. Intel deprecates RealSense hardware frequently. Driver support is appropriate for a prototyping tool, but it is not sufficient for a production ready tool.
We have tried to build our tools around Intel's changing APIs and patchwork driver support, but this became difficult and confusing for users.
You can still use RealSense Video Backend as a custom plugin. You can load it directly in the plugin folder of Pupil Capture. Check it out here.
Player: Add Gaze History to Polyline Visualization - #1785
We decided to use the Polyline Visualization as a replacement for the previously removed Scan Path Plugin. You can now visualize past gaze points up to a certain duration by adjusting the duration slider in the Vis Polyline
Plugin. When first dragging the slider, the history for all frames will be computed, which will take some time. Afterwards you can see the gaze history and adjust the duration instantly.
Capture: Improved Region of Interest (ROI) selection - #1788
The ROI selector for the eye windows has been improved for better stability and better user experience. In doing so, we standardized the format of ROI boundaries across Pupil to the following order: (minx, miny, maxx, maxy)
. Please note that this changes the format for adjusting the ROI via network messages.
Player: Add 3D Eyeball Visualization - #1810
We have added the green circle overlay for the 3D eyeball to Player. The overlay will be added to the Eye Overlay
and Eye Video Exporter
plugins, when 3D pupil data is available. You can disable the visualization (including the pupil outline) via the corresponding plugin menus.
Bug Fixes
Changed Windows disconnection log-level from warning to debug - #1786
Fixed some of small bugs in the ROI selector - #1788
Fixed mixed up export columns for monocular
eye_center
andgaze_normal
- #1805Prevent a crash caused by incorrect UVC timestamps - #1812
Developer Notes
Getting Rid of C++ Dependencies - #1789
We have made the next step towards an easier installation process of Pupil. After the externalization of the Pupil Detectors in v1.20, we have been able to replace the last remaining C++ module with a Scipy re-implementation. This means that you do not have to compile any Python Extension Modules when starting Pupil.
As a result, you can now run Pupil from source on Windows without having to go through the trouble of setting up CERES. While the pupil-detectors still depend on CERES, we offer prebuilt wheels for Windows that you can install without having to set up the dependencies.
Changed Requirements
We updated the pyglui
dependency to v1.27. Please update with:
pip install -U git+https://github.com/pupil-labs/pyglui
We Are Hiring Python & DevOps Engineers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for Senior Full Stack/DevOps engineers that have experience with kubernetes, docker, and server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
Release Notes Update
25.02.2019 v1.22.3
#1817 - Fixed a regression which caused all frames to be dropped after changing Pupil Capture's time base.
27.02.2019 v1.22.7
Commit bcfbf7d - Fixed a typo that caused the
v1.22.3
fix to not work correctly.
February 20, 2020
We are pleased to announce the release of Pupil v1.22!
Download the latest bundle (scroll down to the end of the release notes and see Assets).
Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Features and Changes
Video Source Rework - #1792
We have reworked the video source input system to greatly improve stability, user experience, and maintainability of the related features:
Single Plugin UI
All previous plugins for managing the video source (managers and sources) have been unified under a single plugin menu: Video Source, which handles device/camera selection as well as adjusting camera settings.
Automatic vs Manual Camera Selection
The new source system offers two ways for selecting cameras: automatic and manual:
In automatic mode (default) you can activate source devices, e.g. Local USB or Pupil Mobile. Activating a device selects the best matching available cameras as input for all windows in Pupil Capture.
Enabling manual mode allows you to select cameras individually for every window.
UI Cleanup
The following non-essential user interface elements have been removed to provide a better overall user experience:
The UI for selecting a file source has been removed. Starting a file source via network notification or via drag-and-drop is still possible.
Manually selecting HMD-Eyes as input source has been removed, as the plugin is started from Unity via network notification.
The Capture plugin UI in the eye windows has been removed entirely from Pupil Player as the displayed information was incomplete and the interactivity was limited.
The Test Image source has been removed as it was only intended for development use.
Removed Built-in RealSense Video Backend
Intel RealSense sensors are designed as a prototyping/development platform. Intel deprecates RealSense hardware frequently. Driver support is appropriate for a prototyping tool, but it is not sufficient for a production ready tool.
We have tried to build our tools around Intel's changing APIs and patchwork driver support, but this became difficult and confusing for users.
You can still use RealSense Video Backend as a custom plugin. You can load it directly in the plugin folder of Pupil Capture. Check it out here.
Player: Add Gaze History to Polyline Visualization - #1785
We decided to use the Polyline Visualization as a replacement for the previously removed Scan Path Plugin. You can now visualize past gaze points up to a certain duration by adjusting the duration slider in the Vis Polyline
Plugin. When first dragging the slider, the history for all frames will be computed, which will take some time. Afterwards you can see the gaze history and adjust the duration instantly.
Capture: Improved Region of Interest (ROI) selection - #1788
The ROI selector for the eye windows has been improved for better stability and better user experience. In doing so, we standardized the format of ROI boundaries across Pupil to the following order: (minx, miny, maxx, maxy)
. Please note that this changes the format for adjusting the ROI via network messages.
Player: Add 3D Eyeball Visualization - #1810
We have added the green circle overlay for the 3D eyeball to Player. The overlay will be added to the Eye Overlay
and Eye Video Exporter
plugins, when 3D pupil data is available. You can disable the visualization (including the pupil outline) via the corresponding plugin menus.
Bug Fixes
Changed Windows disconnection log-level from warning to debug - #1786
Fixed some of small bugs in the ROI selector - #1788
Fixed mixed up export columns for monocular
eye_center
andgaze_normal
- #1805Prevent a crash caused by incorrect UVC timestamps - #1812
Developer Notes
Getting Rid of C++ Dependencies - #1789
We have made the next step towards an easier installation process of Pupil. After the externalization of the Pupil Detectors in v1.20, we have been able to replace the last remaining C++ module with a Scipy re-implementation. This means that you do not have to compile any Python Extension Modules when starting Pupil.
As a result, you can now run Pupil from source on Windows without having to go through the trouble of setting up CERES. While the pupil-detectors still depend on CERES, we offer prebuilt wheels for Windows that you can install without having to set up the dependencies.
Changed Requirements
We updated the pyglui
dependency to v1.27. Please update with:
pip install -U git+https://github.com/pupil-labs/pyglui
We Are Hiring Python & DevOps Engineers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for Senior Full Stack/DevOps engineers that have experience with kubernetes, docker, and server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
Release Notes Update
25.02.2019 v1.22.3
#1817 - Fixed a regression which caused all frames to be dropped after changing Pupil Capture's time base.
27.02.2019 v1.22.7
Commit bcfbf7d - Fixed a typo that caused the
v1.22.3
fix to not work correctly.
February 16, 2020
Improved stability of video transcoding pipeline.
February 16, 2020
Improved stability of video transcoding pipeline.
January 9, 2020
We are pleased to announce the release of Pupil v1.21!
Download the latest bundle (scroll down to the end of the release notes and see Assets). Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Features and Changes
Support for Pupil Invisible Audio Recordings - #1748
Pupil Player now supports audio playback and audio export for Pupil Invisible recordings with audio recordings.
Stream Head Pose over Network - #1774
You can now subscribe to the the topic head_pose
to receive live data from the Online Head Pose Tracker plugin.
Bug Fixes
Fixed eye processes always displayed the algorithm view - #1766
Fixed potential crash when changing the eye video size - #1770
Made pyrealsense warning a debug-level message in order to be less obtrusive - #1768
Fixed broken interactivity of the ROI mask in the eye windows - #1778
Fixed multiple stability issues with the fullscreen calibration window - #1767
Developer Notes
Changed Requirements
We updated the PyAV
dependency to v0.4.4 for the PI audio support. On Linux and macOS, you can update with:
pip install -U git+https://github.com/pupil-labs/PyAV
On Windows, download and install the latest wheel from GitHub.
We Are Hiring Python & DevOps Engineers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for Senior Full Stack/DevOps engineers that have experience with kubernetes, docker, and server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
January 9, 2020
We are pleased to announce the release of Pupil v1.21!
Download the latest bundle (scroll down to the end of the release notes and see Assets). Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Features and Changes
Support for Pupil Invisible Audio Recordings - #1748
Pupil Player now supports audio playback and audio export for Pupil Invisible recordings with audio recordings.
Stream Head Pose over Network - #1774
You can now subscribe to the the topic head_pose
to receive live data from the Online Head Pose Tracker plugin.
Bug Fixes
Fixed eye processes always displayed the algorithm view - #1766
Fixed potential crash when changing the eye video size - #1770
Made pyrealsense warning a debug-level message in order to be less obtrusive - #1768
Fixed broken interactivity of the ROI mask in the eye windows - #1778
Fixed multiple stability issues with the fullscreen calibration window - #1767
Developer Notes
Changed Requirements
We updated the PyAV
dependency to v0.4.4 for the PI audio support. On Linux and macOS, you can update with:
pip install -U git+https://github.com/pupil-labs/PyAV
On Windows, download and install the latest wheel from GitHub.
We Are Hiring Python & DevOps Engineers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for Senior Full Stack/DevOps engineers that have experience with kubernetes, docker, and server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
December 12, 2019
We are pleased to announce the release of Pupil v1.20!
Download the latest bundle (scroll down to the end of the release notes and see Assets). Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Features and Changes
Deprecated Fingertip Calibration - #1753, #1757
We removed the fingertip calibration method.
The fingertip calibration method was rarely used and provided lower accuracy compared to other calibration methods. Additionally, dependencies needed for the fingertip detection required some work to set up correctly and accounted for almost 50% of the size of our application bundles.
Bug Fixes
Fixed a crash when trying to open empty (but existing) recordings - #1751
Fixed a crash on Windows when missing the latest Visual Studio redistributables - #1756
Fixed visualization of circle markers in offline calibration for recordings with gaps - #1758
Fixed broken combination of
--hide-ui
and--profile
CLI flags - #1760
Developer Notes
Externalized Pupil Detectors - #1642
We have extracted the Pupil Detectors into their own repository: https://github.com/pupil-labs/pupil-detectors
This will make it very easy to use our pupil detectors in any standalone application or experimental setup without having to deal with all of Pupil's intricacies.
Additionally this will make it much easier to run Pupil from source (especially on Windows), since we are distributing pupil-detectors as package via PyPI and even provide precompiled binaries for Windows!
OpenGL Debugging - #1752
We occasionally receive crash reports involving error messages from PyOpenGL, which we cannot reproduce on our machines. We wrapped the PyOpenGL errors to prevent crashes and log additional messages in these cases. This can result in visual UI errors (instead of a crash) when there are OpenGL issues. If you notice any weird UI behavior, please save the log file and report back to us.
We Are Hiring Python & DevOps Engineers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for Senior Full Stack/DevOps engineers that have experience with kubernetes, docker, and server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
December 12, 2019
We are pleased to announce the release of Pupil v1.20!
Download the latest bundle (scroll down to the end of the release notes and see Assets). Please feel free to get in touch with feedback and questions via the #pupil channel on Discord 😄
Features and Changes
Deprecated Fingertip Calibration - #1753, #1757
We removed the fingertip calibration method.
The fingertip calibration method was rarely used and provided lower accuracy compared to other calibration methods. Additionally, dependencies needed for the fingertip detection required some work to set up correctly and accounted for almost 50% of the size of our application bundles.
Bug Fixes
Fixed a crash when trying to open empty (but existing) recordings - #1751
Fixed a crash on Windows when missing the latest Visual Studio redistributables - #1756
Fixed visualization of circle markers in offline calibration for recordings with gaps - #1758
Fixed broken combination of
--hide-ui
and--profile
CLI flags - #1760
Developer Notes
Externalized Pupil Detectors - #1642
We have extracted the Pupil Detectors into their own repository: https://github.com/pupil-labs/pupil-detectors
This will make it very easy to use our pupil detectors in any standalone application or experimental setup without having to deal with all of Pupil's intricacies.
Additionally this will make it much easier to run Pupil from source (especially on Windows), since we are distributing pupil-detectors as package via PyPI and even provide precompiled binaries for Windows!
OpenGL Debugging - #1752
We occasionally receive crash reports involving error messages from PyOpenGL, which we cannot reproduce on our machines. We wrapped the PyOpenGL errors to prevent crashes and log additional messages in these cases. This can result in visual UI errors (instead of a crash) when there are OpenGL issues. If you notice any weird UI behavior, please save the log file and report back to us.
We Are Hiring Python & DevOps Engineers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for Senior Full Stack/DevOps engineers that have experience with kubernetes, docker, and server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
November 21, 2019
We are pleased to announce the release of Pupil v1.19!
Download the latest bundle (at the end of the release notes under Assets) and let us know what you think via the #pupil channel on Discord 😄
Features and Changes
Improved Fixation Detector - #1743
We made improvements to the Fixation Detector to improve detection accuracy:
We were able to get rid of large gaps in between detected fixations. Now there will be more sequential fixations detected.
We decreased the false-positive detection rate by omitting low-confidence data.
We fixed a bug that would sometimes result in wrong fixations on 3D gaze data.
API Changes
Due to the improvements mentioned above, the Fixation Detector now requires calibrated gaze in both 2D and 3D mode. To reflect these changes, the method
field of a fixation datum now contains either 2d gaze
or 3d gaze
(previously pupil
vs gaze
). This effects both the real-time fixations published through the network API and the fixations.csv
export file.
Removed Fixation-Boosted Calibration - #1744
Since the Fixation Detector now requires calibrated gaze, it will not be used anymore to speed up the calibration process.
Improved Binocular Gaze Mapping - #1731
We improved the mapping of monocular gaze data streams by using a dynamic windows size dependent on the frame rate. This will result in better temporal accuracy when matching binocular gaze data for eyes with frequent low-confidence phases.
We also investigated some other variants of binocular gaze mapping, but chose to stick with this simple improvement. For a full evaluation and discussion, see this jupyter notebook.
Bug Fixes
Add/remove markers for legacy square markers works again - #1737
Fixed some bad UX with switching surface detection modes - #1738
Player is now able to load recordings with square brackets in the name - #1740
Fixed a crash when setting 2D detector properties via network API - #1742
Improved inconsistent background process behavior - #1741
Developer Notes
Automated Testing - #1721, #1724, #1730, #1735
We have done some work to consolidate old tests in the codebase and set it all up nicely with pytest. You can run the test suite from the repository root with:
pip install pytest
pytest
We set up a travis-ci integration to run pytest on all commits and pull requests. Now you will get feedback if you break something 😄 The test coverage is currently low. But from this point on, we - and you - are encouraged to write tests when adding new features and when fixing bugs.
We Are Hiring Python & DevOps Engineers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for Senior Full Stack/DevOps engineers that have experience with kubernetes, docker, and server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
Release Notes Update
27.11.2019 17:00: v1.19.2
We have fixed an issue in v1.19.0
where multi-part Pupil Mobile and Pupil Invisible recordings were not displayed correctly in Pupil Player - #1747. If you have opened such a recording in Pupil Player v1.19.0
then delete all *_lookup.npy
files and reopen the recording in Pupil Player v1.19.2
.
November 21, 2019
We are pleased to announce the release of Pupil v1.19!
Download the latest bundle (at the end of the release notes under Assets) and let us know what you think via the #pupil channel on Discord 😄
Features and Changes
Improved Fixation Detector - #1743
We made improvements to the Fixation Detector to improve detection accuracy:
We were able to get rid of large gaps in between detected fixations. Now there will be more sequential fixations detected.
We decreased the false-positive detection rate by omitting low-confidence data.
We fixed a bug that would sometimes result in wrong fixations on 3D gaze data.
API Changes
Due to the improvements mentioned above, the Fixation Detector now requires calibrated gaze in both 2D and 3D mode. To reflect these changes, the method
field of a fixation datum now contains either 2d gaze
or 3d gaze
(previously pupil
vs gaze
). This effects both the real-time fixations published through the network API and the fixations.csv
export file.
Removed Fixation-Boosted Calibration - #1744
Since the Fixation Detector now requires calibrated gaze, it will not be used anymore to speed up the calibration process.
Improved Binocular Gaze Mapping - #1731
We improved the mapping of monocular gaze data streams by using a dynamic windows size dependent on the frame rate. This will result in better temporal accuracy when matching binocular gaze data for eyes with frequent low-confidence phases.
We also investigated some other variants of binocular gaze mapping, but chose to stick with this simple improvement. For a full evaluation and discussion, see this jupyter notebook.
Bug Fixes
Add/remove markers for legacy square markers works again - #1737
Fixed some bad UX with switching surface detection modes - #1738
Player is now able to load recordings with square brackets in the name - #1740
Fixed a crash when setting 2D detector properties via network API - #1742
Improved inconsistent background process behavior - #1741
Developer Notes
Automated Testing - #1721, #1724, #1730, #1735
We have done some work to consolidate old tests in the codebase and set it all up nicely with pytest. You can run the test suite from the repository root with:
pip install pytest
pytest
We set up a travis-ci integration to run pytest on all commits and pull requests. Now you will get feedback if you break something 😄 The test coverage is currently low. But from this point on, we - and you - are encouraged to write tests when adding new features and when fixing bugs.
We Are Hiring Python & DevOps Engineers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for Senior Full Stack/DevOps engineers that have experience with kubernetes, docker, and server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
Release Notes Update
27.11.2019 17:00: v1.19.2
We have fixed an issue in v1.19.0
where multi-part Pupil Mobile and Pupil Invisible recordings were not displayed correctly in Pupil Player - #1747. If you have opened such a recording in Pupil Player v1.19.0
then delete all *_lookup.npy
files and reopen the recording in Pupil Player v1.19.2
.
November 5, 2019
We are pleased to announce the release of Pupil v1.18!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Features and Changes
Player: Deprecated Recordings - #1711
We have removed support for opening recordings made with:
Pupil Core < v1.3 or
Pupil Mobile < r0.21.0
If you need to open a recording made with one of these versions, you will have to open it once in Pupil Player v1.16 or v1.17. This will upgrade the recording format to the new version. You can find more information about this in the v1.16 release notes.
Surface Tracker Export: Distorted Image <-> Surface Coordinates - #1678
We added the homographies to the surface tracker export, which can be used to convert pixel positions between surface coordinates and coordinates of the original camera image. You can use these to crop the surface from the distorted scene image, and get something similar to our surface debug view.
The homographies are exported as two additional columns in the surf_positions_<name>.csv
export file:
dist_img_to_surf_trans
surf_to_dist_img_trans
Exporting Frame PTS - #1682
We added a column pts
to the timestamps export in <name>_timestamps.csv
. This information can be used to seek to and retrieve single frames from the exported video file without the need for sequentially iterating over the entire video.
Service: Start Binocular - #1714
For consistency with Pupil Capture, Pupil Service will now start in binocular mode.
Pupil Remote Plugin: Port No Longer Saved - #1715
The selected port for the Pupil Remote Plugin will no longer be stored in the Capture/Service settings. The default port is always 50020. If the port is already in use, Capture will choose a random other port, while Service will shut down again. You can choose a different port on startup with the -P
command line flag.
Bug Fixes
Eye movements overlayed in Player will now be correctly exported - #1675
Shutting down eye processes while recording will correctly finish the recording - #1677
Fixed some UX issues with the Head Pose Tracker visualization window - #1669
Improved performance of apriltag detection for the Head Pose Tracker - #1669
Fixed Pupil Service UI showing port
None
- #1715Pupil Service will now be shut down correctly if the selected port is already in use - #1716
Windows run*.bat files will now correctly forward CLI arguments - #1717
Fixed Pupil Invisible recordings displaying erroneous gaze positions at (0,0) - #1719
Developer Notes
Eye-Process Plugin List - #1674, #1681
The Eye processes are now using the Plugin_List
class that we are already using in Capture, Player and Service. It takes care of managing the plugin lifecycle and makes it easier to add, remove or change plugins on the Eye process.
Command Line Interface Overhaul
We improved the handling of the Command Line Interface (CLI) of Pupil apps and added some additional start flags:
App Arguments Only - #1709
We changed the general structure of Pupil command line invocation to:
python main.py [-h] {Capture,Player,Service} [app-specific-arguments]
For the full list of arguments, see the README of the Pupil repository.
main.py
takes no arguments except -h/--help
; resulting in an info message. The <app>
argument has been made required and will not default to Capture
.
Version Flag - #1679
You can start any Pupil App with the --version
flag. This will display the current version and exit the program afterwards.
Hide UI Flag - #1712
You can now start Pupil Capture and Pupil Service with the --hide-ui
flag. This will suppress all UI creation while running the program. Use remote messages to control the behavior of the program. To stop the app gracefully, you can either send a SIGTERM signal or (when running via console) hit CTRL-C
to send a SIGINT signal (see point below for additional info).
CTRL-C Shutdown - #1708, #1713
We added support for gracefully shutting down all apps (Capture, Player, Service) via hitting CTRL-C
in the console from where it is running. All running operations and processes will be correctly finalized and terminated.
Recording Version 2.1
The info.player.json
meta-version has been bumped to 2.1. This was necessary to reverse the effects of a bug that corrupted some recordings (#1719).
We Are Hiring Python & DevOps Engineers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for Senior Full Stack/DevOps engineers that have experience with kubernetes, docker, and server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
Release Note Updates
06.11.2019 11:10: Fix opening Pupil Mobile recordings - #1722
In Pupil v1.18-4
, we fixed an issue that prevented users from opening newly created Pupil Mobile recordings in Pupil Player.
11.11.2019 13:10: Fix Offline Pupil Detection on Windows - #1727
In Pupil v.1.18-35
, we fixed an issue that caused Pupil Player to crash when running the Offline Pupil Detection on Windows.
November 5, 2019
We are pleased to announce the release of Pupil v1.18!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Features and Changes
Player: Deprecated Recordings - #1711
We have removed support for opening recordings made with:
Pupil Core < v1.3 or
Pupil Mobile < r0.21.0
If you need to open a recording made with one of these versions, you will have to open it once in Pupil Player v1.16 or v1.17. This will upgrade the recording format to the new version. You can find more information about this in the v1.16 release notes.
Surface Tracker Export: Distorted Image <-> Surface Coordinates - #1678
We added the homographies to the surface tracker export, which can be used to convert pixel positions between surface coordinates and coordinates of the original camera image. You can use these to crop the surface from the distorted scene image, and get something similar to our surface debug view.
The homographies are exported as two additional columns in the surf_positions_<name>.csv
export file:
dist_img_to_surf_trans
surf_to_dist_img_trans
Exporting Frame PTS - #1682
We added a column pts
to the timestamps export in <name>_timestamps.csv
. This information can be used to seek to and retrieve single frames from the exported video file without the need for sequentially iterating over the entire video.
Service: Start Binocular - #1714
For consistency with Pupil Capture, Pupil Service will now start in binocular mode.
Pupil Remote Plugin: Port No Longer Saved - #1715
The selected port for the Pupil Remote Plugin will no longer be stored in the Capture/Service settings. The default port is always 50020. If the port is already in use, Capture will choose a random other port, while Service will shut down again. You can choose a different port on startup with the -P
command line flag.
Bug Fixes
Eye movements overlayed in Player will now be correctly exported - #1675
Shutting down eye processes while recording will correctly finish the recording - #1677
Fixed some UX issues with the Head Pose Tracker visualization window - #1669
Improved performance of apriltag detection for the Head Pose Tracker - #1669
Fixed Pupil Service UI showing port
None
- #1715Pupil Service will now be shut down correctly if the selected port is already in use - #1716
Windows run*.bat files will now correctly forward CLI arguments - #1717
Fixed Pupil Invisible recordings displaying erroneous gaze positions at (0,0) - #1719
Developer Notes
Eye-Process Plugin List - #1674, #1681
The Eye processes are now using the Plugin_List
class that we are already using in Capture, Player and Service. It takes care of managing the plugin lifecycle and makes it easier to add, remove or change plugins on the Eye process.
Command Line Interface Overhaul
We improved the handling of the Command Line Interface (CLI) of Pupil apps and added some additional start flags:
App Arguments Only - #1709
We changed the general structure of Pupil command line invocation to:
python main.py [-h] {Capture,Player,Service} [app-specific-arguments]
For the full list of arguments, see the README of the Pupil repository.
main.py
takes no arguments except -h/--help
; resulting in an info message. The <app>
argument has been made required and will not default to Capture
.
Version Flag - #1679
You can start any Pupil App with the --version
flag. This will display the current version and exit the program afterwards.
Hide UI Flag - #1712
You can now start Pupil Capture and Pupil Service with the --hide-ui
flag. This will suppress all UI creation while running the program. Use remote messages to control the behavior of the program. To stop the app gracefully, you can either send a SIGTERM signal or (when running via console) hit CTRL-C
to send a SIGINT signal (see point below for additional info).
CTRL-C Shutdown - #1708, #1713
We added support for gracefully shutting down all apps (Capture, Player, Service) via hitting CTRL-C
in the console from where it is running. All running operations and processes will be correctly finalized and terminated.
Recording Version 2.1
The info.player.json
meta-version has been bumped to 2.1. This was necessary to reverse the effects of a bug that corrupted some recordings (#1719).
We Are Hiring Python & DevOps Engineers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for Senior Full Stack/DevOps engineers that have experience with kubernetes, docker, and server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
Release Note Updates
06.11.2019 11:10: Fix opening Pupil Mobile recordings - #1722
In Pupil v1.18-4
, we fixed an issue that prevented users from opening newly created Pupil Mobile recordings in Pupil Player.
11.11.2019 13:10: Fix Offline Pupil Detection on Windows - #1727
In Pupil v.1.18-35
, we fixed an issue that caused Pupil Player to crash when running the Offline Pupil Detection on Windows.
October 24, 2019
We are pleased to announce the release of Pupil v1.17!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Features & Changes
Surface Tracker Updates
We have fixed stability issues with surfaces defined with AprilTag markers. See issue #1696 for more information.
We now support more AprilTag marker types (#1700)! Checkout the docs for more info about tag types and settings.
Legacy Square Markers
If you have recordings with Legacy Markers, and would like to use the latest release to redefine surfaces, please get in touch with us via chat.
Heatmap Image Export Size - #1702
Heatmap images are now exported with a minimum size of 2000px. This will make it easier to use them out of the box without having to upscale them manually.
Streaming Pupil Invisible into Pupil Capture- #1693
You can now stream live video and gaze data from Pupil Invisible to Pupil Capture!
Streaming Scene Video
Select Pupil Mobile backend in Pupil Capture's Backend Manager. Select your Companion Device as Remote Host. Note that the Pupil Invisible Companion Device and Pupil Capture need to be connected to the same network and recording is not supported.
Streaming Gaze Data
Install the PI Preview Plugin.
Removed NSLR Eye-Movement Classifier - #1698
We found that the library did not perform well enough to be usefull and removed it from Pupil Player and Capture. Let us know if you need it back!
Bug Fixes
Fixed Eye Movement Visualizations not being exported every time - #1675
Fixed broken recordings when closing the eye windows while recording - #1677
Fixed inaccurate UI when editing surface corners - #1692
Fixed unresponsive UI for the gaze history length of online heatmaps - #1697
Fixed an error with loading some Pupil Invisible recordings - #1704
Fixed that surfaces would not get saved to recordings when they had never been modified - #1706
Developer Notes
We Are Hiring Python Developers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for Senior Full Stack/DevOps engineers that have experience with kubernetes, docker, and server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
October 24, 2019
We are pleased to announce the release of Pupil v1.17!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Features & Changes
Surface Tracker Updates
We have fixed stability issues with surfaces defined with AprilTag markers. See issue #1696 for more information.
We now support more AprilTag marker types (#1700)! Checkout the docs for more info about tag types and settings.
Legacy Square Markers
If you have recordings with Legacy Markers, and would like to use the latest release to redefine surfaces, please get in touch with us via chat.
Heatmap Image Export Size - #1702
Heatmap images are now exported with a minimum size of 2000px. This will make it easier to use them out of the box without having to upscale them manually.
Streaming Pupil Invisible into Pupil Capture- #1693
You can now stream live video and gaze data from Pupil Invisible to Pupil Capture!
Streaming Scene Video
Select Pupil Mobile backend in Pupil Capture's Backend Manager. Select your Companion Device as Remote Host. Note that the Pupil Invisible Companion Device and Pupil Capture need to be connected to the same network and recording is not supported.
Streaming Gaze Data
Install the PI Preview Plugin.
Removed NSLR Eye-Movement Classifier - #1698
We found that the library did not perform well enough to be usefull and removed it from Pupil Player and Capture. Let us know if you need it back!
Bug Fixes
Fixed Eye Movement Visualizations not being exported every time - #1675
Fixed broken recordings when closing the eye windows while recording - #1677
Fixed inaccurate UI when editing surface corners - #1692
Fixed unresponsive UI for the gaze history length of online heatmaps - #1697
Fixed an error with loading some Pupil Invisible recordings - #1704
Fixed that surfaces would not get saved to recordings when they had never been modified - #1706
Developer Notes
We Are Hiring Python Developers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for Senior Full Stack/DevOps engineers that have experience with kubernetes, docker, and server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
October 2, 2019
We are pleased to announce the release of Pupil v1.16!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Features
Pupil Invisible Support - #1596, #1623
Preparing for the upcoming public release of Pupil Invisible (PI), we are adding support for recordings made with PI in Pupil Player.
You will be able to download the raw recording from Pupil Cloud or copy them directly from the Companion device. Drag and drop the recordings onto Pupil Player as usual.
Most features of Player will support recordings made with PI. Currently not supported are: The iMotions exporter and blink detector.
Surface Tracker
Removed Robust Detection from SurfaceTracker - #1600
The robust detection mode was a remnant of the legacy surface marker system. The new Apriltags markers are by default more robust than even the old robust detection mode. This option did not have an effect anymore, so it was removed.
Bugfixes
Disabled Hardware Timestamps on Windows Operating System - #1621
In the past we have received multiple reports of timestamps being out of sync on Windows. After some testing we discovered that the timestamps from the hardware were not always reliable on Windows. The exact cause of this is still unknown but we have disabled the use of hardware timestamps on Windows for now. We are now using software timestamps with an appropriate offset-mapping.
Improved Eye Overlay Visualizations - #1609
Previously, the Eye Overlay plugin would always display the closest eye frame to the current playback position. Therefore, in cases where the eye camera disconnected during a recording, the eye video was visualized incorrectly. Instead, now, we show a gray frame to indicate missing data.
Smaller Fixes
Fixed minor performance issues in Apriltag surface marker detector
Fixed broken SurfaceTracker heatmap rendering when no gaze data is available - #1584
Fixed SurfaceTracker timeline drawing performance issues - #1595
Fixed issues with EyeMovementDetector when using inconsistent 3D gaze data - #1587
Fixed UI/UX issues when editing surfaces in SurfaceTracker - #1591
Fixed flickering surface markers - #1600
Updated the app icons for the Windows bundles to the latest design
Fixed incorrect handling of missing projection matrices for HMD - #1603
Fixed performance-related crashes of the SurfaceTracker on Windows - #1605, #1606
Improved user feedback when opening invalid recordings - #1585
Fixed crashing offline detection in recordings with timing irregularities - #1615
Improved user feedback when trying to load an unknown plugin - #1613
Fixed a crash when trying to use Pupil Invisible with Capture - #1620
Fixed a crash when setting color values for CircleViz to 1.0 - #1624
Improved user feedback when recording from Pupil Mobile without TimeSync - #1629
Deprecated Recordings
Previously released versions of Pupil Player v1.16 do not support recordings created with
Pupil Capture
v1.2
or earlier, andPupil Mobile
r0.22.x
or earlier.
This was due to the fact that these recordings are missing meta information that is required for the upgrade to our new Pupil Player Recording Format 2.0
. For details see "Missing Meta Information" below.
With this PR, instead of aborting with a "Recording too old" warning, Pupil Player will attempt to infer the missing values from other data in the recording. This inference is not perfect and results in imprecise values and might cause issues when converting recorded Pupil time to wall clock time. For details see "Missing Meta Information" below.
Upgrading will only work in Player v1.16
Since this inference is limited, we will remove the upgrade functionality for deprecated recordings in v1.17. This means, that if you want to open a deprecated recording in v1.17 or later, you will have to convert them to the new format using v1.16 first.
Missing Meta Information
Specifically the following keys in the info.csv
file are missing:
Start Time (System)
: Recording start time in Unix epochStart Time (Synced)
: Recording start time in Pupil time
Pupil Player assumes that these timestamps were measured at the same time. This allows for the after-the-effect synchronization of Pupil data with externally recorded data that uses the Unix epoch.
Their value can be inferred from the existing recording, given a roughly known imprecision:
Start Time (System)
: The system start time can be inferred using the existingStart Date
andStart Time
fields (precision: one second). Unfortunately, theStart Time
is subject to the system's timezone whileStart Time (System)
is not. Therefore, its inference is off by the recording's amount of timezone offset.Start Time (Synced)
: This timestamp can be replaced by the earliest known recorded video timestamp in the given recording. In this case, the inference error depends on the startup delay of the cameras. This delay is typically less than a second for Pupil Capture recordings, and 2-5 seconds for Pupil Mobile recordings.
Incorrect Playback
In some cases, deprecated recordings might not be played back correctly in Pupil Player. If this is the case for you, please contact info@pupil-labs.com
for support.
Developers Notes
Changed Requirements
Updated
pyndsi
dependency to v1.0
pip install git+https://github.com/pupil-labs/pyndsi -U
Added
packaging
dependency
pip install packaging
Pupil Player Recording Format 2.0
Variable Framerate Videos
Instead of writing videos with equally distant presentation timestamps, Capture now saves videos with their original timing. This means, that newly recorded videos can be played back correctly in VLC without having to export them first with Pupil Player.
Recording Info Files
Due to slight differences in how Capture, Mobile and Invisible store recording meta info, we introduced a unified recording meta info file for Pupil Player: info.player.json
. When opening a recording made with Pupil Mobile, Pupil Invisible, or older versions of Pupil Capture, we are transforming the recording into the new format (as we did before already) and aggregate all necessary meta infos in info.player.json
without changing the original meta info file. The original meta info files will be renamed in order to uniquely identify the origin of this meta info:
Older versions of Pupil Capture*:
info.csv
->info.old_style.csv
Pupil Mobile:
info.csv
->info.mobile.csv
Pupil Invisible:
info.json
->info.invisible.json
*Note that e.g. when a Pupil Mobile recording has been opened in an older version of Pupil Player before, we also rename to info.old_style.csv
as the original info.csv
will have been adjusted by Player before.
You can read about the exact specification of the new info.player.json
format here.
Pretty Plugin Class Names - #1597
Custom plugin classes don't have to rely on underscores in their names anymore for displaying them nicely in the UI. You can now name your plugin classes in accordance with PEP 8 - Class Names:
class MyAwesomePlugin(Analysis_Plugin_Base):
@classmethod
def parse_pretty_class_name(cls) -> str:
return "Awesome Plugin"
Note that currently all existing pupil classes (e.g. Analysis_Plugin_Base
) keep their names with underscores for backward compatibility.
We Are Hiring Python Developers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for Dev Ops engineers that have experience with kubernetes, docker, and server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
Release Note Updates
17.09.2019 22:40: Missing Microsoft Visual C++ Redistributable
We have received reports from two users that Pupil Capture failed to start on Windows with the following error message:
ImportError: DLL load failed: The specified module could not be found.
In both cases the "Microsoft Visual C++ Redistributable for Visual Studio" was missing on their system. If you should encounter the same error message, please download and run the vc_redist.x64.exe
file from the official Microsoft support page. Afterwards, Pupil should start up as expected.
23.09.2019 17:22: Additional Bug Fixes
The following issues have been fixed in Pupil v1.16-43
:
Remove remaining dependencies on the
info.csv
file - #1652Fix frameless/worldless recordings in Pupil Capture - #1653
30.09.2019 15:30: Offline Calibration Bug Fix
The v1.16-71
release fixes #1656 Player will not create/ allow edits on new calibrations.
02.10.2019 15:35: Deprecated Recordings and Offline Calibration Bug Fix
The following issues have been addressed in Pupil v1.16-80
:
October 2, 2019
We are pleased to announce the release of Pupil v1.16!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Features
Pupil Invisible Support - #1596, #1623
Preparing for the upcoming public release of Pupil Invisible (PI), we are adding support for recordings made with PI in Pupil Player.
You will be able to download the raw recording from Pupil Cloud or copy them directly from the Companion device. Drag and drop the recordings onto Pupil Player as usual.
Most features of Player will support recordings made with PI. Currently not supported are: The iMotions exporter and blink detector.
Surface Tracker
Removed Robust Detection from SurfaceTracker - #1600
The robust detection mode was a remnant of the legacy surface marker system. The new Apriltags markers are by default more robust than even the old robust detection mode. This option did not have an effect anymore, so it was removed.
Bugfixes
Disabled Hardware Timestamps on Windows Operating System - #1621
In the past we have received multiple reports of timestamps being out of sync on Windows. After some testing we discovered that the timestamps from the hardware were not always reliable on Windows. The exact cause of this is still unknown but we have disabled the use of hardware timestamps on Windows for now. We are now using software timestamps with an appropriate offset-mapping.
Improved Eye Overlay Visualizations - #1609
Previously, the Eye Overlay plugin would always display the closest eye frame to the current playback position. Therefore, in cases where the eye camera disconnected during a recording, the eye video was visualized incorrectly. Instead, now, we show a gray frame to indicate missing data.
Smaller Fixes
Fixed minor performance issues in Apriltag surface marker detector
Fixed broken SurfaceTracker heatmap rendering when no gaze data is available - #1584
Fixed SurfaceTracker timeline drawing performance issues - #1595
Fixed issues with EyeMovementDetector when using inconsistent 3D gaze data - #1587
Fixed UI/UX issues when editing surfaces in SurfaceTracker - #1591
Fixed flickering surface markers - #1600
Updated the app icons for the Windows bundles to the latest design
Fixed incorrect handling of missing projection matrices for HMD - #1603
Fixed performance-related crashes of the SurfaceTracker on Windows - #1605, #1606
Improved user feedback when opening invalid recordings - #1585
Fixed crashing offline detection in recordings with timing irregularities - #1615
Improved user feedback when trying to load an unknown plugin - #1613
Fixed a crash when trying to use Pupil Invisible with Capture - #1620
Fixed a crash when setting color values for CircleViz to 1.0 - #1624
Improved user feedback when recording from Pupil Mobile without TimeSync - #1629
Deprecated Recordings
Previously released versions of Pupil Player v1.16 do not support recordings created with
Pupil Capture
v1.2
or earlier, andPupil Mobile
r0.22.x
or earlier.
This was due to the fact that these recordings are missing meta information that is required for the upgrade to our new Pupil Player Recording Format 2.0
. For details see "Missing Meta Information" below.
With this PR, instead of aborting with a "Recording too old" warning, Pupil Player will attempt to infer the missing values from other data in the recording. This inference is not perfect and results in imprecise values and might cause issues when converting recorded Pupil time to wall clock time. For details see "Missing Meta Information" below.
Upgrading will only work in Player v1.16
Since this inference is limited, we will remove the upgrade functionality for deprecated recordings in v1.17. This means, that if you want to open a deprecated recording in v1.17 or later, you will have to convert them to the new format using v1.16 first.
Missing Meta Information
Specifically the following keys in the info.csv
file are missing:
Start Time (System)
: Recording start time in Unix epochStart Time (Synced)
: Recording start time in Pupil time
Pupil Player assumes that these timestamps were measured at the same time. This allows for the after-the-effect synchronization of Pupil data with externally recorded data that uses the Unix epoch.
Their value can be inferred from the existing recording, given a roughly known imprecision:
Start Time (System)
: The system start time can be inferred using the existingStart Date
andStart Time
fields (precision: one second). Unfortunately, theStart Time
is subject to the system's timezone whileStart Time (System)
is not. Therefore, its inference is off by the recording's amount of timezone offset.Start Time (Synced)
: This timestamp can be replaced by the earliest known recorded video timestamp in the given recording. In this case, the inference error depends on the startup delay of the cameras. This delay is typically less than a second for Pupil Capture recordings, and 2-5 seconds for Pupil Mobile recordings.
Incorrect Playback
In some cases, deprecated recordings might not be played back correctly in Pupil Player. If this is the case for you, please contact info@pupil-labs.com
for support.
Developers Notes
Changed Requirements
Updated
pyndsi
dependency to v1.0
pip install git+https://github.com/pupil-labs/pyndsi -U
Added
packaging
dependency
pip install packaging
Pupil Player Recording Format 2.0
Variable Framerate Videos
Instead of writing videos with equally distant presentation timestamps, Capture now saves videos with their original timing. This means, that newly recorded videos can be played back correctly in VLC without having to export them first with Pupil Player.
Recording Info Files
Due to slight differences in how Capture, Mobile and Invisible store recording meta info, we introduced a unified recording meta info file for Pupil Player: info.player.json
. When opening a recording made with Pupil Mobile, Pupil Invisible, or older versions of Pupil Capture, we are transforming the recording into the new format (as we did before already) and aggregate all necessary meta infos in info.player.json
without changing the original meta info file. The original meta info files will be renamed in order to uniquely identify the origin of this meta info:
Older versions of Pupil Capture*:
info.csv
->info.old_style.csv
Pupil Mobile:
info.csv
->info.mobile.csv
Pupil Invisible:
info.json
->info.invisible.json
*Note that e.g. when a Pupil Mobile recording has been opened in an older version of Pupil Player before, we also rename to info.old_style.csv
as the original info.csv
will have been adjusted by Player before.
You can read about the exact specification of the new info.player.json
format here.
Pretty Plugin Class Names - #1597
Custom plugin classes don't have to rely on underscores in their names anymore for displaying them nicely in the UI. You can now name your plugin classes in accordance with PEP 8 - Class Names:
class MyAwesomePlugin(Analysis_Plugin_Base):
@classmethod
def parse_pretty_class_name(cls) -> str:
return "Awesome Plugin"
Note that currently all existing pupil classes (e.g. Analysis_Plugin_Base
) keep their names with underscores for backward compatibility.
We Are Hiring Python Developers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for Dev Ops engineers that have experience with kubernetes, docker, and server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
Release Note Updates
17.09.2019 22:40: Missing Microsoft Visual C++ Redistributable
We have received reports from two users that Pupil Capture failed to start on Windows with the following error message:
ImportError: DLL load failed: The specified module could not be found.
In both cases the "Microsoft Visual C++ Redistributable for Visual Studio" was missing on their system. If you should encounter the same error message, please download and run the vc_redist.x64.exe
file from the official Microsoft support page. Afterwards, Pupil should start up as expected.
23.09.2019 17:22: Additional Bug Fixes
The following issues have been fixed in Pupil v1.16-43
:
Remove remaining dependencies on the
info.csv
file - #1652Fix frameless/worldless recordings in Pupil Capture - #1653
30.09.2019 15:30: Offline Calibration Bug Fix
The v1.16-71
release fixes #1656 Player will not create/ allow edits on new calibrations.
02.10.2019 15:35: Deprecated Recordings and Offline Calibration Bug Fix
The following issues have been addressed in Pupil v1.16-80
:
September 5, 2019
We are pleased to announce the release of Pupil v1.15!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Features
Apriltags support for Surface Tracking
Starting with v1.15
, the Surface Tracker will detect apriltags instead of "square markers".
We highly recommend to replace "square markers" with apriltags in your setup since their improved detection performance translates to gaze being mapped to surfaces with higher accuracy and surfaces being detected more reliably, especially during movements.
You can still use the "square markers" by setting the Marker Detector Mode
to Legacy square markers
. This is especially useful, if you want to process recordings that were made previous to this release.
Apriltags version 3 and support for Windows
We started to maintain our own fork of @duckietown
's apriltags Python bindings for @AprilRobotics
' AprilTag 3 library. Forking the projects was mostly necessary in order to support them on Windows.
This change allows us to finally enable the Head Pose Tracker feature (initially released in v1.12
) on Windows!
Freeze the current 3d eye model
You now have the option to freeze the current eye model. Doing so will prevent any changes to the current eye model and discard any alternative models that might have built up in the background.
Warning: Freezing the eye model will disable its ability to compensate for any kind of slippage.
Bugfixes
Drop invalid frame after disconnect - #1573
Revert change that caused eye process to crash when minimizing
Developers notes
Changed requirements
Please install apriltags via
pip install git+https://github.com/pupil-labs/apriltags
API changes
Improved fixations_on_surface
export
A fixation is based on gaze data that full-fills the maximum-dispersion-minimum-duration-criterion. We use it to infer e.g. fixation's position. When mapping a fixation to a surface, we use the surface's homography to calculate the fixation's position in surface coordinates. The surface's homography is calculated based on the detected surface markers for a given world frame. In most cases, the fixation appears during multiple world frames, i.e. there are multiple surface homographies which could be used to map the fixation to the surface. Until now, we only exported fixations for the first frame during which the fixation appeared. This is not a problem, as long the surface does not move in relation to the world camera during the fixation. Since this is an assumption, that does not always hold true, we will export all fixation mappings starting with v1.15. This means, that if a fixation spans multiple world frames during which a surface was detected, we will export the fixation and its position for each of these world frames.
Therefore, we added two new columns, world_timestamp
and world_index
, to identify the world frame which was used to map the fixation.
Eye model freezing
You can un/freeze the 3d eye models via a notification:
{
"subject": "pupil_detector.set_property.3d",
"name": "model_is_frozen",
"value": True, # set to False to unfreeze
# remove this line if you want
# to freeze the model both eyes
"target": "eye0",
}
See this example on how to use the Pupil Detector Network API.
Change pupil detection ROI by notification - #1576
You can change the pupil detector's region of interest via a notification:
{
"subject": "pupil_detector.set_property",
"target": "eye0",
"name": "roi",
# value: (minX, maxX, minY, maxY)
"value": [0, 100, 0, 200],
}
HMD-Eyes Streaming - #1501
This new video backend allows to stream a virtual Unity camera to Pupil Capture.
HMD-Eyes Streaming is an experimental feature for the hmd-eyes project and can only be activated via a network notification.
More news on this topic will follow soon over at the hmd-eyes
project!
We are hiring Python developers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for Dev Ops engineers that have experience with kubernetes, docker, and server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
Release Updated
Update: Versions v1.15-65
and higher include multiple fixes for surface tracking. Please update Pupil if you have been using versions v1.15-4
or v1.15-5
.
Update 2019-08-27 10:07: Please find the macOS release attached below.Update 2019-09-05 12:05: We updated the Windows bundle to include fixes for the HMD Streaming backend.
September 5, 2019
We are pleased to announce the release of Pupil v1.15!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Features
Apriltags support for Surface Tracking
Starting with v1.15
, the Surface Tracker will detect apriltags instead of "square markers".
We highly recommend to replace "square markers" with apriltags in your setup since their improved detection performance translates to gaze being mapped to surfaces with higher accuracy and surfaces being detected more reliably, especially during movements.
You can still use the "square markers" by setting the Marker Detector Mode
to Legacy square markers
. This is especially useful, if you want to process recordings that were made previous to this release.
Apriltags version 3 and support for Windows
We started to maintain our own fork of @duckietown
's apriltags Python bindings for @AprilRobotics
' AprilTag 3 library. Forking the projects was mostly necessary in order to support them on Windows.
This change allows us to finally enable the Head Pose Tracker feature (initially released in v1.12
) on Windows!
Freeze the current 3d eye model
You now have the option to freeze the current eye model. Doing so will prevent any changes to the current eye model and discard any alternative models that might have built up in the background.
Warning: Freezing the eye model will disable its ability to compensate for any kind of slippage.
Bugfixes
Drop invalid frame after disconnect - #1573
Revert change that caused eye process to crash when minimizing
Developers notes
Changed requirements
Please install apriltags via
pip install git+https://github.com/pupil-labs/apriltags
API changes
Improved fixations_on_surface
export
A fixation is based on gaze data that full-fills the maximum-dispersion-minimum-duration-criterion. We use it to infer e.g. fixation's position. When mapping a fixation to a surface, we use the surface's homography to calculate the fixation's position in surface coordinates. The surface's homography is calculated based on the detected surface markers for a given world frame. In most cases, the fixation appears during multiple world frames, i.e. there are multiple surface homographies which could be used to map the fixation to the surface. Until now, we only exported fixations for the first frame during which the fixation appeared. This is not a problem, as long the surface does not move in relation to the world camera during the fixation. Since this is an assumption, that does not always hold true, we will export all fixation mappings starting with v1.15. This means, that if a fixation spans multiple world frames during which a surface was detected, we will export the fixation and its position for each of these world frames.
Therefore, we added two new columns, world_timestamp
and world_index
, to identify the world frame which was used to map the fixation.
Eye model freezing
You can un/freeze the 3d eye models via a notification:
{
"subject": "pupil_detector.set_property.3d",
"name": "model_is_frozen",
"value": True, # set to False to unfreeze
# remove this line if you want
# to freeze the model both eyes
"target": "eye0",
}
See this example on how to use the Pupil Detector Network API.
Change pupil detection ROI by notification - #1576
You can change the pupil detector's region of interest via a notification:
{
"subject": "pupil_detector.set_property",
"target": "eye0",
"name": "roi",
# value: (minX, maxX, minY, maxY)
"value": [0, 100, 0, 200],
}
HMD-Eyes Streaming - #1501
This new video backend allows to stream a virtual Unity camera to Pupil Capture.
HMD-Eyes Streaming is an experimental feature for the hmd-eyes project and can only be activated via a network notification.
More news on this topic will follow soon over at the hmd-eyes
project!
We are hiring Python developers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for Dev Ops engineers that have experience with kubernetes, docker, and server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
Release Updated
Update: Versions v1.15-65
and higher include multiple fixes for surface tracking. Please update Pupil if you have been using versions v1.15-4
or v1.15-5
.
Update 2019-08-27 10:07: Please find the macOS release attached below.Update 2019-09-05 12:05: We updated the Windows bundle to include fixes for the HMD Streaming backend.
August 5, 2019
We are pleased to announce the release of Pupil v1.14! This update primarily contains bugfixes and optimizations (no new features).
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Bugfixes
Surface Tracker stability improvements - #1547, #1555
Includes re-introduction of the
fixations_on_surface_<name>.csv
export
Head Pose Tracker stability improvements - #1561, #1562, #1569
Calculate eye movement segment frame indices - #1539
Work around negative time-jump issue on Windows - #1551
Correctly disable Stripe Detector for Pupil Cam3 - #1554
GLFW: Raise with detailed error message - #1559
Developers notes
Changed Requirements
We removed our dependency on boost !
We are hiring Python developers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for Dev Ops engineers that have experience with kubernetes, docker, and server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
August 5, 2019
We are pleased to announce the release of Pupil v1.14! This update primarily contains bugfixes and optimizations (no new features).
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Bugfixes
Surface Tracker stability improvements - #1547, #1555
Includes re-introduction of the
fixations_on_surface_<name>.csv
export
Head Pose Tracker stability improvements - #1561, #1562, #1569
Calculate eye movement segment frame indices - #1539
Work around negative time-jump issue on Windows - #1551
Correctly disable Stripe Detector for Pupil Cam3 - #1554
GLFW: Raise with detailed error message - #1559
Developers notes
Changed Requirements
We removed our dependency on boost !
We are hiring Python developers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for Dev Ops engineers that have experience with kubernetes, docker, and server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
June 24, 2019
We are pleased to announce the release of Pupil v1.13!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Features
Eye Movement Detector - #1512, #1516, #1519
Our new eye movement detector is based on Naive Segmented Linear Regression and classifies fixation, saccade, post-saccadic oscillation, and smooth pursuit movements.
Event identification is based on segmentation that simultaneously de-noises the signal and determines event boundaries. The full gaze position time-series is segmented into an approximately optimal piecewise linear function in O(n) time. Gaze feature parameters for classification into fixations, saccades, smooth pursuits and post-saccadic oscillations are derived from human labeling in a data-driven manner.[1]
More details about this approach can be found here.
The open source implementation can be found here.
See this Pupil Helper script to filter eye movements in real time.
Surface Tracker Revamp - #1268, #1515
We have rewritten the surface tracker and included many improvements, e.g.:
Compensate camera distortion correctly when mapping gaze onto the surface
Do not block when recomputing surface location cache in post-hoc surface tracker in Pupil Player
Improve heatmap generation
General UI improvements
Improved timeline visualizations
Option to freeze the image in Capture for easier surface setup and editing
Use more than a single video frame to define a surface in post-hoc surface tracker in Pupil Player
See the pull request description for a full list.
Please be aware that the names of the exported csv files has changed slightly. See the documentation for the new names.
Caveat: You might need to readjust your surface definitions after upgrading pre-v1.13 recordings. This is due to the internal surface definition having changed and not being fully compatible with previously saved surface definitions.
Pupil Invisible Companion Compatibility Improvements
We are gearing up for a wider release of our new eye tracking glasses Pupil Invisible, and are adding some features to Pupil desktop to support Pupil Invisible. Pupil Invisible will ship with its own Companion Android app. This app will enable one to stream video over WiFi for monitoring purposes.
Bugfixes
Improved ui responsiveness - pyglui#105, #1527
Fix Pupil Mobile intrinsics lookup - #1510
Fix audio playback in cases where the audio is shorter than the world video - #1514
Pupil data fields
theta
andphi
changed to be consistently floats - #1526
Developers notes
Changed Requirements
Eye movement detector:
nslr
andnslr-hmm
pip install git+https://github.com/pupil-labs/nslr
pip install git+https://github.com/pupil-labs/nslr-hmm
ndsi v0.5
pip install git+https://github.com/pupil-labs/pyndsi -U
pyglui v1.24
pip install git+https://github.com/pupil-labs/pyglui -U
API changes
Eye movement detector: See the format documentation for information.
New
world_process.adapt_window_size
notification. Request world process to fit window size to most recent world video frame - #1524
We are hiring Python developers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
June 24, 2019
We are pleased to announce the release of Pupil v1.13!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Features
Eye Movement Detector - #1512, #1516, #1519
Our new eye movement detector is based on Naive Segmented Linear Regression and classifies fixation, saccade, post-saccadic oscillation, and smooth pursuit movements.
Event identification is based on segmentation that simultaneously de-noises the signal and determines event boundaries. The full gaze position time-series is segmented into an approximately optimal piecewise linear function in O(n) time. Gaze feature parameters for classification into fixations, saccades, smooth pursuits and post-saccadic oscillations are derived from human labeling in a data-driven manner.[1]
More details about this approach can be found here.
The open source implementation can be found here.
See this Pupil Helper script to filter eye movements in real time.
Surface Tracker Revamp - #1268, #1515
We have rewritten the surface tracker and included many improvements, e.g.:
Compensate camera distortion correctly when mapping gaze onto the surface
Do not block when recomputing surface location cache in post-hoc surface tracker in Pupil Player
Improve heatmap generation
General UI improvements
Improved timeline visualizations
Option to freeze the image in Capture for easier surface setup and editing
Use more than a single video frame to define a surface in post-hoc surface tracker in Pupil Player
See the pull request description for a full list.
Please be aware that the names of the exported csv files has changed slightly. See the documentation for the new names.
Caveat: You might need to readjust your surface definitions after upgrading pre-v1.13 recordings. This is due to the internal surface definition having changed and not being fully compatible with previously saved surface definitions.
Pupil Invisible Companion Compatibility Improvements
We are gearing up for a wider release of our new eye tracking glasses Pupil Invisible, and are adding some features to Pupil desktop to support Pupil Invisible. Pupil Invisible will ship with its own Companion Android app. This app will enable one to stream video over WiFi for monitoring purposes.
Bugfixes
Improved ui responsiveness - pyglui#105, #1527
Fix Pupil Mobile intrinsics lookup - #1510
Fix audio playback in cases where the audio is shorter than the world video - #1514
Pupil data fields
theta
andphi
changed to be consistently floats - #1526
Developers notes
Changed Requirements
Eye movement detector:
nslr
andnslr-hmm
pip install git+https://github.com/pupil-labs/nslr
pip install git+https://github.com/pupil-labs/nslr-hmm
ndsi v0.5
pip install git+https://github.com/pupil-labs/pyndsi -U
pyglui v1.24
pip install git+https://github.com/pupil-labs/pyglui -U
API changes
Eye movement detector: See the format documentation for information.
New
world_process.adapt_window_size
notification. Request world process to fit window size to most recent world video frame - #1524
We are hiring Python developers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
May 6, 2019
We are pleased to announce the release of Pupil v1.12!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Features
Head Pose Tracking - #1484
The plugin builds a 3d model by detecting markers (apriltag) in your environment. After building the model, you can track the Pupil wearer's head pose within this environment.
See our video tutorial on how to setup and use this feature. Enable captions on the youtube video for the tutorial commentary. Also, checkout the documentation (Detailed Data Format section, last paragraph) on how the exported data format looks like.
Caveat: Head Pose Tracking is not yet available on Windows since the used apriltag implementation is incomaptible with Windows.
Auto Camera Selection - #1470
We introduced a new button in the Backend Manager: Start with default devices
After switching to a different video backend, you can click the Start with default devices button
. This will automatically select the correct sensor and start capturing for corresponding world and eye windows.
This feature is particularly useful if you need to the change the capture often.
Video Overlay - #1489
The Video Overlay
plugin enables you to overlay any video with synchronized timestamps.
You can easily record videos with timestamps by using a second instance of Pupil Capture or Pupil Mobile.
Example use case: Use another camera to record a "third person" view of a participant in your eye tracking experiment and overlay this video with the first person world video + gaze visualization.
The Vis Eye Video Overlay
has been renamed to Eye Overlay
.
Improvements
Full support for the new 200Hz HTC Vive Binocular Add-on - #1497
Consistent export ranges - #1480
Opencv 4 compatibility - #1495
Eye video export - Fixed a bug where the data from the incorrect eye was rendered - #1488
Fix offline gaze mapper crash when using a recorded calibration - #1499
Developer notes
Changed Requirements
apriltag - The Head Pose Tracker uses the apriltag detector. Specifically, Pupil uses
@swatbotics
's C library via a modified version of their Python bindings.
API changes
Plugins can consume UI events - #1481
May 6, 2019
We are pleased to announce the release of Pupil v1.12!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Features
Head Pose Tracking - #1484
The plugin builds a 3d model by detecting markers (apriltag) in your environment. After building the model, you can track the Pupil wearer's head pose within this environment.
See our video tutorial on how to setup and use this feature. Enable captions on the youtube video for the tutorial commentary. Also, checkout the documentation (Detailed Data Format section, last paragraph) on how the exported data format looks like.
Caveat: Head Pose Tracking is not yet available on Windows since the used apriltag implementation is incomaptible with Windows.
Auto Camera Selection - #1470
We introduced a new button in the Backend Manager: Start with default devices
After switching to a different video backend, you can click the Start with default devices button
. This will automatically select the correct sensor and start capturing for corresponding world and eye windows.
This feature is particularly useful if you need to the change the capture often.
Video Overlay - #1489
The Video Overlay
plugin enables you to overlay any video with synchronized timestamps.
You can easily record videos with timestamps by using a second instance of Pupil Capture or Pupil Mobile.
Example use case: Use another camera to record a "third person" view of a participant in your eye tracking experiment and overlay this video with the first person world video + gaze visualization.
The Vis Eye Video Overlay
has been renamed to Eye Overlay
.
Improvements
Full support for the new 200Hz HTC Vive Binocular Add-on - #1497
Consistent export ranges - #1480
Opencv 4 compatibility - #1495
Eye video export - Fixed a bug where the data from the incorrect eye was rendered - #1488
Fix offline gaze mapper crash when using a recorded calibration - #1499
Developer notes
Changed Requirements
apriltag - The Head Pose Tracker uses the apriltag detector. Specifically, Pupil uses
@swatbotics
's C library via a modified version of their Python bindings.
API changes
Plugins can consume UI events - #1481
March 5, 2019
We are pleased to announce the release of Pupil v1.11!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Features
Offline Calibration Refactor
Author: @ckbaumannPull Requests: #1389, #1400, #1407, #1415, #1434
Calibration export and import: Export a calibration from one recording and use it to map gaze in an other recording. See #1003 for a possible use case.
Accuracy and precision: Evaluate accuracy and precision for separate gaze mappings.
Caching of generated gaze data: Offline calibrated gaze is session-persistent and does not need to be recalculated after restarting Pupil Player.
Streamlined usability: Detect reference locations, calibrate, map gaze, and calculate accuracy with the click of a single button.
Split Recording Support
Author: @WindsooonPull Requests: #1417, #1452, #1456
Starting with version v0.37
, Pupil Mobile creates recordings with multiple video files. This ensures that we do not reach/exceed the file size limit on SD cards and additionally ensures that we can gracefully handle sensor disconnects of the headset without compromising the recording session.
Pupil Player will behave in the same way as it did previously, with the difference that it will fill any gaps within the recording, e.g. during a a disconnect, with gray frames. World-less recordings as well as corrupted video files are also being handled as gaps.
Developers notes
Added
debug
-level logs for Pupil Remote requests and responses - #1436
Changed Requirements
numpy >= 1.13
API changes
Raw Data Export: Changed timestamp column names - #1444
pupil_positions.csv
:world_timestamp
->pupil_timestamp
gaze_positions.csv
:world_timestamp
->gaze_timestamp
We are hiring Python developers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for full stack Python developers that have experience with kubernetes, docker, and async server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
March 5, 2019
We are pleased to announce the release of Pupil v1.11!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Features
Offline Calibration Refactor
Author: @ckbaumannPull Requests: #1389, #1400, #1407, #1415, #1434
Calibration export and import: Export a calibration from one recording and use it to map gaze in an other recording. See #1003 for a possible use case.
Accuracy and precision: Evaluate accuracy and precision for separate gaze mappings.
Caching of generated gaze data: Offline calibrated gaze is session-persistent and does not need to be recalculated after restarting Pupil Player.
Streamlined usability: Detect reference locations, calibrate, map gaze, and calculate accuracy with the click of a single button.
Split Recording Support
Author: @WindsooonPull Requests: #1417, #1452, #1456
Starting with version v0.37
, Pupil Mobile creates recordings with multiple video files. This ensures that we do not reach/exceed the file size limit on SD cards and additionally ensures that we can gracefully handle sensor disconnects of the headset without compromising the recording session.
Pupil Player will behave in the same way as it did previously, with the difference that it will fill any gaps within the recording, e.g. during a a disconnect, with gray frames. World-less recordings as well as corrupted video files are also being handled as gaps.
Developers notes
Added
debug
-level logs for Pupil Remote requests and responses - #1436
Changed Requirements
numpy >= 1.13
API changes
Raw Data Export: Changed timestamp column names - #1444
pupil_positions.csv
:world_timestamp
->pupil_timestamp
gaze_positions.csv
:world_timestamp
->gaze_timestamp
We are hiring Python developers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for full stack Python developers that have experience with kubernetes, docker, and async server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
January 2, 2019
We are pleased to announce the release of Pupil v1.10!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Features
Capture
RealSense D400 Support - #1409
Please be aware, that it is recommended to keep your D400's firmware update-to-date. See the Intel documentation on how to do that.
Intel does not currently provide a Python wrapper for macOS. Until they do, we are not able to support the Pupil Capture D400 backend on macOS.
Player
Export timestamps as csv
on video export
Timestamps will now be exported as a csv when you export a video #1411
Command-line arguments - #1384
Previously, only simple arguments could be passed to Pupil via command-line. This included the debug
and profiled
mode. Motivated by #1315, we decided to start parsing command-line arguments properly.
See the pull request description for details on the available command-line arguments.
Pupil Remote port command-line argument
Using the --port PORT
argument, it is now possible to set the Pupil Remote port for Capture and Service.
Bugfixes
Fake Backend: Fix playback after seeking - #1396
Fixation Detector: Fix export format - #1395
Improved compatibility with macOS 10.14 Mojave - #1381
Developers notes
New dependencies
Optional, required for RealSense D400 backend:
pyrealsense2
API changes
Pupil Detector Network API, see PR for details - #1395
Pupil Remote: Forward IPC/multipart messages - #1385
We are hiring Python developers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for full stack Python developers that have experience with kubernetes, docker, and async server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
January 2, 2019
We are pleased to announce the release of Pupil v1.10!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Features
Capture
RealSense D400 Support - #1409
Please be aware, that it is recommended to keep your D400's firmware update-to-date. See the Intel documentation on how to do that.
Intel does not currently provide a Python wrapper for macOS. Until they do, we are not able to support the Pupil Capture D400 backend on macOS.
Player
Export timestamps as csv
on video export
Timestamps will now be exported as a csv when you export a video #1411
Command-line arguments - #1384
Previously, only simple arguments could be passed to Pupil via command-line. This included the debug
and profiled
mode. Motivated by #1315, we decided to start parsing command-line arguments properly.
See the pull request description for details on the available command-line arguments.
Pupil Remote port command-line argument
Using the --port PORT
argument, it is now possible to set the Pupil Remote port for Capture and Service.
Bugfixes
Fake Backend: Fix playback after seeking - #1396
Fixation Detector: Fix export format - #1395
Improved compatibility with macOS 10.14 Mojave - #1381
Developers notes
New dependencies
Optional, required for RealSense D400 backend:
pyrealsense2
API changes
Pupil Detector Network API, see PR for details - #1395
Pupil Remote: Forward IPC/multipart messages - #1385
We are hiring Python developers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for full stack Python developers that have experience with kubernetes, docker, and async server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
November 7, 2018
We are pleased to announce the release of Pupil v1.9!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Features
Capture
Remote Recorder - #1194
The Remote Recorder
plugin no longer requires Pupil Mobile streams to be active in Capture. Instead, it lists all available devices and allows you to start and stop recordings with the click of a button. It is also possible to change the recording session names for all available devices at the same time.
This feature requires Pupil Mobile version 0.25.1
or higher.
Annotation Plugin
Annotations are no longer special types of notifications. Annotations are now sent via the IPC under the topic annotation
. Recorded annotations are now stored in annotation.pldata
.
Pupil Detectors - #1358
Reduced the default "maximum pupil size" setting in order to decrease false-positive detections.
Time Sync Group Members - #1277
All Time Sync
actors (e.g. Capture or Pupil Mobile) join a network group for discovery and time sync announcements. We added the functionality of listing all present group members in the Time Sync
plugin menu.
Be aware that all Pupil Mobile instances will be listed as pupil-mobile-follower
.
Player
Buffered Playback - #1279
We added buffering to Pupil Player. Instead of decoding each frame on demand, Player decodes multiple frames ahead of time. This improves smooth playback, even at high playback speeds of h264 encoded videos.
Be aware, that this will require a bit more of memory as a trade-off.
Video Exporters
We improved the different video export plugins.
There is a new
Eye Video Exporter
plugin (#1301). This plugin will export eye videos for your recording.Video Export Launcher
was renamed toWorld Video Exporter
to make its task clearer. By default, the exported video is no longer namedworld_viz.mp4
butworld.mp4
(#1295).We implemented a new export management UI that is now used in
World Video Exporter
,Eye Video Exporter
, andiMotitions Exporter
. You can add as many export tasks as needed and they will all be processed in order (#1322).
Raw Data Exporter
@fneitzel added the possibility to individually turn off pupil and gaze exports in the Raw Data Exporter
(#1239).
The exported timestamp
, index
, and id
csv columns have been renamed to more explicit titles (#1352).
iMotions Exporter: Support for world-less recordings - #1308
We added support for world-less recordings to the iMotions Exporter. World-less recordings do not include a scene (world) video and are common for AR/VR setups.
Annotation Plugin
After opening a recording in Player, all annotations (recorded or added in Player) are stored in annotation_player.pldata
instead of the offline data
directory. This does not override annotation.pldata
, so you can reset annotations to the recorded ones by deleting annotation_player.pldata
.
Recordings created by former versions of Pupil will be updated to the new format.
Bugfixes
Surface edit bug on Macs with Retina displays - #1252
Crash when recording during Frame Publisher enabled - #1263
Synchronization Drift between Cameras and Recording Computer - #1266
Incorrect
Frame Publisher
world frame topic - #1276Fixation Detector
bug where binocular 3d gaze data was ignored - #1286Race condition during Player exports - #1304
Incorrect log message handling in background tasks - #1305
Timebase changes break fixation and blink detection in Capture - #1324
Use of deprecated fields in
Blink Detection
- #1283
Developers notes
New dependencies
pyav 0.4.2
cysignal (on macOS and Linux only)
black
format
The entire code base has been changed to the black
format (#1343, #1344, #1346).
API changes
seek_control.trim_indeces_changed
notification - #1329set_min_calibration_confidence
notification - #1361Fixed field consistency for monocularly mapped gaze - #1291
We are hiring Python developers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for full stack Python developers that have experience with kubernetes, docker, and async server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
November 7, 2018
We are pleased to announce the release of Pupil v1.9!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
Features
Capture
Remote Recorder - #1194
The Remote Recorder
plugin no longer requires Pupil Mobile streams to be active in Capture. Instead, it lists all available devices and allows you to start and stop recordings with the click of a button. It is also possible to change the recording session names for all available devices at the same time.
This feature requires Pupil Mobile version 0.25.1
or higher.
Annotation Plugin
Annotations are no longer special types of notifications. Annotations are now sent via the IPC under the topic annotation
. Recorded annotations are now stored in annotation.pldata
.
Pupil Detectors - #1358
Reduced the default "maximum pupil size" setting in order to decrease false-positive detections.
Time Sync Group Members - #1277
All Time Sync
actors (e.g. Capture or Pupil Mobile) join a network group for discovery and time sync announcements. We added the functionality of listing all present group members in the Time Sync
plugin menu.
Be aware that all Pupil Mobile instances will be listed as pupil-mobile-follower
.
Player
Buffered Playback - #1279
We added buffering to Pupil Player. Instead of decoding each frame on demand, Player decodes multiple frames ahead of time. This improves smooth playback, even at high playback speeds of h264 encoded videos.
Be aware, that this will require a bit more of memory as a trade-off.
Video Exporters
We improved the different video export plugins.
There is a new
Eye Video Exporter
plugin (#1301). This plugin will export eye videos for your recording.Video Export Launcher
was renamed toWorld Video Exporter
to make its task clearer. By default, the exported video is no longer namedworld_viz.mp4
butworld.mp4
(#1295).We implemented a new export management UI that is now used in
World Video Exporter
,Eye Video Exporter
, andiMotitions Exporter
. You can add as many export tasks as needed and they will all be processed in order (#1322).
Raw Data Exporter
@fneitzel added the possibility to individually turn off pupil and gaze exports in the Raw Data Exporter
(#1239).
The exported timestamp
, index
, and id
csv columns have been renamed to more explicit titles (#1352).
iMotions Exporter: Support for world-less recordings - #1308
We added support for world-less recordings to the iMotions Exporter. World-less recordings do not include a scene (world) video and are common for AR/VR setups.
Annotation Plugin
After opening a recording in Player, all annotations (recorded or added in Player) are stored in annotation_player.pldata
instead of the offline data
directory. This does not override annotation.pldata
, so you can reset annotations to the recorded ones by deleting annotation_player.pldata
.
Recordings created by former versions of Pupil will be updated to the new format.
Bugfixes
Surface edit bug on Macs with Retina displays - #1252
Crash when recording during Frame Publisher enabled - #1263
Synchronization Drift between Cameras and Recording Computer - #1266
Incorrect
Frame Publisher
world frame topic - #1276Fixation Detector
bug where binocular 3d gaze data was ignored - #1286Race condition during Player exports - #1304
Incorrect log message handling in background tasks - #1305
Timebase changes break fixation and blink detection in Capture - #1324
Use of deprecated fields in
Blink Detection
- #1283
Developers notes
New dependencies
pyav 0.4.2
cysignal (on macOS and Linux only)
black
format
The entire code base has been changed to the black
format (#1343, #1344, #1346).
API changes
seek_control.trim_indeces_changed
notification - #1329set_min_calibration_confidence
notification - #1361Fixed field consistency for monocularly mapped gaze - #1291
We are hiring Python developers!
Hey - you're reading the developer notes, so this is for you! We're looking to hire developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
We are also looking for full stack Python developers that have experience with kubernetes, docker, and async server-side Python.
Send an email to jobs@pupil-labs.com with a CV to start a discussion. We look forward to hearing from you.
August 2, 2018
We are pleased to announce the release of Pupil v1.8!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
We have been working hard to significantly reduce the memory usage of Pupil Capture and Player. See incremental data serialization and deferred deserialization for details.
Features
Capture
Auto-exposure mode -- #1210
We have added an auto-exposure mode for the 200Hz Pupil cameras. You can enable it in the UVC Source
menu of the eye windows.
Incremental data serialization -- #1141
Prior to this release, data was cached in memory during recordings and written to disk after the recording had finished. This resulted in large memory consumption during recording.
Starting in v1.8
, Pupil Capture will store data directory to the disk as it becomes available during the recording. This reduces the memory footprint and improves reliability of Pupil Capture. See the New Recording Format section on our documentation.
Automatic recording stop on low disk space
The recorder will show a warning if less than 5GB of disk space is available to the user. Recordings will be stopped gracefully as soon as less than 1GB of disk space is available.
Fingertip Calibration -- #1218
We introduced a proof-of-concept fingertip calibration with Pupil Capture v1.6
. It was based on traditional computer vision approaches and the fingertip detection was not very stable.
Now, we are releasing a revised version that uses convolutional neural networks (CNN) for the hand and fingertip detection. For details checkout our documentation.
Note - The current bundle support CPU inference only. If you install from source and have an NVIDIA GPU with CUDA 9.0 drivers installed, you can install pytorch and our fingertip detector will use your GPU!
Player
Deferred Deserialization
Before v1.8
, opening a recording in Pupil Player would read the entire pupil_data
file and deserialize the data into Python objects. This enabled fast processing of the data but also used excessive amounts of memory. This led to software instabilities for users who were trying to process recordings with long durations.
Starting with v1.8
, Pupil Player only deserializes data if required. This reduces memory consumption dramatically and improves software stability.
Please be aware that the initial upgrade of recordings to the new format can take a bit of time. Please be patient while the recording is converted.
Temporally disabled features
We had to disable the following features due to changes on how we handle data within Pupil Player:
Vis Scan Path
Manual gaze correction for
Gaze From Recording
We are working on a solution and will hopefully by able ro re-enable these with in the next release.
Bugfixes
Fixed a bug were Player crashed if
info.csv
included non-ASCII chracters -- #1224Correctly reset the last known reference location after stopping the manual marker calibration in Capture -- #1206
Developers notes
New dependencies
We have added PyTorch to our dependencies. If you want to make use of GPU acceleration you will have to run Pupil Capture from source and install the GPU version of PyTorch. We will work on bundling GPU supported versions in the future.
New recording format
We had to make changes to our recording format in order to make the incremental serialization and deferred deserialization features possible. Please, see our documentation for more details on the New Recording Format.
API changes
zmq_tools.Msg_Streamer.send()
has been reworked. The previous required two arguments: topic
, payload
. The new version only accepts the payload
argument and expects the payload to have a topic
field instead.
Real-time fixation format changes -- #1231
Online fixations are published in a high frequency. Each fixation has a base_data
field that includes gaze data related to this fixation. In turn, gaze data has a base_data
field on its own including pupil data. As a result, recordings grew unreasonably fast in size if the Online Fixation Detector
was enabled. E.g. for an eleven minute long recording, the pupil_data
file grew to 1.4GB of which 1.1GB were fixations alone.
As a consequence, we are replacing each gaze datum in the base_data
field of online fixations with (topic, timestamp)
tuples. These uniquely identify their corresponding gaze datum within the gaze data stream.
We are hiring developers!
Hey - you're reading the developer notes, so this is for you!
Pupil Core Contributors - We're hiring developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
Full Stack Developers - We are also looking for full stack developers that have experience with one ore more of the following tools/platforms: kubernetes, docker, vue.js, and flask.
Send an email to jobs@pupil-labs.com
with a CV to start a discussion. We look forward to hearing from you.
August 2, 2018
We are pleased to announce the release of Pupil v1.8!
Download the latest bundle and let us know what you think via the #pupil channel on Discord 😄
We have been working hard to significantly reduce the memory usage of Pupil Capture and Player. See incremental data serialization and deferred deserialization for details.
Features
Capture
Auto-exposure mode -- #1210
We have added an auto-exposure mode for the 200Hz Pupil cameras. You can enable it in the UVC Source
menu of the eye windows.
Incremental data serialization -- #1141
Prior to this release, data was cached in memory during recordings and written to disk after the recording had finished. This resulted in large memory consumption during recording.
Starting in v1.8
, Pupil Capture will store data directory to the disk as it becomes available during the recording. This reduces the memory footprint and improves reliability of Pupil Capture. See the New Recording Format section on our documentation.
Automatic recording stop on low disk space
The recorder will show a warning if less than 5GB of disk space is available to the user. Recordings will be stopped gracefully as soon as less than 1GB of disk space is available.
Fingertip Calibration -- #1218
We introduced a proof-of-concept fingertip calibration with Pupil Capture v1.6
. It was based on traditional computer vision approaches and the fingertip detection was not very stable.
Now, we are releasing a revised version that uses convolutional neural networks (CNN) for the hand and fingertip detection. For details checkout our documentation.
Note - The current bundle support CPU inference only. If you install from source and have an NVIDIA GPU with CUDA 9.0 drivers installed, you can install pytorch and our fingertip detector will use your GPU!
Player
Deferred Deserialization
Before v1.8
, opening a recording in Pupil Player would read the entire pupil_data
file and deserialize the data into Python objects. This enabled fast processing of the data but also used excessive amounts of memory. This led to software instabilities for users who were trying to process recordings with long durations.
Starting with v1.8
, Pupil Player only deserializes data if required. This reduces memory consumption dramatically and improves software stability.
Please be aware that the initial upgrade of recordings to the new format can take a bit of time. Please be patient while the recording is converted.
Temporally disabled features
We had to disable the following features due to changes on how we handle data within Pupil Player:
Vis Scan Path
Manual gaze correction for
Gaze From Recording
We are working on a solution and will hopefully by able ro re-enable these with in the next release.
Bugfixes
Fixed a bug were Player crashed if
info.csv
included non-ASCII chracters -- #1224Correctly reset the last known reference location after stopping the manual marker calibration in Capture -- #1206
Developers notes
New dependencies
We have added PyTorch to our dependencies. If you want to make use of GPU acceleration you will have to run Pupil Capture from source and install the GPU version of PyTorch. We will work on bundling GPU supported versions in the future.
New recording format
We had to make changes to our recording format in order to make the incremental serialization and deferred deserialization features possible. Please, see our documentation for more details on the New Recording Format.
API changes
zmq_tools.Msg_Streamer.send()
has been reworked. The previous required two arguments: topic
, payload
. The new version only accepts the payload
argument and expects the payload to have a topic
field instead.
Real-time fixation format changes -- #1231
Online fixations are published in a high frequency. Each fixation has a base_data
field that includes gaze data related to this fixation. In turn, gaze data has a base_data
field on its own including pupil data. As a result, recordings grew unreasonably fast in size if the Online Fixation Detector
was enabled. E.g. for an eleven minute long recording, the pupil_data
file grew to 1.4GB of which 1.1GB were fixations alone.
As a consequence, we are replacing each gaze datum in the base_data
field of online fixations with (topic, timestamp)
tuples. These uniquely identify their corresponding gaze datum within the gaze data stream.
We are hiring developers!
Hey - you're reading the developer notes, so this is for you!
Pupil Core Contributors - We're hiring developers to contribute to Pupil source code. If you love Python and enjoy writing code that is a joy to read, get in touch. Experience with the scientific Python stack is a plus, but not required. We have a lot of exciting projects in the pipeline.
Full Stack Developers - We are also looking for full stack developers that have experience with one ore more of the following tools/platforms: kubernetes, docker, vue.js, and flask.
Send an email to jobs@pupil-labs.com
with a CV to start a discussion. We look forward to hearing from you.
May 4, 2018
Overview
We are pleased to announce the release of Pupil v1.7
!
Download the latest bundle and let us know what you think via the #pupil channel 😄
Features
Adjustable audio playback volume - #1131
Audio wave visualization - #1131
Online Surface Tracker: Map and publish fixations - 4bb1aeb
2d and 3d detector improvements - #1168
Capture opens both eye windows by default - e2d9576
Bugfixes
May 4, 2018
Overview
We are pleased to announce the release of Pupil v1.7
!
Download the latest bundle and let us know what you think via the #pupil channel 😄
Features
Adjustable audio playback volume - #1131
Audio wave visualization - #1131
Online Surface Tracker: Map and publish fixations - 4bb1aeb
2d and 3d detector improvements - #1168
Capture opens both eye windows by default - e2d9576
Bugfixes
March 21, 2018
Overview
We are pleased to announce the release of Pupil v1.6
!
Download the latest bundle and let us know what you think via the #pupil channel 😄
Features
Audio Playback in Player -- #1111
Starting with v1.6
, Pupil Player supports playback of recorded audio. Audio playback only works if the playback speed is set to 1x
. Mute functionality and audio wave visualization will be implemented in the next release.
Developer notes
The audio playback implementation is based on portaudio
and its pyaudio
wrapper. Installing these will be required to run from source. See the developer doc changes for details.
Audio Capture for Windows -- #1110
We are very happy to add support for audio capture on Windows!
iMotions Exporter -- #1118
We added a plugin in Pupil Player that will enable you to export Pupil data to iMotions data format. You can import the exported Pupil data into iMotions software.
Experimental Fingertip Calibration -- #1108
This experimental calibration routine allows you to self-calibrate by using your fingertip as a reference point. This method should be used for prototyping and experimentation only. The detection robustness is not yet production grade, but we will put a lot more effort into this going forward. We wanted to release the idea so that you can try it out, suggest improvements, and provide feedback feedback.
Accuracy Visualizer -- #1126
Accuracy Visualizer
is now a default plugin. This plugin will visualize the calibrated area. You can turn the visualization off in the plugin's settings menu.
Bugfixes
HoloLens Relay: Fix gaze topic format change regression -- 6534ec
UVC Source: Use hardware timestamps for Pupil Labs cameras only -- #1123
Automatic detection and reinitialization if 200Hz eye camera exhibits stripes: issue #1116 -- #1024. If you are still experiencing this behavior after updating to
v1.6
, please open an issue and we will follow up with you to debug.
March 21, 2018
Overview
We are pleased to announce the release of Pupil v1.6
!
Download the latest bundle and let us know what you think via the #pupil channel 😄
Features
Audio Playback in Player -- #1111
Starting with v1.6
, Pupil Player supports playback of recorded audio. Audio playback only works if the playback speed is set to 1x
. Mute functionality and audio wave visualization will be implemented in the next release.
Developer notes
The audio playback implementation is based on portaudio
and its pyaudio
wrapper. Installing these will be required to run from source. See the developer doc changes for details.
Audio Capture for Windows -- #1110
We are very happy to add support for audio capture on Windows!
iMotions Exporter -- #1118
We added a plugin in Pupil Player that will enable you to export Pupil data to iMotions data format. You can import the exported Pupil data into iMotions software.
Experimental Fingertip Calibration -- #1108
This experimental calibration routine allows you to self-calibrate by using your fingertip as a reference point. This method should be used for prototyping and experimentation only. The detection robustness is not yet production grade, but we will put a lot more effort into this going forward. We wanted to release the idea so that you can try it out, suggest improvements, and provide feedback feedback.
Accuracy Visualizer -- #1126
Accuracy Visualizer
is now a default plugin. This plugin will visualize the calibrated area. You can turn the visualization off in the plugin's settings menu.
Bugfixes
HoloLens Relay: Fix gaze topic format change regression -- 6534ec
UVC Source: Use hardware timestamps for Pupil Labs cameras only -- #1123
Automatic detection and reinitialization if 200Hz eye camera exhibits stripes: issue #1116 -- #1024. If you are still experiencing this behavior after updating to
v1.6
, please open an issue and we will follow up with you to debug.
March 2, 2018
Overview
We are pleased to announce the release of Pupil v1.5
!
Download the latest application bundle and let us know what you think via the #pupil channel 😄
Features
Capture
Indicate annotation firing with a short log message. This provides realtime feedback for users that fire annotations via
Pupil Remote
-- #1080Screen-based calibration plugins: Session-persistent monitor selection -- #1091
Capture allows creation of recordings without an active world video source, requires
pyndsi>=0.4
-- #1101.Enable eye video recordings by default - #1101
Enable/disable eye video recordings using the
recording.should_start
notification -- #1101
Player
Trim section and export improvements -- #1093
Remove trim section indicators from video and raw data exporters
Add trim section indicators to general settings, showing both:
relative recording time
frame indices
Add
export_info.csv
file to export folder. This csv file contains the following information:Player version
Data format version
Export start date and time
Frame index trim range
Relative time trim range
Absolute time trim range (
world
timestamp range)
We no longer use trim range with frame indices as a naming convention
We now use incrementally increasing folder names with format
%d%d%d
, e.g.:<recording>/exports/000/
<recording>/exports/001/
<recording>/exports/002/
Confidence threshold consistency -- #1100
Previously, all binocular mappers dismissed pupil data with confidence lower than
0.6
to prevent bad binocular mapping. Now, low confidence pupil data is mapped monocularly.Introduction of a user settable global minimum calibration confidence threshold. This threshold sets the lower bounds of the pupil data confidence that is used for calibration. This value was previously fixed to
0.8
in Pupil Capture and to0.0
in Pupil Player -- effectively using low confidence pupil data for calibration. This threshold can be set in all calibration plugin menus.
These changes were introduced based on #1052 and #1047.
Minimum data confidence: Lower bounds gaze data by confidence that is used to generate visualizations and high level data, e.g. fixations. This does not effect the raw data export. Following data points are effected by it:
Fixations
Offline surface heatmaps
Circle, cross, light points and polyline visualizations
Bugfixes
Fix
Fake_Source
compatibility with Frame Publisher -- #1085Service does not crash when starting
Frame_Publisher
via a notification -- #1099Frame_Publisher
only warns once on encountering an incompatible format instead of warning each time -- #1099Fix an issue where
Fake_Source
would return frames at 2 fps in Capture if it was restored from the session settings. -- #1085Offline Fixation Detector: Correct upper limit of fixation lenghts to 4000ms -- #1084
Avoid
ZeroDivisionError
when calculating hdpi factors on Windows -- #1090Avoid positioning windows by default behind the Windows task bar -- #1103
Fix bug where export folders were written into the recording folder itself instead of into the
exports
subfolderUVC control settings are correctly restored from session settings and on reconnect -- #1099 and #1101
Avoid crash on recordings with less than 10 world frames -- #1101
Player always checks for recordings without world videos instead of only checking on upgrading the recording format - #1101
Fixed a bug that crashed Player when removing a surface - #1106
Fixed a bug that crashed Capture on restoring a minimized window on MS Windows 10 when log display was loaded - #1097
March 2, 2018
Overview
We are pleased to announce the release of Pupil v1.5
!
Download the latest application bundle and let us know what you think via the #pupil channel 😄
Features
Capture
Indicate annotation firing with a short log message. This provides realtime feedback for users that fire annotations via
Pupil Remote
-- #1080Screen-based calibration plugins: Session-persistent monitor selection -- #1091
Capture allows creation of recordings without an active world video source, requires
pyndsi>=0.4
-- #1101.Enable eye video recordings by default - #1101
Enable/disable eye video recordings using the
recording.should_start
notification -- #1101
Player
Trim section and export improvements -- #1093
Remove trim section indicators from video and raw data exporters
Add trim section indicators to general settings, showing both:
relative recording time
frame indices
Add
export_info.csv
file to export folder. This csv file contains the following information:Player version
Data format version
Export start date and time
Frame index trim range
Relative time trim range
Absolute time trim range (
world
timestamp range)
We no longer use trim range with frame indices as a naming convention
We now use incrementally increasing folder names with format
%d%d%d
, e.g.:<recording>/exports/000/
<recording>/exports/001/
<recording>/exports/002/
Confidence threshold consistency -- #1100
Previously, all binocular mappers dismissed pupil data with confidence lower than
0.6
to prevent bad binocular mapping. Now, low confidence pupil data is mapped monocularly.Introduction of a user settable global minimum calibration confidence threshold. This threshold sets the lower bounds of the pupil data confidence that is used for calibration. This value was previously fixed to
0.8
in Pupil Capture and to0.0
in Pupil Player -- effectively using low confidence pupil data for calibration. This threshold can be set in all calibration plugin menus.
These changes were introduced based on #1052 and #1047.
Minimum data confidence: Lower bounds gaze data by confidence that is used to generate visualizations and high level data, e.g. fixations. This does not effect the raw data export. Following data points are effected by it:
Fixations
Offline surface heatmaps
Circle, cross, light points and polyline visualizations
Bugfixes
Fix
Fake_Source
compatibility with Frame Publisher -- #1085Service does not crash when starting
Frame_Publisher
via a notification -- #1099Frame_Publisher
only warns once on encountering an incompatible format instead of warning each time -- #1099Fix an issue where
Fake_Source
would return frames at 2 fps in Capture if it was restored from the session settings. -- #1085Offline Fixation Detector: Correct upper limit of fixation lenghts to 4000ms -- #1084
Avoid
ZeroDivisionError
when calculating hdpi factors on Windows -- #1090Avoid positioning windows by default behind the Windows task bar -- #1103
Fix bug where export folders were written into the recording folder itself instead of into the
exports
subfolderUVC control settings are correctly restored from session settings and on reconnect -- #1099 and #1101
Avoid crash on recordings with less than 10 world frames -- #1101
Player always checks for recordings without world videos instead of only checking on upgrading the recording format - #1101
Fixed a bug that crashed Player when removing a surface - #1106
Fixed a bug that crashed Capture on restoring a minimized window on MS Windows 10 when log display was loaded - #1097
February 16, 2018
v1.4 Overview
We are pleased to announce the release of Pupil v1.4
!
Download the latest bundle and let us know what you think via the #pupil channel 😄
Features
Pupil Player
Support for Pupil Mobile recordings without world video -- #1071
Annotation csv exports include user-defined fields -- #1061
Trim calibration and mapping ranges using trim marks -- #1057
Add Jump to previous fixation thumb button to the Offline Fixation Detector plugin -- #1072
Export ranges are based on time instead of frame indices. The export folder will be named appropriately -- #1072
Resuming playback at the end of the trim section will seek to the beginning of the trim section instead of continuing playing after the trim section. This allows you to play back the trimmed section repeatedly without having to seek manually -- #1072
Pupil Player default plugin list -- d40a3b
Remove the
Vis Scan Path
Add the
Raw Data Exporter
FakeCapture (aka Test Image) frame content now displays the Pupil Labs gradient.
Bug Fixes
Improve Player timeline drawing performance by caching the drawn data -- #1063, #1068
Improve performance while playback in Player is paused -- 2df34d
Remove bug where the
Vis Scan Path
plugin would generate increasingly bad results with each playback iteration -- bf1977Catch one of reasons that lead to the
GLFW: Cannot create window
error on Windows
Feedback
As always we hope you enjoy the new release and please let us know if you discover any bugs. We look forward to your feedback!
February 16, 2018
v1.4 Overview
We are pleased to announce the release of Pupil v1.4
!
Download the latest bundle and let us know what you think via the #pupil channel 😄
Features
Pupil Player
Support for Pupil Mobile recordings without world video -- #1071
Annotation csv exports include user-defined fields -- #1061
Trim calibration and mapping ranges using trim marks -- #1057
Add Jump to previous fixation thumb button to the Offline Fixation Detector plugin -- #1072
Export ranges are based on time instead of frame indices. The export folder will be named appropriately -- #1072
Resuming playback at the end of the trim section will seek to the beginning of the trim section instead of continuing playing after the trim section. This allows you to play back the trimmed section repeatedly without having to seek manually -- #1072
Pupil Player default plugin list -- d40a3b
Remove the
Vis Scan Path
Add the
Raw Data Exporter
FakeCapture (aka Test Image) frame content now displays the Pupil Labs gradient.
Bug Fixes
Improve Player timeline drawing performance by caching the drawn data -- #1063, #1068
Improve performance while playback in Player is paused -- 2df34d
Remove bug where the
Vis Scan Path
plugin would generate increasingly bad results with each playback iteration -- bf1977Catch one of reasons that lead to the
GLFW: Cannot create window
error on Windows
Feedback
As always we hope you enjoy the new release and please let us know if you discover any bugs. We look forward to your feedback!
February 2, 2018
v1.3 Overview
We are pleased to announce the release of Pupil v1.3
!
There are a lot of new features - especially for Pupil Player - that we are excited to share with the Pupil community.
![screen shot 2018-02-02 at 13 34 22](https://framerusercontent.com/images/c5OQfGSGlsvZ4RaD4ZQl8JyNCA.png)
Screenshot of Pupil Player demonstrating the timeline graphs features during a "blink" event.
Download the latest bundle and let us know what you think via the #pupil channel 😄
Features
General
Text input - Support for cut/copy/paste and double click to select all in text input fields - #1027
Collapsed menu - Main menu is now collapsed by default#1019. You can expand the menu by clicking any of the the icons on the right hand side of the screen.
Pupil detection - Disable coarse detection for eye videos with width less than 200 pixels #1022
Pupil Service UI #1016
![Pupil Service UI](https://framerusercontent.com/images/IFomVkUpzoSTSPjae8ZGA4H8AE.png)
Pupil Capture
200hz eye camera - Improved support for the new Pupil Labs 200Hz eye cameras
Online pupil detection opt-out - This feature enables one to opt-out of online pupil detection in Pupil Capture #1009. If you use this feature, you should record eye videos so that you can use offline pupil detection and calibration in Pupil Player (
Recorder > Record eye
).Real world start time - Add record start time as Unix timestamp to
info.csv
#1017 - this enables users to calculate when the recording started in real-world time.Single marker calibration - Add display modes for Single Marker Calibration #1021
Pupil Player
Time based seeking in Player #1041
Improved Player seek control
Add more Player timelines
Offline blink detection #1040
Offline Calibration Improvements
Disabled automatic search for calibration markers
Added option to cancel calibration marker seach
Added option to use detected calibration marker as natural features for manual filtering
Improved caching of detected data
Improved Player Seek Control
Highlight elements on hover that can be activated when being clicked or dragged
Playback modes and speeds #1023
Frame-by-frame seeking when paused
Decrease and increase playback speed while playing
Point & click somewhere on the seek bar to jump to this position
Player Timelines
Added pupil confidence graph in the timeline #1040
Added 2d pupil diameter (pixels) graph in the timeline diameter, 2d, in pixels #1040
Added blink events graph in the timeline #1040
Added recorded FPS, for world and eye videos as a graph in the timeline #1051
Bug Fixes
Fixed visual bug where offline gaze mapping would be stuck at 99% #1014
Fixed crash when changing a text field in the
Accuracy Visualizer
menu #1007Fix passing a recording to Pupil Player as a command line argument #1006
Fix bug where main menu would not expand #1006
Improved ROI handling #1031
Feedback
As always we hope you enjoy the new release and please let us know of any bugs. We look forward to your feedback!
200hz video Pupil mobile bundle support
This has been added with v1.3-9
! Please update to this version and re-download the recordings from your device.
February 2, 2018
v1.3 Overview
We are pleased to announce the release of Pupil v1.3
!
There are a lot of new features - especially for Pupil Player - that we are excited to share with the Pupil community.
![screen shot 2018-02-02 at 13 34 22](https://framerusercontent.com/images/c5OQfGSGlsvZ4RaD4ZQl8JyNCA.png)
Screenshot of Pupil Player demonstrating the timeline graphs features during a "blink" event.
Download the latest bundle and let us know what you think via the #pupil channel 😄
Features
General
Text input - Support for cut/copy/paste and double click to select all in text input fields - #1027
Collapsed menu - Main menu is now collapsed by default#1019. You can expand the menu by clicking any of the the icons on the right hand side of the screen.
Pupil detection - Disable coarse detection for eye videos with width less than 200 pixels #1022
Pupil Service UI #1016
![Pupil Service UI](https://framerusercontent.com/images/IFomVkUpzoSTSPjae8ZGA4H8AE.png)
Pupil Capture
200hz eye camera - Improved support for the new Pupil Labs 200Hz eye cameras
Online pupil detection opt-out - This feature enables one to opt-out of online pupil detection in Pupil Capture #1009. If you use this feature, you should record eye videos so that you can use offline pupil detection and calibration in Pupil Player (
Recorder > Record eye
).Real world start time - Add record start time as Unix timestamp to
info.csv
#1017 - this enables users to calculate when the recording started in real-world time.Single marker calibration - Add display modes for Single Marker Calibration #1021
Pupil Player
Time based seeking in Player #1041
Improved Player seek control
Add more Player timelines
Offline blink detection #1040
Offline Calibration Improvements
Disabled automatic search for calibration markers
Added option to cancel calibration marker seach
Added option to use detected calibration marker as natural features for manual filtering
Improved caching of detected data
Improved Player Seek Control
Highlight elements on hover that can be activated when being clicked or dragged
Playback modes and speeds #1023
Frame-by-frame seeking when paused
Decrease and increase playback speed while playing
Point & click somewhere on the seek bar to jump to this position
Player Timelines
Added pupil confidence graph in the timeline #1040
Added 2d pupil diameter (pixels) graph in the timeline diameter, 2d, in pixels #1040
Added blink events graph in the timeline #1040
Added recorded FPS, for world and eye videos as a graph in the timeline #1051
Bug Fixes
Fixed visual bug where offline gaze mapping would be stuck at 99% #1014
Fixed crash when changing a text field in the
Accuracy Visualizer
menu #1007Fix passing a recording to Pupil Player as a command line argument #1006
Fix bug where main menu would not expand #1006
Improved ROI handling #1031
Feedback
As always we hope you enjoy the new release and please let us know of any bugs. We look forward to your feedback!
200hz video Pupil mobile bundle support
This has been added with v1.3-9
! Please update to this version and re-download the recordings from your device.
December 14, 2017
v1.2 Overview
The new v1.2
release improves overall stability and includes support for the new 200Hz eye cameras. We highly recommend upgrading Pupil Player, Service and Capture to v1.2
.
Features
Support for the new 200Hz eye cameras (#983)
Default eye video size to qVGA or smaller for better performance.
Bundle support for RealSense 3D on macOS and Windows
Add
Hololens Relay
- UDP relay for Pupil to HoloLens communicationImprove calibration marker detection, see below for details (#961, #965)
Offline Surface Tracker - Add confidence column to gaze export (#941, #945)
Refactor
Video Export
procedure (#927)Refactor
Batch Exporter
(#931)
Bug Fixes
Improve Retina display support (#935)
Improve
Camera Intrinsics Estimation
stability (2028f1e6db21c32cdc84bd4ef0d685dd4ace01e1, #969)Fix
Offline Fixation Detector
(#950)Improve
HMD Calibration (3d)
stability (#970)Include
pyndsi
v0.3.3
which fixes recording Pupil Mobile streams (#910 #976 #981)
Calibration marker detection
The calibration marker detection accuracy and efficiency has been improved. The procedure works best with our new markers:
![](https://framerusercontent.com/images/0Q9ipwFx6iBZxJm2BW5bFARB9rA.png?scale-down-to=4096)
Feedback
As always we hope you enjoy the new release and please do let us know of any bugs. We look forward to your feedback!
December 14, 2017
v1.2 Overview
The new v1.2
release improves overall stability and includes support for the new 200Hz eye cameras. We highly recommend upgrading Pupil Player, Service and Capture to v1.2
.
Features
Support for the new 200Hz eye cameras (#983)
Default eye video size to qVGA or smaller for better performance.
Bundle support for RealSense 3D on macOS and Windows
Add
Hololens Relay
- UDP relay for Pupil to HoloLens communicationImprove calibration marker detection, see below for details (#961, #965)
Offline Surface Tracker - Add confidence column to gaze export (#941, #945)
Refactor
Video Export
procedure (#927)Refactor
Batch Exporter
(#931)
Bug Fixes
Improve Retina display support (#935)
Improve
Camera Intrinsics Estimation
stability (2028f1e6db21c32cdc84bd4ef0d685dd4ace01e1, #969)Fix
Offline Fixation Detector
(#950)Improve
HMD Calibration (3d)
stability (#970)Include
pyndsi
v0.3.3
which fixes recording Pupil Mobile streams (#910 #976 #981)
Calibration marker detection
The calibration marker detection accuracy and efficiency has been improved. The procedure works best with our new markers:
![](https://framerusercontent.com/images/0Q9ipwFx6iBZxJm2BW5bFARB9rA.png?scale-down-to=4096)
Feedback
As always we hope you enjoy the new release and please do let us know of any bugs. We look forward to your feedback!
November 9, 2017
v1.1 Overview
We are pleased to announce the release of Pupil v1.1
- this release happily coincides with the 100th release of Pupil software!
There are a lot of new features in this release and we are especially excited to introduce Pupil's new GUI!
![](https://framerusercontent.com/images/xS27mfdMzIrjtBy4wQOZM3o.png)
We highly recommend upgrading Pupil Player, Service and Capture to v1.1
.
A note on versioning
TL;DR Please don’t read too deeply into version numbers. We will be deploying new software on a continuous basis and version numbers will continue to increment accordingly.
The core team had a healthy debate on about version numbering. Should we release v1.0
or v0.10.
? This could seem like a trivial debate about naming conventions, but we also realize that there is a lot of “baggage” associated with versioning. We acknowledge that many people see v1.0
as a big step for a project. It often signifies a level of maturity of the project and community. Pupil is certainly growing into a platform with a healthy group of contributors and researchers around the world who depend on Pupil software and hardware. Pupil is certainly more mature now compared to where it was 5 years ago. But we want emphasize that Pupil is an active project that will always be in flux. We will always need feedback from the community, there will be bugs and together we will fix them! This version is just one small step forward of many to come. 😄
Prior to this release we were (ab)using Semantic Versioning. For those not familiar with the Semantic Versioning, the version numbers follow the format major.minor.patch
. From a technical perspective, we should have released a major
version when we first defined the plugin structure, and another major
version whenever we redefined the plugin interface, and yet another major
version when we defined the “API” with the IPC Backbone in v0.8.
Where’s v1.0
?
With all the changes that we made with the new UI, it looks like we flew right past v1.0
and went straight to v1.1
! There are more than 100 commits between the v1.0
and v1.1
tags, so a minor bump was required.
Features
New GUI - A lot of work went into redesigning the GUI in Pupil Capture and Pupil Player. There are ton of new features; too many to list! Just dive in and let us know what you think via chat or make an issue if you find a bug!
Pupil Capture - Checkout the docs for a high level overview of the new GUI structure here
Pupil Player - The most significant GUI changes are in Pupil Player. The new GUI greatly simplifies the interface. No more floating plugin windows cluttering the window! We also introduce an expandable events timeline so that all temporal events can be clearly labeled and correlated with the timeline. For a high level overview check out the docs.
Plugin Manager - The central place for loading and unloading other plugins. No more tedious
Select to load
drop down menus! 😄Pupil diameter history graph - You can now visualize changes in pupil diameter over time in Pupil.
Improvements to libuvc, our usb video backend. Considerably more accurate timestamp generation and more robust failure behaviour on usb disconnect and recovery.
Bug Fixes
Pupil Mobile file import - Pupil Player can now properly read and display files recorded by Pupil Mobile that were saved on the Android device (see #873)
Fixation detector - Substantially decreased number of false negative detections
Realsense backend - Improved failure behaviour on usb disconnect and recovery. Added pointcloud visualizer window.
HMD calibration - Improved Network API based on feedback from HMD-eyes developers.
Manual marker calibration - Improved detection accuracy of markers.
Backwards incompatible changes
We revised the plugin API with v1.0
in order to ensure that all plugins work consistently with the new GUI. Please be aware that some of these changes are incompatible with the old API. This means that old plugins need updating before they can run correctly in v1.1
. Please see the these pull request notes on details about the changes. We also updated the plugin guide accordingly.
Disabled Plugins
These plugins will be disabled due to legacy incompatibilities:
Batch Exporter
Marker Auto Trim Marks
We will re-add them on demand and when we have time to refactor them.
We will work on refactoring these plugins and will release an up-to-date version with the next release. Follow development progress and express your interest in these features in the following issues:
Windows Driver Installation
Drivers are automatically installed and updated! Please let us know if this does not work for you so that we can improve the installer!
Feedback
As always we hope you enjoy the new release and please do let us know of any bugs. We look forward to your feedback!
Linux bundle has been updated to support Ubuntu 17.10.
November 9, 2017
v1.1 Overview
We are pleased to announce the release of Pupil v1.1
- this release happily coincides with the 100th release of Pupil software!
There are a lot of new features in this release and we are especially excited to introduce Pupil's new GUI!
![](https://framerusercontent.com/images/xS27mfdMzIrjtBy4wQOZM3o.png)
We highly recommend upgrading Pupil Player, Service and Capture to v1.1
.
A note on versioning
TL;DR Please don’t read too deeply into version numbers. We will be deploying new software on a continuous basis and version numbers will continue to increment accordingly.
The core team had a healthy debate on about version numbering. Should we release v1.0
or v0.10.
? This could seem like a trivial debate about naming conventions, but we also realize that there is a lot of “baggage” associated with versioning. We acknowledge that many people see v1.0
as a big step for a project. It often signifies a level of maturity of the project and community. Pupil is certainly growing into a platform with a healthy group of contributors and researchers around the world who depend on Pupil software and hardware. Pupil is certainly more mature now compared to where it was 5 years ago. But we want emphasize that Pupil is an active project that will always be in flux. We will always need feedback from the community, there will be bugs and together we will fix them! This version is just one small step forward of many to come. 😄
Prior to this release we were (ab)using Semantic Versioning. For those not familiar with the Semantic Versioning, the version numbers follow the format major.minor.patch
. From a technical perspective, we should have released a major
version when we first defined the plugin structure, and another major
version whenever we redefined the plugin interface, and yet another major
version when we defined the “API” with the IPC Backbone in v0.8.
Where’s v1.0
?
With all the changes that we made with the new UI, it looks like we flew right past v1.0
and went straight to v1.1
! There are more than 100 commits between the v1.0
and v1.1
tags, so a minor bump was required.
Features
New GUI - A lot of work went into redesigning the GUI in Pupil Capture and Pupil Player. There are ton of new features; too many to list! Just dive in and let us know what you think via chat or make an issue if you find a bug!
Pupil Capture - Checkout the docs for a high level overview of the new GUI structure here
Pupil Player - The most significant GUI changes are in Pupil Player. The new GUI greatly simplifies the interface. No more floating plugin windows cluttering the window! We also introduce an expandable events timeline so that all temporal events can be clearly labeled and correlated with the timeline. For a high level overview check out the docs.
Plugin Manager - The central place for loading and unloading other plugins. No more tedious
Select to load
drop down menus! 😄Pupil diameter history graph - You can now visualize changes in pupil diameter over time in Pupil.
Improvements to libuvc, our usb video backend. Considerably more accurate timestamp generation and more robust failure behaviour on usb disconnect and recovery.
Bug Fixes
Pupil Mobile file import - Pupil Player can now properly read and display files recorded by Pupil Mobile that were saved on the Android device (see #873)
Fixation detector - Substantially decreased number of false negative detections
Realsense backend - Improved failure behaviour on usb disconnect and recovery. Added pointcloud visualizer window.
HMD calibration - Improved Network API based on feedback from HMD-eyes developers.
Manual marker calibration - Improved detection accuracy of markers.
Backwards incompatible changes
We revised the plugin API with v1.0
in order to ensure that all plugins work consistently with the new GUI. Please be aware that some of these changes are incompatible with the old API. This means that old plugins need updating before they can run correctly in v1.1
. Please see the these pull request notes on details about the changes. We also updated the plugin guide accordingly.
Disabled Plugins
These plugins will be disabled due to legacy incompatibilities:
Batch Exporter
Marker Auto Trim Marks
We will re-add them on demand and when we have time to refactor them.
We will work on refactoring these plugins and will release an up-to-date version with the next release. Follow development progress and express your interest in these features in the following issues:
Windows Driver Installation
Drivers are automatically installed and updated! Please let us know if this does not work for you so that we can improve the installer!
Feedback
As always we hope you enjoy the new release and please do let us know of any bugs. We look forward to your feedback!
Linux bundle has been updated to support Ubuntu 17.10.
September 2, 2017
v0.9.14 Overview
New feature release. We recommend updating Pupil Player and Capture to v0.9.14
!
Features
3d World camera - We have added support for librealsense which enables Pupil Capture to capture depth and rgb streams from the Intel RealSense R200. Read more about this feature in the docs. Currently only supported when running from source. Bundle support is coming soon!
Widows Driver Auto-Installer - Windows drivers are automatically installed when you run Pupil Capture on Windows 10.
Accuracy Test - The accuracy test and visualizer has been refactored. This plugin enables one to easily test and visualize accuracy directly after a calibration. See #808 and #812.
Diameter History Plugin - New plugin for Pupil Capture that displays a graph of pupil diameter for each eye using diameter data from the 3d model. See #820.
Camera intrinsics - Refactored all distortion code. Added fisheye distortion models to support wide angle lenses and added more pre-set calibrations. See #789 for further explanation and discussion.
Real-time Fixation Detector for Pupil Capture - refactored fixation detector to work in 2d and 3d mode alike. Reduced CPU load and added visual feedback.
Real-time heat maps in Pupil Player - test the heatmap feature of our surface-tracker in real-time.
Blink detector visualization - Update for the blink detector plugin. Blinks are now visualized in Pupil Capture by a screen shade on blink onset.
Drag and drop support for video files in Pupil Capture - Select the
Video File Source
manager and drag and drop video files into the world and eye windows in Pupil Capture.Looped video playback for file source
Stability and performance improvements of the libuvc usb video backend.
Pupil detector size settings now scale with video resolution.
Improved monocular 3d gaze mapper accuracy by allowing one more degree of freedom in the mapping function.
Numerous small improvements in the Eye Video Overlay plugin for Pupil Player
Bug Fixes
Frame publisher - a number of smaller fixes
.mkv
file support in the offline pupil detectorSurface tracking marker detector fixes - a number of smaller fixes
ROI settings are persistent again.
Windows Driver Installation
Drivers are now automatically installed and updated! Please let us know if this does not work for you so that we can improve the installer!
Feedback
As always we hope you enjoy the new release and please do let us know of any bugs. We look forward to your feedback!
September 2, 2017
v0.9.14 Overview
New feature release. We recommend updating Pupil Player and Capture to v0.9.14
!
Features
3d World camera - We have added support for librealsense which enables Pupil Capture to capture depth and rgb streams from the Intel RealSense R200. Read more about this feature in the docs. Currently only supported when running from source. Bundle support is coming soon!
Widows Driver Auto-Installer - Windows drivers are automatically installed when you run Pupil Capture on Windows 10.
Accuracy Test - The accuracy test and visualizer has been refactored. This plugin enables one to easily test and visualize accuracy directly after a calibration. See #808 and #812.
Diameter History Plugin - New plugin for Pupil Capture that displays a graph of pupil diameter for each eye using diameter data from the 3d model. See #820.
Camera intrinsics - Refactored all distortion code. Added fisheye distortion models to support wide angle lenses and added more pre-set calibrations. See #789 for further explanation and discussion.
Real-time Fixation Detector for Pupil Capture - refactored fixation detector to work in 2d and 3d mode alike. Reduced CPU load and added visual feedback.
Real-time heat maps in Pupil Player - test the heatmap feature of our surface-tracker in real-time.
Blink detector visualization - Update for the blink detector plugin. Blinks are now visualized in Pupil Capture by a screen shade on blink onset.
Drag and drop support for video files in Pupil Capture - Select the
Video File Source
manager and drag and drop video files into the world and eye windows in Pupil Capture.Looped video playback for file source
Stability and performance improvements of the libuvc usb video backend.
Pupil detector size settings now scale with video resolution.
Improved monocular 3d gaze mapper accuracy by allowing one more degree of freedom in the mapping function.
Numerous small improvements in the Eye Video Overlay plugin for Pupil Player
Bug Fixes
Frame publisher - a number of smaller fixes
.mkv
file support in the offline pupil detectorSurface tracking marker detector fixes - a number of smaller fixes
ROI settings are persistent again.
Windows Driver Installation
Drivers are now automatically installed and updated! Please let us know if this does not work for you so that we can improve the installer!
Feedback
As always we hope you enjoy the new release and please do let us know of any bugs. We look forward to your feedback!
July 28, 2017
v0.9.13 Overview
New feature release. We recommend updating Pupil Player and Capture to v0.9.13
!
Features
Fully synced audio: Audio is recorded with timestamps and merged with video when exported in Pupil Player. (Mac and Linux only for now).
Pupil Player now supports recordings made locally with Pupil Mobile.
Offline pupil detection: You can now do post-recording pupil (re)-detection. Just make sure you record the eye video in capture.
Offline Calibration using markers: If you include calibrations in your recording you can calibrate after using this method.
Offline Calibration using natural features: You can calibrate post-recording by clicking known gaze locations.
Pupil Capture now has a
reset-settings-to-defaults
button.Reverted back to old presets for 3d detection as they create more stable models.
Pupil Time Sync is no longer a default plugin. This should be turned on when using groups and/or Pupil Mobile!
Bug Fixes
Fixed a bug that allowed more than 2 eye processes to run.
Improved libuvc performance on Windows (reduces CPU load).
Numerous small fixes.
Deprecated/removed features:
Removed manual gaze correction. Let us know if this feature is really required.
Hotfixes within this release (v0.9.13-48)
Fixed offline surface Tracker
Fixed demo recording import
Improved libuvc shutdown procedure
Documentation and Demo
We are still short on docs for the new player methods. Check out the demo video here:
Windows Driver Installation
If you are installing Pupil on Windows 10 for the first time or after a windows update, please see Windows 10 driver setup instructions here.
Feedback
As always we hope you enjoy the new release and please do let us know of any bugs. We look forward to your feedback!
Windows release coming soon!
July 28, 2017
v0.9.13 Overview
New feature release. We recommend updating Pupil Player and Capture to v0.9.13
!
Features
Fully synced audio: Audio is recorded with timestamps and merged with video when exported in Pupil Player. (Mac and Linux only for now).
Pupil Player now supports recordings made locally with Pupil Mobile.
Offline pupil detection: You can now do post-recording pupil (re)-detection. Just make sure you record the eye video in capture.
Offline Calibration using markers: If you include calibrations in your recording you can calibrate after using this method.
Offline Calibration using natural features: You can calibrate post-recording by clicking known gaze locations.
Pupil Capture now has a
reset-settings-to-defaults
button.Reverted back to old presets for 3d detection as they create more stable models.
Pupil Time Sync is no longer a default plugin. This should be turned on when using groups and/or Pupil Mobile!
Bug Fixes
Fixed a bug that allowed more than 2 eye processes to run.
Improved libuvc performance on Windows (reduces CPU load).
Numerous small fixes.
Deprecated/removed features:
Removed manual gaze correction. Let us know if this feature is really required.
Hotfixes within this release (v0.9.13-48)
Fixed offline surface Tracker
Fixed demo recording import
Improved libuvc shutdown procedure
Documentation and Demo
We are still short on docs for the new player methods. Check out the demo video here:
Windows Driver Installation
If you are installing Pupil on Windows 10 for the first time or after a windows update, please see Windows 10 driver setup instructions here.
Feedback
As always we hope you enjoy the new release and please do let us know of any bugs. We look forward to your feedback!
Windows release coming soon!
May 23, 2017
v0.9.12 Overview
Bugfix and feature release. We recommend updating Pupil Player and Capture to v0.9.12
!
Features
Audio capture is now its own plugin. Audio packets are now timestamped and made available as events.
Pupil Mobile now syncs its time with Pupil Capture.
Protocol versioning for time-sync and NDSI (the protocol we use to talk to pupil mobile).
H264 capture to file from pupil-mobile network stream is now officially supported.
Bug Fixes
Increased timestamp granularity for Windows capture backend to avoid frames with the same timestamp.
Fixed a bug that double applied gaze correction in the exported video.
Windows Driver Installation
If you are installing Pupil on Windows 10 for the first time, please see Windows 10 driver setup instructions here.
May 23, 2017
v0.9.12 Overview
Bugfix and feature release. We recommend updating Pupil Player and Capture to v0.9.12
!
Features
Audio capture is now its own plugin. Audio packets are now timestamped and made available as events.
Pupil Mobile now syncs its time with Pupil Capture.
Protocol versioning for time-sync and NDSI (the protocol we use to talk to pupil mobile).
H264 capture to file from pupil-mobile network stream is now officially supported.
Bug Fixes
Increased timestamp granularity for Windows capture backend to avoid frames with the same timestamp.
Fixed a bug that double applied gaze correction in the exported video.
Windows Driver Installation
If you are installing Pupil on Windows 10 for the first time, please see Windows 10 driver setup instructions here.
May 11, 2017
v0.9.10 Overview
Bugfix and feature release. We recommend updating Pupil Player and Capture to v0.9.10
!
Rewrite of Pupil time sync
Rewrote the time sync plugin and defined a public protocol draft.
We have added example implementation scripts of clients in pupil-helpers.
Bug Fixes
Fixed three bugs in the offline surface tracker.
Windows Driver Installation
If you are installing Pupil on Windows 10 for the first time, please see Windows 10 driver setup instructions here.
May 11, 2017
v0.9.10 Overview
Bugfix and feature release. We recommend updating Pupil Player and Capture to v0.9.10
!
Rewrite of Pupil time sync
Rewrote the time sync plugin and defined a public protocol draft.
We have added example implementation scripts of clients in pupil-helpers.
Bug Fixes
Fixed three bugs in the offline surface tracker.
Windows Driver Installation
If you are installing Pupil on Windows 10 for the first time, please see Windows 10 driver setup instructions here.
April 20, 2017
v0.9.6 Overview
Bugfix release. We recommend updating Pupil Player to v0.9.6
!
Bug Fixes
Fixed video export in player for eye video overlay
Windows Driver Installation
If you are installing Pupil on Windows 10 for the first time, please see Windows 10 driver setup instructions here.
April 20, 2017
v0.9.6 Overview
Bugfix release. We recommend updating Pupil Player to v0.9.6
!
Bug Fixes
Fixed video export in player for eye video overlay
Windows Driver Installation
If you are installing Pupil on Windows 10 for the first time, please see Windows 10 driver setup instructions here.
April 13, 2017
v0.9.5 Overview
New feature and bugfix release. We recommend updating Pupil Capture and Pupil Player to v0.9.5
!
New features
Pupil Player now lets you know when a recording in updated. So you know that it may take a bit!
Bug Fixes
Fixed video export and offline surface detection for Pupil Player under MacOS (#693)
HMD mapper bug fixed.
Various smaller fixes and improvements.
April 13, 2017
v0.9.5 Overview
New feature and bugfix release. We recommend updating Pupil Capture and Pupil Player to v0.9.5
!
New features
Pupil Player now lets you know when a recording in updated. So you know that it may take a bit!
Bug Fixes
Fixed video export and offline surface detection for Pupil Player under MacOS (#693)
HMD mapper bug fixed.
Various smaller fixes and improvements.
April 5, 2017
v0.9.4 Overview
New feature and bugfix release. We recommend updating Pupil Capture and Pupil Player to v0.9.4
!
New features
Pupil Mobile now supports
h264
streaming and saving. This is a great feature because it considerably reduces file size and bandwidth requirements. Warning - this feature is still in Beta.Use
msgpack
instead of pickle for saving data to files. This makes saved files smaller and loading faster.Use only Python built-in types on the IPC and only save Python built-in types in
pupil_data
. With this convention Pupil data is self-sufficient and can be interpreted easily outside of Python.Improved high DPI display support (e.g. for Retina displays)
Added ability to load runtime calibration plugins.
Calibration data and plugin parameters are now reported on the IPC and saved during recording.
macOS Application Bundles
We are able to offer macOS application bundles again! (finally 😄)
Bug Fixes
Offline and Online Surface Tracker now report identical events.
Numerous small fixes.
April 5, 2017
v0.9.4 Overview
New feature and bugfix release. We recommend updating Pupil Capture and Pupil Player to v0.9.4
!
New features
Pupil Mobile now supports
h264
streaming and saving. This is a great feature because it considerably reduces file size and bandwidth requirements. Warning - this feature is still in Beta.Use
msgpack
instead of pickle for saving data to files. This makes saved files smaller and loading faster.Use only Python built-in types on the IPC and only save Python built-in types in
pupil_data
. With this convention Pupil data is self-sufficient and can be interpreted easily outside of Python.Improved high DPI display support (e.g. for Retina displays)
Added ability to load runtime calibration plugins.
Calibration data and plugin parameters are now reported on the IPC and saved during recording.
macOS Application Bundles
We are able to offer macOS application bundles again! (finally 😄)
Bug Fixes
Offline and Online Surface Tracker now report identical events.
Numerous small fixes.
March 21, 2017
v0.9.3 Overview
A third bug fix release for v0.9.0
.
Fixed various bugs in player and Capture
Improved exporter memory consumption
Fix buggy gaze data from v0.8.7 recordings when upgrading version formats in player
Improve player file loading speed.
We recommend updating Pupil Capture and Pupil Player to v0.9.3
.
Windows Driver Installation
If you are installing Pupil on Windows 10 for the first time, please see Windows 10 driver setup instructions here.
March 21, 2017
v0.9.3 Overview
A third bug fix release for v0.9.0
.
Fixed various bugs in player and Capture
Improved exporter memory consumption
Fix buggy gaze data from v0.8.7 recordings when upgrading version formats in player
Improve player file loading speed.
We recommend updating Pupil Capture and Pupil Player to v0.9.3
.
Windows Driver Installation
If you are installing Pupil on Windows 10 for the first time, please see Windows 10 driver setup instructions here.
March 10, 2017
v0.9.2 Overview
A second bugfix release for v0.9.0
.
Fixed various bugs in player and Capture
We recommend updating Pupil Capture and Pupil Player to v0.9.2
.
New features
Inverted marker tracking in Player
Windows Driver Installation
We made a driver installer so that you can get up and running with Pupil much faster on Windows 10. Please see Windows 10 driver setup instructions here.
March 10, 2017
v0.9.2 Overview
A second bugfix release for v0.9.0
.
Fixed various bugs in player and Capture
We recommend updating Pupil Capture and Pupil Player to v0.9.2
.
New features
Inverted marker tracking in Player
Windows Driver Installation
We made a driver installer so that you can get up and running with Pupil much faster on Windows 10. Please see Windows 10 driver setup instructions here.
February 24, 2017
v0.9.1 Overview
A bugfix release for v0.9.0
.
Fixed various bugs in player.
Its recommended to update Capture and Player to v0.9.1
.
Windows Driver Installation
We made a driver installer so that you can get up and running with Pupil much faster on Windows 10. Please see Windows 10 driver setup instructions here.
February 24, 2017
v0.9.1 Overview
A bugfix release for v0.9.0
.
Fixed various bugs in player.
Its recommended to update Capture and Player to v0.9.1
.
Windows Driver Installation
We made a driver installer so that you can get up and running with Pupil much faster on Windows 10. Please see Windows 10 driver setup instructions here.
February 17, 2017
v0.9.0 Overview
We are happy to release Pupil software v0.9.0
.
There are a lot of new features and improvements that we have added to Pupil software in this release along with the great migration of the entire codebase from Python2 to Python3!
Python2 --> Python3 - The great migration
We have successfully transitioned the entire Pupil codebase from Python2 to Python3. There are many reasons why we decided to transition to Python3. While we could have comfortably continued with Python2 on macOS and Linux, it was not feasible to support Pupil and dependencies on Windows with Python2. Some of you may have noticed that with v0.8.7-w
(Windows release) we already started using Python v3.5.2
. This introduced a schism in the codebase, with Pupil development for Windows using Python3 and macOS and Linux using Python2.
In order to unify the codebase for cross-platform development, we decided to make the great migration to Python3. The migration hasn't been as easy as we had anticipated. But we are happy that we decided to make the move. Now the Pupil community can take advantage of all the new Python3 features, performance enhancements, and work with other Python3 supported libraries.
OpenCV2 --> OpenCV3
We have moved to OpenCV3 and will start adding new futures like fisheye lens distorion modelling in the near future.
Non-blocking capture
We have restructured the video input source logic. All video sources are now normal plugins. This means you can create your own custom video source more easily. This change has also simplified the source code.
We now allow the capture source to not return an image. This means that the application UI and app will remain responsive even if capture fails to stream images.
Note To plugin developers: This change also means that we had to relax the constraints on what data is guaranteed to be available during the plugin update loop. This resulted in two Plugin API changes:
We moved the
frame
object into events.We introduce
recent_events(events)
to replaceupdate(frame, events)
. You can still useupdate(frame, events)
in your plugin for the time being but note that it is officially deprecated.
Pupil Detector 2d - Parameter Optimization
The default parameters of the 2d pupil detection algorithm have been significantly improved. We have optimized the default parameters using a grid-search approach to perform well on the challenging "Labeled pupils in the wild (LPW)" dataset, while maintaining high speed performance (>300 FPS). In particular the performance in difficult lighting scenarios (e.g. outdoors) improved over the old default parameter settings.
Blink detection
We have added a simple blink detection plugin to Pupil Capture. The plugin detects the onset of a blink via the joint and sudden falloff in pupil detection confidence in both eyes occurring when the pupils are occluded by the falling eyelids. Similarly the end of a blink can be detected via the sudden rise in pupil detection confidence when the pupils are uncovered again.
Fixation tuned calibration
We have added a real-time fixation detector to Pupil Capture. The fixation detector uses the pupil normal vector provided by the 3d model (in 3d detection and mapping mode) to measure dispersion. The fixation detector plugin is now loaded by default in Pupil Capture and is used by calibration procedures to speed up calibration.
Notes
macOS
- bundles are coming soonTODOs and Known Issues
Tested and fixed for Linux and Windows
February 17, 2017
v0.9.0 Overview
We are happy to release Pupil software v0.9.0
.
There are a lot of new features and improvements that we have added to Pupil software in this release along with the great migration of the entire codebase from Python2 to Python3!
Python2 --> Python3 - The great migration
We have successfully transitioned the entire Pupil codebase from Python2 to Python3. There are many reasons why we decided to transition to Python3. While we could have comfortably continued with Python2 on macOS and Linux, it was not feasible to support Pupil and dependencies on Windows with Python2. Some of you may have noticed that with v0.8.7-w
(Windows release) we already started using Python v3.5.2
. This introduced a schism in the codebase, with Pupil development for Windows using Python3 and macOS and Linux using Python2.
In order to unify the codebase for cross-platform development, we decided to make the great migration to Python3. The migration hasn't been as easy as we had anticipated. But we are happy that we decided to make the move. Now the Pupil community can take advantage of all the new Python3 features, performance enhancements, and work with other Python3 supported libraries.
OpenCV2 --> OpenCV3
We have moved to OpenCV3 and will start adding new futures like fisheye lens distorion modelling in the near future.
Non-blocking capture
We have restructured the video input source logic. All video sources are now normal plugins. This means you can create your own custom video source more easily. This change has also simplified the source code.
We now allow the capture source to not return an image. This means that the application UI and app will remain responsive even if capture fails to stream images.
Note To plugin developers: This change also means that we had to relax the constraints on what data is guaranteed to be available during the plugin update loop. This resulted in two Plugin API changes:
We moved the
frame
object into events.We introduce
recent_events(events)
to replaceupdate(frame, events)
. You can still useupdate(frame, events)
in your plugin for the time being but note that it is officially deprecated.
Pupil Detector 2d - Parameter Optimization
The default parameters of the 2d pupil detection algorithm have been significantly improved. We have optimized the default parameters using a grid-search approach to perform well on the challenging "Labeled pupils in the wild (LPW)" dataset, while maintaining high speed performance (>300 FPS). In particular the performance in difficult lighting scenarios (e.g. outdoors) improved over the old default parameter settings.
Blink detection
We have added a simple blink detection plugin to Pupil Capture. The plugin detects the onset of a blink via the joint and sudden falloff in pupil detection confidence in both eyes occurring when the pupils are occluded by the falling eyelids. Similarly the end of a blink can be detected via the sudden rise in pupil detection confidence when the pupils are uncovered again.
Fixation tuned calibration
We have added a real-time fixation detector to Pupil Capture. The fixation detector uses the pupil normal vector provided by the 3d model (in 3d detection and mapping mode) to measure dispersion. The fixation detector plugin is now loaded by default in Pupil Capture and is used by calibration procedures to speed up calibration.
Notes
macOS
- bundles are coming soonTODOs and Known Issues
Tested and fixed for Linux and Windows
December 14, 2016
Pupil on Windows
We are very happy to announce that the latest version of Pupil is now available on Windows 10!
A lot of work went into getting the usb and low level video backend running robustly for Windows, and now Pupil is equally well supported on macOS, Linux, and Windows.
📢 Note📢 The v0.8.7
release for macOS and Linux can be found :point_right: here
It is also important to note that this is the first Pupil release that uses Python v3.5.2, and OpenCV v3.1. You can run the python3_opencv3_merged branch from source on macOS, Linux, or Windows if you are using Python 3.5.2 and OpenCV 3.1.
Windows Driver Setup
In order to support isochronous USB transfer on Windows, you will need to install drivers for your Pupil headset cameras. You can follow the steps from this short guide on our wiki.
Running Bundles on Windows
Download the bundle.
Extract the archive with 7zip
Launch
pupil_capture.exe
orpupil_player.exe
from within thePupil Capture
orPupil Player
bundle.
You may also want to right click on the .exe
and Create shortcut
so you can launch the applications from desktop. For debugging, you can run the .exe
with cmd prompt.
Known Issues
CPU performance Graph is not accurate. The CPU graph in World, Eye, and Player windows displays a much greater percentage than is actually being used. Use
task manager
to see actual cpu performance.Audio feedback for calibration routines is not yet implemented in Windows
Audio Source - Menu yet working
~~Pupil Remote - not yet functional~~ fixed in latest bundle below
Updates
Updated timing function to
timeGetTime
in pyuvc in latest Capture bundlePupil Service bundle is now ready for testing 😄
Fixed multiprocessing bug for batch exporter plugin on Pupil Player
v0.8.7-w-20
fixed calibration issues, plugin window location issues
Please open issues on github if you find any bugs, so that we can incorporate changes in the next bundle/release. We hope look forward to your feedback!
Best,
The Pupil Dev Team
December 14, 2016
Pupil on Windows
We are very happy to announce that the latest version of Pupil is now available on Windows 10!
A lot of work went into getting the usb and low level video backend running robustly for Windows, and now Pupil is equally well supported on macOS, Linux, and Windows.
📢 Note📢 The v0.8.7
release for macOS and Linux can be found :point_right: here
It is also important to note that this is the first Pupil release that uses Python v3.5.2, and OpenCV v3.1. You can run the python3_opencv3_merged branch from source on macOS, Linux, or Windows if you are using Python 3.5.2 and OpenCV 3.1.
Windows Driver Setup
In order to support isochronous USB transfer on Windows, you will need to install drivers for your Pupil headset cameras. You can follow the steps from this short guide on our wiki.
Running Bundles on Windows
Download the bundle.
Extract the archive with 7zip
Launch
pupil_capture.exe
orpupil_player.exe
from within thePupil Capture
orPupil Player
bundle.
You may also want to right click on the .exe
and Create shortcut
so you can launch the applications from desktop. For debugging, you can run the .exe
with cmd prompt.
Known Issues
CPU performance Graph is not accurate. The CPU graph in World, Eye, and Player windows displays a much greater percentage than is actually being used. Use
task manager
to see actual cpu performance.Audio feedback for calibration routines is not yet implemented in Windows
Audio Source - Menu yet working
~~Pupil Remote - not yet functional~~ fixed in latest bundle below
Updates
Updated timing function to
timeGetTime
in pyuvc in latest Capture bundlePupil Service bundle is now ready for testing 😄
Fixed multiprocessing bug for batch exporter plugin on Pupil Player
v0.8.7-w-20
fixed calibration issues, plugin window location issues
Please open issues on github if you find any bugs, so that we can incorporate changes in the next bundle/release. We hope look forward to your feedback!
Best,
The Pupil Dev Team
December 1, 2016
This release incorporates our latest round of improvements and bugfixes.
Improvements to pupil mobile capture backend.
added better initial pose for calibration visualiser.
Fixed a bug that occurred when using marker tracking and monocular 3d gaze mapping.
Fixed an overflow error.
December 1, 2016
This release incorporates our latest round of improvements and bugfixes.
Improvements to pupil mobile capture backend.
added better initial pose for calibration visualiser.
Fixed a bug that occurred when using marker tracking and monocular 3d gaze mapping.
Fixed an overflow error.
November 3, 2016
This release incorporates our latest round of improvements and bugfixes.
Improved the surface tracker performance and interface. We recommend re-running offline surface tracker on your previous recordings!
Optimised the player UI by adding categories to the growing list of plugins.
Turn on 3d detection and mapping by default.
Auto rewind the player.
Added a topic field to all data dictionaries
Minor fixes.
We hope you find these new features useful and look forward to feedback!
Best,The Pupil Dev Team
November 3, 2016
This release incorporates our latest round of improvements and bugfixes.
Improved the surface tracker performance and interface. We recommend re-running offline surface tracker on your previous recordings!
Optimised the player UI by adding categories to the growing list of plugins.
Turn on 3d detection and mapping by default.
Auto rewind the player.
Added a topic field to all data dictionaries
Minor fixes.
We hope you find these new features useful and look forward to feedback!
Best,The Pupil Dev Team
September 27, 2016
This release incorporates our latest round of improvements and bugfixes.
Add support for MacOS 10.12 "Sierra"
New normal fixation detector.
Improved user interface for
Pupil Remote
settings.Implemented z-ordering of menus in Pupil Player.
Added Icons for
Thumb
elements in Pupil Player.Added realtime fixation detector.
Minor fixes.
We hope you find these new features useful and look forward to feedback!
Best,The Pupil Dev Team
September 27, 2016
This release incorporates our latest round of improvements and bugfixes.
Add support for MacOS 10.12 "Sierra"
New normal fixation detector.
Improved user interface for
Pupil Remote
settings.Implemented z-ordering of menus in Pupil Player.
Added Icons for
Thumb
elements in Pupil Player.Added realtime fixation detector.
Minor fixes.
We hope you find these new features useful and look forward to feedback!
Best,The Pupil Dev Team
August 29, 2016
This release fixes a few issues and improves the binocular 3d gaze mapper.
August 29, 2016
This release fixes a few issues and improves the binocular 3d gaze mapper.
July 13, 2016
This release fixes a few bugs of the v0.8 release in Pupil Capture.
The surface tracker did not work with with the IPC
Inverted markers did not work when using robust detection.
Shutdown of the IPC has been improved.
Pupil Player exports work again (this was broken for MacOS only.)
July 13, 2016
This release fixes a few bugs of the v0.8 release in Pupil Capture.
The surface tracker did not work with with the IPC
Inverted markers did not work when using robust detection.
Shutdown of the IPC has been improved.
Pupil Player exports work again (this was broken for MacOS only.)
July 13, 2016
We are happy to release v0.8
of the Pupil platform!
Introduction of the IPC Backbone
To allow realtime interaction with other actors on the same machine and across machine boundaries we have completely rewritten the way Pupil exchanges data internally.
From now on all exchanged data (pupil data, control messages, log messages) flow through a message bus we call the IPC Backbone
. All messaging has been unified and is now sent using the ZMQ PUBSUB pattern. One of the greatest features of the IPC backbone is that it can be accessed from outside Pupil Capture and across the network. As an outside actor you can write to the IPC Backbone and subscribe to (parts of) the message stream.
Message Format
Messages are defined as multi-frame zmq messages. Some message topics we are using as of now are logging
, pupil
, gaze
, notify
, delayed_notify
. More info can be found here: Pupil-Interprocess-and-Network-Communication
Tapping into the IPC from outside
Any external actor (on the same box or remote) can now tap into the IPC Backbone and subscribe to all (or parts) of the message stream. It can also send messages that will be received by all subscribers in the app.
Deprecation of Pupil Server
Since this IPC Backbone is basically Pupil Server on steroids we have no need for it any more in Pupil Capture and it has been removed from the app.
This also means that the Pupil Server data format is deprecated. The IPC Backbone will give you the similar data. The only difference is that gaze, surface, and pupil data are no longer grouped. All pupil helper scripts have been updated accordingly.
Gaze Mapping
All gaze mappers have been restructured to handle temporal correlation of pupil data explicitly. Two base classes for mono and binocular mappers have been added.
Plugin API changes
Plugins will continue working as before. There are no plugin API changes except delayed notifications. See plugin.py
for details.
Plugin notifications are now published on the IPC Backbone. Note, that all notification subjects have changed!
More structure to Notification names
Notification subjects are grouped by categories category.command_or_statement
. Example: recording.should_stop
.
New Dependencies
We use msg_pack
for message serialization
pip install msgpack-python
New App: Pupil Service
We have added a third app to the Pupil project: Pupil Service
. It is like Pupil Capture except it does not have a world video feed or GUI.
It is intended for AR and VR eye tracking setups. Pupil Service is meant to run in the background and controlled via network commands only. The service process has no GUI.
The tools introduced in the HMD-eyes project are made to work with Pupil Service
and Pupil Capture
alike.
Bugfixes and Enhancements
Tweak USB Camera bandwidth settings.
Make eye window ROI interaction more intuitive.
Use new marker design for screen marker calibration.
Renamed
surfaces
event tosurface
.Added 3d location data to surface events.
Add jump to next fixation button in Pupil Player
Added option to use inverted markers in realtime surface tracker.
We hope you find these new features useful and look forward to feedback!
Best,
The Pupil Dev Team
July 13, 2016
We are happy to release v0.8
of the Pupil platform!
Introduction of the IPC Backbone
To allow realtime interaction with other actors on the same machine and across machine boundaries we have completely rewritten the way Pupil exchanges data internally.
From now on all exchanged data (pupil data, control messages, log messages) flow through a message bus we call the IPC Backbone
. All messaging has been unified and is now sent using the ZMQ PUBSUB pattern. One of the greatest features of the IPC backbone is that it can be accessed from outside Pupil Capture and across the network. As an outside actor you can write to the IPC Backbone and subscribe to (parts of) the message stream.
Message Format
Messages are defined as multi-frame zmq messages. Some message topics we are using as of now are logging
, pupil
, gaze
, notify
, delayed_notify
. More info can be found here: Pupil-Interprocess-and-Network-Communication
Tapping into the IPC from outside
Any external actor (on the same box or remote) can now tap into the IPC Backbone and subscribe to all (or parts) of the message stream. It can also send messages that will be received by all subscribers in the app.
Deprecation of Pupil Server
Since this IPC Backbone is basically Pupil Server on steroids we have no need for it any more in Pupil Capture and it has been removed from the app.
This also means that the Pupil Server data format is deprecated. The IPC Backbone will give you the similar data. The only difference is that gaze, surface, and pupil data are no longer grouped. All pupil helper scripts have been updated accordingly.
Gaze Mapping
All gaze mappers have been restructured to handle temporal correlation of pupil data explicitly. Two base classes for mono and binocular mappers have been added.
Plugin API changes
Plugins will continue working as before. There are no plugin API changes except delayed notifications. See plugin.py
for details.
Plugin notifications are now published on the IPC Backbone. Note, that all notification subjects have changed!
More structure to Notification names
Notification subjects are grouped by categories category.command_or_statement
. Example: recording.should_stop
.
New Dependencies
We use msg_pack
for message serialization
pip install msgpack-python
New App: Pupil Service
We have added a third app to the Pupil project: Pupil Service
. It is like Pupil Capture except it does not have a world video feed or GUI.
It is intended for AR and VR eye tracking setups. Pupil Service is meant to run in the background and controlled via network commands only. The service process has no GUI.
The tools introduced in the HMD-eyes project are made to work with Pupil Service
and Pupil Capture
alike.
Bugfixes and Enhancements
Tweak USB Camera bandwidth settings.
Make eye window ROI interaction more intuitive.
Use new marker design for screen marker calibration.
Renamed
surfaces
event tosurface
.Added 3d location data to surface events.
Add jump to next fixation button in Pupil Player
Added option to use inverted markers in realtime surface tracker.
We hope you find these new features useful and look forward to feedback!
Best,
The Pupil Dev Team
May 11, 2016
We are happy to release v0.7.6
of the Pupil platform!
New Features
This is a minor release, it reflects the latest changes and bug fixes made to the pupil source code since v0.7.5.
Calibration Marker Detection
The detector has been improved to work with smaller markers like the one below
![](https://framerusercontent.com/images/mBlyc7VKsFfmBINBXWdHQxekU.png)
The simple marker above printed with a diameter of 40mm
can be used at distances of 2m
with current Pupil hardware. The old marker is still supported and will continue working.
Capture Robustification
The pupil video_backend now tries to restart the capture device after all attempts to read from the device have failed.
The world process with fall back to a fake capture if all attempts to restart and recover communications with the camera have failed. This behavior provides better feedback to the user than a shutdown of the capture program.
The world process will stop recording when the fallback to fake capture occurs.
Revived Plugin: Pupil Remote
Motivated by #352 and conversations with pupil community members ( @papr, @cpicanco, and others ).
A new plugin that offers a simple way of controlling Pupil from non python environments like Matlab.
It handles requests in a separate thread to reduce latency. (As opposed to the initial version posted in #352)
See: https://github.com/pupil-labs/pupil-helpers/blob/master/pupil_remote/pupil_remote_controls.py for an example of how to control Pupil Capture via Pupil Remote.
Improved Logging
All Pupil Capture logging is now globally received by the world process, displayed in the world window, and saved to
capture.log
New Plugin:
Log History
shows the last 50 log messages in the world window.
We hope you find these new features useful and look forward to feedback!
Best,
The Pupil Dev Team
May 11, 2016
We are happy to release v0.7.6
of the Pupil platform!
New Features
This is a minor release, it reflects the latest changes and bug fixes made to the pupil source code since v0.7.5.
Calibration Marker Detection
The detector has been improved to work with smaller markers like the one below
![](https://framerusercontent.com/images/mBlyc7VKsFfmBINBXWdHQxekU.png)
The simple marker above printed with a diameter of 40mm
can be used at distances of 2m
with current Pupil hardware. The old marker is still supported and will continue working.
Capture Robustification
The pupil video_backend now tries to restart the capture device after all attempts to read from the device have failed.
The world process with fall back to a fake capture if all attempts to restart and recover communications with the camera have failed. This behavior provides better feedback to the user than a shutdown of the capture program.
The world process will stop recording when the fallback to fake capture occurs.
Revived Plugin: Pupil Remote
Motivated by #352 and conversations with pupil community members ( @papr, @cpicanco, and others ).
A new plugin that offers a simple way of controlling Pupil from non python environments like Matlab.
It handles requests in a separate thread to reduce latency. (As opposed to the initial version posted in #352)
See: https://github.com/pupil-labs/pupil-helpers/blob/master/pupil_remote/pupil_remote_controls.py for an example of how to control Pupil Capture via Pupil Remote.
Improved Logging
All Pupil Capture logging is now globally received by the world process, displayed in the world window, and saved to
capture.log
New Plugin:
Log History
shows the last 50 log messages in the world window.
We hope you find these new features useful and look forward to feedback!
Best,
The Pupil Dev Team
April 19, 2016
We are happy to release v0.7.5
of the Pupil platform!
New Features
This is a minor release, it reflects the latest changes and bug fixes made to the pupil source code since v0.7.4.
Surface Tracker
We have improved the surface tracker.
Renamed
marker detector
tosurface tracker
.'min_markers_perimeter' is settable in the offline surface tracker.
Individual markers can now be added and removed from surfaces.
The UI has been improved.
A few bugs in the marker detector have been fixed.
The surface tracker now compensates for lens distortion.
3D gaze mapping data
3D gaze mappers now report 3d gaze data.
Performance improvements
Thanks to @mikeboers work a critical memory leak in PyAV has been fixed: Pupil Player and Pupil Capture now use considerably less memory on multiprocessor linux machines.
Small improvements and bug fixes
Improved notifications for recording and calibration.
Small tweaks and bug fixes.
We hope you find these new features useful and look forward to feedback!
Best,
The Pupil Dev Team
April 19, 2016
We are happy to release v0.7.5
of the Pupil platform!
New Features
This is a minor release, it reflects the latest changes and bug fixes made to the pupil source code since v0.7.4.
Surface Tracker
We have improved the surface tracker.
Renamed
marker detector
tosurface tracker
.'min_markers_perimeter' is settable in the offline surface tracker.
Individual markers can now be added and removed from surfaces.
The UI has been improved.
A few bugs in the marker detector have been fixed.
The surface tracker now compensates for lens distortion.
3D gaze mapping data
3D gaze mappers now report 3d gaze data.
Performance improvements
Thanks to @mikeboers work a critical memory leak in PyAV has been fixed: Pupil Player and Pupil Capture now use considerably less memory on multiprocessor linux machines.
Small improvements and bug fixes
Improved notifications for recording and calibration.
Small tweaks and bug fixes.
We hope you find these new features useful and look forward to feedback!
Best,
The Pupil Dev Team
March 24, 2016
We are happy to release v.0.7
of the Pupil platform! These notes are for new features introduced in v0.7
and v0.7.3
. and serve as the official v0.7
release notes.
Contributors
We want to thank everybody that has contributed! Be it in code, bug reports or feature request and feedback! Pupil would not be possible without you! Check out the list of contributors here.
New features
Pupil Detection
Ported the existing detector to C++
and improved its performance. The detector CPU load has been cut in half!v0.7 introduces our new 3d model based pupil detector. We call it 3d detector
and renamed the old detector 2d detector
.
3d Pupil Detector
The old pupil detector and gaze mapper hit the limits of regression based calibration: It assumed the headset was "screwed to the head". When the headset moves the gaze mapping shifts. This is usually referred to as drift
.Additionally, a model-less search for the ellipse yields poor results when the pupil is partially obstructed by reflections, eyelashes, and/or eyelids.
With the use of a 3d model of the eye and a pinhole model of the camera based on Swirski’s work in “A fully-automatic, temporal approach to single camera, glint-free 3d eye model fitting - PETMEI 2013 we model the eyeball as a sphere and the pupil as a disk on the sphere. The sphere size used is based on an average human eyeball diameter of 24mm. The state of the model is the position of the sphere in eye camera space and two rotation vectors that describe the location of the pupil on the sphere.
Using temporal constraints and competing eye models we can detect and compensate for slippage events when 2d pupil evidence is strong. In case of weak 2d evidence we can use constraints from existing models to robustly fit pupils with considerably less evidence than before.
Using the 3d model we can now compute some exciting and very useful properties from the eye video footage:
3d pupil normal: The normal vector of the pupil in relation to the 3d location of the eye camera.
3d pupil diameter: Using the 3d model we correct the pupil diameter for perspective scale and transform it to millimeter (mm) space using the assumption that the eye ball size has a diameter of 24mm.
3d sphere location: The location of the eyeball in relation to the eye camera.
Augmented 2d detector confidence: Using a strong 3d model we have a better handle on false positives.
This new 3d pupil detector still has areas that we will continue to work on, but we feel that it already outperforms the 2d detector and recommend giving it a try!
3d Gaze Mapper & Calibration
With a 3d location of the eye and 3d vectors of gaze we don’t have to rely on a generic polynomial or similar for gaze mapping.
Instead we use a geometric gaze mapping approach: We model the world camera as a pinhole camera with distortion, and project pupil line of sight vectors onto the world image. For this we need to know the rotation translation of world and eye camera. This rigid transformation is obtained in a 5 point calibration routine. Under the hood we use a custom bundle adjustment routine.
Since we are now detecting and modeling the location of the eyeball we are taking movement of the headset into account. This means that the 3d model in combination with the geometric gaze mapper can compensate for slippage. You can take off the headset and put it back on without having to re-calibrate! We are still working on this feature, and you can expect this to become better over time.
For robust 3d pupil detection and fast slippage compensation we found that detecting the eye(s) at 90 Hz or greater is very helpful.
Restructured Pupil Server & Updated pupil-helper
scripts
Pupil server now sends
JSON
data, that conforms to a standard format structured topic.Every subject in the events like
pupil
andgaze
data is broadcast as a separate topic.Motivated by @zenithlight’s pull request: https://github.com/pupil-labs/pupil/pull/326
See updated pupil (and much easier to use) helper scripts: https://github.com/pupil-labs/pupil-helpers
Introducing Plugin Notifications
This scheme allows plugins to notify their in-app environment and when using pupil sync to notify other programs and machines.
This simplifies developing more intertwined behaviour while keeping plugins isolated.
Plugin notifications that carry the flag
record
are saved during recording and can be retrieved.
Pupil Sync
This is not a new feature but we added some cool new stuff!
Added proper network time synchronization for multiple source and multi user synchronous recordings. Check out the updated pupil sync examples: https://github.com/pupil-labs/pupil-helpers/tree/master/pupil_sync -- check out a short video here: https://youtu.be/2a6awbn9Cao
Plugin notifications that carry the flag
network propagate
are shared with all other Pupil Sync nodes.
Full Support for Binocular Operation
Start and stop left and right eye during runtime.
2d and 3d gaze mappers support binocular mapping.
Better Visual Feedback in Pupil Capture
Pupil and Gaze confidence is represented as transparency in the gaze dot in the world window and the pupil dot in the eye window(s). More opaque dot = higher confidence.
Changes to the Recording Format
We have depreciated the pupil_positions.npy
and gaze_postions.npy
files and now record all events and notifications that happen during recording into the pupil_data file. pupil_data
is a pickled python dictionary. All raw data can be easily exported as .csv
from Pupil Player :point_down:
Exporting Data
We have restructured export logic in Pupil Player.
Now you can press
e
or click the UI buttone
and all open plugins that have export capability will export.All exports are separated from your raw data and contained in the
exports
directory. Theexports
directory will live within your recording folder. Structural change prompted by @cpicanco's issue.We have added an exporter in Pupil Player that will give you access to all raw data that was acquired during the recording.
Bugfixes and Small improvements
GLFW
MacOS fixes for render issues and crashesPlayer GUI design is more consistent
Various
pyglui
fixes and improvementsAudio modes are settable
Camera default values for 120fps capture
Fixed bugs in libuvc and
pyuvc
that appeared when using LogitechC525/B525
cameras.Fixed bug in Pupil Player that prevented exports when world.wav file was present.
Licence Change
On 2016-12-15 we transitioned to a new license. Pupil is now licensed under LGPLv3. For more information see the License page on the wiki.
We hope you find these new features useful and look forward to feedback!
Best,
The Pupil Dev Team
March 24, 2016
We are happy to release v.0.7
of the Pupil platform! These notes are for new features introduced in v0.7
and v0.7.3
. and serve as the official v0.7
release notes.
Contributors
We want to thank everybody that has contributed! Be it in code, bug reports or feature request and feedback! Pupil would not be possible without you! Check out the list of contributors here.
New features
Pupil Detection
Ported the existing detector to C++
and improved its performance. The detector CPU load has been cut in half!v0.7 introduces our new 3d model based pupil detector. We call it 3d detector
and renamed the old detector 2d detector
.
3d Pupil Detector
The old pupil detector and gaze mapper hit the limits of regression based calibration: It assumed the headset was "screwed to the head". When the headset moves the gaze mapping shifts. This is usually referred to as drift
.Additionally, a model-less search for the ellipse yields poor results when the pupil is partially obstructed by reflections, eyelashes, and/or eyelids.
With the use of a 3d model of the eye and a pinhole model of the camera based on Swirski’s work in “A fully-automatic, temporal approach to single camera, glint-free 3d eye model fitting - PETMEI 2013 we model the eyeball as a sphere and the pupil as a disk on the sphere. The sphere size used is based on an average human eyeball diameter of 24mm. The state of the model is the position of the sphere in eye camera space and two rotation vectors that describe the location of the pupil on the sphere.
Using temporal constraints and competing eye models we can detect and compensate for slippage events when 2d pupil evidence is strong. In case of weak 2d evidence we can use constraints from existing models to robustly fit pupils with considerably less evidence than before.
Using the 3d model we can now compute some exciting and very useful properties from the eye video footage:
3d pupil normal: The normal vector of the pupil in relation to the 3d location of the eye camera.
3d pupil diameter: Using the 3d model we correct the pupil diameter for perspective scale and transform it to millimeter (mm) space using the assumption that the eye ball size has a diameter of 24mm.
3d sphere location: The location of the eyeball in relation to the eye camera.
Augmented 2d detector confidence: Using a strong 3d model we have a better handle on false positives.
This new 3d pupil detector still has areas that we will continue to work on, but we feel that it already outperforms the 2d detector and recommend giving it a try!
3d Gaze Mapper & Calibration
With a 3d location of the eye and 3d vectors of gaze we don’t have to rely on a generic polynomial or similar for gaze mapping.
Instead we use a geometric gaze mapping approach: We model the world camera as a pinhole camera with distortion, and project pupil line of sight vectors onto the world image. For this we need to know the rotation translation of world and eye camera. This rigid transformation is obtained in a 5 point calibration routine. Under the hood we use a custom bundle adjustment routine.
Since we are now detecting and modeling the location of the eyeball we are taking movement of the headset into account. This means that the 3d model in combination with the geometric gaze mapper can compensate for slippage. You can take off the headset and put it back on without having to re-calibrate! We are still working on this feature, and you can expect this to become better over time.
For robust 3d pupil detection and fast slippage compensation we found that detecting the eye(s) at 90 Hz or greater is very helpful.
Restructured Pupil Server & Updated pupil-helper
scripts
Pupil server now sends
JSON
data, that conforms to a standard format structured topic.Every subject in the events like
pupil
andgaze
data is broadcast as a separate topic.Motivated by @zenithlight’s pull request: https://github.com/pupil-labs/pupil/pull/326
See updated pupil (and much easier to use) helper scripts: https://github.com/pupil-labs/pupil-helpers
Introducing Plugin Notifications
This scheme allows plugins to notify their in-app environment and when using pupil sync to notify other programs and machines.
This simplifies developing more intertwined behaviour while keeping plugins isolated.
Plugin notifications that carry the flag
record
are saved during recording and can be retrieved.
Pupil Sync
This is not a new feature but we added some cool new stuff!
Added proper network time synchronization for multiple source and multi user synchronous recordings. Check out the updated pupil sync examples: https://github.com/pupil-labs/pupil-helpers/tree/master/pupil_sync -- check out a short video here: https://youtu.be/2a6awbn9Cao
Plugin notifications that carry the flag
network propagate
are shared with all other Pupil Sync nodes.
Full Support for Binocular Operation
Start and stop left and right eye during runtime.
2d and 3d gaze mappers support binocular mapping.
Better Visual Feedback in Pupil Capture
Pupil and Gaze confidence is represented as transparency in the gaze dot in the world window and the pupil dot in the eye window(s). More opaque dot = higher confidence.
Changes to the Recording Format
We have depreciated the pupil_positions.npy
and gaze_postions.npy
files and now record all events and notifications that happen during recording into the pupil_data file. pupil_data
is a pickled python dictionary. All raw data can be easily exported as .csv
from Pupil Player :point_down:
Exporting Data
We have restructured export logic in Pupil Player.
Now you can press
e
or click the UI buttone
and all open plugins that have export capability will export.All exports are separated from your raw data and contained in the
exports
directory. Theexports
directory will live within your recording folder. Structural change prompted by @cpicanco's issue.We have added an exporter in Pupil Player that will give you access to all raw data that was acquired during the recording.
Bugfixes and Small improvements
GLFW
MacOS fixes for render issues and crashesPlayer GUI design is more consistent
Various
pyglui
fixes and improvementsAudio modes are settable
Camera default values for 120fps capture
Fixed bugs in libuvc and
pyuvc
that appeared when using LogitechC525/B525
cameras.Fixed bug in Pupil Player that prevented exports when world.wav file was present.
Licence Change
On 2016-12-15 we transitioned to a new license. Pupil is now licensed under LGPLv3. For more information see the License page on the wiki.
We hope you find these new features useful and look forward to feedback!
Best,
The Pupil Dev Team
January 26, 2016
This release fixes a bug that crippled the eye model lifecycle management.
January 26, 2016
This release fixes a bug that crippled the eye model lifecycle management.
January 23, 2016
This is the biggest release of Pupil to-date!
Some details below:
Limits of regression based calibration: It assumes the headset is "screwed to the head". You need to compensate for the movement of the headset. Addionally, a model-less search for the ellipse yields poor results when the pupil is partially obstructed by reflections or eyelashes or eyelids.With the use of a 3D model of the eye and a pinhole model of the camera based on Swirski’s work in [“A fully-automatic, temporal approach to single camera, glint-free 3D eye model fitting” PETMEI 2013] we can model the eyeball as a sphere and the pupil as a disk on that sphere. The sphere used is based on an average human eyeball diameter is 24mm. The state of the model is the position of the sphere in eye camera space and two rotation vectors that describe the location of the pupil on the sphere.
Using temporal constraints and competing eye models we can detect and compensate for slippage events when 2D pupil evidence is strong. In case of weak 2D evidence we can use constraint from existing models to robustly fit pupils with considerably less evidence than before.
With a 3D location of the eye and 3D vectors of gaze we don’t have to rely of polynomials for gaze mapping. Instead we use a geometric gaze mapping approach. We model the world camera as a pinhole camera with distortion, and project pupil line of sight vectors onto the world image. For this we need to know the rotation translation of world and eye camera. This rigid transformation is obtained in a 3 point calibration routine. At the time of writing we simply assume that the targets are equally far away and minimize the distance of the obtained point pairs. This will be extended to infer distances during calibration.
January 23, 2016
This is the biggest release of Pupil to-date!
Some details below:
Limits of regression based calibration: It assumes the headset is "screwed to the head". You need to compensate for the movement of the headset. Addionally, a model-less search for the ellipse yields poor results when the pupil is partially obstructed by reflections or eyelashes or eyelids.With the use of a 3D model of the eye and a pinhole model of the camera based on Swirski’s work in [“A fully-automatic, temporal approach to single camera, glint-free 3D eye model fitting” PETMEI 2013] we can model the eyeball as a sphere and the pupil as a disk on that sphere. The sphere used is based on an average human eyeball diameter is 24mm. The state of the model is the position of the sphere in eye camera space and two rotation vectors that describe the location of the pupil on the sphere.
Using temporal constraints and competing eye models we can detect and compensate for slippage events when 2D pupil evidence is strong. In case of weak 2D evidence we can use constraint from existing models to robustly fit pupils with considerably less evidence than before.
With a 3D location of the eye and 3D vectors of gaze we don’t have to rely of polynomials for gaze mapping. Instead we use a geometric gaze mapping approach. We model the world camera as a pinhole camera with distortion, and project pupil line of sight vectors onto the world image. For this we need to know the rotation translation of world and eye camera. This rigid transformation is obtained in a 3 point calibration routine. At the time of writing we simply assume that the targets are equally far away and minimize the distance of the obtained point pairs. This will be extended to infer distances during calibration.
January 5, 2016
Updated glfw binary to fix fullscreen crashes.Testing new icon design.
January 5, 2016
Updated glfw binary to fix fullscreen crashes.Testing new icon design.
January 4, 2016
increase export video quality.
January 4, 2016
increase export video quality.
December 31, 2015
This is an incremental release that solves many issues. Note that the Windows pre-releases are not fully up-to-date with the Linux or Mac releases.
This bundle is based on commit 5a27c0bbd5c9abacab10efad4172d09f25e91043
The requirements to run the Windows bundle can be found in the Windows documentation
December 31, 2015
This is an incremental release that solves many issues. Note that the Windows pre-releases are not fully up-to-date with the Linux or Mac releases.
This bundle is based on commit 5a27c0bbd5c9abacab10efad4172d09f25e91043
The requirements to run the Windows bundle can be found in the Windows documentation
December 22, 2015
This is an incremental release containing enhancements and bugfixes.
added annotations to pupil capture
fixed numerous bugs.
Mac bundles are now code signed
December 22, 2015
This is an incremental release containing enhancements and bugfixes.
added annotations to pupil capture
fixed numerous bugs.
Mac bundles are now code signed
December 4, 2015
This release has a number of small bugfixes and enhancements.
December 4, 2015
This release has a number of small bugfixes and enhancements.
October 30, 2015
This is an incremental release. It contains enhancements and bug fixes.
October 30, 2015
This is an incremental release. It contains enhancements and bug fixes.
September 24, 2015
This is an incremental release of Pupil Capture for Linux and MacOS and Pupil Player for Linux
Shader based color conversion saves 30% CPU time in the world process.
Audio Capture has been properly implemented for Mac and Linux
Video Capture has been optimised to reduce CPU load when using
more CPU smaller file
setting.Various smaller fixes
All media file IO is done with PyAV
September 24, 2015
This is an incremental release of Pupil Capture for Linux and MacOS and Pupil Player for Linux
Shader based color conversion saves 30% CPU time in the world process.
Audio Capture has been properly implemented for Mac and Linux
Video Capture has been optimised to reduce CPU load when using
more CPU smaller file
setting.Various smaller fixes
All media file IO is done with PyAV
September 9, 2015
For all Windows users of the Pupil Community we present the Windows bundle v0.6 that solves many issues and that introduces some new features.
The most important feature we want to put emphasize on is the camera selection.
![camera_menu](https://framerusercontent.com/images/PZ4C5nbHeIfaBivv4uSk6ZSpeg.png)
The requirements to run the Windows bundle can be found in the Windows documentation
September 9, 2015
For all Windows users of the Pupil Community we present the Windows bundle v0.6 that solves many issues and that introduces some new features.
The most important feature we want to put emphasize on is the camera selection.
![camera_menu](https://framerusercontent.com/images/PZ4C5nbHeIfaBivv4uSk6ZSpeg.png)
The requirements to run the Windows bundle can be found in the Windows documentation
September 8, 2015
Dear Pupil Community, we are very happy to push our latest release of Pupil Capture and Player!
New Features
Pupil Sync -- Synchronise and control recording and timebase across all Pupil Capture instances running in the same network. We think this will be useful for multiuser and multi-camera setups. (You can run Pupil Capture with a webcam (we recommend c930e) to incorporate additional synchronised recordings from other viewpoints.)
Added visible logging feedback within the World window.
You can auto load your own plugin while running from the application bundle. Check out documentation on the wiki
Restructured video capture -- Pupil Capture now checks if a camera is already in use and allows to switch between all connected cameras and an additional 'fake capture'. You can now start Pupil Capture, connect the headset later and select the camera you want while the program runs. Multiple sessions of Pupil Capture can now be opened without it trying to open cameras that are already in use from a running Pupil Capture session.
Added slider enabling one to adjust the minimum marker size. This way you can use much smaller markers.
Linux Player memory leak has been fixed.
Changes for developers
Recording meta info has been made more robust
Added a
time spent between frames
message inevents
for better counters and animations.Added plugin notification scheme. Use
notify_all
to let other plugins know what your plugin does. Listen to other plugins withon_notify
. It all happens here and here
Installation of Bundles is the usual. Windows release will trail this release by about a week. Please let us know if you spot anything buggy or have great ideas for improvement (small and big).
September 8, 2015
Dear Pupil Community, we are very happy to push our latest release of Pupil Capture and Player!
New Features
Pupil Sync -- Synchronise and control recording and timebase across all Pupil Capture instances running in the same network. We think this will be useful for multiuser and multi-camera setups. (You can run Pupil Capture with a webcam (we recommend c930e) to incorporate additional synchronised recordings from other viewpoints.)
Added visible logging feedback within the World window.
You can auto load your own plugin while running from the application bundle. Check out documentation on the wiki
Restructured video capture -- Pupil Capture now checks if a camera is already in use and allows to switch between all connected cameras and an additional 'fake capture'. You can now start Pupil Capture, connect the headset later and select the camera you want while the program runs. Multiple sessions of Pupil Capture can now be opened without it trying to open cameras that are already in use from a running Pupil Capture session.
Added slider enabling one to adjust the minimum marker size. This way you can use much smaller markers.
Linux Player memory leak has been fixed.
Changes for developers
Recording meta info has been made more robust
Added a
time spent between frames
message inevents
for better counters and animations.Added plugin notification scheme. Use
notify_all
to let other plugins know what your plugin does. Listen to other plugins withon_notify
. It all happens here and here
Installation of Bundles is the usual. Windows release will trail this release by about a week. Please let us know if you spot anything buggy or have great ideas for improvement (small and big).
August 31, 2015
This release adds the v0.5.7 features as mentioned for Linux and Mac. Some Windows-only errors are solved, e.g. there's a camera selector now.
August 31, 2015
This release adds the v0.5.7 features as mentioned for Linux and Mac. Some Windows-only errors are solved, e.g. there's a camera selector now.
August 12, 2015
New Pupil Hardware support
added new Pupil Camera IDs.
improved 120fps preformance
Bug fixes
improved GUI touchpad behaviour
fixed various smaller bugs with section exports.
fixed #187
August 12, 2015
New Pupil Hardware support
added new Pupil Camera IDs.
improved 120fps preformance
Bug fixes
improved GUI touchpad behaviour
fixed various smaller bugs with section exports.
fixed #187
July 24, 2015
Beta Release
This is a beta release for Pupil Player and Capture. Pupil Player is provided as Windows bundle for the first time and might still be unstable. Please raise an issue for every problem you experience!
Usage
To use Capture and Player you have to download and extract the corresponding archive. The archive format is 7-Zip.
Run Pupil Capture by double clicking on
Pupil Capture\pupil_capture.exe
Run Pupil Player by double clicking on
Pupil Player\pupil_player.exe
. Afterwards you can drag'n'drop a record folder to the Player window from the Windows file explorer that was previously created by Pupil Capture.
July 24, 2015
Beta Release
This is a beta release for Pupil Player and Capture. Pupil Player is provided as Windows bundle for the first time and might still be unstable. Please raise an issue for every problem you experience!
Usage
To use Capture and Player you have to download and extract the corresponding archive. The archive format is 7-Zip.
Run Pupil Capture by double clicking on
Pupil Capture\pupil_capture.exe
Run Pupil Player by double clicking on
Pupil Player\pupil_player.exe
. Afterwards you can drag'n'drop a record folder to the Player window from the Windows file explorer that was previously created by Pupil Capture.
July 23, 2015
Fixed issue #178
July 23, 2015
Fixed issue #178
July 21, 2015
Updated test bundle for Pupil Capture on Windows.
July 21, 2015
Updated test bundle for Pupil Capture on Windows.
July 8, 2015
This release fixed a bug that prevented mjpeg recordings to be exported in player.
July 8, 2015
This release fixed a bug that prevented mjpeg recordings to be exported in player.
July 6, 2015
This is a quick update following release of version 0.5.
MJPEG recording is now officially implemented and supported on Mac and Linux both in bundle and running from source. Unlike version 0.5.3 we no longer rely on a ffmpeg executable to be present for this. Its back to simply download and install the bundle
.
New Feature: MJPEG direct recording
By setting bigger file, less CPU
in the recorder options, CPU load of recording is reduced to almost 0. While the file size of this recoding type is more that the MPEG4 compressed format we usually use, we can save substantial CPU time by simply saving camera jpeg frames into a .mkv container. This also yields the best possible image quality.
We really recommend this option when disk space is plentiful (most cases) while CPU load directly affects battery lifetime on mobile computers.
If you are running Pupil from source please make sure to use our fork of PyAV
July 6, 2015
This is a quick update following release of version 0.5.
MJPEG recording is now officially implemented and supported on Mac and Linux both in bundle and running from source. Unlike version 0.5.3 we no longer rely on a ffmpeg executable to be present for this. Its back to simply download and install the bundle
.
New Feature: MJPEG direct recording
By setting bigger file, less CPU
in the recorder options, CPU load of recording is reduced to almost 0. While the file size of this recoding type is more that the MPEG4 compressed format we usually use, we can save substantial CPU time by simply saving camera jpeg frames into a .mkv container. This also yields the best possible image quality.
We really recommend this option when disk space is plentiful (most cases) while CPU load directly affects battery lifetime on mobile computers.
If you are running Pupil from source please make sure to use our fork of PyAV
July 3, 2015
Pupil Capture and Player release version 0.5
We are happy to release the latest version of Pupil software. As usual the installers for Mac OS and Linux are below. Windows will follow.
New Features
New unified video capture backend: pyUVC - No more patching of Linux. Single USB support everywhere!
MJPEG recording option. (Uses no CPU but makes bigger files, currently for Linux only -Mac works when running from source).
Fixation detector for Pupil Player.
Pupil and gaze data is now also saved in a pickled file. We use this internally for Pupil Player
Now you can open a different recording in Pupil Player simply by dragging it onto an existing window.
Pupil Player now opens a blank window when started without recording. You can drag and drop a recording into this "dummy" to open a recording.
Bugfixes minor Improvements
Windows and menus always remember their size and position
Settings from previous software versions are ignored.
Many minor UI tweaks
Mac OS cursor does not hide on resize
Fixed bug that occurred on some Ubuntu machines on window resize
Simplified hdpi handling
Multiple Player export bugfixes
Various other bugfixes
Now using GLFW3.1
Updated and streamlined GLFW bindings
Updated pyglui to v0.3
Simply install the bundle below. Please raise an issue if you find a bug!
July 3, 2015
Pupil Capture and Player release version 0.5
We are happy to release the latest version of Pupil software. As usual the installers for Mac OS and Linux are below. Windows will follow.
New Features
New unified video capture backend: pyUVC - No more patching of Linux. Single USB support everywhere!
MJPEG recording option. (Uses no CPU but makes bigger files, currently for Linux only -Mac works when running from source).
Fixation detector for Pupil Player.
Pupil and gaze data is now also saved in a pickled file. We use this internally for Pupil Player
Now you can open a different recording in Pupil Player simply by dragging it onto an existing window.
Pupil Player now opens a blank window when started without recording. You can drag and drop a recording into this "dummy" to open a recording.
Bugfixes minor Improvements
Windows and menus always remember their size and position
Settings from previous software versions are ignored.
Many minor UI tweaks
Mac OS cursor does not hide on resize
Fixed bug that occurred on some Ubuntu machines on window resize
Simplified hdpi handling
Multiple Player export bugfixes
Various other bugfixes
Now using GLFW3.1
Updated and streamlined GLFW bindings
Updated pyglui to v0.3
Simply install the bundle below. Please raise an issue if you find a bug!
June 11, 2015
bug-fixes in player and capture.
June 11, 2015
bug-fixes in player and capture.
June 11, 2015
The is a pre-release. Proper release notes will follow.
June 11, 2015
The is a pre-release. Proper release notes will follow.
May 29, 2015
This is a test bundle for the new unified capture backend.
May 29, 2015
This is a test bundle for the new unified capture backend.
May 21, 2015
Bugfixes & Enhancements
Player
Manual gaze correction now applies to offline marker surfaces too
Seek bar works on retina displays
Renamed player variables to reflect changes in capture.
Capture:
Recording directory is now remembered and user settable.
Calibration routines now have user settable sample duration.
Fixed bug in Pupil remote and pupil server plugin.
May 21, 2015
Bugfixes & Enhancements
Player
Manual gaze correction now applies to offline marker surfaces too
Seek bar works on retina displays
Renamed player variables to reflect changes in capture.
Capture:
Recording directory is now remembered and user settable.
Calibration routines now have user settable sample duration.
Fixed bug in Pupil remote and pupil server plugin.
April 23, 2015
added audio capture.
April 23, 2015
added audio capture.
April 22, 2015
New test bundle with bugfixes.
April 22, 2015
New test bundle with bugfixes.
April 17, 2015
New test bundle with bugfixes.
Now able to record.
April 17, 2015
New test bundle with bugfixes.
Now able to record.
April 16, 2015
This is another incremental release with a few newly added feature and various bug fixes:
New Features
option to rotate eye image display for binocular headsets
added eye video overlay as plugin for Pupil Player
added watermark as plugin for Pupil Player
Bug Fixes / Improvements
removed unnecessary call to frame.img -> speed gain
recorder plug-in now works under windows too
ROI handles in eye window are now bigger
fixed bugs in marker detector
made version handling more robust
removed unused scripts
typo fixes in comments
make sound module failures not crash the app.
various small UI handling improvmens
April 16, 2015
This is another incremental release with a few newly added feature and various bug fixes:
New Features
option to rotate eye image display for binocular headsets
added eye video overlay as plugin for Pupil Player
added watermark as plugin for Pupil Player
Bug Fixes / Improvements
removed unnecessary call to frame.img -> speed gain
recorder plug-in now works under windows too
ROI handles in eye window are now bigger
fixed bugs in marker detector
made version handling more robust
removed unused scripts
typo fixes in comments
make sound module failures not crash the app.
various small UI handling improvmens
March 10, 2015
This is a test bundle for windows.
User deployment survey
Please try it out and raise an issue with feedback (machine,windows verison etc.) even when its working on your machine so we know!
Pupil Player Bundle
This is in the making we still need to find a way to build PyAV on windows.
March 10, 2015
This is a test bundle for windows.
User deployment survey
Please try it out and raise an issue with feedback (machine,windows verison etc.) even when its working on your machine so we know!
Pupil Player Bundle
This is in the making we still need to find a way to build PyAV on windows.
March 5, 2015
added user customisable info field for recordings.
session name can now contain
/
to create subdirectories.
March 5, 2015
added user customisable info field for recordings.
session name can now contain
/
to create subdirectories.
March 2, 2015
This update contains bug fixes and small improvements.
But most importantly from now on we have proper MacOS and Linux (Debian) installers.
Mac
Pupil Capture and Pupil Player come in .dmg files. The applications are installed in Applications
. User settings and recordings live in the user directory.
Linux
Pupil Capture and Pupil Player are installed via .deb bundles. They usually open in the OS software center and can be installed there.The application is installed to /opt
We create Desktop entries in /usr/share/applicaitons
We create a terminal shortcut in 'usr/bin'
Settings and recording live in ~/
March 2, 2015
This update contains bug fixes and small improvements.
But most importantly from now on we have proper MacOS and Linux (Debian) installers.
Mac
Pupil Capture and Pupil Player come in .dmg files. The applications are installed in Applications
. User settings and recordings live in the user directory.
Linux
Pupil Capture and Pupil Player are installed via .deb bundles. They usually open in the OS software center and can be installed there.The application is installed to /opt
We create Desktop entries in /usr/share/applicaitons
We create a terminal shortcut in 'usr/bin'
Settings and recording live in ~/
February 25, 2015
Content
February 25, 2015
Content
December 16, 2014
added batch exporting plugin
December 16, 2014
added batch exporting plugin
October 15, 2014
added option to specify recording path.
fixed linux capture backend bug.
October 15, 2014
added option to specify recording path.
fixed linux capture backend bug.
September 26, 2014
Incremental release
added Pupil Remote Plugin:
Send messages to control Pupil Capture functions using ZMQ:
'R' toggle recording
'R rec_name' toggle recording and name new recording rec_name
'T' set timebase to 0
'C' start currently selected calibration
Have a look here for additional helper scripts: https://gist.github.com/mkassner/c66bce144846358d8f4f
September 26, 2014
Incremental release
added Pupil Remote Plugin:
Send messages to control Pupil Capture functions using ZMQ:
'R' toggle recording
'R rec_name' toggle recording and name new recording rec_name
'T' set timebase to 0
'C' start currently selected calibration
Have a look here for additional helper scripts: https://gist.github.com/mkassner/c66bce144846358d8f4f
July 31, 2014
This is a proof on concept release for Pupil Player with variable frame rate video export.
Much to be done still. But this version will save out videos that have been recorded at varying frame rates with appropriate presentations timestamps.
July 31, 2014
This is a proof on concept release for Pupil Player with variable frame rate video export.
Much to be done still. But this version will save out videos that have been recorded at varying frame rates with appropriate presentations timestamps.
July 24, 2014
Added sanity check to hardware timestamps. Because on very rare occasions these can be false (and this happens on the camera).
GUI improvements for player.
July 24, 2014
Added sanity check to hardware timestamps. Because on very rare occasions these can be false (and this happens on the camera).
GUI improvements for player.
July 10, 2014
Incremental work release on Pupil Player
added surface and gaze metrics report and visualisation.
added option to use markers as trim in and trim out marks for automated export of multiple sections in one recording.
a few extra fixes.
July 10, 2014
Incremental work release on Pupil Player
added surface and gaze metrics report and visualisation.
added option to use markers as trim in and trim out marks for automated export of multiple sections in one recording.
a few extra fixes.
June 9, 2014
Fixes:
made manual marker calibration more robust.
fix draw ROI interface in eye window.
June 9, 2014
Fixes:
made manual marker calibration more robust.
fix draw ROI interface in eye window.
May 21, 2014
We are pleased to announce another release of Pupil Player and Pupil Capture Software. We usually aim to push new features every month, but we had been working on our technical report and improved Pupil Pro Headsets (soon to be available, we are really excited!), thus skipping one release date. Here the list of new features and fixes:
New Features and Bugfiexs Pupil Capture:
Hardware time stamping: Running Pupil Pro with Linux now uses hardware timestamps taken by the camera hardware at the start of exposure. Allowing temporal correlation of world and eye cam at unprecedented accuracy.
Audio capture. On Mac OS (requires 'brew install sox') and Ubuntu (just works) we can now record audio alongside video. Audio sources are selectable as a drop-down menu. Saved as world.wav file alongside world.avi. This is still quite beta and we don't do audio playback in player yet.
Frame rate for world camera is now forced to 24/30fps irrespective of environment lighting conditions.
Various small GUI tweaks and fixes
Added named textures for easy upload and re-use on GPU.
Settings files and recording data are now cross platform compatible.
New Features and Bugfixes Pupil Player:
Offline Marker Tracker: A new plugin as been added, the offline marker tracker. We had gotten many requests for this and hope that you will find it useful! Functionally similar to the Capture marker tracker, this plugin searches all frames in the background and adds useful extras for analysis. Check out the heat-maps and data export! Have a a look at our demo video. Try it out in the latest Pupil player with our demo dataset.
Various small GUI tweaks and other bug fixes.
Settings files and recording data are now cross platform compatible.
Bundles:
Now support Ubuntu 14.04
MacOS bundles still have some issues with older hardware. 2012+ MacBooks appear to be fine.
May 21, 2014
We are pleased to announce another release of Pupil Player and Pupil Capture Software. We usually aim to push new features every month, but we had been working on our technical report and improved Pupil Pro Headsets (soon to be available, we are really excited!), thus skipping one release date. Here the list of new features and fixes:
New Features and Bugfiexs Pupil Capture:
Hardware time stamping: Running Pupil Pro with Linux now uses hardware timestamps taken by the camera hardware at the start of exposure. Allowing temporal correlation of world and eye cam at unprecedented accuracy.
Audio capture. On Mac OS (requires 'brew install sox') and Ubuntu (just works) we can now record audio alongside video. Audio sources are selectable as a drop-down menu. Saved as world.wav file alongside world.avi. This is still quite beta and we don't do audio playback in player yet.
Frame rate for world camera is now forced to 24/30fps irrespective of environment lighting conditions.
Various small GUI tweaks and fixes
Added named textures for easy upload and re-use on GPU.
Settings files and recording data are now cross platform compatible.
New Features and Bugfixes Pupil Player:
Offline Marker Tracker: A new plugin as been added, the offline marker tracker. We had gotten many requests for this and hope that you will find it useful! Functionally similar to the Capture marker tracker, this plugin searches all frames in the background and adds useful extras for analysis. Check out the heat-maps and data export! Have a a look at our demo video. Try it out in the latest Pupil player with our demo dataset.
Various small GUI tweaks and other bug fixes.
Settings files and recording data are now cross platform compatible.
Bundles:
Now support Ubuntu 14.04
MacOS bundles still have some issues with older hardware. 2012+ MacBooks appear to be fine.
May 17, 2014
Fixed a bug that created corrupted hardware timestamps in version v0.3.8.2
May 17, 2014
Fixed a bug that created corrupted hardware timestamps in version v0.3.8.2
April 23, 2014
Added option to record audio.
Made bundle compatible with Ubuntu 14.04 LTS
April 23, 2014
Added option to record audio.
Made bundle compatible with Ubuntu 14.04 LTS
April 7, 2014
This is a Pupil Capture test bundle for MacOS 10.9 'Mavericks'
April 7, 2014
This is a Pupil Capture test bundle for MacOS 10.9 'Mavericks'
March 25, 2014
New features for Pupil Capture
Adjustable timebase for capture. We've changed the reference time point for timestamps to now.
Screen marker based accuracy test. Enables one to evaluate gaze accuracy and precision. (See screenshot below)
Added option to disable the coarse pupil detector, as it may fail when the subject wears mascara.
Detector variables have be tweaked to improve detector effectiveness and speed.
Camera capture is now more robust. If the camera should fail while Pupil Capture is running, (e.g. if a USB cable is disconnected) the software will clean up any ongoing action (recordings etc.) and do a proper shutdown so that data is preserved.
Capture app starts even without cameras connected. When cameras are not detected Pupil Capture software still starts, but respective windows without cameras will display a test pattern (random color squares).
Added option to disable screen updates for world and eye window. This helps weak systems record without dropping frame rate.
Retina screens are now supported. The mouse cursor is no longer offset.
Pupil Server to OSC bridge script. After getting a lot of requests for OSC streams, we made a small script to allow Pupil to interface with OSC devices. Script can be found in this gist.
Performance improvements for Pupil Capture
Various small improvements have been made as we are trying to make pupil capture run on smaller and cheaper laptops. Using Ubuntu 13.10 we are now able to make recordings with Pupil Capture on a 2010 11inch Macbook Air. When disabling screen updates we are no longer dropping frames. The CPU of our test machine is a Intel Core2Duo SU9400 -- this means that Pupil should run on any "i" type intel processor now.
This test was done using Linux, we know that some internals on MacOS are slightly less efficient so you may need a little bit more steam if you run MacOS (any recent Mac will do fine though.) If you have tested minimum specs for Windows please send us an update!
New features for Pupil Player
Marker based surface tracker, has been added but still need some visualization methods
Added a simple fixations filter. Current simple filter, although not scientific, will make the data look better by filtering data based on distance between points. We are currently working on a proper fixation and saccade classifier -- stay tuned!
Added cross as gaze visualization.
Added a simple / hacky gaze offset plugin.
Added pupil server plugin.
Small fixes and improvements
Pupil Player now closes via GUI.
The capture class now has proper exceptions
Texture upload have be optimized for capture and player
Pupil Player meta info reader is more robust
Fixed a bug that messed up the log files on MacOS
Fixed a bug that led to crashes when entering fullscreen mode.
Pupil Player now avoids export overwrites by adding a timestamp to the out file name
Many small GUI improvements and more documentation strings
Player plugins can no longer make permanent changes to recent pupil positions.
Fixed a bug in capture selection list on Linux
Pupil player plugins now remember their GUI state.
Added support for Logitech c510 camera
Pupil Player exporter now logs is a more proper way
Various minor fixes and tweaks
Bundle support
Ubuntu has been working quite well for a while now, so no real news here.
Various fixes have been made to the MacOS bundle for player and capture. Everything should work now. (If not please let us know!)
March 25, 2014
New features for Pupil Capture
Adjustable timebase for capture. We've changed the reference time point for timestamps to now.
Screen marker based accuracy test. Enables one to evaluate gaze accuracy and precision. (See screenshot below)
Added option to disable the coarse pupil detector, as it may fail when the subject wears mascara.
Detector variables have be tweaked to improve detector effectiveness and speed.
Camera capture is now more robust. If the camera should fail while Pupil Capture is running, (e.g. if a USB cable is disconnected) the software will clean up any ongoing action (recordings etc.) and do a proper shutdown so that data is preserved.
Capture app starts even without cameras connected. When cameras are not detected Pupil Capture software still starts, but respective windows without cameras will display a test pattern (random color squares).
Added option to disable screen updates for world and eye window. This helps weak systems record without dropping frame rate.
Retina screens are now supported. The mouse cursor is no longer offset.
Pupil Server to OSC bridge script. After getting a lot of requests for OSC streams, we made a small script to allow Pupil to interface with OSC devices. Script can be found in this gist.
Performance improvements for Pupil Capture
Various small improvements have been made as we are trying to make pupil capture run on smaller and cheaper laptops. Using Ubuntu 13.10 we are now able to make recordings with Pupil Capture on a 2010 11inch Macbook Air. When disabling screen updates we are no longer dropping frames. The CPU of our test machine is a Intel Core2Duo SU9400 -- this means that Pupil should run on any "i" type intel processor now.
This test was done using Linux, we know that some internals on MacOS are slightly less efficient so you may need a little bit more steam if you run MacOS (any recent Mac will do fine though.) If you have tested minimum specs for Windows please send us an update!
New features for Pupil Player
Marker based surface tracker, has been added but still need some visualization methods
Added a simple fixations filter. Current simple filter, although not scientific, will make the data look better by filtering data based on distance between points. We are currently working on a proper fixation and saccade classifier -- stay tuned!
Added cross as gaze visualization.
Added a simple / hacky gaze offset plugin.
Added pupil server plugin.
Small fixes and improvements
Pupil Player now closes via GUI.
The capture class now has proper exceptions
Texture upload have be optimized for capture and player
Pupil Player meta info reader is more robust
Fixed a bug that messed up the log files on MacOS
Fixed a bug that led to crashes when entering fullscreen mode.
Pupil Player now avoids export overwrites by adding a timestamp to the out file name
Many small GUI improvements and more documentation strings
Player plugins can no longer make permanent changes to recent pupil positions.
Fixed a bug in capture selection list on Linux
Pupil player plugins now remember their GUI state.
Added support for Logitech c510 camera
Pupil Player exporter now logs is a more proper way
Various minor fixes and tweaks
Bundle support
Ubuntu has been working quite well for a while now, so no real news here.
Various fixes have been made to the MacOS bundle for player and capture. Everything should work now. (If not please let us know!)
March 19, 2014
release of incremental work
various fixes for hi dpi monitor support and faulty capture devices.
added options to stop screen upload to save CPU time
improvements on pupil detector settings
detailed release notes to follow...
known issues
the bundle for pupil player on mac does not export.
March 19, 2014
release of incremental work
various fixes for hi dpi monitor support and faulty capture devices.
added options to stop screen upload to save CPU time
improvements on pupil detector settings
detailed release notes to follow...
known issues
the bundle for pupil player on mac does not export.
March 7, 2014
Bundle for 64 bit Linux of apparent pupil size bug fix
Pupil apparent size was not reported when pupil was found during quick evaluation shortcut in canny detector.
March 7, 2014
Bundle for 64 bit Linux of apparent pupil size bug fix
Pupil apparent size was not reported when pupil was found during quick evaluation shortcut in canny detector.
February 26, 2014
added option to disable coarse pupil detection.
February 26, 2014
added option to disable coarse pupil detection.
February 24, 2014
Release
This is just a release of the bundles that reflect the current state. Since the last major release of Player and Capture more improvements and bug fixes have been made.
February 24, 2014
Release
This is just a release of the bundles that reflect the current state. Since the last major release of Player and Capture more improvements and bug fixes have been made.
February 11, 2014
Pupil Player
With the App we finally add the crucial missing part of the toolchain. You can now visualize Pupil Capture recordings after the fact using a proper App with GUI.
A platform for offline processing
Pupil Player uses the plugin system found in Pupil Capture. This allows developers and contributors to roll their own new awesome ideas into Pupil. Currently we have a few simple Visualizations but more powerful plugins are in the Pipeline!
Exporting
The current visualization plugins now support exporting.
Bundles
Sadly still only the Linux bundles are available. Mac and Windows will work running from source.
February 11, 2014
Pupil Player
With the App we finally add the crucial missing part of the toolchain. You can now visualize Pupil Capture recordings after the fact using a proper App with GUI.
A platform for offline processing
Pupil Player uses the plugin system found in Pupil Capture. This allows developers and contributors to roll their own new awesome ideas into Pupil. Currently we have a few simple Visualizations but more powerful plugins are in the Pipeline!
Exporting
The current visualization plugins now support exporting.
Bundles
Sadly still only the Linux bundles are available. Mac and Windows will work running from source.
February 3, 2014
Second work in progress release of Pupil Player
added plugins
all plugins are started using the atb drop down menu.
some plugins can be added more than once.
plugins do not yet work with the exporter.
February 3, 2014
Second work in progress release of Pupil Player
added plugins
all plugins are started using the atb drop down menu.
some plugins can be added more than once.
plugins do not yet work with the exporter.
January 28, 2014
This is a test release of the Pupil Player. More a proof of concept than anything else. It does work!
Bundle release
Bundle only available for Ubuntu X64 for now. MAC will follow but Quicktime is not playing along so well.
Features
seek bar with frame accurate seeking.
simple gaze dot overlay
exporter that will export the video with a simple red circle as gaze overlay.
each export runs in a separate process: this means you can continue looking at the recording and run multiple exports simultaneously
you can specify start and stop frame - much like trim marks
you can specify the file location and name
Soon to Come
scan path
marker tracking
more visualizations
bundle release for MacOS
Bundle version use
Just drag a pupil recording dir onto the launcher icon and the player will start.
Source version use
cd pupil/pupil_src/player
python main.py /path/to/data/dir/
January 28, 2014
This is a test release of the Pupil Player. More a proof of concept than anything else. It does work!
Bundle release
Bundle only available for Ubuntu X64 for now. MAC will follow but Quicktime is not playing along so well.
Features
seek bar with frame accurate seeking.
simple gaze dot overlay
exporter that will export the video with a simple red circle as gaze overlay.
each export runs in a separate process: this means you can continue looking at the recording and run multiple exports simultaneously
you can specify start and stop frame - much like trim marks
you can specify the file location and name
Soon to Come
scan path
marker tracking
more visualizations
bundle release for MacOS
Bundle version use
Just drag a pupil recording dir onto the launcher icon and the player will start.
Source version use
cd pupil/pupil_src/player
python main.py /path/to/data/dir/
January 19, 2014
This release introduces a few improvements.
Manual Marker Calibration
We now have two markers both can be printed out from the pdf below.
circular calibration marker
: Used to calibrate as usual. Check the Pupil Github wiki for instructions.stop calibration marker
: Similar in looks but with white center. Show this to the scene camera and the calibration will stop.
We added a progress indicator so the operator can see when they are collecting samples. We also improved detection robustness as some users had trouble with false positives.
Selectable Capture
During runtime you can now select which camera to capture from (Mac&Linux only).This is integrated into the GUI via a drop down menu. Known issues: On Linux you should not select the same capture in more than one window simultaneously (crash).
Support for new Pupil Pro Headset.
A few tweaks have been made to enable the use of our soon to be released improved Pupil Pro mobile eye tracking headset. Of course doing this we did not drop any other hardware from the support list.
Minor bugfix and tweaks
See commits.
January 19, 2014
This release introduces a few improvements.
Manual Marker Calibration
We now have two markers both can be printed out from the pdf below.
circular calibration marker
: Used to calibrate as usual. Check the Pupil Github wiki for instructions.stop calibration marker
: Similar in looks but with white center. Show this to the scene camera and the calibration will stop.
We added a progress indicator so the operator can see when they are collecting samples. We also improved detection robustness as some users had trouble with false positives.
Selectable Capture
During runtime you can now select which camera to capture from (Mac&Linux only).This is integrated into the GUI via a drop down menu. Known issues: On Linux you should not select the same capture in more than one window simultaneously (crash).
Support for new Pupil Pro Headset.
A few tweaks have been made to enable the use of our soon to be released improved Pupil Pro mobile eye tracking headset. Of course doing this we did not drop any other hardware from the support list.
Minor bugfix and tweaks
See commits.
December 2, 2013
Attempting to fix missing frames issue on Linux machines.
December 2, 2013
Attempting to fix missing frames issue on Linux machines.
November 6, 2013
New Pupil Capture Routine release:
Improved Pupil Detection:
The new Pupil detector is more sophisticated and should preform much better. The False Positive rate (ghost pupils) has been dramatically reduced.
Pupil detection now pre-filters contours based on curvature properties.
Ellipse fitting now utilizes fitting error to discard bad ellipses
Our detector now matches and combines contour segments to find a maximally supported ellipse.
The result is now augmented with a confidence measure (from 0 to 1).
Improved Point Display
GL points now have custom shaders for improved display. This means that Linux users that experienced square points should now see proper circle points.
Proper Logging
Pupil now uses a proper logging system and stream information into logfiles in the user_settings dir.
Bugfixes
A few small bugs have been smashed.
Installation using the bundles
If you want to just use Pupil Software and don't need to make changes to the source code, you can download and run our latest standalone Pupil app bundles below.
bundle installation Linux:
download pupil ubuntu zipped bundle
unzip to location of choice
run make_shortcut.sh
move shortcut to launcher and wherever else you want
done
User Directories on Linux:
On Linux your data will be next to the pupil bundle folder.
bundle installation Mac:
download pupil mac zipped file
unzip to location of choice
User directories on Mac:
On Mac Pupil will create two directories on your Desktop:
pupil_settings
pupil_recordings
simple player application for mac
Because you need some way to look at data and we are working on a proper player. Here a minimalistic app to play pupil data.
install Pupil Simple Player
drag a recording folder onto the app (the one that contains the actual files)
the app will start and play back your recording
a new video with overlay will be saved in the recoding dir
November 6, 2013
New Pupil Capture Routine release:
Improved Pupil Detection:
The new Pupil detector is more sophisticated and should preform much better. The False Positive rate (ghost pupils) has been dramatically reduced.
Pupil detection now pre-filters contours based on curvature properties.
Ellipse fitting now utilizes fitting error to discard bad ellipses
Our detector now matches and combines contour segments to find a maximally supported ellipse.
The result is now augmented with a confidence measure (from 0 to 1).
Improved Point Display
GL points now have custom shaders for improved display. This means that Linux users that experienced square points should now see proper circle points.
Proper Logging
Pupil now uses a proper logging system and stream information into logfiles in the user_settings dir.
Bugfixes
A few small bugs have been smashed.
Installation using the bundles
If you want to just use Pupil Software and don't need to make changes to the source code, you can download and run our latest standalone Pupil app bundles below.
bundle installation Linux:
download pupil ubuntu zipped bundle
unzip to location of choice
run make_shortcut.sh
move shortcut to launcher and wherever else you want
done
User Directories on Linux:
On Linux your data will be next to the pupil bundle folder.
bundle installation Mac:
download pupil mac zipped file
unzip to location of choice
User directories on Mac:
On Mac Pupil will create two directories on your Desktop:
pupil_settings
pupil_recordings
simple player application for mac
Because you need some way to look at data and we are working on a proper player. Here a minimalistic app to play pupil data.
install Pupil Simple Player
drag a recording folder onto the app (the one that contains the actual files)
the app will start and play back your recording
a new video with overlay will be saved in the recoding dir
October 9, 2013
Release of bundled version of the first bilateral version of pupil. No correlation done at this point, just dual eye pupil detection.
See our wiki for bandwidth patches if you are short on usb controllers.
October 9, 2013
Release of bundled version of the first bilateral version of pupil. No correlation done at this point, just dual eye pupil detection.
See our wiki for bandwidth patches if you are short on usb controllers.
October 8, 2013
changes
multiple bugfixes for improved stability
slight UI tweaks
changes versioning naming convention to: vX.Y.Z (Major, Minor, Development Iteration/Hot-fix)
bundle installation Linux:
download pupil ubuntu zipped bundle
unzip to location of choice
run make_shortcut.sh
move shortcut to launcher and wherever else you want
done
User Directories on Linux:
On Linux your data will be next to the pupil bundle folder.
bundle installation Mac:
download pupil mac zipped file
unzip to location of choice
running on Mac:
We have one bug that prevents us from running the app without a terminal, here is the tmp workaround
Open "run_pupil_capture_from_mac_terminal" to start Pupil Capture
User directories on Mac:
On Mac Pupil will create two directories on your Desktop:
pupil_settings
pupil_recordings
simple player application for mac
Because you need some way to look at data and we are working on a proper player. Here a minimalistic app to play pupil data.
install Pupil Simple Player
drag a recording folder onto the app (the one that contains the actual files)
the app will start and play back your recording
a new video with overlay will be saved in the recoding dir
October 8, 2013
changes
multiple bugfixes for improved stability
slight UI tweaks
changes versioning naming convention to: vX.Y.Z (Major, Minor, Development Iteration/Hot-fix)
bundle installation Linux:
download pupil ubuntu zipped bundle
unzip to location of choice
run make_shortcut.sh
move shortcut to launcher and wherever else you want
done
User Directories on Linux:
On Linux your data will be next to the pupil bundle folder.
bundle installation Mac:
download pupil mac zipped file
unzip to location of choice
running on Mac:
We have one bug that prevents us from running the app without a terminal, here is the tmp workaround
Open "run_pupil_capture_from_mac_terminal" to start Pupil Capture
User directories on Mac:
On Mac Pupil will create two directories on your Desktop:
pupil_settings
pupil_recordings
simple player application for mac
Because you need some way to look at data and we are working on a proper player. Here a minimalistic app to play pupil data.
install Pupil Simple Player
drag a recording folder onto the app (the one that contains the actual files)
the app will start and play back your recording
a new video with overlay will be saved in the recoding dir
October 6, 2013
This is a test release for Linux x64 only.
user settings and tmp files are now saved in "settings"
all recordings are saved in "recordings"
both folders are created when needed and reside next to the binary app/pupil_src folder when data is written by the app.
pupil capture now uses glfw3 with multi monitor and multi window support.
calibration workflow is improved:
full screen option
monitor selection
Pupil can now be fully bundled and distributed in a more app like fashion:
Pupil can be run in two ways from now on:
a) use the self contained bundle with pre-compiled application.
b) install all required python packages and run pupil "from source" using python
Option a:On linux all you need is to download the .zip file, extract it to where you wish and run 'make_shortcut.sh' to create a shortcut to start pupil_capture. This shortcut can be dragged into the launcher.
Mac to follow soon.
Option b:Please have a look at https://code.google.com/p/pupil/wiki/Installation
October 6, 2013
This is a test release for Linux x64 only.
user settings and tmp files are now saved in "settings"
all recordings are saved in "recordings"
both folders are created when needed and reside next to the binary app/pupil_src folder when data is written by the app.
pupil capture now uses glfw3 with multi monitor and multi window support.
calibration workflow is improved:
full screen option
monitor selection
Pupil can now be fully bundled and distributed in a more app like fashion:
Pupil can be run in two ways from now on:
a) use the self contained bundle with pre-compiled application.
b) install all required python packages and run pupil "from source" using python
Option a:On linux all you need is to download the .zip file, extract it to where you wish and run 'make_shortcut.sh' to create a shortcut to start pupil_capture. This shortcut can be dragged into the launcher.
Mac to follow soon.
Option b:Please have a look at https://code.google.com/p/pupil/wiki/Installation