Pupil Core releases


Pupil core

March 27, 2020

Please feel free to get in touch with feedback and questions via the #pupil channel on Discord :smile:

Improvements

Windows Driver Installation Stability - #1833

We have greatly improved the driver installation workflow on Windows. You should not experience problems with spaces or non-english characters in the path to your Pupil installation anymore. Additionally, you do no longer have to run the application as administrator. You still need an administrator to authorize the driver installation.

Changes

On-demand Loading of Multi-part Recordings - #1835

Multiple users have reported problems when opening Pupil Mobile recordings with a large number of video parts. We were able to fix this issue by loading the video parts on demand instead of opening them all at once. As a result, Pupil Player will open Pupil Mobile or Pupil Invisible recordings with multiple parts much quicker than before.

As a side-effect, during playback in Pupil Player, you might experience short frame jumps when a new video part needs to be loaded. This does not affect the underlying data or any exports.

Bug Fixes

  • Fixed a crash when changing the ROI without a connected headset - #1827
  • Fixed a crash when opening recordings with empty video parts - #1832
  • Gracefully stop recordings when receiving non-monotonic timestamps - #1829
  • Fixed an issue with calibration file storage on macOS - #1836
  • Fixed an issue with gaze history settings restoration from session settings - #1836

Developer Notes

Changed Requirements - #1834

We updated the pyndsi dependency on v1.3. On Linux and macOS, you can update with:

pip install -U git+https://github.com/pupil-labs/pyndsi

On Windows, please download and install the latest wheel from GitHub.

Removed All Direct C++ Dependencies - #1828

After the latest rework of the calibration codebase to remove C++ dependencies in v1.22, we have re-evaluated all dependencies for Pupil Core software – especially on Windows. We are very pleased to announce that we have finally externalized or gotten rid of all C++ dependencies! :tada:

This greatly simplifies the developer setup on Windows. The biggest changes include:

  • You no longer have to install Visual Studio to run from source
  • You no longer need to download OpenCV
  • You are no longer bound to a fixed path C:\work

We have updated the developer docs with the updated setup instructions. We hope that this simplifies your setup and enables you to develop awesome plugins for Pupil Capture and Pupil Player that can be shared with the community!

Release Note Updates

30.03.2020 14:00: Fix driver installation issue in Windows bundle - #1842

In Pupil v1.23-4, we fixed an issue that prevented users from installing the camera drivers on Windows.

06.04.2020 11:50: Include missing libuvc.dylib in macOS bundle

In Pupil v1.23-5, we have included missing library files for Pupil Player and Pupil Service on macOS.

08.04.2020 17:00: Include missing libusb.dylib in macOS bundle

In Pupil v1.23-5-fix0, we have included missing library files for Pupil Player and Pupil Service on macOS.

24.04.2020 14:00: Improve driver installation stability on Windows - #1856

In Pupil v1.23-10, we have made changes to the driver installation procedure that ensure a successful installation on a larger variety of Windows setups.

Pupil core

February 20, 2020

Please feel free to get in touch with feedback and questions via the #pupil channel on Discord :smile:

Features and Changes

Video Source Rework - #1792

We have reworked the video source input system to greatly improve stability, user experience, and maintainability of the related features:

Single Plugin UI

All previous plugins for managing the video source (managers and sources) have been unified under a single plugin menu: Video Source, which handles device/camera selection as well as adjusting camera settings.

Automatic vs Manual Camera Selection

The new source system offers two ways for selecting cameras: automatic and manual:

  • In automatic mode (default) you can activate source devices, e.g. Local USB or Pupil Mobile. Activating a device selects the best matching available cameras as input for all windows in Pupil Capture.
  • Enabling manual mode allows you to select cameras individually for every window.

UI Cleanup

The following non-essential user interface elements have been removed to provide a better overall user experience:

  • The UI for selecting a file source has been removed. Starting a file source via network notification or via drag-and-drop is still possible.
  • Manually selecting HMD-Eyes as input source has been removed, as the plugin is started from Unity via network notification.
  • The Capture plugin UI in the eye windows has been removed entirely from Pupil Player as the displayed information was incomplete and the interactivity was limited.
  • The Test Image source has been removed as it was only intended for development use.

Removed Built-in RealSense Video Backend

Intel RealSense sensors are designed as a prototyping/development platform. Intel deprecates RealSense hardware frequently. Driver support is appropriate for a prototyping tool, but it is not sufficient for a production ready tool.

We have tried to build our tools around Intel’s changing APIs and patchwork driver support, but this became difficult and confusing for users.

You can still use RealSense Video Backend as a custom plugin. You can load it directly in the plugin folder of Pupil Capture. Check it out here.

Player: Add Gaze History to Polyline Visualization - #1785

We decided to use the Polyline Visualization as a replacement for the previously removed Scan Path Plugin. You can now visualize past gaze points up to a certain duration by adjusting the duration slider in the Vis Polyline Plugin. When first dragging the slider, the history for all frames will be computed, which will take some time. Afterwards you can see the gaze history and adjust the duration instantly.

Capture: Improved Region of Interest (ROI) selection - #1788

The ROI selector for the eye windows has been improved for better stability and better user experience. In doing so, we standardized the format of ROI boundaries across Pupil to the following order: (minx, miny, maxx, maxy). Please note that this changes the format for adjusting the ROI via network messages.

Player: Add 3D Eyeball Visualization - #1810

We have added the green circle overlay for the 3D eyeball to Player. The overlay will be added to the Eye Overlay and Eye Video Exporter plugins, when 3D pupil data is available. You can disable the visualization (including the pupil outline) via the corresponding plugin menus.

Bug Fixes

  • Changed Windows disconnection log-level from warning to debug - #1786
  • Fixed some of small bugs in the ROI selector - #1788
  • Fixed mixed up export columns for monocular eye_center and gaze_normal - #1805
  • Prevent a crash caused by incorrect UVC timestamps - #1812

Developer Notes

Getting Rid of C++ Dependencies - #1789

We have made the next step towards an easier installation process of Pupil. After the externalization of the Pupil Detectors in v1.20, we have been able to replace the last remaining C++ module with a Scipy re-implementation. This means that you do not have to compile any Python Extension Modules when starting Pupil.

As a result, you can now run Pupil from source on Windows without having to go through the trouble of setting up CERES. While the pupil-detectors still depend on CERES, we offer prebuilt wheels for Windows that you can install without having to set up the dependencies.

Changed Requirements

We updated the pyglui dependency to v1.27. Please update with:

pip install -U git+https://github.com/pupil-labs/pyglui

Release Notes Update

25.02.2019 v1.22.3

  • #1817 - Fixed a regression which caused all frames to be dropped after changing Pupil Capture’s time base.

27.02.2019 v1.22.7

  • Commit bcfbf7d - Fixed a typo that caused the v1.22.3 fix to not work correctly.

Pupil core

January 9, 2020

Features and Changes

Support for Pupil Invisible Audio Recordings - #1748

Pupil Player now supports audio playback and audio export for Pupil Invisible recordings with audio recordings.

Stream Head Pose over Network - #1774

You can now subscribe to the the topic head_pose to receive live data from the Online Head Pose Tracker plugin.

Bug Fixes

  • Fixed eye processes always displayed the algorithm view - #1766
  • Fixed potential crash when changing the eye video size - #1770
  • Made pyrealsense warning a debug-level message in order to be less obtrusive - #1768
  • Fixed broken interactivity of the ROI mask in the eye windows - #1778
  • Fixed multiple stability issues with the fullscreen calibration window - #1767

Developer Notes

Changed Requirements

We updated the PyAV dependency to v0.4.4 for the PI audio support. On Linux and macOS, you can update with:

pip install -U git+https://github.com/pupil-labs/PyAV

On Windows, download and install the latest wheel from GitHub.

Pupil core

December 12, 2019

Features and Changes

Deprecated Fingertip Calibration - #1753, #1757

We removed the fingertip calibration method.

The fingertip calibration method was rarely used and provided lower accuracy compared to other calibration methods. Additionally, dependencies needed for the fingertip detection required some work to set up correctly and accounted for almost 50% of the size of our application bundles.

Bug Fixes

  • Fixed a crash when trying to open empty (but existing) recordings - #1751
  • Fixed a crash on Windows when missing the latest Visual Studio redistributables - #1756
  • Fixed visualization of circle markers in offline calibration for recordings with gaps - #1758
  • Fixed broken combination of --hide-ui and --profile CLI flags - #1760

Developer Notes

Externalized Pupil Detectors - #1642

We have extracted the Pupil Detectors into their own repository: https://github.com/pupil-labs/pupil-detectors

This will make it very easy to use our pupil detectors in any standalone application or experimental setup without having to deal with all of Pupil’s intricacies.

Additionally this will make it much easier to run Pupil from source (especially on Windows), since we are distributing pupil-detectors as package via PyPI and even provide precompiled binaries for Windows!

OpenGL Debugging - #1752

We occasionally receive crash reports involving error messages from PyOpenGL, which we cannot reproduce on our machines. We wrapped the PyOpenGL errors to prevent crashes and log additional messages in these cases. This can result in visual UI errors (instead of a crash) when there are OpenGL issues. If you notice any weird UI behavior, please save the log file and report back to us.

Pupil core

November 21, 2019

Features and Changes

Improved Fixation Detector - #1743

We made improvements to the Fixation Detector to improve detection accuracy:

  • We were able to get rid of large gaps in between detected fixations. Now there will be more sequential fixations detected.
  • We decreased the false-positive detection rate by omitting low-confidence data.
  • We fixed a bug that would sometimes result in wrong fixations on 3D gaze data.

API Changes

Due to the improvements mentioned above, the Fixation Detector now requires calibrated gaze in both 2D and 3D mode. To reflect these changes, the method field of a fixation datum now contains either 2d gaze or 3d gaze (previously pupil vs gaze). This effects both the real-time fixations published through the network API and the fixations.csv export file.

Removed Fixation-Boosted Calibration - #1744

Since the Fixation Detector now requires calibrated gaze, it will not be used anymore to speed up the calibration process.

Improved Binocular Gaze Mapping - #1731

We improved the mapping of monocular gaze data streams by using a dynamic windows size dependent on the frame rate. This will result in better temporal accuracy when matching binocular gaze data for eyes with frequent low-confidence phases.

We also investigated some other variants of binocular gaze mapping, but chose to stick with this simple improvement. For a full evaluation and discussion, see this jupyter notebook.

Bug Fixes

  • Add/remove markers for legacy square markers works again - #1737
  • Fixed some bad UX with switching surface detection modes - #1738
  • Player is now able to load recordings with square brackets in the name - #1740
  • Fixed a crash when setting 2D detector properties via network API - #1742
  • Improved inconsistent background process behavior - #1741

Developer Notes

Automated Testing - #1721, #1724, #1730, #1735

We have done some work to consolidate old tests in the codebase and set it all up nicely with pytest. You can run the test suite from the repository root with:

pip install pytest
pytest

We set up a travis-ci integration to run pytest on all commits and pull requests. Now you will get feedback if you break something :smile: The test coverage is currently low. But from this point on, we - and you - are encouraged to write tests when adding new features and when fixing bugs.

Release Notes Update

27.11.2019 17:00: v1.19.2

We have fixed an issue in v1.19.0 where multi-part Pupil Mobile and Pupil Invisible recordings were not displayed correctly in Pupil Player - #1747. If you have opened such a recording in Pupil Player v1.19.0 then delete all *_lookup.npy files and reopen the recording in Pupil Player v1.19.2.

  • ...