mikelgg93

This instructions will guide you on how to use Pupil VR Add-on with 3rd party content in Steam VR. Such as playing a game, for which you won't have access to the camera feed and properties.

1. Record the VR content using an external tool:

To capture the scene camera feed without having to develop anything, there are 3rd party plugins for OBS, like this one or this, that allows you to capture your VR content through the OpenVR API.

Please note that there is no OpenXR solution yet.

2. Recording with the eye tracker using Pupil Core Software:

You will need to record the data without calibration using Pupil Capture.

mikelgg93

Post-Hoc Apply multiple offset corrections.

In this gist, we will learn all we need to apply different offset corrections on a Pupil Invisible recording.

To do this you will need to do either some manual adjustings, and use either a spreadsheet or programming to modify the database. By the end of the guide, you will obtain the gaze corrected positions but you will still need to create a visualisation.

To help you to follow these instructions, we will illustrate this with a driving case. In this case, we will like to apply an offset for the road (far distance) and a different one to the cockpit (close distance and where parallax errors may occur).

Get the offset value for a specific distance (near distance)

mikelgg93

import os
import platform
import cv2  # For selecting AOIs

import numpy as np  # For numerical operations
import pandas as pd  # For data manipulation

pd.set_option("mode.chained_assignment", None)

import matplotlib as mpl  # For plotting

mikelgg93

import os
import tkinter as tk
from tkinter import filedialog
import logging
import argparse
import pandas as pd
import numpy as np

# Preparing the logger
logging.getLogger("remove_confidence_below_threshold")

mikelgg93

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

events_df = pd.read_csv("path/events.csv")
blinks_df = pd.read_csv("path/blinks.csv")

# filter blinks only occurring between event name A and B
event_name_A = "recording.begin"
event_name_B = "recording.end"

mikelgg93

To install it:

pip install -r requirements.txt

To execute it:

python read_video.py path/to/video.mp4

mikelgg93

This Gist provides instructions on how to use a custom plugin that replaces the default Display Recent Gaze plugin in Pupil Capture.

Installation and Use

Like any other plugin, place it pupil_capture_settings > plugins and restart Pupil Capture, you should see a new plugin on the plugin menu named Custom Display Recent Gaze, this will allow you change the size, color and transparency of the circle drawn with the latest gaze points after calibration.

mikelgg93

import glob
import logging
import os
import shutil
from pathlib import Path
import pandas as pd
import requests

API_URL = "https://api.cloud.pupil-labs.com/v2"

mikelgg93

import glob
import logging
import os
import shutil
from pathlib import Path
import pandas as pd
import requests

API_URL = "https://api.cloud.pupil-labs.com/v2"

mikelgg93

Install it and run it:

1. Set up your Python environment and API key using OpenAI's quick start guide
2. Clone this gist or download it
3. pip install -r requirements.txt
4. python assistant.py

Using it

If you have your computer and Companion Device (Neon or Pupil Invisible) connected to the same network, it will be automatically linked and start streaming the scene camera with the gaze circle overlay.