Skip to main content
Ctrl+K
pymovements  documentation - Home pymovements  documentation - Home
  • Getting started
  • Tutorials
  • Datasets
  • API Reference
  • Contributing
    • Bibliography
  • GitHub
  • Getting started
  • Tutorials
  • Datasets
  • API Reference
  • Contributing
  • Bibliography
  • GitHub

Section Navigation

  • pymovements in 10 minutes
  • Downloading Public Datasets
  • Working with a Local Dataset
  • Parsing SR Research EyeLink Data
  • Plotting Gaze Data
  • Preprocessing Raw Gaze Data
  • Saving and Loading Preprocessed Data
  • Handling Gaze Events
  • Creating Synthetic Data
  • How to use pymovements in R
  • Tutorials
  • Parsing SR Research EyeLink Data

Parsing SR Research EyeLink Data#

What you will learn in this tutorial:#

  • how to parse raw eye tracking files created with SR Research EyeLink

  • how to extract experiment information using patterns

  • how to create a custom dataset definition to load a complete dataset of multiple files

Preparations#

We import pymovements as the alias pm for convenience.

import pymovements as pm

Let’s start by downloading a toy dataset ToyDatasetEyeLink that contains *.asc files:

dataset = pm.Dataset("ToyDatasetEyeLink", path='data/ToyDatasetEyeLink')
dataset.download()
INFO:pymovements.dataset.dataset:
        You are downloading the pymovements Toy Dataset EyeLink. Please be aware that pymovements does not
        host or distribute any dataset resources and only provides a convenient interface to
        download the public dataset resources that were published by their respective authors.

        Please cite the referenced publication if you intend to use the dataset in your research.
        
Downloading https://github.com/pymovements/pymovements-toy-dataset-eyelink/archive/refs/heads/main.zip to data/ToyDatasetEyeLink/downloads/pymovements-toy-dataset-eyelink.zip
Checking integrity of pymovements-toy-dataset-eyelink.zip
Extracting pymovements-toy-dataset-eyelink.zip to data/ToyDatasetEyeLink/raw
Extracting archive:   0%|          | 0/4 [00:00<?, ?file/s]
Extracting archive: 100%|██████████| 4/4 [00:00<00:00, 123.43file/s]

Dataset
  • DatasetDefinition
    DatasetDefinition
    • None
      None
    • None
      None
    • None
      None
    • None
      None
    • Experiment
      Experiment
      • EyeTracker
        EyeTracker
        • True
          True
        • 'EyeLink Portable Duo'
          'EyeLink Portable Duo'
        • None
          None
        • False
          False
        • 1000.0
          1000.0
        • 'EyeLink'
          'EyeLink'
        • None
          None
      • 1000.0
        1000.0
      • Screen
        Screen
        • 68
          68
        • 30.2
          30.2
        • 1024
          1024
        • 'upper left'
          'upper left'
        • 38
          38
        • 1280
          1280
        • 15.599386487782953
          15.599386487782953
        • -15.599386487782953
          -15.599386487782953
        • 12.508044410882546
          12.508044410882546
        • -12.508044410882546
          -12.508044410882546
    • None
      None
    • dict (1 items)
      • str
        'subject_{subject_id:d}_session_{session_id:d}.asc'
    • dict (1 items)
      • dict (2 items)
        • <class 'int'>
          <class 'int'>
        • <class 'int'>
          <class 'int'>
    • True
      True
    • 'pymovements Toy Dataset EyeLink'
      'pymovements Toy Dataset EyeLink'
    • dict (0 items)
      • 'ToyDatasetEyeLink'
        'ToyDatasetEyeLink'
      • None
        None
      • None
        None
      • list (1 items)
        • ResourceDefinition
          • 'gaze'
            'gaze'
          • 'pymovements-toy-dataset-eyelink.zip'
            'pymovements-toy-dataset-eyelink.zip'
          • str
            'subject_{subject_id:d}_session_{session_id:d}.asc'
          • dict (2 items)
            • <class 'int'>
              <class 'int'>
            • <class 'int'>
              <class 'int'>
          • None
            None
          • dict (4 items)
            • list (2 items)
              • 'task'
              • 'trial_id'
            • list (9 items)
                • 'SYNCTIME_READING_SCREEN'
                  'SYNCTIME_READING_SCREEN'
                • 'task'
                  'task'
                • (1 more)
                • 'SYNCTIME_JUDO'
                  'SYNCTIME_JUDO'
                • 'task'
                  'task'
                • (1 more)
              • (7 more)
            • (2 more)
          • '966c0b6aefe61f32942366ed719454d3'
            '966c0b6aefe61f32942366ed719454d3'
          • None
            None
          • str
            'https://github.com/pymovements/pymovements-toy-dataset-eyelink/archive/refs/heads/main.zip'
      • None
        None
      • None
        None
      • None
        None
      • None
        None
    • ()
      ()
    • DataFrame (0 columns, 0 rows)
      shape: (0, 0)
    • list (0 items)
      • PosixPath('data/ToyDatasetEyeLink')
        PosixPath('data/ToyDatasetEyeLink')
      • DatasetPaths
        DatasetPaths
        • PosixPath('data/ToyDatasetEyeLink')
          PosixPath('data/ToyDatasetEyeLink')
        • PosixPath('data/ToyDatasetEyeLink/downloads')
          PosixPath('data/ToyDatasetEyeLink/downloads')
        • PosixPath('data/ToyDatasetEyeLink/events')
          PosixPath('data/ToyDatasetEyeLink/events')
        • PosixPath
          PosixPath('data/ToyDatasetEyeLink/precomputed_events')
        • PosixPath
          PosixPath('data/ToyDatasetEyeLink/precomputed_reading_measures')
        • PosixPath('data/ToyDatasetEyeLink/preprocessed')
          PosixPath('data/ToyDatasetEyeLink/preprocessed')
        • PosixPath('data/ToyDatasetEyeLink/raw')
          PosixPath('data/ToyDatasetEyeLink/raw')
        • PosixPath('data/ToyDatasetEyeLink')
          PosixPath('data/ToyDatasetEyeLink')
      • list (0 items)
        • list (0 items)

          This dataset includes *.asc files that store raw eye-tracking data along with synchronization messages. Below, we’ll inspect the files included in the dataset:

          asc_files = list(dataset.path.glob('**/*.asc'))
          asc_files
          
          [PosixPath('data/ToyDatasetEyeLink/raw/pymovements-toy-dataset-eyelink-main/raw/subject_2_session_1.asc'),
           PosixPath('data/ToyDatasetEyeLink/raw/pymovements-toy-dataset-eyelink-main/raw/subject_1_session_1.asc')]
          

          Let’s display the first 20 lines of one of the files to get a sense of its structure:

          !head -n 20 data/ToyDatasetEyeLink/raw/aeye-lab-pymovements-toy-dataset-eyelink-a970d09/raw/subject_1_session_1.asc
          
          head: cannot open 'data/ToyDatasetEyeLink/raw/aeye-lab-pymovements-toy-dataset-eyelink-a970d09/raw/subject_1_session_1.asc' for reading: No such file or directory
          

          We can see that this file is a converted version of an *.edf file created by EyeLink.

          Let’s try loading one of these files directly using pm.gaze.from_asc:

          Loading eye-tracking data from a file#

          Loading eye-tracking data is straightforward. You can load an .asc file with a single call to pm.gaze.from_asc:

          gaze = pm.gaze.from_asc(file=asc_files[0])
          gaze
          
          Gaze
          • DataFrame (3 columns, 109216 rows)
            shape: (109_216, 3)
            timepupilpixel
            i64f64list[f64]
            2762704783.0[139.1, 142.8]
            2762705783.0[139.3, 142.8]
            2762706783.0[139.5, 142.4]
            2762707783.0[139.6, 141.9]
            2762708783.0[139.5, 141.3]
            ………
            2903401705.0[762.7, 605.5]
            2903402706.0[762.6, 605.2]
            2903403706.0[762.5, 605.0]
            2903404706.0[762.7, 604.9]
            2903405705.0[763.0, 604.9]
          • Events
            Events
            • DataFrame (4 columns, 0 rows)
              shape: (0, 4)
              nameonsetoffsetduration
              stri64i64i64
            • None
              None
          • None
            None
          • Experiment
            Experiment
            • EyeTracker
              EyeTracker
              • True
                True
              • 'EyeLink Portable Duo'
                'EyeLink Portable Duo'
              • 'Desktop'
                'Desktop'
              • False
                False
              • 1000.0
                1000.0
              • 'EyeLink'
                'EyeLink'
              • '6.12'
                '6.12'
            • 1000.0
              1000.0
            • Screen
              Screen
              • None
                None
              • None
                None
              • 1024
                1024
              • None
                None
              • None
                None
              • 1280
                1280

          This function automatically loads the raw eye-tracking data and attempts to infer the experimental settings used.

          Let’s inspect a few rows from the resulting GazeDataFrame:

          gaze.samples
          
          shape: (109_216, 3)
          timepupilpixel
          i64f64list[f64]
          2762704783.0[139.1, 142.8]
          2762705783.0[139.3, 142.8]
          2762706783.0[139.5, 142.4]
          2762707783.0[139.6, 141.9]
          2762708783.0[139.5, 141.3]
          ………
          2903401705.0[762.7, 605.5]
          2903402706.0[762.6, 605.2]
          2903403706.0[762.5, 605.0]
          2903404706.0[762.7, 604.9]
          2903405705.0[763.0, 604.9]

          We can see that timestamps (column time), pupil diameter (column pupil), and raw pixel coordinates (column pixel) are extracted automatically.

          Let’s now take a look at the experimental metadata that was retrieved:

          gaze.experiment
          
          Experiment
          • EyeTracker
            EyeTracker
            • True
              True
            • 'EyeLink Portable Duo'
              'EyeLink Portable Duo'
            • 'Desktop'
              'Desktop'
            • False
              False
            • 1000.0
              1000.0
            • 'EyeLink'
              'EyeLink'
            • '6.12'
              '6.12'
          • 1000.0
            1000.0
          • Screen
            Screen
            • None
              None
            • None
              None
            • 1024
              1024
            • None
              None
            • None
              None
            • 1280
              1280

          All relevant experimental metadata have been successfully extracted, such as the eye tracker model and the screen resolution used during recording.

          Loading eye-tracking data along with SR Research recording messages#

          To extract all MSG-prefixed SR Research messages, simply pass True to the pm.gaze.from_asc. The messages are stored in gaze.messages:

          gaze = pm.gaze.from_asc(file=asc_files[0], messages=True)
          gaze.messages
          
          shape: (2_365, 2)
          timecontent
          f64str
          2.695217e6"!CMD 0 select_parser_configura…
          2.695543e6"!CMD 0 fixation_update_interva…
          2.695544e6"!CMD 0 fixation_update_accumul…
          2.695546e6"!CMD 0 auto_calibration_messag…
          2.700119e6"DISPLAY_COORDS 0 0 1279 1023"
          ……
          2.904096e6"!V TRIAL_VAR forid "
          2.904097e6"!V TRIAL_VAR sessiontype "
          2.904098e6"!V TRIAL_VAR combinationid -32…
          2.904099e6"TRIAL_RESULT 0"
          2.904548e6"JUDO.STOP"

          We can also control which messages are parsed by specifying them in the messages argument. For example, to extract only trial-related messages containing the keyword TRIAL, we can do the following:

          gaze = pm.gaze.from_asc(file=asc_files[0], messages=['TRIAL'])
          gaze.messages
          
          shape: (2_027, 2)
          timecontent
          f64str
          2.762689e6"TRIALID 0"
          2.811758e6"!V TRIAL_VAR Session_Name_ sub…
          2.811759e6"!V TRIAL_VAR Trial_Index_ 1"
          2.81176e6"!V TRIAL_VAR RT_KAROLINSKA -1"
          2.811761e6"!V TRIAL_VAR RESPONSE_KAROLINS…
          ……
          2.904095e6"!V TRIAL_VAR q00corrans "
          2.904096e6"!V TRIAL_VAR forid "
          2.904097e6"!V TRIAL_VAR sessiontype "
          2.904098e6"!V TRIAL_VAR combinationid -32…
          2.904099e6"TRIAL_RESULT 0"

          Defining custom patterns for data extraction#

          Now let’s define our own patterns to extract additional information from the *.asc files and add them to the GazeDataFrame. We can do this using the parameter patterns using pm.gaze.from_asc.

          patterns accepts either a list of custom patterns to match additional columns or a key identifying predefined and eye-tracker-specific patterns.

          Let’s define a set of custom patterns to extract more information from parsed messages and show the resulting GazeDataFrame:

          patterns = [
              {
                  'pattern': 'SYNCTIME_READING_SCREEN',
                  'column': 'task',
                  'value': 'reading',
              },
              {
                  'pattern': 'SYNCTIME_JUDO',
                  'column': 'task',
                  'value': 'judo',
              },
              r'TRIALID (?P<trial_id>\d+)',
          ]
          
          gaze = pm.gaze.from_asc(file=asc_files[0], patterns=patterns)
          gaze.samples
          
          shape: (109_216, 5)
          timepupiltrial_idtaskpixel
          i64f64strstrlist[f64]
          2762704783.0"0"null[139.1, 142.8]
          2762705783.0"0"null[139.3, 142.8]
          2762706783.0"0"null[139.5, 142.4]
          2762707783.0"0"null[139.6, 141.9]
          2762708783.0"0"null[139.5, 141.3]
          ……………
          2903401705.0"12""judo"[762.7, 605.5]
          2903402706.0"12""judo"[762.6, 605.2]
          2903403706.0"12""judo"[762.5, 605.0]
          2903404706.0"12""judo"[762.7, 604.9]
          2903405705.0"12""judo"[763.0, 604.9]

          The examples above illustrate that patterns can be defined in different forms. Some patterns simply match a message and assign a fixed column value (see the first pattern above), while others use regular expressions to capture dynamic information—for instance, the trial_id in the last pattern.

          Given the patterns defined above, we can see that the columns for task and trial_id has been added.

          The trial_id was extracted from messages such as MSG 2762689 TRIALID 0, while the task value was obtained from messages like MSG 2814942 SYNCTIME_JUDO.

          Writing a DatasetDefinition to parse the complete dataset#

          Let’s create a custom DatasetDefinition to load all asc files, including the patterns we defined earlier.

          First we create a ResourceDefinition that specifies how we want to load our asc files. We can use the patterns that we identified and specify them as one of the load keyword arguments (load_kwargs).

          In addition, we also define the filename pattern, which represents subject and session information encoded in the filename. The datatypes of the additional metadata parsed from the filename can be specified via filename_pattern_schema_overrides.

          resource_definition = pm.ResourceDefinition(
              content='gaze',
              filename_pattern=r'subject_{subject_id:d}_session_{session_id:d}.asc',
              filename_pattern_schema_overrides={
                  'subject_id': int,
                  'session_id': int,
              },
              load_kwargs={
                  'patterns': patterns,
                  'schema': {'trial_id': int},
              },
          )
          

          Next, we need to define the experiment:

          experiment = pm.Experiment(
              screen_width_px=1280,
              screen_height_px=1024,
              screen_width_cm=38,
              screen_height_cm=30.2,
              distance_cm=68,
              origin='lower left',
              sampling_rate=1000,
          )
          

          We now use these do write our DatasetDefinition. We choose ToyDatasetEyeLink as the name.

          dataset_definition = pm.DatasetDefinition(
              name='ToyDatasetEyeLink',
              experiment=experiment,
              resources=[resource_definition],
          )
          

          Let’s initialize a new Dataset and load the data using the dataset definition we just set up:

          dataset = pm.Dataset(
              definition=dataset_definition,
              path='data/ToyDatasetEyeLink',
          )
          dataset.load()
          
          Dataset
          • DatasetDefinition
            DatasetDefinition
            • None
              None
            • None
              None
            • None
              None
            • None
              None
            • Experiment
              Experiment
              • EyeTracker
                EyeTracker
                • None
                  None
                • None
                  None
                • None
                  None
                • None
                  None
                • 1000
                  1000
                • None
                  None
                • None
                  None
              • 1000
                1000
              • Screen
                Screen
                • 68
                  68
                • 30.2
                  30.2
                • 1024
                  1024
                • 'lower left'
                  'lower left'
                • 38
                  38
                • 1280
                  1280
            • None
              None
            • dict (1 items)
              • str
                'subject_{subject_id:d}_session_{session_id:d}.asc'
            • dict (1 items)
              • dict (2 items)
                • <class 'int'>
                  <class 'int'>
                • <class 'int'>
                  <class 'int'>
            • True
              True
            • None
              None
            • dict (0 items)
              • 'ToyDatasetEyeLink'
                'ToyDatasetEyeLink'
              • None
                None
              • None
                None
              • list (1 items)
                • ResourceDefinition
                  • 'gaze'
                    'gaze'
                  • None
                    None
                  • str
                    'subject_{subject_id:d}_session_{session_id:d}.asc'
                  • dict (2 items)
                    • <class 'int'>
                      <class 'int'>
                    • <class 'int'>
                      <class 'int'>
                  • None
                    None
                  • dict (2 items)
                    • list (3 items)
                        • 'SYNCTIME_READING_SCREEN'
                          'SYNCTIME_READING_SCREEN'
                        • 'task'
                          'task'
                        • (1 more)
                        • 'SYNCTIME_JUDO'
                          'SYNCTIME_JUDO'
                        • 'task'
                          'task'
                        • (1 more)
                      • (1 more)
                    • dict (1 items)
                      • <class 'int'>
                        <class 'int'>
                  • None
                    None
                  • None
                    None
                  • None
                    None
              • None
                None
              • None
                None
              • None
                None
              • None
                None
            • tuple
              (shape: (0, 4) ┌──────┬───────┬────────┬──────────┐ │ name ┆ onset ┆ offset ┆ duration │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 ┆ i64 │ ╞══════╪═══════╪════════╪══════════╡ └──────┴───────┴────────┴──────────┘, shape: (0, 4) ┌──────┬───────┬────────┬──────────┐ │ name ┆ onset ┆ offset ┆ duration │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 ┆ i64 │ ╞══════╪═══════╪════════╪══════════╡ └──────┴───────┴────────┴──────────┘)
            • dict (1 items)
              • DataFrame (3 columns, 2 rows)
                shape: (2, 3)
                subject_idsession_idfilepath
                i64i64str
                11"pymovements-toy-dataset-eyelin…
                21"pymovements-toy-dataset-eyelin…
            • list (2 items)
              • Gaze
                • DataFrame (5 columns, 128342 rows)
                  shape: (128_342, 5)
                  timepupiltrial_idtaskpixel
                  i64f64i64strlist[f64]
                  2154556778.00null[138.1, 132.8]
                  2154557778.00null[138.2, 132.7]
                  2154558778.00null[138.2, 132.3]
                  2154559778.00null[138.1, 131.9]
                  2154560777.00null[137.9, 131.6]
                  ……………
                  2339287619.012"judo"[637.7, 531.7]
                  2339288619.012"judo"[637.9, 531.8]
                  2339289618.012"judo"[637.8, 531.6]
                  2339290618.012"judo"[637.6, 531.4]
                  2339291618.012"judo"[637.3, 531.2]
                • Events
                  Events
                  • DataFrame (4 columns, 0 rows)
                    shape: (0, 4)
                    nameonsetoffsetduration
                    stri64i64i64
                  • None
                    None
                • None
                  None
                • Experiment
                  Experiment
                  • EyeTracker
                    EyeTracker
                    • True
                      True
                    • 'EyeLink Portable Duo'
                      'EyeLink Portable Duo'
                    • 'Desktop'
                      'Desktop'
                    • False
                      False
                    • 1000
                      1000
                    • 'EyeLink'
                      'EyeLink'
                    • '6.12'
                      '6.12'
                  • 1000
                    1000
                  • Screen
                    Screen
                    • 68
                      68
                    • 30.2
                      30.2
                    • 1024
                      1024
                    • 'lower left'
                      'lower left'
                    • 38
                      38
                    • 1280
                      1280
              • Gaze
                • DataFrame (5 columns, 109216 rows)
                  shape: (109_216, 5)
                  timepupiltrial_idtaskpixel
                  i64f64i64strlist[f64]
                  2762704783.00null[139.1, 142.8]
                  2762705783.00null[139.3, 142.8]
                  2762706783.00null[139.5, 142.4]
                  2762707783.00null[139.6, 141.9]
                  2762708783.00null[139.5, 141.3]
                  ……………
                  2903401705.012"judo"[762.7, 605.5]
                  2903402706.012"judo"[762.6, 605.2]
                  2903403706.012"judo"[762.5, 605.0]
                  2903404706.012"judo"[762.7, 604.9]
                  2903405705.012"judo"[763.0, 604.9]
                • Events
                  Events
                  • DataFrame (4 columns, 0 rows)
                    shape: (0, 4)
                    nameonsetoffsetduration
                    stri64i64i64
                  • None
                    None
                • None
                  None
                • Experiment
                  Experiment
                  • EyeTracker
                    EyeTracker
                    • True
                      True
                    • 'EyeLink Portable Duo'
                      'EyeLink Portable Duo'
                    • 'Desktop'
                      'Desktop'
                    • False
                      False
                    • 1000
                      1000
                    • 'EyeLink'
                      'EyeLink'
                    • '6.12'
                      '6.12'
                  • 1000
                    1000
                  • Screen
                    Screen
                    • 68
                      68
                    • 30.2
                      30.2
                    • 1024
                      1024
                    • 'lower left'
                      'lower left'
                    • 38
                      38
                    • 1280
                      1280
            • PosixPath('data/ToyDatasetEyeLink')
              PosixPath('data/ToyDatasetEyeLink')
            • DatasetPaths
              DatasetPaths
              • PosixPath('data/ToyDatasetEyeLink')
                PosixPath('data/ToyDatasetEyeLink')
              • PosixPath('data/ToyDatasetEyeLink/downloads')
                PosixPath('data/ToyDatasetEyeLink/downloads')
              • PosixPath('data/ToyDatasetEyeLink/events')
                PosixPath('data/ToyDatasetEyeLink/events')
              • PosixPath
                PosixPath('data/ToyDatasetEyeLink/precomputed_events')
              • PosixPath
                PosixPath('data/ToyDatasetEyeLink/precomputed_reading_measures')
              • PosixPath('data/ToyDatasetEyeLink/preprocessed')
                PosixPath('data/ToyDatasetEyeLink/preprocessed')
              • PosixPath('data/ToyDatasetEyeLink/raw')
                PosixPath('data/ToyDatasetEyeLink/raw')
              • PosixPath('data/ToyDatasetEyeLink')
                PosixPath('data/ToyDatasetEyeLink')
            • list (0 items)
              • list (0 items)

                Let’s inspect the first Gaze in this dataset:

                dataset.gaze[0].samples
                
                shape: (128_342, 5)
                timepupiltrial_idtaskpixel
                i64f64i64strlist[f64]
                2154556778.00null[138.1, 132.8]
                2154557778.00null[138.2, 132.7]
                2154558778.00null[138.2, 132.3]
                2154559778.00null[138.1, 131.9]
                2154560777.00null[137.9, 131.6]
                ……………
                2339287619.012"judo"[637.7, 531.7]
                2339288619.012"judo"[637.9, 531.8]
                2339289618.012"judo"[637.8, 531.6]
                2339290618.012"judo"[637.6, 531.4]
                2339291618.012"judo"[637.3, 531.2]

                What you have learned in this tutorial:#

                • how to handle *.asc files

                • how to create a custom dataset loading all files and parsing custom messages

                • how to load the dataset into your working memory

                previous

                Working with a Local Dataset

                next

                Plotting Gaze Data

                On this page
                • What you will learn in this tutorial:
                • Preparations
                  • Loading eye-tracking data from a file
                  • Loading eye-tracking data along with SR Research recording messages
                  • Defining custom patterns for data extraction
                  • Writing a DatasetDefinition to parse the complete dataset
                • What you have learned in this tutorial:

                This Page

                • Show Source

                © Copyright 2022-2025 The pymovements Project Authors.

                Created using Sphinx 8.2.3.

                Built with the PyData Sphinx Theme 0.16.1.