Saving and Loading Event Data

What you will learn in this tutorial:

• how to save your event data

• how to load your event data

Preparations

We import pymovements as the alias pm for convenience.

import pymovements as pm

/home/docs/checkouts/readthedocs.org/user_builds/pymovements/envs/v0.22.1/lib/python3.9/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Let’s start by downloading our ToyDataset and loading in its data:

dataset = pm.Dataset('ToyDataset', path='data/ToyDataset')
dataset.download()
dataset.load()

INFO:pymovements.dataset.dataset:        You are downloading the pymovements Toy Dataset. Please be aware that pymovements does not
        host or distribute any dataset resources and only provides a convenient interface to
        download the public dataset resources that were published by their respective authors.

        Please cite the referenced publication if you intend to use the dataset in your research.

Using already downloaded and verified file: data/ToyDataset/downloads/pymovements-toy-dataset.zip
Extracting pymovements-toy-dataset.zip to data/ToyDataset/raw

100%|██████████| 23/23 [00:00<00:00, 343.97it/s]
100%|██████████| 20/20 [00:00<00:00, 107.17it/s]

Dataset

• definition:
  DatasetDefinition
  DatasetDefinition

  • acceleration_columns:
    None
    None
  • column_map:
    dict (0 items)
  • custom_read_kwargs:
    dict (1 items)
    • gaze:
      dict (4 items)
      • columns:
        list (5 items)
        • 'timestamp'
        • 'x'
        • (3 more)
      • schema_overrides:
        dict (5 items)
        • timestamp:
          Float64
          Float64
        • x:
          Float64
          Float64
        • (3 more)
      • (2 more)
  • distance_column:
    None
    None
  • experiment:
    Experiment
    Experiment

    • eyetracker:
      EyeTracker
      EyeTracker

      • left:
        None
        None
      • model:
        None
        None
      • mount:
        None
        None
      • right:
        None
        None
      • sampling_rate:
        1000
        1000
      • vendor:
        None
        None
      • version:
        None
        None
    • sampling_rate:
      1000
      1000
    • screen:
      Screen
      Screen

      • distance_cm:
        68
        68
      • height_cm:
        30.2
        30.2
      • height_px:
        1024
        1024
      • origin:
        'upper left'
        'upper left'
      • width_cm:
        38
        38
      • width_px:
        1280
        1280
      • x_max_dva:
        15.599386487782953
        15.599386487782953
      • x_min_dva:
        -15.599386487782953
        -15.599386487782953
      • y_max_dva:
        12.508044410882546
        12.508044410882546
      • y_min_dva:
        -12.508044410882546
        -12.508044410882546
  • extract:
    None
    None
  • filename_format:
    dict (1 items)
    • gaze:
      'trial_{text_id:d}_{page_id:d}.csv'
      'trial_{text_id:d}_{page_id:d}.csv'
  • filename_format_schema_overrides:
    dict (1 items)
    • gaze:
      dict (2 items)
      • text_id:
        <class 'int'>
        <class 'int'>
      • page_id:
        <class 'int'>
        <class 'int'>
  • has_files:
    dict (3 items)
    • gaze:
      True
      True
    • precomputed_events:
      False
      False
    • (1 more)
  • has_resources:
    True
    True
  • long_name:
    'pymovements Toy Dataset'
    'pymovements Toy Dataset'
  • mirrors:
    dict (0 items)
  • name:
    'ToyDataset'
    'ToyDataset'
  • pixel_columns:
    list (2 items)
    • 'x'
    • 'y'
  • position_columns:
    None
    None
  • resources:
    dict (1 items)
    • gaze:
      list (1 items)
      • • resource:
          str
          'http://github.com/aeye-lab/pymovements-toy-dataset/zipball/6cb5d663317bf418cec0c9abe1dde5085a8a8ebd/'
        • filename:
          'pymovements-toy-dataset.zip'
          'pymovements-toy-dataset.zip'
        • (1 more)
  • time_column:
    'timestamp'
    'timestamp'
  • time_unit:
    'ms'
    'ms'
  • trial_columns:
    None
    None
  • velocity_columns:
    None
    None
• events:
  list (0 items)
• fileinfo:
  dict (1 items)
  • gaze:
    DataFrame (3 columns, 20 rows)
    shape: (20, 3)

    text_id	page_id	filepath
    i64	i64	str
    0	1	"aeye-lab-pymovements-toy-datas…
    0	2	"aeye-lab-pymovements-toy-datas…
    0	3	"aeye-lab-pymovements-toy-datas…
    0	4	"aeye-lab-pymovements-toy-datas…
    0	5	"aeye-lab-pymovements-toy-datas…
    …	…	…
    3	1	"aeye-lab-pymovements-toy-datas…
    3	2	"aeye-lab-pymovements-toy-datas…
    3	3	"aeye-lab-pymovements-toy-datas…
    3	4	"aeye-lab-pymovements-toy-datas…
    3	5	"aeye-lab-pymovements-toy-datas…

• gaze:
  list (20 items)
  • GazeDataFrame
    • frame:
      DataFrame (6 columns, 17223 rows)
      shape: (17_223, 6)

      time	stimuli_x	stimuli_y	text_id	page_id	pixel
      i64	f64	f64	i64	i64	list[f64]
      1988145	-1.0	-1.0	0	1	[206.8, 152.4]
      1988146	-1.0	-1.0	0	1	[206.9, 152.1]
      1988147	-1.0	-1.0	0	1	[207.0, 151.8]
      1988148	-1.0	-1.0	0	1	[207.1, 151.7]
      1988149	-1.0	-1.0	0	1	[207.0, 151.5]
      …	…	…	…	…	…
      2005363	-1.0	-1.0	0	1	[361.0, 415.4]
      2005364	-1.0	-1.0	0	1	[358.0, 414.5]
      2005365	-1.0	-1.0	0	1	[355.8, 413.8]
      2005366	-1.0	-1.0	0	1	[353.1, 413.2]
      2005367	-1.0	-1.0	0	1	[351.2, 412.9]

    • events:
      EventDataFrame
      EventDataFrame

      • frame:
        DataFrame (6 columns, 0 rows)
        shape: (0, 6)

        text_id	page_id	name	onset	offset	duration
        i64	i64	str	i64	i64	i64

      • trial_columns:
        list (2 items)
        • 'text_id'
        • 'page_id'
    • trial_columns:
      list (2 items)
      • 'text_id'
      • 'page_id'
    • experiment:
      Experiment
      Experiment

      • eyetracker:
        EyeTracker
        EyeTracker

        • left:
          None
          None
        • model:
          None
          None
        • mount:
          None
          None
        • right:
          None
          None
        • sampling_rate:
          1000
          1000
        • vendor:
          None
          None
        • version:
          None
          None
      • sampling_rate:
        1000
        1000
      • screen:
        Screen
        Screen

        • distance_cm:
          68
          68
        • height_cm:
          30.2
          30.2
        • height_px:
          1024
          1024
        • origin:
          'upper left'
          'upper left'
        • width_cm:
          38
          38
        • width_px:
          1280
          1280
        • x_max_dva:
          15.599386487782953
          15.599386487782953
        • x_min_dva:
          -15.599386487782953
          -15.599386487782953
        • y_max_dva:
          12.508044410882546
          12.508044410882546
        • y_min_dva:
          -12.508044410882546
          -12.508044410882546
  • GazeDataFrame
    • frame:
      DataFrame (6 columns, 29799 rows)
      shape: (29_799, 6)

      time	stimuli_x	stimuli_y	text_id	page_id	pixel
      i64	f64	f64	i64	i64	list[f64]
      2008305	-1.0	-1.0	0	2	[141.4, 153.6]
      2008306	-1.0	-1.0	0	2	[141.1, 153.2]
      2008307	-1.0	-1.0	0	2	[140.7, 152.8]
      2008308	-1.0	-1.0	0	2	[140.6, 152.7]
      2008309	-1.0	-1.0	0	2	[140.5, 152.6]
      …	…	…	…	…	…
      2038099	-1.0	-1.0	0	2	[273.8, 773.8]
      2038100	-1.0	-1.0	0	2	[273.8, 774.1]
      2038101	-1.0	-1.0	0	2	[273.9, 774.5]
      2038102	-1.0	-1.0	0	2	[274.0, 774.4]
      2038103	-1.0	-1.0	0	2	[274.0, 773.9]

    • events:
      EventDataFrame
      EventDataFrame

      • frame:
        DataFrame (6 columns, 0 rows)
        shape: (0, 6)

        text_id	page_id	name	onset	offset	duration
        i64	i64	str	i64	i64	i64

      • trial_columns:
        list (2 items)
        • 'text_id'
        • 'page_id'
    • trial_columns:
      list (2 items)
      • 'text_id'
      • 'page_id'
    • experiment:
      Experiment
      Experiment

      • eyetracker:
        EyeTracker
        EyeTracker

        • left:
          None
          None
        • model:
          None
          None
        • mount:
          None
          None
        • right:
          None
          None
        • sampling_rate:
          1000
          1000
        • vendor:
          None
          None
        • version:
          None
          None
      • sampling_rate:
        1000
        1000
      • screen:
        Screen
        Screen

        • distance_cm:
          68
          68
        • height_cm:
          30.2
          30.2
        • height_px:
          1024
          1024
        • origin:
          'upper left'
          'upper left'
        • width_cm:
          38
          38
        • width_px:
          1280
          1280
        • x_max_dva:
          15.599386487782953
          15.599386487782953
        • x_min_dva:
          -15.599386487782953
          -15.599386487782953
        • y_max_dva:
          12.508044410882546
          12.508044410882546
        • y_min_dva:
          -12.508044410882546
          -12.508044410882546
  • (18 more)
• path:
  PosixPath('data/ToyDataset')
  PosixPath('data/ToyDataset')
• paths:
  DatasetPaths
  DatasetPaths

  • dataset:
    PosixPath('data/ToyDataset')
    PosixPath('data/ToyDataset')
  • downloads:
    PosixPath('data/ToyDataset/downloads')
    PosixPath('data/ToyDataset/downloads')
  • events:
    PosixPath('data/ToyDataset/events')
    PosixPath('data/ToyDataset/events')
  • precomputed_events:
    PosixPath('data/ToyDataset/precomputed_events')
    PosixPath('data/ToyDataset/precomputed_events')
  • precomputed_reading_measures:
    PosixPath
    PosixPath('data/ToyDataset/precomputed_reading_measures')
  • preprocessed:
    PosixPath('data/ToyDataset/preprocessed')
    PosixPath('data/ToyDataset/preprocessed')
  • raw:
    PosixPath('data/ToyDataset/raw')
    PosixPath('data/ToyDataset/raw')
  • root:
    PosixPath('data/ToyDataset')
    PosixPath('data/ToyDataset')
• precomputed_events:
  list (0 items)
• precomputed_reading_measures:
  list (0 items)

Now let’s do some preprocessing:

dataset.pix2deg()
dataset.pos2vel()

dataset.gaze[0]

100%|██████████| 20/20 [00:00<00:00, 45.61it/s]
100%|██████████| 20/20 [00:00<00:00, 79.53it/s]

GazeDataFrame

• frame:
  DataFrame (8 columns, 17223 rows)
  shape: (17_223, 8)

  time	stimuli_x	stimuli_y	text_id	page_id	pixel	position	velocity
  i64	f64	f64	i64	i64	list[f64]	list[f64]	list[f64]
  1988145	-1.0	-1.0	0	1	[206.8, 152.4]	[-10.697598, -8.852399]	[null, null]
  1988146	-1.0	-1.0	0	1	[206.9, 152.1]	[-10.695183, -8.859678]	[null, null]
  1988147	-1.0	-1.0	0	1	[207.0, 151.8]	[-10.692768, -8.866956]	[1.610194, -5.256267]
  1988148	-1.0	-1.0	0	1	[207.1, 151.7]	[-10.690352, -8.869381]	[0.402548, -4.447465]
  1988149	-1.0	-1.0	0	1	[207.0, 151.5]	[-10.692768, -8.874233]	[0.402561, -3.234462]
  …	…	…	…	…	…	…	…
  2005363	-1.0	-1.0	0	1	[361.0, 415.4]	[-6.932438, -2.386672]	[-63.266374, -21.085616]
  2005364	-1.0	-1.0	0	1	[358.0, 414.5]	[-7.006376, -2.408998]	[-63.249652, -19.431326]
  2005365	-1.0	-1.0	0	1	[355.8, 413.8]	[-7.060582, -2.426362]	[-60.359624, -15.710061]
  2005366	-1.0	-1.0	0	1	[353.1, 413.2]	[-7.12709, -2.441245]	[null, null]
  2005367	-1.0	-1.0	0	1	[351.2, 412.9]	[-7.173881, -2.448686]	[null, null]

• events:
  EventDataFrame
  EventDataFrame

  • frame:
    DataFrame (6 columns, 0 rows)
    shape: (0, 6)

    text_id	page_id	name	onset	offset	duration
    i64	i64	str	i64	i64	i64

  • trial_columns:
    list (2 items)
    • 'text_id'
    • 'page_id'
• trial_columns:
  list (2 items)
  • 'text_id'
  • 'page_id'
• experiment:
  Experiment
  Experiment

  • eyetracker:
    EyeTracker
    EyeTracker

    • left:
      None
      None
    • model:
      None
      None
    • mount:
      None
      None
    • right:
      None
      None
    • sampling_rate:
      1000
      1000
    • vendor:
      None
      None
    • version:
      None
      None
  • sampling_rate:
    1000
    1000
  • screen:
    Screen
    Screen

    • distance_cm:
      68
      68
    • height_cm:
      30.2
      30.2
    • height_px:
      1024
      1024
    • origin:
      'upper left'
      'upper left'
    • width_cm:
      38
      38
    • width_px:
      1280
      1280
    • x_max_dva:
      15.599386487782953
      15.599386487782953
    • x_min_dva:
      -15.599386487782953
      -15.599386487782953
    • y_max_dva:
      12.508044410882546
      12.508044410882546
    • y_min_dva:
      -12.508044410882546
      -12.508044410882546

Detect some events:

dataset.detect_events('ivt')

dataset.events[0]

20it [00:00, 113.39it/s]

EventDataFrame

• frame:
  DataFrame (6 columns, 73 rows)
  shape: (73, 6)

  text_id	page_id	name	onset	offset	duration
  i64	i64	str	i64	i64	i64
  0	1	"fixation"	1988147	1988322	175
  0	1	"fixation"	1988351	1988546	195
  0	1	"fixation"	1988592	1988736	144
  0	1	"fixation"	1988788	1989013	225
  0	1	"fixation"	1989060	1989170	110
  …	…	…	…	…	…
  0	1	"fixation"	2004132	2004331	199
  0	1	"fixation"	2004399	2004687	288
  0	1	"fixation"	2004714	2004878	164
  0	1	"fixation"	2004931	2005109	178
  0	1	"fixation"	2005138	2005287	149

• trial_columns:
  list (2 items)
  • 'text_id'
  • 'page_id'

dataset.detect_events('microsaccades')

dataset.events[0]

20it [00:00, 60.83it/s]

EventDataFrame

• frame:
  DataFrame (6 columns, 215 rows)
  shape: (215, 6)

  text_id	page_id	name	onset	offset	duration
  i64	i64	str	i64	i64	i64
  0	1	"fixation"	1988147	1988322	175
  0	1	"fixation"	1988351	1988546	195
  0	1	"fixation"	1988592	1988736	144
  0	1	"fixation"	1988788	1989013	225
  0	1	"fixation"	1989060	1989170	110
  …	…	…	…	…	…
  0	1	"saccade"	2005110	2005126	16
  0	1	"saccade"	2005128	2005138	10
  0	1	"saccade"	2005288	2005345	57
  0	1	"saccade"	2005347	2005356	9
  0	1	"saccade"	2005359	2005365	6

• trial_columns:
  list (2 items)
  • 'text_id'
  • 'page_id'

We now have event dataframes available with our detected event data.

Saving

Saving your event data is as simple as:

dataset.save_events()

100%|██████████| 20/20 [00:00<00:00, 3261.38it/s]

Dataset

• definition:
  DatasetDefinition
  DatasetDefinition

  • acceleration_columns:
    None
    None
  • column_map:
    dict (0 items)
  • custom_read_kwargs:
    dict (1 items)
    • gaze:
      dict (4 items)
      • columns:
        list (5 items)
        • 'timestamp'
        • 'x'
        • (3 more)
      • schema_overrides:
        dict (5 items)
        • timestamp:
          Float64
          Float64
        • x:
          Float64
          Float64
        • (3 more)
      • (2 more)
  • distance_column:
    None
    None
  • experiment:
    Experiment
    Experiment

    • eyetracker:
      EyeTracker
      EyeTracker

      • left:
        None
        None
      • model:
        None
        None
      • mount:
        None
        None
      • right:
        None
        None
      • sampling_rate:
        1000
        1000
      • vendor:
        None
        None
      • version:
        None
        None
    • sampling_rate:
      1000
      1000
    • screen:
      Screen
      Screen

      • distance_cm:
        68
        68
      • height_cm:
        30.2
        30.2
      • height_px:
        1024
        1024
      • origin:
        'upper left'
        'upper left'
      • width_cm:
        38
        38
      • width_px:
        1280
        1280
      • x_max_dva:
        15.599386487782953
        15.599386487782953
      • x_min_dva:
        -15.599386487782953
        -15.599386487782953
      • y_max_dva:
        12.508044410882546
        12.508044410882546
      • y_min_dva:
        -12.508044410882546
        -12.508044410882546
  • extract:
    None
    None
  • filename_format:
    dict (1 items)
    • gaze:
      'trial_{text_id:d}_{page_id:d}.csv'
      'trial_{text_id:d}_{page_id:d}.csv'
  • filename_format_schema_overrides:
    dict (1 items)
    • gaze:
      dict (2 items)
      • text_id:
        <class 'int'>
        <class 'int'>
      • page_id:
        <class 'int'>
        <class 'int'>
  • has_files:
    dict (3 items)
    • gaze:
      True
      True
    • precomputed_events:
      False
      False
    • (1 more)
  • has_resources:
    True
    True
  • long_name:
    'pymovements Toy Dataset'
    'pymovements Toy Dataset'
  • mirrors:
    dict (0 items)
  • name:
    'ToyDataset'
    'ToyDataset'
  • pixel_columns:
    list (2 items)
    • 'x'
    • 'y'
  • position_columns:
    None
    None
  • resources:
    dict (1 items)
    • gaze:
      list (1 items)
      • • resource:
          str
          'http://github.com/aeye-lab/pymovements-toy-dataset/zipball/6cb5d663317bf418cec0c9abe1dde5085a8a8ebd/'
        • filename:
          'pymovements-toy-dataset.zip'
          'pymovements-toy-dataset.zip'
        • (1 more)
  • time_column:
    'timestamp'
    'timestamp'
  • time_unit:
    'ms'
    'ms'
  • trial_columns:
    None
    None
  • velocity_columns:
    None
    None
• events:
  list (20 items)
  • EventDataFrame
    • frame:
      DataFrame (6 columns, 215 rows)
      shape: (215, 6)

      text_id	page_id	name	onset	offset	duration
      i64	i64	str	i64	i64	i64
      0	1	"fixation"	1988147	1988322	175
      0	1	"fixation"	1988351	1988546	195
      0	1	"fixation"	1988592	1988736	144
      0	1	"fixation"	1988788	1989013	225
      0	1	"fixation"	1989060	1989170	110
      …	…	…	…	…	…
      0	1	"saccade"	2005110	2005126	16
      0	1	"saccade"	2005128	2005138	10
      0	1	"saccade"	2005288	2005345	57
      0	1	"saccade"	2005347	2005356	9
      0	1	"saccade"	2005359	2005365	6

    • trial_columns:
      list (2 items)
      • 'text_id'
      • 'page_id'
  • EventDataFrame
    • frame:
      DataFrame (6 columns, 352 rows)
      shape: (352, 6)

      text_id	page_id	name	onset	offset	duration
      i64	i64	str	i64	i64	i64
      0	2	"fixation"	2008307	2008474	167
      0	2	"fixation"	2008498	2008603	105
      0	2	"fixation"	2008628	2008806	178
      0	2	"fixation"	2008862	2009051	189
      0	2	"fixation"	2009064	2009201	137
      …	…	…	…	…	…
      0	2	"saccade"	2037638	2037654	16
      0	2	"saccade"	2037656	2037666	10
      0	2	"saccade"	2037812	2037830	18
      0	2	"saccade"	2037832	2037839	7
      0	2	"saccade"	2037845	2037851	6

    • trial_columns:
      list (2 items)
      • 'text_id'
      • 'page_id'
  • (18 more)
• fileinfo:
  dict (1 items)
  • gaze:
    DataFrame (3 columns, 20 rows)
    shape: (20, 3)

    text_id	page_id	filepath
    i64	i64	str
    0	1	"aeye-lab-pymovements-toy-datas…
    0	2	"aeye-lab-pymovements-toy-datas…
    0	3	"aeye-lab-pymovements-toy-datas…
    0	4	"aeye-lab-pymovements-toy-datas…
    0	5	"aeye-lab-pymovements-toy-datas…
    …	…	…
    3	1	"aeye-lab-pymovements-toy-datas…
    3	2	"aeye-lab-pymovements-toy-datas…
    3	3	"aeye-lab-pymovements-toy-datas…
    3	4	"aeye-lab-pymovements-toy-datas…
    3	5	"aeye-lab-pymovements-toy-datas…

• gaze:
  list (20 items)
  • GazeDataFrame
    • frame:
      DataFrame (8 columns, 17223 rows)
      shape: (17_223, 8)

      time	stimuli_x	stimuli_y	text_id	page_id	pixel	position	velocity
      i64	f64	f64	i64	i64	list[f64]	list[f64]	list[f64]
      1988145	-1.0	-1.0	0	1	[206.8, 152.4]	[-10.697598, -8.852399]	[null, null]
      1988146	-1.0	-1.0	0	1	[206.9, 152.1]	[-10.695183, -8.859678]	[null, null]
      1988147	-1.0	-1.0	0	1	[207.0, 151.8]	[-10.692768, -8.866956]	[1.610194, -5.256267]
      1988148	-1.0	-1.0	0	1	[207.1, 151.7]	[-10.690352, -8.869381]	[0.402548, -4.447465]
      1988149	-1.0	-1.0	0	1	[207.0, 151.5]	[-10.692768, -8.874233]	[0.402561, -3.234462]
      …	…	…	…	…	…	…	…
      2005363	-1.0	-1.0	0	1	[361.0, 415.4]	[-6.932438, -2.386672]	[-63.266374, -21.085616]
      2005364	-1.0	-1.0	0	1	[358.0, 414.5]	[-7.006376, -2.408998]	[-63.249652, -19.431326]
      2005365	-1.0	-1.0	0	1	[355.8, 413.8]	[-7.060582, -2.426362]	[-60.359624, -15.710061]
      2005366	-1.0	-1.0	0	1	[353.1, 413.2]	[-7.12709, -2.441245]	[null, null]
      2005367	-1.0	-1.0	0	1	[351.2, 412.9]	[-7.173881, -2.448686]	[null, null]

    • events:
      EventDataFrame
      EventDataFrame

      • frame:
        DataFrame (6 columns, 215 rows)
        shape: (215, 6)

        text_id	page_id	name	onset	offset	duration
        i64	i64	str	i64	i64	i64
        0	1	"fixation"	1988147	1988322	175
        0	1	"fixation"	1988351	1988546	195
        0	1	"fixation"	1988592	1988736	144
        0	1	"fixation"	1988788	1989013	225
        0	1	"fixation"	1989060	1989170	110
        …	…	…	…	…	…
        0	1	"saccade"	2005110	2005126	16
        0	1	"saccade"	2005128	2005138	10
        0	1	"saccade"	2005288	2005345	57
        0	1	"saccade"	2005347	2005356	9
        0	1	"saccade"	2005359	2005365	6

      • trial_columns:
        list (2 items)
        • 'text_id'
        • 'page_id'
    • trial_columns:
      list (2 items)
      • 'text_id'
      • 'page_id'
    • experiment:
      Experiment
      Experiment

      • eyetracker:
        EyeTracker
        EyeTracker

        • left:
          None
          None
        • model:
          None
          None
        • mount:
          None
          None
        • right:
          None
          None
        • sampling_rate:
          1000
          1000
        • vendor:
          None
          None
        • version:
          None
          None
      • sampling_rate:
        1000
        1000
      • screen:
        Screen
        Screen

        • distance_cm:
          68
          68
        • height_cm:
          30.2
          30.2
        • height_px:
          1024
          1024
        • origin:
          'upper left'
          'upper left'
        • width_cm:
          38
          38
        • width_px:
          1280
          1280
        • x_max_dva:
          15.599386487782953
          15.599386487782953
        • x_min_dva:
          -15.599386487782953
          -15.599386487782953
        • y_max_dva:
          12.508044410882546
          12.508044410882546
        • y_min_dva:
          -12.508044410882546
          -12.508044410882546
  • GazeDataFrame
    • frame:
      DataFrame (8 columns, 29799 rows)
      shape: (29_799, 8)

      time	stimuli_x	stimuli_y	text_id	page_id	pixel	position	velocity
      i64	f64	f64	i64	i64	list[f64]	list[f64]	list[f64]
      2008305	-1.0	-1.0	0	2	[141.4, 153.6]	[-12.268583, -8.823284]	[null, null]
      2008306	-1.0	-1.0	0	2	[141.1, 153.2]	[-12.275749, -8.832989]	[null, null]
      2008307	-1.0	-1.0	0	2	[140.7, 152.8]	[-12.285302, -8.842695]	[-5.572617, -6.065816]
      2008308	-1.0	-1.0	0	2	[140.6, 152.7]	[-12.28769, -8.845121]	[-3.582268, -4.043733]
      2008309	-1.0	-1.0	0	2	[140.5, 152.6]	[-12.290078, -8.847547]	[-2.388085, -2.021821]
      …	…	…	…	…	…	…	…
      2038099	-1.0	-1.0	0	2	[273.8, 773.8]	[-9.071149, 6.490168]	[1.21962, 1.635403]
      2038100	-1.0	-1.0	0	2	[273.8, 774.1]	[-9.071149, 6.497527]	[1.626175, 4.497406]
      2038101	-1.0	-1.0	0	2	[273.9, 774.5]	[-9.06871, 6.50734]	[1.626186, 1.635423]
      2038102	-1.0	-1.0	0	2	[274.0, 774.4]	[-9.066271, 6.504886]	[null, null]
      2038103	-1.0	-1.0	0	2	[274.0, 773.9]	[-9.066271, 6.492621]	[null, null]

    • events:
      EventDataFrame
      EventDataFrame

      • frame:
        DataFrame (6 columns, 352 rows)
        shape: (352, 6)

        text_id	page_id	name	onset	offset	duration
        i64	i64	str	i64	i64	i64
        0	2	"fixation"	2008307	2008474	167
        0	2	"fixation"	2008498	2008603	105
        0	2	"fixation"	2008628	2008806	178
        0	2	"fixation"	2008862	2009051	189
        0	2	"fixation"	2009064	2009201	137
        …	…	…	…	…	…
        0	2	"saccade"	2037638	2037654	16
        0	2	"saccade"	2037656	2037666	10
        0	2	"saccade"	2037812	2037830	18
        0	2	"saccade"	2037832	2037839	7
        0	2	"saccade"	2037845	2037851	6

      • trial_columns:
        list (2 items)
        • 'text_id'
        • 'page_id'
    • trial_columns:
      list (2 items)
      • 'text_id'
      • 'page_id'
    • experiment:
      Experiment
      Experiment

      • eyetracker:
        EyeTracker
        EyeTracker

        • left:
          None
          None
        • model:
          None
          None
        • mount:
          None
          None
        • right:
          None
          None
        • sampling_rate:
          1000
          1000
        • vendor:
          None
          None
        • version:
          None
          None
      • sampling_rate:
        1000
        1000
      • screen:
        Screen
        Screen

        • distance_cm:
          68
          68
        • height_cm:
          30.2
          30.2
        • height_px:
          1024
          1024
        • origin:
          'upper left'
          'upper left'
        • width_cm:
          38
          38
        • width_px:
          1280
          1280
        • x_max_dva:
          15.599386487782953
          15.599386487782953
        • x_min_dva:
          -15.599386487782953
          -15.599386487782953
        • y_max_dva:
          12.508044410882546
          12.508044410882546
        • y_min_dva:
          -12.508044410882546
          -12.508044410882546
  • (18 more)
• path:
  PosixPath('data/ToyDataset')
  PosixPath('data/ToyDataset')
• paths:
  DatasetPaths
  DatasetPaths

  • dataset:
    PosixPath('data/ToyDataset')
    PosixPath('data/ToyDataset')
  • downloads:
    PosixPath('data/ToyDataset/downloads')
    PosixPath('data/ToyDataset/downloads')
  • events:
    PosixPath('data/ToyDataset/events')
    PosixPath('data/ToyDataset/events')
  • precomputed_events:
    PosixPath('data/ToyDataset/precomputed_events')
    PosixPath('data/ToyDataset/precomputed_events')
  • precomputed_reading_measures:
    PosixPath
    PosixPath('data/ToyDataset/precomputed_reading_measures')
  • preprocessed:
    PosixPath('data/ToyDataset/preprocessed')
    PosixPath('data/ToyDataset/preprocessed')
  • raw:
    PosixPath('data/ToyDataset/raw')
    PosixPath('data/ToyDataset/raw')
  • root:
    PosixPath('data/ToyDataset')
    PosixPath('data/ToyDataset')
• precomputed_events:
  list (0 items)
• precomputed_reading_measures:
  list (0 items)

All of the event data is saved into this directory:

dataset.paths.events

PosixPath('data/ToyDataset/events')

Let’s confirm it by printing all the new files in this directory:

print(list(dataset.paths.events.glob('*/*/*')))

[PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_1_1.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_2_2.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_2_3.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_0_3.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_1_4.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_2_5.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_3_2.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_1_5.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_0_4.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_3_3.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_3_5.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_3_4.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_2_1.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_0_1.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_3_1.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_1_2.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_0_2.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_1_3.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_2_4.feather'), PosixPath('data/ToyDataset/events/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_0_5.feather')]

All of the files have been saved into the Dataset.paths.events as feather files.

If we want to save the data into an alternative directory and also use a different file format like csv we can use the following:

dataset.save_events(events_dirname='events_csv', extension='csv')

100%|██████████| 20/20 [00:00<00:00, 3150.53it/s]

Dataset

• definition:
  DatasetDefinition
  DatasetDefinition

  • acceleration_columns:
    None
    None
  • column_map:
    dict (0 items)
  • custom_read_kwargs:
    dict (1 items)
    • gaze:
      dict (4 items)
      • columns:
        list (5 items)
        • 'timestamp'
        • 'x'
        • (3 more)
      • schema_overrides:
        dict (5 items)
        • timestamp:
          Float64
          Float64
        • x:
          Float64
          Float64
        • (3 more)
      • (2 more)
  • distance_column:
    None
    None
  • experiment:
    Experiment
    Experiment

    • eyetracker:
      EyeTracker
      EyeTracker

      • left:
        None
        None
      • model:
        None
        None
      • mount:
        None
        None
      • right:
        None
        None
      • sampling_rate:
        1000
        1000
      • vendor:
        None
        None
      • version:
        None
        None
    • sampling_rate:
      1000
      1000
    • screen:
      Screen
      Screen

      • distance_cm:
        68
        68
      • height_cm:
        30.2
        30.2
      • height_px:
        1024
        1024
      • origin:
        'upper left'
        'upper left'
      • width_cm:
        38
        38
      • width_px:
        1280
        1280
      • x_max_dva:
        15.599386487782953
        15.599386487782953
      • x_min_dva:
        -15.599386487782953
        -15.599386487782953
      • y_max_dva:
        12.508044410882546
        12.508044410882546
      • y_min_dva:
        -12.508044410882546
        -12.508044410882546
  • extract:
    None
    None
  • filename_format:
    dict (1 items)
    • gaze:
      'trial_{text_id:d}_{page_id:d}.csv'
      'trial_{text_id:d}_{page_id:d}.csv'
  • filename_format_schema_overrides:
    dict (1 items)
    • gaze:
      dict (2 items)
      • text_id:
        <class 'int'>
        <class 'int'>
      • page_id:
        <class 'int'>
        <class 'int'>
  • has_files:
    dict (3 items)
    • gaze:
      True
      True
    • precomputed_events:
      False
      False
    • (1 more)
  • has_resources:
    True
    True
  • long_name:
    'pymovements Toy Dataset'
    'pymovements Toy Dataset'
  • mirrors:
    dict (0 items)
  • name:
    'ToyDataset'
    'ToyDataset'
  • pixel_columns:
    list (2 items)
    • 'x'
    • 'y'
  • position_columns:
    None
    None
  • resources:
    dict (1 items)
    • gaze:
      list (1 items)
      • • resource:
          str
          'http://github.com/aeye-lab/pymovements-toy-dataset/zipball/6cb5d663317bf418cec0c9abe1dde5085a8a8ebd/'
        • filename:
          'pymovements-toy-dataset.zip'
          'pymovements-toy-dataset.zip'
        • (1 more)
  • time_column:
    'timestamp'
    'timestamp'
  • time_unit:
    'ms'
    'ms'
  • trial_columns:
    None
    None
  • velocity_columns:
    None
    None
• events:
  list (20 items)
  • EventDataFrame
    • frame:
      DataFrame (6 columns, 215 rows)
      shape: (215, 6)

      text_id	page_id	name	onset	offset	duration
      i64	i64	str	i64	i64	i64
      0	1	"fixation"	1988147	1988322	175
      0	1	"fixation"	1988351	1988546	195
      0	1	"fixation"	1988592	1988736	144
      0	1	"fixation"	1988788	1989013	225
      0	1	"fixation"	1989060	1989170	110
      …	…	…	…	…	…
      0	1	"saccade"	2005110	2005126	16
      0	1	"saccade"	2005128	2005138	10
      0	1	"saccade"	2005288	2005345	57
      0	1	"saccade"	2005347	2005356	9
      0	1	"saccade"	2005359	2005365	6

    • trial_columns:
      list (2 items)
      • 'text_id'
      • 'page_id'
  • EventDataFrame
    • frame:
      DataFrame (6 columns, 352 rows)
      shape: (352, 6)

      text_id	page_id	name	onset	offset	duration
      i64	i64	str	i64	i64	i64
      0	2	"fixation"	2008307	2008474	167
      0	2	"fixation"	2008498	2008603	105
      0	2	"fixation"	2008628	2008806	178
      0	2	"fixation"	2008862	2009051	189
      0	2	"fixation"	2009064	2009201	137
      …	…	…	…	…	…
      0	2	"saccade"	2037638	2037654	16
      0	2	"saccade"	2037656	2037666	10
      0	2	"saccade"	2037812	2037830	18
      0	2	"saccade"	2037832	2037839	7
      0	2	"saccade"	2037845	2037851	6

    • trial_columns:
      list (2 items)
      • 'text_id'
      • 'page_id'
  • (18 more)
• fileinfo:
  dict (1 items)
  • gaze:
    DataFrame (3 columns, 20 rows)
    shape: (20, 3)

    text_id	page_id	filepath
    i64	i64	str
    0	1	"aeye-lab-pymovements-toy-datas…
    0	2	"aeye-lab-pymovements-toy-datas…
    0	3	"aeye-lab-pymovements-toy-datas…
    0	4	"aeye-lab-pymovements-toy-datas…
    0	5	"aeye-lab-pymovements-toy-datas…
    …	…	…
    3	1	"aeye-lab-pymovements-toy-datas…
    3	2	"aeye-lab-pymovements-toy-datas…
    3	3	"aeye-lab-pymovements-toy-datas…
    3	4	"aeye-lab-pymovements-toy-datas…
    3	5	"aeye-lab-pymovements-toy-datas…

• gaze:
  list (20 items)
  • GazeDataFrame
    • frame:
      DataFrame (8 columns, 17223 rows)
      shape: (17_223, 8)

      time	stimuli_x	stimuli_y	text_id	page_id	pixel	position	velocity
      i64	f64	f64	i64	i64	list[f64]	list[f64]	list[f64]
      1988145	-1.0	-1.0	0	1	[206.8, 152.4]	[-10.697598, -8.852399]	[null, null]
      1988146	-1.0	-1.0	0	1	[206.9, 152.1]	[-10.695183, -8.859678]	[null, null]
      1988147	-1.0	-1.0	0	1	[207.0, 151.8]	[-10.692768, -8.866956]	[1.610194, -5.256267]
      1988148	-1.0	-1.0	0	1	[207.1, 151.7]	[-10.690352, -8.869381]	[0.402548, -4.447465]
      1988149	-1.0	-1.0	0	1	[207.0, 151.5]	[-10.692768, -8.874233]	[0.402561, -3.234462]
      …	…	…	…	…	…	…	…
      2005363	-1.0	-1.0	0	1	[361.0, 415.4]	[-6.932438, -2.386672]	[-63.266374, -21.085616]
      2005364	-1.0	-1.0	0	1	[358.0, 414.5]	[-7.006376, -2.408998]	[-63.249652, -19.431326]
      2005365	-1.0	-1.0	0	1	[355.8, 413.8]	[-7.060582, -2.426362]	[-60.359624, -15.710061]
      2005366	-1.0	-1.0	0	1	[353.1, 413.2]	[-7.12709, -2.441245]	[null, null]
      2005367	-1.0	-1.0	0	1	[351.2, 412.9]	[-7.173881, -2.448686]	[null, null]

    • events:
      EventDataFrame
      EventDataFrame

      • frame:
        DataFrame (6 columns, 215 rows)
        shape: (215, 6)

        text_id	page_id	name	onset	offset	duration
        i64	i64	str	i64	i64	i64
        0	1	"fixation"	1988147	1988322	175
        0	1	"fixation"	1988351	1988546	195
        0	1	"fixation"	1988592	1988736	144
        0	1	"fixation"	1988788	1989013	225
        0	1	"fixation"	1989060	1989170	110
        …	…	…	…	…	…
        0	1	"saccade"	2005110	2005126	16
        0	1	"saccade"	2005128	2005138	10
        0	1	"saccade"	2005288	2005345	57
        0	1	"saccade"	2005347	2005356	9
        0	1	"saccade"	2005359	2005365	6

      • trial_columns:
        list (2 items)
        • 'text_id'
        • 'page_id'
    • trial_columns:
      list (2 items)
      • 'text_id'
      • 'page_id'
    • experiment:
      Experiment
      Experiment

      • eyetracker:
        EyeTracker
        EyeTracker

        • left:
          None
          None
        • model:
          None
          None
        • mount:
          None
          None
        • right:
          None
          None
        • sampling_rate:
          1000
          1000
        • vendor:
          None
          None
        • version:
          None
          None
      • sampling_rate:
        1000
        1000
      • screen:
        Screen
        Screen

        • distance_cm:
          68
          68
        • height_cm:
          30.2
          30.2
        • height_px:
          1024
          1024
        • origin:
          'upper left'
          'upper left'
        • width_cm:
          38
          38
        • width_px:
          1280
          1280
        • x_max_dva:
          15.599386487782953
          15.599386487782953
        • x_min_dva:
          -15.599386487782953
          -15.599386487782953
        • y_max_dva:
          12.508044410882546
          12.508044410882546
        • y_min_dva:
          -12.508044410882546
          -12.508044410882546
  • GazeDataFrame
    • frame:
      DataFrame (8 columns, 29799 rows)
      shape: (29_799, 8)

      time	stimuli_x	stimuli_y	text_id	page_id	pixel	position	velocity
      i64	f64	f64	i64	i64	list[f64]	list[f64]	list[f64]
      2008305	-1.0	-1.0	0	2	[141.4, 153.6]	[-12.268583, -8.823284]	[null, null]
      2008306	-1.0	-1.0	0	2	[141.1, 153.2]	[-12.275749, -8.832989]	[null, null]
      2008307	-1.0	-1.0	0	2	[140.7, 152.8]	[-12.285302, -8.842695]	[-5.572617, -6.065816]
      2008308	-1.0	-1.0	0	2	[140.6, 152.7]	[-12.28769, -8.845121]	[-3.582268, -4.043733]
      2008309	-1.0	-1.0	0	2	[140.5, 152.6]	[-12.290078, -8.847547]	[-2.388085, -2.021821]
      …	…	…	…	…	…	…	…
      2038099	-1.0	-1.0	0	2	[273.8, 773.8]	[-9.071149, 6.490168]	[1.21962, 1.635403]
      2038100	-1.0	-1.0	0	2	[273.8, 774.1]	[-9.071149, 6.497527]	[1.626175, 4.497406]
      2038101	-1.0	-1.0	0	2	[273.9, 774.5]	[-9.06871, 6.50734]	[1.626186, 1.635423]
      2038102	-1.0	-1.0	0	2	[274.0, 774.4]	[-9.066271, 6.504886]	[null, null]
      2038103	-1.0	-1.0	0	2	[274.0, 773.9]	[-9.066271, 6.492621]	[null, null]

    • events:
      EventDataFrame
      EventDataFrame

      • frame:
        DataFrame (6 columns, 352 rows)
        shape: (352, 6)

        text_id	page_id	name	onset	offset	duration
        i64	i64	str	i64	i64	i64
        0	2	"fixation"	2008307	2008474	167
        0	2	"fixation"	2008498	2008603	105
        0	2	"fixation"	2008628	2008806	178
        0	2	"fixation"	2008862	2009051	189
        0	2	"fixation"	2009064	2009201	137
        …	…	…	…	…	…
        0	2	"saccade"	2037638	2037654	16
        0	2	"saccade"	2037656	2037666	10
        0	2	"saccade"	2037812	2037830	18
        0	2	"saccade"	2037832	2037839	7
        0	2	"saccade"	2037845	2037851	6

      • trial_columns:
        list (2 items)
        • 'text_id'
        • 'page_id'
    • trial_columns:
      list (2 items)
      • 'text_id'
      • 'page_id'
    • experiment:
      Experiment
      Experiment

      • eyetracker:
        EyeTracker
        EyeTracker

        • left:
          None
          None
        • model:
          None
          None
        • mount:
          None
          None
        • right:
          None
          None
        • sampling_rate:
          1000
          1000
        • vendor:
          None
          None
        • version:
          None
          None
      • sampling_rate:
        1000
        1000
      • screen:
        Screen
        Screen

        • distance_cm:
          68
          68
        • height_cm:
          30.2
          30.2
        • height_px:
          1024
          1024
        • origin:
          'upper left'
          'upper left'
        • width_cm:
          38
          38
        • width_px:
          1280
          1280
        • x_max_dva:
          15.599386487782953
          15.599386487782953
        • x_min_dva:
          -15.599386487782953
          -15.599386487782953
        • y_max_dva:
          12.508044410882546
          12.508044410882546
        • y_min_dva:
          -12.508044410882546
          -12.508044410882546
  • (18 more)
• path:
  PosixPath('data/ToyDataset')
  PosixPath('data/ToyDataset')
• paths:
  DatasetPaths
  DatasetPaths

  • dataset:
    PosixPath('data/ToyDataset')
    PosixPath('data/ToyDataset')
  • downloads:
    PosixPath('data/ToyDataset/downloads')
    PosixPath('data/ToyDataset/downloads')
  • events:
    PosixPath('data/ToyDataset/events')
    PosixPath('data/ToyDataset/events')
  • precomputed_events:
    PosixPath('data/ToyDataset/precomputed_events')
    PosixPath('data/ToyDataset/precomputed_events')
  • precomputed_reading_measures:
    PosixPath
    PosixPath('data/ToyDataset/precomputed_reading_measures')
  • preprocessed:
    PosixPath('data/ToyDataset/preprocessed')
    PosixPath('data/ToyDataset/preprocessed')
  • raw:
    PosixPath('data/ToyDataset/raw')
    PosixPath('data/ToyDataset/raw')
  • root:
    PosixPath('data/ToyDataset')
    PosixPath('data/ToyDataset')
• precomputed_events:
  list (0 items)
• precomputed_reading_measures:
  list (0 items)

Let’s confirm again by printing all the new files in this alternative directory:

alternative_dirpath = dataset.path / 'events_csv'
print(list(alternative_dirpath.glob('*/*/*')))

[PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_3_1.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_0_2.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_0_3.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_1_4.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_1_3.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_0_4.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_0_5.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_2_5.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_1_5.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_3_4.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_2_2.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_2_1.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_0_1.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_3_2.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_1_1.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_3_3.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_1_2.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_3_5.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_2_4.csv'), PosixPath('data/ToyDataset/events_csv/aeye-lab-pymovements-toy-dataset-6cb5d66/data/trial_2_3.csv')]

Loading

Now let’s imagine that this preprocessing and saving was done in another file and we only want to load the preprocessed data.

We simulate this by initializing a new dataset object. We don’t need to download any additional data.

preprocessed_dataset = pm.Dataset('ToyDataset', path='data/ToyDataset')

The preprocessed data can now simply be loaded by setting preprocessed to True:

preprocessed_dataset.load(events=True)

dataset.events[0]

100%|██████████| 20/20 [00:00<00:00, 115.66it/s]
100%|██████████| 20/20 [00:00<00:00, 1507.09it/s]

EventDataFrame

• frame:
  DataFrame (6 columns, 215 rows)
  shape: (215, 6)

  text_id	page_id	name	onset	offset	duration
  i64	i64	str	i64	i64	i64
  0	1	"fixation"	1988147	1988322	175
  0	1	"fixation"	1988351	1988546	195
  0	1	"fixation"	1988592	1988736	144
  0	1	"fixation"	1988788	1989013	225
  0	1	"fixation"	1989060	1989170	110
  …	…	…	…	…	…
  0	1	"saccade"	2005110	2005126	16
  0	1	"saccade"	2005128	2005138	10
  0	1	"saccade"	2005288	2005345	57
  0	1	"saccade"	2005347	2005356	9
  0	1	"saccade"	2005359	2005365	6

• trial_columns:
  list (2 items)
  • 'text_id'
  • 'page_id'

By default, the events directory and the feather extension will be chosen.

In case of alternative directory names or other file formats you can use the following:

preprocessed_dataset.load(
    events=True,
    events_dirname='events_csv',
    extension='csv',
)
dataset.events[0]

100%|██████████| 20/20 [00:00<00:00, 107.84it/s]
100%|██████████| 20/20 [00:00<00:00, 1214.00it/s]

EventDataFrame

• frame:
  DataFrame (6 columns, 215 rows)
  shape: (215, 6)

  text_id	page_id	name	onset	offset	duration
  i64	i64	str	i64	i64	i64
  0	1	"fixation"	1988147	1988322	175
  0	1	"fixation"	1988351	1988546	195
  0	1	"fixation"	1988592	1988736	144
  0	1	"fixation"	1988788	1989013	225
  0	1	"fixation"	1989060	1989170	110
  …	…	…	…	…	…
  0	1	"saccade"	2005110	2005126	16
  0	1	"saccade"	2005128	2005138	10
  0	1	"saccade"	2005288	2005345	57
  0	1	"saccade"	2005347	2005356	9
  0	1	"saccade"	2005359	2005365	6

• trial_columns:
  list (2 items)
  • 'text_id'
  • 'page_id'

What you have learned in this tutorial:

• saving your preprocesed data using Dataset.save_preprocessed()

• load your preprocesed data using Dataset.load(preprocessed=True)

• using custom directory names by specifying preprocessed_dirname

• using other file formats than the default feather format by specifying extension