{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "011263bd",
   "metadata": {},
   "source": [
    "# Saving and Loading Event Data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d010630",
   "metadata": {},
   "source": [
    "## What you will learn in this tutorial:\n",
    "\n",
    "* how to save your event data\n",
    "* how to load your event data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bacf49e8",
   "metadata": {},
   "source": [
    "## Preparations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d973b685",
   "metadata": {},
   "source": [
    "We import `pymovements` as the alias `pm` for convenience."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33a914a5",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pymovements as pm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26acca82",
   "metadata": {},
   "source": [
    "Let's start by downloading our `ToyDataset` and loading in its data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "375b5f97",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = pm.Dataset('ToyDataset', path='data/ToyDataset')\n",
    "dataset.download()\n",
    "dataset.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1cfa35f",
   "metadata": {},
   "source": [
    "Now let's do some preprocessing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "45a2df85",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset.pix2deg()\n",
    "dataset.pos2vel()\n",
    "\n",
    "dataset.gaze[0].frame.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "416592bc",
   "metadata": {},
   "source": [
    "Detect some events:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "02000e37",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset.detect_events('ivt')\n",
    "\n",
    "dataset.events[0].frame.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8dccdc40",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset.detect_events('microsaccades')\n",
    "\n",
    "dataset.events[0].frame.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1358c312",
   "metadata": {},
   "source": [
    "We now have event dataframes available with our detected event data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d1fd4b3",
   "metadata": {},
   "source": [
    "## Saving"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b757a5ec",
   "metadata": {},
   "source": [
    "Saving your event data is as simple as:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1cbab15d",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset.save_events()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1715cf46",
   "metadata": {},
   "source": [
    "All of the event data is saved into this directory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a5a847da",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset.paths.events"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c704f4d",
   "metadata": {},
   "source": [
    "Let's confirm it by printing all the new files in this directory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4660504b",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(list(dataset.paths.events.glob('*/*/*')))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8a95584",
   "metadata": {},
   "source": [
    "All of the files have been saved into the `Dataset.paths.events` as `feather` files.\n",
    "\n",
    "If we want to save the data into an alternative directory and also use a different file format like `csv` we can use the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "058d6695",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset.save_events(events_dirname='events_csv', extension='csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17ba8399",
   "metadata": {},
   "source": [
    "Let's confirm again by printing all the new files in this alternative directory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ac60a00",
   "metadata": {},
   "outputs": [],
   "source": [
    "alternative_dirpath = dataset.path / 'events_csv'\n",
    "print(list(alternative_dirpath.glob('*/*/*')))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "991cea37",
   "metadata": {},
   "source": [
    "## Loading"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aba6cb0f",
   "metadata": {},
   "source": [
    "Now let's imagine that this preprocessing and saving was done in another file and we only want to load the preprocessed data.\n",
    "\n",
    "We simulate this by initializing a new dataset object. We don't need to download any additional data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73566af7",
   "metadata": {},
   "outputs": [],
   "source": [
    "preprocessed_dataset = pm.Dataset('ToyDataset', path='data/ToyDataset')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "143b6988",
   "metadata": {},
   "source": [
    "The preprocessed data can now simply be loaded by setting `preprocessed` to `True`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "291816f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "preprocessed_dataset.load(events=True)\n",
    "\n",
    "dataset.events[0].frame.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "baa790d0",
   "metadata": {},
   "source": [
    "By default, the `events` directory and the `feather` extension will be chosen."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66355c46",
   "metadata": {},
   "source": [
    "In case of alternative directory names or other file formats you can use the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "89b8cfd3",
   "metadata": {},
   "outputs": [],
   "source": [
    "preprocessed_dataset.load(\n",
    "    events=True,\n",
    "    events_dirname='events_csv',\n",
    "    extension='csv',\n",
    ")\n",
    "dataset.events[0].frame.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62c67791",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eeea42fc",
   "metadata": {},
   "source": [
    "## What you have learned in this tutorial:\n",
    "\n",
    "* saving your preprocesed data using `Dataset.save_preprocessed()`\n",
    "* load your preprocesed data using `Dataset.load(preprocessed=True)`\n",
    "* using custom directory names by specifying `preprocessed_dirname`\n",
    "* using other file formats than the default `feather` format by specifying `extension`"
   ]
  }
 ],
 "metadata": {},
 "nbformat": 4,
 "nbformat_minor": 5
}