Matlab and Python Import Utilities ================================== .. contents:: --------------------- Overview ~~~~~~~~ Trodes data can be imported into Matlab or Python for secondary processing and further analysis using the import utility scripts packaged with Trodes. These scripts can be found within the Resources folder in the Trodes installation directory, and are available for Matlab and Python (2 & 3). Trodes import functions import Trodes ``.dat`` binary files extracted using the Trodes Export Utility. Trodes Export can be accessed from within Trodes as well as from the command line. More information about the Trodes Export utility can be found :doc:`here `. The basic workflow is as follows: 1. Open the ``.rec`` file you would like to analyze in Trodes using the Open Playback File option in the Trodes main menu. 2. Navigate to File > Extract... 3. Select the data bands you would like to analyze, then hit Start. The resulting ``.dat`` files can then be imported into Matlab or Python using the appropriate version of the ``readTrodesExtractedDataFile`` script. These scripts are found in the TrodesToMatlab and TrodesToPython subfolders of the Resources folder of the Trodes installation folder. **An important note on voltage scaling:** Neural data exported from Trodes must be scaled to account for amplification. To convert data into uV scale, simply multiply the data by the ``voltage_scaling`` variable provided in the extracted data. Additionally, the voltage scaling value will vary depending on the amplification used to record. For Intan-based headstages this is a fixed value associated with the amplifier chip type. When recording with Neuropixels probes, the voltage scaling value will change depending on the selected gain value used when recording. Camera Module Data ~~~~~~~~~~~~~~~~~~ Camera Module binary files containing animal tracking and geometry data can also be imported into Matlab or Python using ``readTrodesExtractedDataFile``. Camera Module timestamp data (``.videoTimeStamp`` files) should be read in using ``readCameraModuleTimeStamps``. In Matlab, this script outputs a vector containing the timestamps of the video frames in uint32 format. In Python, this script outputs a numpy array containing the video frame timestamps. Additional information about Camera Module export can be found :ref:`here `. Importing Trodes Data Into Matlab (or Octave) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Before data can be imported into Matlab, the specific data bands for analysis must be exported using Trodes Export. This is done by opening a ``.rec`` data file using the Open Playback File option in the Trodes main menu, then exporting the desired data bands using Trodes Export. This can also be done using the Trodes Export command line utility. The exported ``.dat`` data files can now be opened using ``readTrodesExtractedDataFile.m``. Make sure the Trodes install folder has been added to your Matlab path, and you have set your current working directory to the folder containing the data file(s) being imported. Your command will look something like this: ``imported_data = readTrodesExtractedDataFile('example_recording.spikes_nt2.dat');`` This will import the data with additional metadata describing the data as a struct. Many metadata fields will be present for all imported data types, however, type-specific fields do exist for each data types. The data itself will be contained within a sub struct called fields. Data in the fields substruct is formatted in a data type specific manner. The example below shows a file containing spike waveforms exported from Trodes using the 'Spikes' option in the Trodes Export Utility. Data Exported using the 'Spikes' option will contain the following fields: - description: 1x30 sq_string - byte_order: 1x13 sq_string - original_file: 1x21 sq_string - clockrate: 1x1 scalar - trodes_version: 1x5 sq_string - compile_date: 1x11 sq_string - compile_time: 1x8 sq_string - qt_version: 1x5 sq_string - commit_tag: 1x26 sq_string - controller_firmware: 1x1 scalar - headstage_firmware: 1x1 scalar - controller_serialnum: 1x1 scalar - headstage_serialnum: 1x1 scalar - autosettle: 1x1 scalar - smartref: 1x1 scalar - gyro: 1x1 scalar - accelerometer: 1x1 scalar - magnetometer: 1x1 scalar - time_offset: 1x1 scalar - system_time_at_creation: 1x1 scalar - timestamp_at_creation: 1x1 scalar - first_timestamp: 1x1 scalar - ntrode_id: 1x1 scalar - num_channels: 1x1 scalar - voltage_scaling: 1x1 scalar - threshold: 1x1 scalar - spike_invert: 1x3 sq_string - reference: 1x3 sq_string - filter: 1x2 sq_string - lowpassfilter: 1x1 scalar - highpassfilter: 1x1 scalar - spike_trigger_on_1: 1x1 scalar - fields: 1x2 struct Many of the fields above exist for all exported data types and contain metadata describing the data. For instance 'clockrate' contains the sampling rate of the data. The struct also contains type-specific fields such as 'threshold' and 'voltage_scaling.' Threshold is unique to the 'Spikes' data type, while 'voltage_scaling' is specific to voltage data recorded using neural data channels. Data formatting for Matlab -------------------------- Trodes data is exported in int16 format, and neural data values are stored as raw values that must be converted into microvolt scale. The following reformatting steps should be taken before proceeding with additional analysis. 1. Convert data to format Double using ``double()`` 2. Scale neural voltage data to microvolts (uV) by multipling by ``voltage_scaling`` **Example:** :: r = readTrodesExtractedDataFile('myrawdata.dat'); d = double(r.fields(1).data(:,1))*double(r.voltage_scaling); Data structure -------------- The extracted data will always be contained within 'fields,' which will always be a 1-by-N struct, where each struct in the array contains a different type of data. ``imported_data.fields`` The data are always stored in 'fields'. In the example using 'Spikes' data, the 'fields' struct array will be formatted as follows: ans = 1x2 struct array containing the fields: name type columns bytesPerItem data 'Name' describes the data: data.fields(1).name and = 'time' data.fields(2).name ans = waveformCh1 'type' tells you what format the data is stored: data.fields(2).type ans = int16 'data' contains the actual data. For spikes, this will be a N by 40 matrix, where N is the number of spikes in the data. data.fields(1).data contains each spike's time data.fields(2).data contains each spikes' waveform data If the nTrode has more than one channel, the array will have one extra entry per channel. In the case of spikes extracted from tetrode data, you will get a 1-by-5 struct, where the first struct is the time each spike occurred, and the next four structs contain the waveform data for each channel in the tetrode. Importing Trodes Data Into Python ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Once ``.dat`` data has been exported from Trodes, it can be imported into Python for secondary processing and analysis using scripts provided in the Resources > TrodesToPython folder contained within the Trodes installation folder. The scripts contained within will enable you to import your recording binary data and camera module video timestamps into Python for further evaluation and/or analysis. **readTrodesExtractedDataFile3:** - Reads in a single binary data (``.dat``) file exported from Trodes using the export utility. - Returns a Python dict containing the file metatdata as well as a Numpy array containing the binary data stored in the ‘data’ field. - Some Camera Module outputs such as position data also create a binary file which can be parsed using this function. **NOTE: numpy installation is required** For Python 2 support, use readTrodesExtractedDataFile2 contained within the same folder. Information about about the data and data type(s) contained within 'data' can be found in the 'fields' dict key entry. :: fields: