Directory structure and reference.txt files

spike2 <tab> 1 <tab> singleEC

tp <tab> 1 <tab> prairietp

(Again there are no spaces between the entries and the tabs.)

The field ref allows one to indicate different records. For example, suppose one advanced the electrode that was recording spike1 to a new location, but left all the other recording elements in the same positions. Then the next directory's reference.txt file might look like:

name <tab> ref <tab> type

spike1 <tab> 2 <tab> singleEC

spike2 <tab> 1 <tab> singleEC

tp <tab> 1 <tab> prairietp

Usually, I try to organize experiments according to the following structure, which dovetails with the dirstruct object class in Matlab. (History and credit: this is based on the file organization scheme used by Ken Sugino and Chris Hempel in Sacha Nelson's lab, 1999-2005.) This allows me to write code that can use the dirstruct object class to traverse the directory tree and automatically analyze and interpret the contents of the experiment.

Each experiment typically has its own directory, such as

2010-07-14

and 'tests' or episodes of data acquisition are in subdirectories, such as t00001, t00002, etc., although they can have any name.

The contents of these directories can be specified by the content of a file called reference.txt. The reference.txt file is organized as a tab-delimited text file with the first row containing headers:

name <tab> ref <tab> type

The <tab> just indicates a tab character; there should be no spaces between the words name, ref, and type, only tabs, and a return after type. name can be any string with no spaces, ref is a reference number (an integer), and type is some description of the record type that has meaning for your program. For example, a directory that has 2 single unit extracellular recordings and a two-photon recording might have a reference.txt file like the following:

name <tab> ref <tab> type

spike1 <tab> 1 <tab> singleEC

to indicate that the spike1 records in the different directories correspond to different single units (but the same single unit for the spike2 field and the same 2-photon recording location as the previous directory).

The dirstruct object can read and write variables to a '.mat' file called experiment, which is located in a subdirectory called analysis. One can use the Matlab function load to read variables from this file, and one can use saveexpvar to write named variables to this file easily. It is a bit annoying that I did not name this file experiment.mat, because this means one must always pass the '-mat' flag to the load function in order for Matlab to treat it as a '.mat' file. Within this directory, one can use a directory called scratch to write temporary files or files that might be used in analysis (such as spike shapes, cell ROI locations, etc).

For example, a complete (but really short) experiment directory tree might look like, with comments:

2010-07-14/

t00001/ % first episode of acquisition

reference.txt % describes what is in this record

mydata.dat % some data that some device wrote during the acquisition

t00002/

reference.txt

mydata.dat

analysis/ % the 'special' analysis directory

experiment % the 'special' analysis file, in .mat format

scratch/ % a directory that any software can use for writing files

whateverfilesIwant

histology/ % here's a manually-created directory; it has no reference.txt so matlab's dirstruct will ignore it

image1.tiff % but I can put say, some images here so I can keep my data organized

image2.tiff

image3.tiff

2010-07-14log.xls % and I can put a log file in here, too; dirstruct will ignore it