WrightTools : a Python package for multidimensional spectroscopy

• resolving congested states (Donaldson et al., 2008; W. Zhao & Wright, 1999), • extracting spectra that would otherwise be selection-rule disallowed (Boyle, NeffMallon, & Wright, 2013; Boyle, Neff-Mallon, Handali, & Wright, 2014), • resolving fully coherent dynamics (Pakoulev et al., 2009), • measuring coupling (Wright, 2011), • and resolving ultrafast dynamics (Czech et al., 2015; Smallwood & Cundiff, 2018).

In our view, the most exciting aspect of these techniques is the vast number of different approaches that scientists can take to learn about material quantum states.Often, a number of these experiments can be accomplished with a single instrument.The diversity of related-but-unique approaches to interrogating quantum systems is an important strength of MDS.
Advancements in optics and laser science are bringing ultrafast multidimensional spectroscopy to more and more laboratories around the world.At the same time, increasing automation and computer control are allowing traditionally "one-dimensional" spectroscopies to be recorded against other dimensions.
Due to its diversity and dimensionality, MDS data is challenging to process and visualize.The tools that scientists develop to process one experiment may not work when different experimental variables are explored.Historically, MDS practitioners have developed custom, one-off data processing workflows that need to be radically changed when new experiments are undertaken.These changes take time to implement, and can become annoyances or opportunities for error.Even worse, the challenge of designing a new processing workflow may dissuade a scientist from creatively modifying their experimental strategy, or comparing their data with data taken from another instrument.This limit to creativity and flexibility defeats one of the main advantages of the MDS "family approach".
WrightTools is a new Python package that is made specifically for multidimensional spectroscopy.It aims to be a core toolkit that is general enough to handle all MDS datasets and processing workloads.Being built for and by MDS practitioners, WrightTools has an intuitive, high-level, object-oriented interface for spectroscopists.To our knowledge, WrightTools is the first MDS-focused toolkit to be freely avaliable and openly licensed.

Challenges and Implementation
There are several recurring challenges in MDS data processing and representation: • There are no agreed-upon file formats.Files generated by researchers may have inconsistent internal conventions, and they often fail to be fully self-describing.• There is a great diversity of dataset types.The same instrument is capable of producing datasets with many different combinations of scanned hardware.• Dataset size may be large enough to run into computer memory limits.
• Dataset dimensionality is large enough to represent challenges in human interaction and visualization.
The excellent Scientific Python ecosystem is well suited to address all of these challenges (Travis E. Oliphant, 2007).Numpy supports interaction with and manipulation of multidimensional arrays (Travis E Oliphant, 2006).Matplotlib supports one, two, and even three-dimensional plotting (Hunter, 2007).h5py (Collette, 2013) interfaces with hdf5 (The HDF Group, 1997), allowing for storage and memory-safe access to large multidimensional arrays in a binary format that can be accessed from a variety of different popular languages, including MATLAB and Fortran.WrightTools does not intend to replace or reimplement these core libraries.Instead, WrightTools offers an interface that impedance-matches multidimensional spectroscopy and Scientific Python.
WrightTools defines a universal MDS data format: the wt5 file.These are simply hdf5 files with certain internal conventions that are designed for MDS.These internal conventions enable the flexibility and ease-of-use that we discuss in the rest of this section.Instances of WrightTools's classes dynamically interact with the multidimensional spectroscopic data within these files.These classes are children of h5py classes.WrightTools offers a variety of functions that try hard to convert data stored in various other formats to wt5.
WrightTools defines a unique and flexable strategy of storing and manipulating MDS datasets.A single instance of the WrightTools.Data class is implemented as a group containing many separate arrays.There are two principle multidimensional array classes: Channel and Variable.Conceptually, these correspond to independent (scanned) dimensions-"variables"-and dependent (measured) signals-"channels".Channels typically contain measured signals from all of the different sensors that are employed simultaneously during a MDS experiment.Variables contain coordinates of different light manipulation hardware that are scanned against each-other to make up an MDS experiment.All variables are recorded, including coordinates for hardware that are not actually moved during that experiment (an array with one unique value) or other independent variables, such as lab time.
There can be many variables that change in the context of a single MDS experiment.
The typical spectroscopist only really cares about a small subset of these variables, but exactly what subset matters may change as different strategies are used to explore the dataset.Furthermore, it is often useful to "combine" multiple variables using simple algebraic relationships to exploit the natural symmetry of many MDS experiments and to draw comparisons between different members of the MDS family (Neff-Mallon & Wright, 2017).In light of these details, WrightTools provides a high-level Axis class that allows users to transparently define which variables, variable relationships, and unit conventions are important to them for representation and manipulation.Each Axis contains an expression, which dictates its relationship with one or more variables.Given 5 variables with names ['w1', 'w2', 'wm', 'd1', 'd2'] , example valid expressions include 'w1', 'w1=wm', 'w1+w2', '2*w1', 'd1-d2', and 'wm-w1+w2'.Users may treat axes like multidimensional arrays, using __getitem__ syntax and slicing, but axes do not themselves contain arrays.Instead, the appropriate axis value at each dataset coordinate is computed on-the-fly using the given expression.Users may at any time change their axes by simply calling transform with new expressions.
WrightTools offers a suite of data manipulation tools with MDS in mind.Users can access portions of their data using high-level methods like chop, split, and clip.They can process their data using simple mathematical operations or more specific tools like level, gradient, collapse, and smooth.Users can even join multiple datasets together, creating higher-dimensional datasets when appropriate.All of these operations refer to the self-describing internal structure of the wt5 file wherever possible.Users are not asked to refer to the specific shape and indicies of their data arrays.Instead, they deal with simple axis expressions and unit-aware coordinates.
WrightTools offers a set of "artists" to quickly draw typical representations.These make it trivial to make beautiful Matplotlib representations of MDS datasets.Again, the self-describing internal structure is capitalized upon, auto-filling labels (including units, symbols, and expressions) and auto-scaling axes.For higher-than-two dimensional datasets, WrightTools makes it easy to plot many separate figures that can be looped through using an image viewer or stitched into a looping animated gif.A convenience function, interact2D, allows users to explore a complete dataset using matplotlib's builtin widgets.