DBSP_DRP: A Python package for automated spectroscopic data reduction of DBSP data

DBSP_DRP is a python package that provides fully automated data reduction of data taken by the Double Spectrograph (DBSP) at the 200-inch Hale Telescope at Palomar Observatory (Oke&Gunn, 1982). The underlying data reduction functionality to extract 1D spectra, perform flux calibration and correction for atmospheric absorption, and coadd spectra together is provided by PypeIt (Prochaska et al., 2020). The new functionality that DBSP_DRP brings is in orchestrating the complex data reduction process by making smart decisions so that no user input is required after verifying the correctness of the metadata in the raw FITS files in a table-like GUI. Though the primary function of DBSP_DRP is to automatically reduce an entire night of data without user input, it has the flexibility for astronomers to fine-tune the data reduction with GUIs for manually identifying the faintest objects, as well as exposing the full set of PypeIt parameters to be tweaked for users with particular science needs. DBSP_DRP also handles some of the occasional quirks specific to DBSP, such as swapping FITS header cards, adding (an) extra null byte/s to FITS files making them not conform to the FITS specification, and not writing the coordinates of the observation to file. Additionally, DBSP_DRP contains a quicklook script for making real-time decisions during an observing run, and can open a GUI displaying a minimally reduced exposure in under 15 seconds. Docker containers are available for ease of deploying DBSP_DRP in its quicklook configuration (without some large atmospheric model files) or in its full configuration.

DBSP_DRP is a python package that provides fully automated data reduction of data taken by the Double Spectrograph (DBSP) at the 200-inch Hale Telescope at Palomar Observatory (Oke & Gunn, 1982). The underlying data reduction functionality to extract 1D spectra, perform flux calibration and correction for atmospheric absorption, and coadd spectra together is provided by PypeIt (Prochaska et al., 2020). The new functionality that DBSP_DRP brings is in orchestrating the complex data reduction process by making smart decisions so that no user input is required after verifying the correctness of the metadata in the raw FITS files in a table-like GUI. Though the primary function of DBSP_DRP is to autmatically reduce an entire night of data without user input, it has the flexibility for astronomers to fine-tune the data reduction with GUIs for manually identifying the faintest objects, as well as exposing the full set of PypeIt parameters to be tweaked for users with particular science needs. DBSP_DRP also handles some of the occasional quirks specific to DBSP, such as swapping FITS header cards, adding (an) extra null byte/s to FITS files making them not conform to the FITS specification, and not writing the coordinates of the observation to file. Additionally, DBSP_DRP contains a quicklook script for making real-time decisions during an observing run, and can open a GUI displaying a minimally reduced exposure in under 15 seconds. Docker containers are available for ease of deploying DBSP_DRP in its quicklook configuration (without some large atmospheric model files) or in its full configuration.

Statement of Need
Palomar Observatory, located near San Diego, CA, is a multinational observatory with a broad user base. Users come from large and small institutions, and their observing experience ranges 1 arXiv:2107.12339v1 [astro-ph.IM] 26 Jul 2021 from novice to expert. One responsibility for serving such a diverse user base is to provide software data reduction pipelines for the most frequently used instruments, such as the Palomar Double Spectrograph (DBSP). Although DBSP was commissioned in 1982, it remains the workhorse instrument of the 200" Hale Telescope. It is used on 42% of the nights in a year, comprising nearly all of the valuable "dark" (moonless) time. In previous years, standard astronomical practice left the data reduction up to the user. However, attitudes in instrument building have shifted since DBSP was built. The pipeline is now considered an indispensable component of the astronomical instrument. In fact, the difference between a good pipeline and a great pipeline means the difference between counting some of the photons vs. counting all of the photons.
Spectroscopy is a severe bottleneck in time-domain astronomy; currently less than 10% of discoveries are spectroscopically classified. Without a pipeline, data reduction is a difficult process and the standard method without a pipeline is to use IRAF, a 35 year old program on which development and maintenance was discontinued in 2013 and whose use is discouraged by many in the field e.g. Ogaz & Tollerud (2018). Needless to say, data reduction sans pipeline is extremely time-consuming. There is a clear need for a modern and stable automated data reduction pipeline for DBSP.
During observing runs, one would like to be able to quickly inspect data as it is taken, in order to ensure that it is of sufficient quality to do the desired science with. For objects whose brightness may have changed between a previous observation and the observing run, the observer may have uncertainties regarding how long of an exposure is needed to produce quality data. For very faint objects or objects in crowded fields, the observer may not even be sure that the telescope is pointed at the right object! A quicklook functionality, that can do a rudimentary reduction to correct for instrumental signatures and subtract light from the sky, revealing the spectra of the objects observed, can answer questions of exposure time and whether the object observed is the right one.
DBSP_DRP is currently being used by the ZTF Bright Transient Survey Perley et al., 2020), the ZTF Census of the Local Universe , and a program investigating ZTF Superluminous Supernovae (Lunnan et al., 2020;Chen et al., in preparation). Ravi et al. (2021) is the first (known) publication that used DBSP_DRP for data reduction. The development of DBSP_DRP also lays the groundwork towards a fully automated pipeline for the Next Generation Palomar Spectrograph that is planned to be deployed on the Palomar 200-inch Hale Telescope in 2022.