easyaccess: Enhanced SQL command line interpreter for astronomical surveys

easyaccess is an enhanced command line interpreter and Python package created to facilitate access to astronomical catalogs stored in SQL Databases. It provides a custom interface with custom commands and was specifically designed to access data from the Dark Energy Survey Oracle database, including autocompletion of tables, columns, users and commands, simple ways to upload and download tables using csv, fits and HDF5 formats, iterators, search and description of tables among others. It can easily be extended to another surveys or SQL databases. The package was completely written in Python and support customized addition of commands and functionalities.


Summary
easyaccess is an enhanced command line interpreter and Python package created to facilitate access to astronomical catalogs stored in SQL Databases.It provides a custom interface with custom commands and was specifically designed to access data from the Dark Energy Survey Oracle database, although it can easily be extended to another survey or SQL database.The package was completely written in Python and support customized addition of commands and functionalities.Visit https://github.com/mgckind/easyaccessto view installation instructions, tutorials, and the Python source code for easyaccess.

The Dark Energy Survey
The Dark Energy Survey (DES) (DES Collaboration 2005; DES Collaboration 2016) is an international, collaborative effort of over 500 scientists from 26 institutions in seven countries.The primary goals of DES are reveal the nature of the mysterious dark energy and dark matter by mapping hundreds of millions of galaxies, detecting thousands of supernovae, and finding patterns in the large-scale structure of the Universe.Survey operations began on on August 31, 2013 and will conclude in early 2019.For about 500 nights, DES has been taking thousands of deep images of southern sky, which are transferred and processed at the National Center for Supercomputing Applications (NCSA).The images are processed to produce catalogs of astronomical sources with hundreds of millions of entries (billions in the case of individual detections), describing the sources found within the images and other relevant metadata.A significant subset of the DES data was recently made public (DES Collaboration 2018) and can be accessed through several mechanisms including easyaccess and web interfaces that run easyaccess as a backend.This public release includes information for almost 400 million astrophysical sources and complementary tables to allow scientific analysis.

DES users
The first release of easyaccess was on February 17th, 2015 and since then, over 300 DES Collaborators have used it to access the DES databases (Figure 1).We note that roughly 800 DES accounts exist, but this includes all database users including those that created accounts before the release of easyaccess.In August 2018 (version 1.4.4),we added support for the public DES data release, and since then we have increased the number of public users.

Features
easyaccess has a variety of features including a history of past commands and smart tab auto-completion for commands, functions, columns, users, tables, and paths.Tables can be written directly into comma-separated-value (CSV) or white-space separated text files, FITS (Wells, Greisen, and Harten 1981) files, and HDF5 (The HDF Group 1997-2018) files.It provides an iteration scheme to avoid memory constraints when retrieving large tables.Tables can also be displayed on the command line and most of the formatting is done using pandas.Similarly, privileged users can easily upload tables to the database from any of the file format described above in order to share data with other users.The uploading mechanism is done chunk-wise, allowing large tables to be loaded while keeping memory usage low.
In addition, there are a variety of customized functions to search and describe the tables, search for users and user tables, check quota usage, check the Oracle execution plan, and soon the ability to run asynchronous jobs through a dedicated server.There are dozens of other minor features that allow for a seamless experience while exploring and discovering data within the hundreds of tables inside the DB.
One can also load SQL queries from a file into the database, or run SQL queries inside the easyaccess python module in another IDE.Most of the features are also exposed Users can also use easyaccess to submit and request cutouts around specific positions or objects which are generated from the images.This allows better integration with other data services for a richer scientific workflow.

Architecture
We have included a simplified UML diagram describing the architecture and dependencies of easyaccess (Figure 3). Figure 3 shows only the different methods for a given class and the name of the file hosting a given class.The main class, easy_or(), inherits all methods from all different subclasses, making this model flexible and extendable to other surveys or databases.These methods are then converted to command line commands and functions that can be called inside easyaccess.Given that there are some DES specific functions, we have moved DES methods into a separate class DesActions().

Installation
To download easyaccess you can clone the source code from GitHub at https://github.com/mgckind/easyaccess or follow any of the standard installation channels described below.

Figure 1 :
Figure 1: Number of user since first version