21cmFAST v3: A Python-integrated C code for generating 3D realizations of the cosmic 21cm signal.

The ﬁeld of 21-cm cosmology – in which the hyperﬁne spectral line of neutral hydrogen (appearing at the rest-frame wavelength of 21 cm) is mapped over large swathes of the Universe’s history – has developed radically over the last decade. The promise of the ﬁeld is to revolutionize our knowledge of the ﬁrst stars, galaxies, and black holes through the timing and patterns they imprint on the cosmic 21-cm signal. In order to interpret the eventual observational data, a range of physical models have been developed – from simple analytic models of the global history of hydrogen reionization, through to fully hydrodynamical simulations of the 3D evolution of the brightness temperature of the spectral line. Between these extremes lies an especially versatile middle-ground: fast semi-numerical models that approximate the full 3D evolution of the relevant ﬁelds: density, velocity, temperature, ionization


Summary
The field of 21-cm cosmology -in which the hyperfine spectral line of neutral hydrogen (appearing at the rest-frame wavelength of 21 cm) is mapped over large swathes of the Universe's history -has developed radically over the last decade.The promise of the field is to revolutionize our knowledge of the first stars, galaxies, and black holes through the timing and patterns they imprint on the cosmic 21-cm signal.In order to interpret the eventual observational data, a range of physical models have been developed -from simple analytic models of the global history of hydrogen reionization, through to fully hydrodynamical simulations of the 3D evolution of the brightness temperature of the spectral line.Between these extremes lies an especially versatile middle-ground: fast semi-numerical models that approximate the full 3D evolution of the relevant fields: density, velocity, temperature, ionization, and radiation (Lyman-alpha, neutral hydrogen 21-cm, etc.).These have the advantage of being comparable to the full first-principles hydrodynamic simulations, but significantly quicker to run; so much so that they can be used to produce thousands of realizations on scales comparable to those observable by upcoming low-frequency radio telescopes, in order to explore the very wide parameter space that still remains consistent with the data.Amongst practitioners in the field of 21-cm cosmology, the 21cmFAST program has become the de facto standard for such semi-numerical simulators.21cmFAST (Mesinger and Furlanetto 2007;Mesinger, Furlanetto, and Cen 2011) is a high-performance C code that uses the excursion set formalism (Furlanetto, Zaldarriaga, and Hernquist 2004) to identify regions of ionized hydrogen atop a cosmological density field evolved using first-or secondorder Lagrangian perturbation theory (Zel'Dovich 1970;Scoccimarro and Sheth 2002), tracking the thermal and ionization state of the intergalactic medium, and computing X-ray, soft UV and ionizing UV cosmic radiation fields based on parametrized galaxy models.For example, the following figure contains slices of lightcones (3D fields in which one axis corresponds to both spatial and temporal evolution) for the various component fields produced by 21cmFAST.
However, 21cmFAST is a highly specialized code, and its implementation has been quite specific and relatively inflexible.This inflexibility makes it difficult to modify the behaviour of the code without detailed knowledge of the full system, or disrupting its workings.This lack of modularity within the code has led to widespread code "branching" as researchers hack new physical features of interest into the C code; the lack of a streamlined API has led derivative codes which run multiple realizations of 21cmFAST simulations (such as the Monte Carlo simulator, 21CMMC, Greig and Mesinger 2015) to re-write large portions of the code in order to serve their purpose.It is thus of critical importance, as the field moves forward in its understanding -and the range and scale of physical models of interest continues to increase -to reformulate the 21cmFAST code in order to provide a fast, modular, well-documented, well-tested, stable simulator for the community.

Features of 21cmFAST v3
This paper presents 21cmFAST v3+, which is formulated to follow these essential guiding principles.While keeping the same core functionality of previous versions of 21cmFAST, it has been fully integrated into a Python package, with a simple and intuitive interface, and a great deal more flexibility.At a higher level, in order to maintain best practices, a community of users and developers has coalesced into a formal collaboration which maintains the project via a Github organization.This allows the code to be consistently monitored for quality, maintaining high test coverage, stylistic integrity, dependable release strategies and versioning, and peer code review.It also provides a single point-of-reference for the community to obtain the code, report bugs and request new features (or get involved in development).
A significant part of the work of moving to a Python interface has been the development of a robust series of underlying Python structures which handle the passing of data between Python and C via the CFFI library.This foundational work provides a platform for future versions to extend the scientific capabilities of the underlying simulation code.The primary new usability features of 21cmFAST v3+ are: • Convenient (Python) data objects which simplify access to and processing of the various fields that form the brightness temperature.• Enhancement of modularity: the underlying C functions for each step of the simulation have been de-coupled, so that arbitrary functionality can be injected into the process.• Conversion of most global parameters to local structs to enable this modularity, and also to obviate the requirement to re-compile in order to change parameters.• Simple pip-based installation.
• Robust on-disk caching/writing of data, both for efficiency and simplified reading of previously processed data (using HDF5).• Simple high-level API to generate either coeval cubes (purely spatial 3D fields defined at a particular time) or full lightcone data (i.e.those coeval cubes interpolated over cosmic time, mimicking actual observations).• Improved exception handling and debugging.
• Simple configuration management, and also more intuitive management for the remaining C global variables.• Comprehensive API documentation and tutorials.
• Comprehensive test suite (and continuous integration).
While in v3 we have focused on the establishment of a stable and extendable infrastructure, we have also incorporated several new scientific features, appearing in separate papers: • Generate transfer functions using the CLASS Boltzmann code (Lesgourgues 2011).
21cmFAST is still in very active development.Amongst further usability and performance improvements, future versions will see several new physical models implemented, including milli-charged dark matter models (Muñoz, Dvorkin, and Loeb 2018) and forward-modelled CMB auxiliary data (Qin, Poulin, et al. 2020).
In addition, 21cmFAST will be incorporated into large-scale inference codes, such as 21CMMC, and is being used to create large data-sets for inference via machine learning.We hope that with this new framework, 21cmFAST will remain an important component of 21-cm cosmology for years to come.

Examples
21cmFAST supports installation using conda, which means installation is as simple as typing conda install -c conda-forge 21cmFAST.The following example can then be run in a Python interpreter.

Performance
Despite being a Python code, 21cmFAST v3 does not diminish the performance of previous pure-C versions.It utilises CFFI to provide the interface to the C-code through Python, which is managed by some custom Python classes that oversee the construction and memory allocation of each C struct.
OpenMP parallelization is enabled within the C-code, providing excellent speed-up for large simulations when performed on high-performance machines.
Note that while a full light-cone simulation can be expensive to perform, it only takes 2-3min to calculate a Coeval box (excluding the initial conditions).For instance, the aforementioned timing for v3 includes 80 minutes to generate the initial condition, which also dominates the maximum RAM required, with an additional ~4 minutes per snapshot to calculate all required fields of perturbation, ionization, spin temperature and brightness temperature.
To guide the user, we list some performance benchmarks for variations on this simulation, run with 21cmFAST v3.0.2.Note that these benchmarks are subject to change as new minor versions are delivered; in particular, operational modes that reduce maximum memory consumption are planned for the near future.At this time, the 21cmFAST team suggests using 4 or fewer shared-memory cores.However, it is worth noting that as performance does vary on different machines, users are recommended to calculate their own scalability.
retain copyright and release the work under a Creative Commons Attribution 4.0 International License (CC-BY).

Figure 2 :Figure 3 :
Figure 2: Brightness temperature lightcone produced by the example code in this paper.