UltraDark.jl: A Julia package for simulation of cosmological scalar fields

UltraDark.jl is a Julia package for the simulation of cosmological scalar fields. Scalar fields are proposed solutions to two of the fundamental questions in cosmology: the nature of dark matter and the universe's initial conditions. Modeling their dynamics requires solving the Gross-Pitaevskii-Poisson equations, which is analytically challenging. This makes simulations essential to understanding the dynamics of cosmological scalar fields. UltraDark.jl is an open, performant and user friendly option for solving these equations numerically.

Figure 1: Growth of density perturbations in a scalar field in an expanding background, as the universe's scale factor growths from a = 1.0 to a = 100.0.The density is measured in units of the universe's critical density ρcrit.This scenario is similar to that in (Musoke et al., 2020), in which the scalar field is an inflaton fragmenting after the end of primordial inflation.

Statement of need
Scalar fields are ubiquitous in physics, as solutions to partial differential equations describing the spatial variation of physical quantities.As dark matter candidates, scalar fields including axion-like particles (ALPs) would explain the nature of the missing 85% of the universe's matter (Adams et al., 2022;Ade & others, 2016;Hu et al., 2000;Matos et al., 2000).As inflaton candidates, scalar fields are proposed to cause a phase of accelerated expansion that sets the stage for big bang nucleosynthesis (Albrecht & Steinhardt, 1982;Amin et al., 2012;Guth, 1981;Linde, 1982).In each case, a scalar field ψ(t, x) represents the density of particles as a function of space and time.
Subject to reasonable conditions, a cosmological scalar field ψ whose particles have mass m obeys the Gross-Pitaevskii equation for the scalar field, coupled to the Poisson equation for the gravitational potential Φ where a(t) is the scale factor characterising the expansion of the universe.UltraDark.jlsolves Equation 1 and Equation 2 with a pseudo-spectral symmetrized split-step method, in which each time step consists of four sub-steps: where F is a Fourier transform, k are the corresponding frequencies, and h is the time step.UltraDark.jlhas adaptive time steps which allow it to accelerate simulations while preserving numerical convergence.This is particularly useful in an expanding universe, where the time step is roughly h ∝ a 2 .Such time steps result in orders of magnitude speedups when simulating collapse of an inflaton field in the early universe (Musoke et al., 2020).Julia (Bezanson et al., 2017) has seen increasing use in scientific computing; see for example Eschle et al. (2023) and Roesch et al. (2021) for overviews of its use in high energy physics and biology.The use of Julia is one of the choices that separates UltraDark.jlfrom similar codes.UltraDark.jluses Julia's rich parallelism capabilities.The Threads.@threads macro provides simple parallelisation of for loops.Folds.jlenables simple parallelisation of reduction operations (Arakaki, 2020).In a cluster environment, PencilArrays.jland PencilFFTs.jlenable straightforward cross-node parallelism, a capability that is challenging to reproduce in Python (Polanco, 2021a(Polanco, , 2021b)).The scaling of this two approaches to parallelism is demonstrated in Figure 2.
The features described above have allowed collaborators and I to produce results presented in 4 publications, each exploring an aspect of the small scale structure of ultralight dark matter.

Figure 2 :
Figure 2: Wall time for a single time step, as a function of number of CPUs.The points represent measured times and the lines represent theoretical 1/#CPU scalings.The circles and solid line are for grids constructed from Arrays and the triangles and dashed lines are for MPI-distributed PencilArrays.
Frank Wang, Risa Wechsler, Luna Zagorac for discussion and comments on various stages of the development of UltraDark.jl.Particular thanks to Noah Glennon as the first user of UltraDark.jl.Computations were performed on Marvin, a Cray CS500 supercomputer at UNH supported by the NSF MRI program under grant AGS-1919310.The author wishes to acknowledge the use of New Zealand eScience Infrastructure (NeSI) high performance computing facilities, consulting support and/or training services as part of this research.New Zealand's national facilities are provided by NeSI and funded jointly by NeSI's collaborator institutions and through the Ministry of Business, Innovation & Employment's Research Infrastructure programme.This work was performed in part at Aspen Center for Physics, which is supported by National Science Foundation under Grant No. PHY-1607611.This work was partially supported by a grant from the Sloan Foundation.Zhang, J., Liu, H., & Chu, M.-C.(2018).Cosmological simulation for fuzzy dark matter model.Frontiers in Astronomy and Space Sciences, 5. https://doi.org/10.3389/fspas.2018.00048