The Pencil Code, a modular MPI code for partial differential equations and particles: multipurpose and multiuser-maintained

The Pencil Code is a highly modular physics-oriented simulation code that can be adapted to a wide range of applications. It is primarily designed to solve partial differential equations (PDEs) of compressible hydrodynamics and has lots of add-ons ranging from astrophysical magnetohydrodynamics (MHD) to meteorological cloud microphysics and engineering applications in combustion. Nevertheless, the framework is general and can also be applied to situations not related to hydrodynamics or even PDEs, for example when just the message passing interface or input/output strategies of the code are to be used. The code can also evolve Lagrangian (inertial and noninertial) particles, their coagulation and condensation, as well as their interaction with the fluid.


Summary
The Pencil Code is a highly modular physics-oriented simulation code that can be adapted to a wide range of applications.It is primarily designed to solve partial differential equations (PDEs) of compressible hydrodynamics and has lots of add-ons ranging from astrophysical magnetohydrodynamics (MHD) (A.Brandenburg and Dobler 2010) to meteorological cloud microphysics (Li et al. 2017) and engineering applications in combustion (Babkovskaia, Haugen, and Brandenburg 2011).Nevertheless, the framework is general and can also be applied to situations not related to hydrodynamics or even PDEs, for example when just the message passing interface or input/output strategies of the code are to be used.The 1 arXiv:2009.08231v1[astro-ph.IM] 17 Sep 2020 code can also evolve Lagrangian (inertial and noninertial) particles, their coagulation and condensation, as well as their interaction with the fluid.A related module has also been adapted to perform ray tracing and to solve the eikonal equation.
The code is being used for Cartesian, cylindrical, and spherical geometries, but further extensions are possible.One can choose between different time stepping schemes and different spatial derivative operators.High-order first and second derivatives are used to deal with weakly compressible turbulent flows.There are also different diffusion operators to allow for both direct numerical simulations (DNS) and various types of large-eddy simulations (LES).

High-level functionality
An idea about the range of available modules can be obtained by inspecting the examples under pencil-code/samples/.Those are low resolution versions related to applications published in the literature.Some of the run directories of actual production runs are published through Zenodo.Below a list of method papers that describe the various applications and tests: • Coagulation and condensation in turbulence (Johansen et al. 2008;Li et al. 2017), • Radiative transfer (Heinemann et al. 2006;Barekat and Brandenburg 2014; A. Brandenburg and Das 2020), • Chiral magnetic effect in relativistic plasmas (Schober et al. 2018)

Statement of need and purpose of software
The code is an easily adaptable tool for solving both standard MHD equations as well as others, such as the test-field equations.Significant amounts of runtime diagnostics as well as Python and IDL libraries for post-processing are available.
Among the currently 83 developers with check-in permission, there are currently 18 owners who can give others check-in permission.Of the developers, 34 have done more than 34 commits.Users have access to the latest development version and can ask to join the circle of developers by contacting one of the owners.
Every revision on GitHub is verified on 9 tests on travis-ci.com.The current version is also automatically being tested on 59 hourly tests and on 79 daily tests.Continuous progress on the code is driven by the research of individual developers.
Further developments and interactions between developers and users are being promoted through annual user meetings since 2004 and a newsletters since 2020.Since 2016, a steering committee of five elected owners reviews the progress and can take decisions of general concern to the Pencil Code community.

Ongoing research using the Pencil Code
Current research includes topics from stellar physics, interstellar and intercluster medium, the early universe, as well as from meteorology and engineering: small-scale dynamos and reconnection; primordial magnetic fields and decaying turbulence; gravitational waves from turbulent sources; planet formation and inertial particles; accretion discs and shear flows; coronal heating and coronal mass ejections; helical dynamos, helical turbulence, and catastrophic quenching; helioseismology; strongly stratified MHD turbulence and negative effective magnetic pressure instability; convection in Cartesian domains; global convection and dynamo simulations; turbulent transport and test-field methods; hydrodynamic and MHD instabilities and turbulence; chiral MHD; turbulent gaseous and solid combustion, particle clustering and deposition on solid walls, front propagation, radiation & ionization.
As of July 2020, 564 papers have been published that acknowledge use of the Pencil Code (A.Brandenburg 2020).

Key references
The Pencil Code is unique in two ways: the high level of flexibility and modularity, and the way it is organized (open source, distributed ownership, openness of development version).
Other software addressing related needs include: Athena, CO5BOLD, ENZO, MuRAM, NIRVANA, Stagger, ZEUS, and several other LES codes.There are also several other engineering DNS codes such as Sandia-3-Dimensional (S3D), a high-order compressible code, optimized for combustion, which is not open source, however.In addition, there are frameworks like Dedalus or Cactus, which allow one to program the equations in symbolic form.
retain copyright and release the work under a Creative Commons Attribution 4.0 International License (CC BY 4.0).