Dr Elke Roediger – The Largest Cosmic Structures Simulated on Viper

Researchers at the E. A. Milne Centre for Astrophysics at the University of Hull study the universe on all scales, ranging from solar physics to cosmology, using techniques of both observational and computational astrophysics. Within the Milne Centre, I lead a research group studying clusters of galaxies. One of our recent numerical simulations of galaxy cluster growth run on Viper is shown in Fig. 1. Our simulations are part of local, national and international collaborations with astrophysicists observing galaxy clusters across the electromagnetic spectrum at the world’s best facilities. By directly comparing our simulations to our collaborators’ observations, we study the formation of structure in the universe to understand why the present universe looks the way that it does. Our work focuses on three main topics: the growth of galaxy clusters, the plasma properties of the cluster atmospheres, and the evolution of galaxies in clusters.

In the early universe, matter was distributed almost homogeneously. The cosmic structures we observe today grew out of tiny inhomogeneities as gravity led to the contraction and collapse of matter in regions of slightly higher density. This in turn led the formation of the first stars, galaxies, and groups, and eventually the formation of clusters of galaxies, the largest gravitationally bound structures in the universe. Clusters of galaxies are still growing by accreting more galaxies and by colliding and merging with each other. Thus they give us the opportunity to observe cosmic structure formation in action, which is a focus of our research.

The cluster galaxies constitute only a small fraction of the total cluster mass. The majority of the baryonic mass, i.e. normal matter in clusters is found as a hot gas in the space between galaxies. However, the largest part of a galaxy cluster’s mass is found in dark matter, which dominates the motion of the cluster’s galaxies and binds the cluster atmosphere.

At temperatures of tens of millions of degrees the hot cluster, or atmosphere is a fully ionised plasma which is threaded with weak magnetic fields. Despite being a plasma, the cluster atmosphere can be modelled by hydrodynamics, i.e., similar to air. However, due to the complex interactions of
the charged plasma particles with each other and the magnetic fields, properties of the cluster “gas”
such as thermal conductivity and viscosity are still uncertain, and are a focus of our studies.

Galaxies inside and outside of clusters evolve differently. Figure 1 shows that galaxies can easily lose their own gas by moving through the cluster atmosphere. This gas loss is particularly important for spiral galaxies. This type of galaxy usually continuously forms new stars out of galaxy’s supply of cold gas. Thus, the loss of gas leads to star formation shutting down, changing many characteristics of the galaxy. Observations show that this can happen both quietly or in a last spectacular burst of star formation. Determining the conditions for the mode of shutdown is another area of our research.

Figure 1: Simulation of a massive galaxy falling into a galaxy cluster. The galaxy moves through the cluster atmosphere and thus experiences a head wind which strips off the galaxy’s gaseous atmosphere. The movie shows slices in the orbital plane of the galaxy. The panels on the right zoom in on the galaxy, the panels on the left show the cluster-scale view. The top panels colour-code the gas density, the bottom panels colour-code the fraction of galactic gas and thus highlight the mixing of the galactic and cluster gas. The presence of turbulent mixing or its suppression by, e.g., viscosity, can now be observed in real clusters. Our simulations provide the reference point to interpret the observations and thus measure the cluster gas properties. The full simulation box spans 12 Mpc (40 million light years). The peak resolution in the galaxy and its wake is 0.36 kpc (about 1000 light years). Thus, the simulation covers a dynamic range of 3×105 in spatial scales. Full movie can be downloaded from http://hpc.hull.ac.uk/media/MinorMerger_or_GasStrippingInGalaxyCluster.mov (25Mb)

The current members of the Milne Centre galaxy cluster team are PhD students Alexander Sheardown and Matthew Hunt, final year MPhys student Matthew Chan and final year BSc students Alexandra Doherty, Emily Wilcox, and Thomas Fish. Viper provides for us an ideal in-house access to HPC where we can develop new simulations and do production runs. Simulations are led by Alexander, Matthew and Matthew; Alexandra, Emily and Thomas analyse the simulation data to answer a variety of scientific questions.

Our main tool is FLASH (http://flash.uchicago.edu/site/), an adaptive mesh refinement hydrodynamics code. Its adaptive refinement capability is vital to our work because it allows us to cover the large dynamic range of scales involved. For example, the simulation in Fig. 1 uses a simulation box of 40 million light years along a side with smallest grid cells resolving scales of 1000 light years, spanning a dynamic range of a factor of 3×105. However, a large amount of that dynamic range is invested in capturing the cluster environment of the galaxy, and the galaxy’s atmosphere is resolved by only 120 grid cells across (at cluster core passage). The high resolution is maintained along a good part of the galaxy’s wake which allows us to study mixing of the stripped galactic gas with the ambient cluster gas. This simulation consumed 60,000 core hours, somewhat less than 24 hours on 100 nodes.

To showcase the potential of adaptive mesh simulations on FLASH, Alexander Sheardown and Matthew Hunt simulated a zoom-in on the galaxy in 2D, resolving the galaxy’s atmosphere with 8000 grid cells across its diameter. A snapshot and zoom-ins into the mixing regions are shown in Fig. 2. At this high resolution, even the 2D simulation consumes 24 hours on 128 nodes.

Figure 2: 2D high dynamic range simulation of galaxy gas stripping and mixing of the stripped gas with the ambient cluster gas. The image shows a density slice through the galaxy and zoom-ins on several regions. The simulation captures mixing features less than 1000 times smaller than the galactic atmosphere.

3D high dynamic range is essential for our simulations of gas mixing in cluster mergers and in galaxies’ wakes. 3D high dynamic range will be even more important to resolve star formation in a galaxy while following the galaxy’s motion through its host cluster, which requires improving resolution by about a factor of 10. The adaptive mesh capability of FLASH and the access to Viper computing time of the order of millions of core hours will enable us to do so.

We presented our first Viper results at an international conference (Physics of the Intracluster Medium: Theory and Computation, http://www.ftpi.umn.edu/workshops/2016-2017/ICM2016/) and have supported several submitted observational papers. Our current simulations on Viper will result in a series of papers lead by the Hull Milne Centre galaxy cluster researchers.

For more information on Dr Roediger’s work, please contact: e.roediger@hull.ac.uk
If you wish to contribute a case study based on your HPC work, please contact: viper@hull.ac.uk