Sam(oa)²

Sam(oa)² stands for Space-Filling Curves and Adaptive Meshes for Oceanic And Other Applications and has been developed at TU Munich. Sam(oa)² is a PDE framework that can, for example, simulate wave propagations during tsunamis using adaptive meshes and space filling curves. It can either simulate built-in sample scenarios like radial_dam_break (perfectly balanced) and oscillating lake (imbalanced), or be used with datasets from real tsunamis by providing corresponding bathymetric and displacement data files.

The simulation performs a domain decomposition and creates an initial grid of sections and cells that is distributed across the participating processes. The overall number of sections is usually chosen as a multiple of the participating threads to have a sufficient degree of over-decomposition. Each section contains a number of cells. In each timestep of the simulation several computational steps are performed on each cell, where the computational complexity can vary depending on underlying characteristics like water height, wave speed and others. During the simulation, the Adaptive Mesh Refinement (AMR) algorithm might coarsen or refine cells in a section depending on the criteria mentioned above and whether cells represent dry land or water, which is illustrated in the following figure.

Example for Adaptive Mesh Refinement (AMR)

Thus, the number of cells and therefore also the workload per section might change over time and has the potential for creating load imbalances between threads within a process, as well as between processes.

The main time stepping loop in Sam(oa)² consists of several steps, the most important of which are illustrated in the following pseudocode.

do time_step = 1, N_TIME_STEPS
    !traverse all sections for the current rank -> the major part of the computation time is spent in that region
    traverse(sections)

    !wait until all traversals have been completed
    !How that is realized depends on the version of the program
    wait_for_results()

    !exchange halo/boundary data with neigboring sections/processes
    exchange_boundary_data()
    
    !apply an adaptive mesh refinement step to coarsen or refine cells
    apply_AMR()
end do