Sam(oa)² (OpenMP tasking)

Description

This version of Sam(oa)² uses OpenMP tasks to traverse single sections in the simulation. Similar to the work-sharing version, section start and end indices are still calculated per thread. However, now computations are encapsulated in an OpenMP task that will in most cases not be executed directly but enqueued in the OpenMP runtime. All threads in a process then participate in executing enqueued tasks, which allows load balancing between threads. A simplified version is illustrated in the following code snippet:

! executed by every thread in the parallel team
do time_step = 1, N_TIME_STEPS
    sec_idx_start = calculate_start_idx_for_thread()
    sec_idx_end   = calculate_end_idx_for_thread()
    
    do idx = sec_idx_start, sec_idx_end
        !$omp task default(shared) firstprivate(idx)
        traverse( sections(idx) )
        !$omp end task
    end do
    
    ! wait until all section traversals for the current thread have been completed
    !$omp taskwait
    
    exchange_boundary_data()
    apply_AMR()
end do

How to Build & Run

The repository provides two detailed README files with descriptions and instructions on how to build and run the various application versions. Furthermore, it provides some scripts for the experiments that have been carried out here. For the prerequisites, refer to the README file in the repository.

This version focuses on the OpenMP tasking version, that can be built following these steps:

# change to the scripts directoy
cd scripts/claix

# specify the paths to the Sam(oa)² repository and ASAGI installation directory
export SAMOA_DIR=/path/to/samoa
export ASAGI_DIR=/path/to/ASAGI

# make sure the script is executable
chmod u+x ./samoa_build_intel.sh

# compile the OpenMP tasking version
COMPILE_TASKING=1 ./samoa_build_intel.sh

After the application has been built successfully, the executable can be found in the bin directory located in the Sam(oa)² repository.

In order to run the OpenMP tasking version, execute the following steps:

# change to the scripts directoy
cd scripts/claix

# specify the paths to the Sam(oa)² repository and ASAGI installation directory
export SAMOA_DIR=/path/to/samoa
export ASAGI_DATA_DIR=/path/to/asagi/dir
# specify the path to a dummy output folder
export OUTPUT_DIR=/path/to/temp/output/folder # not used in this configuration but still required by program

# make sure the script is executable
chmod u+x ./samoa_run_intel.sh

# run the OpenMP tasking version
OMP_NUM_THREADS_VAR=11 NUM_RANKS=4 RUN_TASKING=1 ./samoa_run_intel.sh