This version of Sam(oa)² uses a classical work-sharing approach to distribute the workload (sections) within a process to the participating threads. The structure of the implementation is equivalent to a static schedule in OpenMP, where the number of sections is equally distributed across threads. A simplified version is illustrated in the following code snippet:
! executed by every thread in the parallel team do time_step = 1, N_TIME_STEPS sec_idx_start = calculate_start_idx_for_thread() sec_idx_end = calculate_end_idx_for_thread() do idx = sec_idx_start, sec_idx_end traverse( sections(idx) ) end do ! After the serial loop has been executed, all section traversals for the current thread have been completed ! Boundary exchange can be initiated for those sections exchange_boundary_data() apply_AMR() end do
As sections might exhibit varying execution times, this might lead to load imbalance within a process and waiting times from idle threads at the barrier. Due to the internal design and implementation, which is a bit more complicated, Sam(oa)² does not use
!$omp do schedule(static). Instead, the parallel region is spawned at the outer level and the time stepping loop is executed by every thread. Start and end indices are manually calculated by every thread. This design choice complicates applying a dynamic schedule to tackle the load imbalance.
The repository provides two detailed README files with descriptions and instructions on how to build and run the various application versions. Furthermore, it provides some scripts for the experiments that have been carried out here. For the prerequisites, refer to the README file in the repository.
This version focuses on the work-sharing version, that can be built following these steps:
# change to the scripts directoy cd scripts/claix # specify the paths to the Sam(oa)² repository and ASAGI installation directory export SAMOA_DIR=/path/to/samoa export ASAGI_DIR=/path/to/ASAGI # make sure the script is executable chmod u+x ./samoa_build_intel.sh # compile the work-sharing version COMPILE_WS=1 ./samoa_build_intel.sh
After the application has been built successfully, the executable can be found in the bin directory located in the Sam(oa)² repository.
In order to run the work-sharing version, execute the following steps:
# change to the scripts directoy cd scripts/claix # specify the paths to the Sam(oa)² repository and ASAGI installation directory export SAMOA_DIR=/path/to/samoa export ASAGI_DATA_DIR=/path/to/asagi/dir # specify the path to a dummy output folder export OUTPUT_DIR=/path/to/temp/output/folder # not used in this configuration but still required by program # make sure the script is executable chmod u+x ./samoa_run_intel.sh # run the work-sharing version OMP_NUM_THREADS_VAR=11 NUM_RANKS=4 RUN_WS=1 ./samoa_run_intel.sh