# Parallel library file I/O

Pattern addressed: Sequential ASCII file I/O

In this pattern data held on all processes is read or written to an ASCII file by a single process. This is inefficient for several reasons: (more...)

This best practice uses a parallel library for file I/O to write to a single binary file. A parallel library can make optimal use of any underlying parallel file system, and will give better performance than serial file I/O. Additionally, reading and writing binary is more efficiency than writing the equivalent ASCII data.

Two obvious choices of parallel libraries are MPI I/O and NetCDF, the latter has advantages in terms of portability.

The following pseudo-code shows the structure of the file I/O when writing data. In parallel all processes open a single file, and then write size elements held in array data to it. In this example size is constant across all processes.

get_my_process_id(proc_id)
offset = proc_id * size * sizeof(double)

filename = "data_file"
open_file(filename)

parallel_write_to_file(data, offset, filename)
close_file(filename)


The process is reversed when reading data from file, as follows.

get_my_process_id(proc_id)
offset = proc_id * size * sizeof(double)

filename = "data_file"
open_file(filename)