Beyond the data parallel paradigm: Issues and options

Abstract

Currently, the predominant approach in compiling a program for parallel execution on a distributed memory multiprocessor is driven by the data parallel paradigm, in which user-specified data mappings are used to derive computation mappings via ad hoc rules such as owner-computes. We explore a more general approach which is driven by the selection of computation mappings from the program dependence constraints, and by the selection of dynamic data mappings from the localization constraints in different computation phases of the program. We state the optimization problems addressed by this approach and outline the solution methods that can be used. We believe that this approach provides promising solutions beyond what can be achieved by the data parallel paradigm. The paper outlines the general program model assumed for this work, states the optimization problems addressed by the approach and presents solutions to these problems.