pMatlab: Parallel Matlab Toolbox

Software Engineer: Ms. Nadya T. Bliss
Architect: Dr. Jeremy Kepner

Contact: pmatlab@ll.mit.edu

Contributors:  Nadya T. Bliss, Jeremy Kepner, Bob Bond, Andy Funk, Ryan Haney, Hahn Kim, Julie Mullen, Albert Reuther, Edmund Wong.

REQUEST TO ALL USERS: Please read all of this 7-page manual (especially
the section on ERROR HANDLING) before starting to use pMatlab.

It is also helpful to read the 7-page MatlabMPI manual
(see pMatlab/MatlabMPI/README).

PC users should read the 1 page manual for running MatlabMPI on a PC
(see pMatlab/MatlabMPI/README.pc).

INTRODUCTION

MATLAB® is the dominant programming language for implementing numerical computations and is widely used for algorithm development, simulation, data reduction, testing and system evaluation. Many of these computations could benefit from faster execution on a parallel computer. There have been many previous attempts to provide an efficient mechanism for running Matlab programs on parallel computers. pMatlab provides a set of Matlab data structures and functions that implement distributed Matlab arrays. Parallel array programming has proven to be an effective programming style for a wide variety of parallel applications and is consistent with standard Matlab programming style. The primary advantages of parallel array programming are:

  • Message passing is done implicitly
  • Existing Matlab program can be made parallel with modifications to a handful of statements

DOWNLOAD

pMatlab_v1.0.tar.gz

REQUIREMENTS
pMatlab uses MatlabMPI for launching programs and communicating between processors and thus has the following requirements

  • Matlab license
  • File system visible to all processors

On shared memory systems, pMatlab only requires a single Matlab license as each user is allowed to have many Matlab sessions. On distributed memory systems, pMatlab will require one Matlab license per machine. Because MatlabMPI uses file i/o for communication, there must be a directory that is visible to every machine (this is usually also required in order to install Matlab). This directory defaults to the directory that the program is launched from, but can be changed when you launch your pMatlab program.

INSTALLING AND RUNNING:

  1. Unpack pMatlab directory into a location that is visible
    to all computers.
  2. Add pMatlab/MatlabMPI/src and pMatlab/src directories to
    matlab path. For example, you might put

    3. addpath ~/pMatlab/MatlabMPI/src
    addpath ~/pMatlab/src
    in your matlab/startup.m file.

  3. PC users should also add
    addpath .\MatMPI
    please see pMatlab/MatlabMPI/README.pc for more info
  4. OPTIONAL: You may need to read and edit
    pMatlab/MatlabMPI/src/MatMPI_Comm_settings.m
    to customize the behavior MatlabMPI for your system.
  5. Start Matlab.
  6. Type "help pMatlab" to get a list of all the functions.
  7. Type "help function_name" to get more info on a specific function.
  8. Go to the "examples" directory and type "RUN" to run the
    first example, which does a parallel FFT.
  9. Look at MatMPI/*.out to output from the different
    processors.
  10. To run other examples, change the "mFile" variable in
    examples/RUN.m to select any of the other examples.

    11. fftTest: Trivially parallel fft.
    pStreams: Parallel Stream Add, Copy, Scale and Triad
    CornerTurn: Redistribution for pipeline parallel processing
    pIO: Scalable parallel file I/O)
    ClutterSim: Basic data parallel processing.
    GeneratorProcessor: Multi-stage pipeline parallel processing.
    pBlurimage: Uses overlap to do parallel convolution
    GetPut: Performance of different Get/Put styles on matrices.

  11. To run using a different number of processors,
    change the "Ncpus" variable in examples/RUN.m. All the
    examples should run with up to 8 processors.
  12. To select which machines to run on change the "machines"
    variable in examples/RUN.m. The machines variable can
    be of the following form:

    machines = {};
       Run on a local processor.

    machines = {'machine1' 'machine2'}) );
       Run on multiprocessors.

    machines = {'machine1:dir1' 'machine2:dir2'}) );
       Run on multiprocessors and communicate via dir1 and dir2,
       which must be visible to both machines.

ERROR HANDLING
pMatlab uses the MatlabMPI approaches to handling errors.

  1. If an error is encountered and your Matlab script has an "exit" statement
    (which it should) then all the Matlab processes should die gracefully.
  2. If a Matlab job is waiting for a message that will never arrive, then
    you will have to kill it by hand by starting a new Matlab session and
    typing:

    3. MPI_Abort

    If this doesn't work, you will need to log into each machine, type
    "top" and kill the Matlab processes one by one.

  3. MatlabMPI can leave a variety of files lying around, which are best
    to delete once an error has occurred by starting Matlab and typing:
    MatMPI_Delete_all

If this doesn't work, you can delete the files by hand. The files can be found in two places - the launching directory and the communication directory (which by default are the same place).

In the launch directory you may have leftover files that look like:

MatMPI/*

In the communication directory you may have leftover files that look like:

p<rank>_p<rank>_t<tag>_buffer.mat
p<rank>_p<rank>_t<tag>_lock.mat

It is very important that you delete these files. In general, if you are using a public directory to do communication (e.g.; /tmp), you should create a subdirectory first (e.g.; /tmp/joe) and use this directory for communication.

WARNING: There is currently a bug with the Matlab GUI on Linux.
If you get the following error (or any other weird error):

??? Error using ==> mkdir
tcsh: No entry for terminal type "'MATLAB Command Window'"tcsh: using
dumb terminal settings.

Then you should try deleting your ~/.matlab file and run your MatlabMPI
programs from the command line using the command: matlab -nojvm

FILES

Description of files/directories:

README This file.
doc/ Additonal documentation on pMatlab
tutorial/ pMatlab tutorial
examples/ Directory containing examples and benchmark programs.
src/ pMatlab source files
MatlabMPI/ MatlabMPI library directory (see MatlabMPI/README)

doc/

pMatlab_into.pdf Introduction to Parallel Programming and pMatlab.
pMatlab_param_sweep.pdf Writing Parameter Sweep Applications with pMatlab.
pMatlab_v0.7_func_ref.pdf pMatlab Function Reference.
Profiling_pMatlab_MatlabMPI.pdf Profiling pMatlab and MatlabMPI.

tutorial/

pMatlabTutotial.ppt pMatlab tutorial power point slides.
ZoomImage.m Serial implementation of zooming in on an image.
mpiZoomImage.m MatlabMPI implementation of zooming in on an image.
pZoomImage.m pMatlab implementation of zooming in on an image.
pBeamformer.m pMatlab implementation of beamformer.
RUN.m Generic scirpt for running pMatlab programs.

examples/

fftTest.m Trivially parallel fft.
pStreams.m Parallel Stream Add, Copy, Scale and Triad.
CornerTurn.m Redistribution for pipeline parallel processing.
pIO.m Scalable parallel file I/O.
ClutterSim.m Basic data parallel processing.
GeneratorProcessor.m Multi-stage pipeline parallel processing.
pBlurimage.m Uses overlap to do parallel convolution.
GetPut.m Performance of different Get/Put styles on matrices.
pMandelbrot.m Computes the Mandelbrot set
RUN.m Generic script for running pMatlab programs

scr/

---------- User Functions/Scripts ----------
pMatlab_Init Initializes all the needed MPI variables.
pMatlab_Finalize Exits non-leader MATLAB processes.
pMatlab_ver Returns the pMatlab version as a string
synch No-op if X is a DOUBLE.
dmat/display Display distributed array.
dmat/double Convert each local part of the DMAT to double precision.
dmat/eq Equal.
dmat/int16 Convert each local part of the DMAT to signed 16-bit integers.
dmat/int32 Convert each local part of the DMAT to signed 32-bit integers.
dmat/int64 Convert each local part of the DMAT to signed 64-bit integers.
dmat/int8 Convert each local part of the DMAT to signed 8-bit integers.
dmat/single Convert each local part of the DMAT to single precision.
dmat/size Size of the distributed array.
dmat/sparse Converts a distributed matrix to a sparse distributed matrix
dmat/subsasgn Subscripted assignment to a distributed object.
Called for syntax A(I) = B.
Should not be called directly.
dmat/subsref Subscripted reference. Called for syntax A(I) = B.
dmat/synch Syncronize the data in the distribute matrix.
dmat/uint16 Convert each local part of the DMAT to unsigned 16-bit integers.
dmat/uint32 Convert each local part of the DMAT to unsigned 32-bit integers.
dmat/uint64 Convert each local part of the DMAT to unsigned 64-bit integers.
dmat/uint8 Convert each local part of the DMAT to unsigned 8-bit integers.
map/display Display map object.
map/eq == Equal.
map/map Map class constructor.
map/ne ~= Not equal.
map/ones Ones distributed array.
map/rand Rand distributed array.
map/spalloc Sparse distributed array.
map/subsasgn Subscripted assignment. Should not be called directly.
map/subsref Subscripted reference. Should not be called directly.
map/zeros Zeros distributed array.

---------- Elementwise Mathematical Functions ----------
dmat/abs Absolute value.
dmat/acos Inverse cosine.
dmat/acosd Inverse cosine, result in degrees.
dmat/acosh Inverse hyperbolic cosine.
dmat/acot Inverse cotangent.
dmat/acotd Inverse cotangent, result in degrees.
dmat/acoth Inverse hyperbolic cotangent.
dmat/acsc Inverse cosecant.
dmat/acscd Inverse cosecant, result in degrees.
dmat/acsch Inverse hyperbolic cosecant.
dmat/angle Phase angle.
dmat/asec Inverse secant.
dmat/asecd Inverse secant, result in degrees.
dmat/asech Inverse hyperbolic secant.
dmat/asin Inverse sine.
dmat/asind Inverse sine, result in degrees.
dmat/asinh Inverse hyperbolic sine.
dmat/atan Inverse tangent.
dmat/atand Inverse tangent, result in degrees.
dmat/atanh Inverse hyperbolic tangent.
dmat/ceil Round towards plus infinity.
dmat/complex Construct complex distributed matrix from real distributed matrix.
dmat/conj Complex conjugate.
dmat/cos Cosine.
dmat/cosd Cosine of argument in degrees.
dmat/cosh Hyperbolic cosine.
dmat/cot Cotangent.
dmat/cotd Cotangent of argument in degrees.
dmat/coth Hyperbolic cotangent.
dmat/csc Cosecant.
dmat/cscd Cosecant of argument in degrees.
dmat/csch Hyperbolic cosecant.
dmat/exp Exponential.
dmat/expm1 Compute exp(x)-1 accurately.
dmat/fix Round towards zero.
dmat/floor Round towards minus infinity.
dmat/imag Complex imaginary part.
dmat/log Natural logarithm.
dmat/log1p Compute log(1+x) accurately.
dmat/log10 Common (base 10) logarithm.
dmat/log2 Base 2 logarithm.
dmat/pow2 Base 2 power.
dmat/real Complex real part.
dmat/reallog Natural logarithm of real number.
dmat/realsqrt Square root of number greater than or equal to zero.
dmat/round Round towards nearest integer.
dmat/sec Secant.
dmat/secd Secant of argument in degrees.
dmat/sech Hyperbolic secant.
dmat/sign Signum.
dmat/sin Sine.
dmat/sind Sine of argument in degrees.
dmat/sinh Hyperbolic sine.
dmat/sqrt Square root.
dmat/tan Tangent.
dmat/tand Tangent of argument in degrees.
dmat/tanh Hyperbolic tangent.

---Please use the following with caution, since implementation is only provided for some special cases.
transpose_grid Redistributes a dmat by transposing its grid.
dmat/conv2 Two dimensional convolution.
dmat/ctranspose ' Complex conjugate transpose.
dmat/dct Distributed discrete cosine transform.
dmat/fft Discrete Fourier transform on a distributed matrix.
dmat/find Find indices of nonzero elements.
dmat/gt > Greater than.
dmat/ldivide .\ Left array divide.
dmat/minus - Minus.
dmat/mtimes * Matrix multiply.
dmat/plus + Plus.
dmat/power .^ Array power.
dmat/rdivide ./ Right array divide.
dmat/summation Takes a sum of several DMATs.
dmat/times .* Array multiply.
dmat/transpose .' Transpose.

---------- Library Functions That a User Might Use ----------
binfun Binary operation for non-distributed arrays.
global_block_range Returns the range of indices in the specified dimension.
global_block_ranges Returns the range of indices in the specified dimension.
global_range Returns the range of indices in the specified dimension.
global_ranges Returns the range of indices in the specified dimension.
global_ind Returns the indices in the specified dimension.
global_inds Returns indices in the specified dimension.
g_mat No-op if the input argument is a DOUBLE.
local No-op if the input argument is a DOUBLE.
put_local Mimics dmat/put_local on a DOUBLE.
remap Remaps a distributed array.
dmat/binfun Binary elementwise operation.
dmat/global_block_range Returns the ranges of global indices local to the current processor.
dmat/global_block_ranges Returns the ranges of global indices for all processors in the map of distributed array D on all processors in communication scope.
dmat/global_ind Returns the global indices local to the current processor.
dmat/global_inds Returns the global indices for all processors in the map of distributed array D.
dmat/global_range Returns the ranges of global indices local to the current processor.
dmat/global_ranges Returns the ranges of global indices for all processors in the map of distributed array D.
dmat/local Returns the local part of the distributed array.
dmat/ndims Returns the number of dimensions of the DMAT.
dmat/put_local Assigns new data to the local part of the distributed array.
map/sparse Sparse distributed array. SPALLOC is the recommended method of creating sparse distributed arrays.

---------- Library Functions ----------
agg No-op if the input argument is a DOUBLE.
agg_all No-op if the input argument is a DOUBLE.
exists_falls_intersection Returns TRUE is the intersection of two FALLS is non-empty, FALSE otherwise.
falls_intersection Given two FALLS, find the intersection FALLS.
gen_falls Generates the local falls structure for processor P from the given PITFALLS structure.
gen_pitfalls Given the number of processors, distribution spec, and the length of the dimension, generates all the PITFALLS information.
get_global_ind Returns a cell array of global indices stored locally given a FALLS object
get_ind_range Creates index ranges.
get_local_falls Given the PITFALLS object, the grid and local processor rank, returns an array of local FALLS objects.
get_local_ind Returns a cell array of local indices given an array of global indices on the current processor and an array of global indices that are being referenced.
get_local_proc Returns the rank of the processor that contains index IND.
local_dims Given the local FALLS and object dimension, calculates a vector of local dimensions of the distributed object.
ls_intersection Finds an intersection of two line segments.
multicast Sends data from a single source to multiple destinations.
n_dim_find N-dimensional FIND functions.
pitfalls_intersection Computes intersection of two PITFALLS.
reconstruct Given collected distributed data in grid layout, reconstructs the original object according to data distribution.
resize Helper function used by distributed mtimes.
dmat/agg Aggregates the parts of a distributed matrix on the leader processor.
dmat/agg_all Aggregates the parts of a distributed matrix on all processors in the map of the distributed matrix
dmat/dmat Distributed matrix constructor.
dmat/g_mat Aggregate parts of the distributed matrix.
dmat/grid Returns the processor grid onto which the distributed array is mapped.
dmat/pitfalls Returns the pitfalls structure associated with the distributed array.
dmat/submat Helper function used by subsref.
dmat/subsasgn_data Helper function for distributed array subsasgn.
dmat/subsasgn1D One dimensional subsasgn.
dmat/subsasgn2D Two dimensional subsasgn.
dmat/subsasgn3D Three dimensional subsasgn.
dmat/subsasgn4D Four dimensional subsasgn.
map/inmap Checks if a processor is in the map.
map/transpose Transposes a map.

Copyright (c) 2005, Massachusetts Institute of Technology.  All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  • Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
  • Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
  • Neither the name of the Massachusetts Institute of Technology nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

MATLAB® is a registered MathWorks trademark. Reference to commercial products, tradenames, trademarks, or manufacturer does not constitute or imply endorsement.

top of page