Newsgroups: comp.parallel.mpi
From: lederman@b125.super.org (Steve Huss-Lederman)
Subject: Re: Reducing overhead
Organization: Supercomputing Research Center, Bowie MD, USA
Date: Fri, 7 Jul 1995 12:46:12 GMT
Message-ID: <LEDERMAN.95Jul7084612@b125.super.org>

The question was raised on how to avoid the three copies in MPI to
reduce the overhead.  The amount of copying done is implementation
dependent.  Thus, it is not the case that all MPIs will do 3 copies.
The difficulity is that the MPI standard (very deliberately and
correctly) does not specify how buffering is done in a standard send.
Some time you will get buffering and thus copying.  However, some
implementation only buffer small messages and don't for longer
messages (where small can sometimes be set by the user).  One way I
have used to aviod buffering is to make sure the recv is posted before
the send begins.  This usually requires an extra message from the
receiver to the sender.  This method may help but it can also hurt in
terms of latency if the MPI implementation is good.  Another point is
that buffering can REDUCE the APPLICATION latency for small message.
If the processes are not well synchronized, then the send can get
buffered on the receiver.  When the recv is posted it can be
fullfilled by a LOCAL memory copy from system space to user space.
This whole issue is tough and make decisions is not easy.

Steve

