Newsgroups: comp.parallel.mpi
From: solt@cs.uiuc.edu (David Solt)
Subject: Re: Performance problems using mpi on the SGI Power Challenge
Organization: University of Illinois at Urbana-Champaign
Date: 8 Oct 1996 17:44:03 GMT
Message-ID: <53e3t3$8a4@vixen.cso.uiuc.edu>

>Gang-scheduling may be the answer for MPI programs exhibiting highly
>synchronous communication, as already reported.  How about also providing
>a MPI_MAX_SPINWAIT environment variable, or something similar?  Then the
>user has control over the "cost" of "performance".
>
>It seems to me that any shared memory implementation of message passing,
>including mpich, could benefit from such a tunable parameter.
>
>Of course, taking advantage of a MAX_SPINWAIT type of parameter
>*effeciently* presupposes some support from the OS, and I don't know
>whether that exists in IRIX.  If not, a fast and flexible message passing
>implementation might be just the incentive SGI needs.
>
>Even a MAX_SPINWAIT parameter is limited because it is static, so there is
>still room for more elegant backoff algorithms.

I am working on my own implementation of MPI on the SGI Power
Challenge.  My approach is to continue to use gang scheduling, but to
allow processes to cooperate in data transfer.  Our initial tests show
that this works well in most circumstances.  Any data transfers over a
constant (MY_MPI_WORK_UNIT :)) are split up into chunks (size of
chunks dependes on the number of available spinning processors) and
placed in the work queues of any spinning processors.

Spinning processors then alternate between checking to see if their
outstanding send or recv is complete and checking their queue for
work.  My implementation is very careful and avoids waiting on locks 
wherever possible.  

The results have been interesting.  Even for data sizes as low as 8K,
it has shown profitable to allow this cooperation in data transfer.  I
know that this does not answer all of the problems of synchronization
and when to spin and when to block, but it is certainly worth
investigating and even if blocking was introduced, it might be worth
keeping this option so that a processor would first spend some cycles
doing any work in its work queue before blocking.  

Dave Solt




